At a Glance
Training is a one-time bill. Inference compounds.GPT-4 cost over $100M to train. Serving GPT-4o at ChatGPT-scale usage costs over $350M per month.
The crossover happens fast.At 100 million daily users, cumulative inference spend overtakes a $100M training budget in 9 days. At 10 million users, it takes 11 months.
After one year at scale, inference owns the ledger.At ChatGPT-level usage on 100M DAU, 97.7% of total spend goes to serving. Training is a rounding error.
Price cuts cannot outrun user growth.Halving per-token cost doubles the crossover window. Doubling users cuts it in half. User growth wins that race.
Inference efficiency is the real moat.The lab that serves cheapest captures deployments. Training gets conference talks. Serving gets customers.
Training a frontier model costs nine figures. Serving it at ChatGPT scale costs nine figures every month. The AI industry tracks the first number because it makes headlines. The second number determines whether the product survives.
Training is fundraising. Inference is unit economics.
The Headline Number
Sam Altman told an MIT audience in April 2023 that GPT-4 cost more than $100 million to train. That figure became the reference point for every article about AI economics. Epoch AI, working from hardware amortization and energy data, estimated the final training run at $40 million, though their methodology notes individual estimates can vary by several-fold. Google's Gemini Ultra landed near $30 million on the same methodology.
These numbers are real. They are also bounded. A training run ends. A serving bill compounds with every user on every request.
A lab raises a funding round, rents a cluster, trains for months, and ships a model. The bill arrives once. Press releases quote it. Training cost is the number everyone knows.
The Number Nobody Tracks
Inference is different. Every query burns GPU cycles. Daily active users multiply that burn by their session count.
Training scales with ambition. Inference scales with success.
We modeled the crossover point using July 2026 list API pricing from OpenAI, Anthropic, and Google. Our base scenario: five queries per user per day, 800 input tokens and 400 output tokens per query ($0.006 per query on GPT-4o). A ChatGPT-scale scenario uses 13 queries per user per day with 1,200 input and 600 output tokens ($0.009 per query), based on OpenAI's reported 2.5 billion prompts per day. We infer 193 million DAU by dividing that figure by 13 prompts per daily user.
At 10 million DAU, GPT-4o inference at $2.50/$10.00 per million tokens crosses a $100 million training budget in 11.1 months. Monthly inference at that scale runs $9 million, or $0.90 per user. One million DAU pushes crossover past nine years.
Scale changes everything.
At 100 million DAU under the ChatGPT-scale scenario ($0.009 per query), the $100 million training line falls in 9 days. Monthly inference exceeds $351 million. After 12 months, inference accounts for 97.7% of total spend: $100 million in training against $4.21 billion in serving costs.
The math is transparent. Cost per query on GPT-4o at list rates: $0.006 in the base scenario, $0.009 in the ChatGPT-scale scenario, multiplied by daily queries, daily active users, and 30 days. List pricing overstates internal serving costs at labs running their own silicon, but the shape holds.
Cut the per-token price in half and the 10M-DAU crossover moves from 11 months to 22. Double the user base and it drops to 5.5.
Why Inference Scales Differently
Training behaves like a fixed cost per model generation. You set the parameter count, the data mix, and the compute budget. The bill is locked before the first token ships to a user.
Inference is variable cost. Each GPT-4o user costs roughly $0.90 per month at list rates under our base scenario. At 100 million users, that is $90 million per month in serving costs alone.
A startup can raise $50 million to train a competitive model. It cannot raise $50 million every month to serve one at a loss. OpenAI reported 900 million weekly active users in February 2026. At that scale and ChatGPT-level usage patterns, list-rate inference would exceed $650 million per month.
Rising training budgets do not change the ratio. A $500 million training run at ChatGPT-scale usage on 100 million DAU still hits the crossover in six weeks. Training costs double with each model generation. User growth moves faster.
the AI Economics series titled, The Prescott Moment, established that commercial AI crossed from capability-constrained to compute-constrained. The training tax is the other half of that shift.
At 10 million DAU, inference and training split roughly even after one year: 51.9% inference, 48.1% training on a $100 million budget. Below 1 million DAU, training still dominates the ledger. Above 100 million, inference takes 91.5% even in the conservative base scenario.
The crossover window is narrow. Most successful deployments blow through it fast.
The Pricing War
API pricing has dropped sharply since GPT-4 launched at $30/$60 per million tokens in 2023. GPT-4o now lists at $2.50/$10, GPT-4.1 at $2.00/$8, Claude Sonnet 5 at $3.00/$15 after its introductory period, Gemini 2.5 Pro at $1.25/$10 for standard-length prompts. Flash-tier models from Google and OpenAI have fallen below $0.20/$0.60.
Price cuts help. They do not repeal the scaling law. If query volume grows faster than per-token costs fall, total inference spend still rises.
A 50% price reduction doubles the DAU needed to hit the same crossover point. Doubling DAU cuts the crossover period in half. User growth wins that race at scale.
Provider competition compresses margins on the serving side. The lab that serves at the lowest cost per query captures the deployments that matter. Training efficiency gets conference talks. Inference efficiency gets customers.
Per-query cost on Claude Sonnet 5 at standard rates runs 40% higher than GPT-4o in our base scenario. Sonnet hits the $100 million crossover at 10M DAU in 7.9 months instead of 11.1.
Who Pays the Tax
Labs that optimize only for training clusters are solving half the problem. The next competitive advantage is inference efficiency: quantization, speculative decoding, mixture-of-experts routing, distillation to preserve capability while cutting per-query compute. Meta open-sourced Llama so others would pay the serving bill. OpenAI ships GPT-4o mini because margin on inference volume beats margin on training prestige.
Inflection AI raised $1.3 billion and shipped Pi, a consumer chatbot. Within a year, its CEO and co-founder moved to Microsoft and the company pivoted to enterprise APIs. Building the model was the fundraising story. Surviving the inference bill was the business story nobody wrote.
Training is fundraising. Inference is unit economics.
A model that costs $100 million to train and $350 million per month to serve needs massive subscription revenue or a path to cheaper inference within weeks of launch. At nine-figure scale, there is no third option. Article 3, The Economics of AI Deployment, which is next in the AI Economics series, takes up who can actually afford to run these models.
The Prescott Moment identified the bottleneck: compute. The training tax pins down where that compute budget burns. Training cost determines who enters the frontier race. At $9 million per month for every 10 million users on GPT-4o, inference cost determines who stays.
Sources