Token Prices Collapsed. Your AI Bill Went Up. Here Is the Arithmetic

Your price per token fell. Your invoice did not. A bill is price multiplied by volume, and volume is where agents live: Gartner predicts inference cost per agentic workflow rises more than fivefold through 2028. This is the arithmetic behind the surprise, and the four line items most budgets never had.

The arithmetic in one line

Your AI bill is three numbers multiplied together. Price per token, tokens per task, and tasks per month.

Vendors publish the first number and compete hard on it. Nobody publishes the other two, because your own architecture sets them.

Between 2023 and 2026 the first number fell by an order of magnitude or more. In the same window the second rose by a larger factor. Reasoning models think in tokens you pay for, and agents repeat the loop. The third rose too, as pilots became production workflows.

Two of the three terms moved against you. That is the whole story, and it is why a price cut announcement is not a budget event.

How far prices actually fell

The price collapse is real and it is large. It is also measured in a way that flatters the headline.

Epoch AI tracked the price of reaching a fixed capability milestone on six benchmarks over three years. It found declines ranging from 9x to 900x per year depending on the task. The price of GPT-4 level performance on PhD-level science questions fell about 40x a year. Epoch adds a caveat most write-ups drop: the fastest drops happened in the most recent year, so persistence is not established.

A cheaper price for the same capability is not a cheaper workload

Those measurements hold capability constant and let price fall. Your team does the opposite. It holds the budget roughly constant and spends the saving on more capability, more context and more attempts.

The OpenRouter study of real traffic found demand showed weak price elasticity. Buyers did not move to the cheapest adequate model when prices fell. They bought better answers instead.

Goldman Sachs Research put the supply side at 60% to 70% cost reduction per token per year, then forecast token consumption multiplying 24 times by 2030. That is this same argument, told from the seller's chair. The margin side of it sits in why AI gross margins run below classic software.

What grew faster than prices fell

This is the part with hard measurement behind it, and the part budget owners rarely see.

Researchers from OpenRouter and a16z analysed over 100 trillion tokens of real inference traffic between November 2024 and November 2025. Two findings matter here.

The prompt grew four times faster than the answer

Prompt tokens per request rose from roughly 1,500 to over 6,000. Completion tokens rose from about 150 to 400. Average sequence length went from under 2,000 tokens in late 2023 to over 5,400 by late 2025.

Read that split carefully. The growth is concentrated in what you send, not in what you get back. Retrieved documents, tool definitions, system instructions and conversation history are now the bulk of the payload.

Reasoning tokens crossed half of all traffic

The same study found the share of tokens routed through reasoning-optimised models climbed from negligible in early 2025 to over 50% by late 2025. Reasoning tokens are billed as output, and the user never reads them.

Pega's operating and finance chief Ken Stillwell described reasoning tokens to AI Business in August 2026 as a black box. That is fair for a cost line you cannot inspect, cannot cap per request, and cannot forecast from last quarter's usage.

What an agent does to one task

A chatbot turn is one call. An agent turn is a loop.

The agent plans, calls a tool, reads the result, checks itself, and tries again. Each step re-sends the accumulated history and tool context. Consumption therefore grows with task complexity instead of staying flat per interaction.

Gartner named this the Inference Paradox on 17 August 2026, defining it as better unit economics escalating the overall cost of AI with no clear path to matching value. Senior director analyst Will Sommer put the mechanism plainly: a chatbot answers, while an agent must constantly reason, negotiate and question itself.

The multiplier is a range, not a number

Published multipliers vary because workloads vary. Gartner's floor is 5x for routing a task to an agentic reasoning model. Coverage of McKinsey's 2026 work reports multi-turn agentic workflows consuming up to 1,000 times the tokens of a single-turn chat or code query.

A three order of magnitude spread is not a measurement, it is a warning. My view is that any multiplier quoted without the workload attached is marketing. The only number worth trusting comes from your own logs.

Token figures come from the OpenRouter traffic study. The call count and output split are qualitative, drawn from how Gartner and McKinsey describe agentic workloads.

Where the money actually goes

If you only budget for the answer, you have budgeted for a fraction of the invoice.

McKinsey's July 2026 analysis of agentic economics found 60% of agentic AI cost sitting in response refinement rather than in the first answer. Refinement is the retries, the self-checks, the reranking and the second pass that makes output usable.

Four line items your budget probably does not have

Each of these is real spend that arrives as an undifferentiated token charge.

On that last line, cost tooling vendor PointFive published a study on 6 August 2026 covering actual provider bills across 2,908 paid coding sessions. It reported roughly 80% of the bill going to prompt cache traffic, and only 1.3% to text the model saw for the first time. Treat that as vendor research rather than independent measurement, which is what the authors themselves say while publishing their methods.

The fix is not to send less. It is to send the same thing in a form the provider charges less for. Anthropic prices cache reads at 0.1 times standard input, with batch processing at half price, and those multipliers stack. If your prefix is stable and uncached, you are paying roughly ten times the necessary rate on the largest part of the payload.

What the spending data shows

Enterprise behaviour matches the arithmetic rather than the price list.

Ramp's August 2026 AI Index, drawn from its token spend management product, found the top 1% of businesses spending a median $7,400 per employee on AI in July. In January the same group spent $2,590. The top 10% went from $281 to $650 over those seven months, while the median firm sat at $11.95.

Note the shape of that distribution before you benchmark against it. The heaviest spenders run budgets hundreds of times larger per head than a typical business, so an average here means very little.

The forecasts point the same way. Gartner puts worldwide AI spending at $2.59 trillion in 2026, up 47%, with agentic AI at $201.9 billion. Goldman Sachs expects 120 quadrillion tokens a month by 2030, a 24-fold rise driven mostly by enterprise agents. Where that build gets funded is the subject of the analysis of hyperscaler capex and its missing payback math.

The only unit that survives contact with an agent

Price per million tokens is now a poor proxy for what you will pay. Two models at the same list price can differ by a large factor on the same job, because one is more verbose or reasons for longer.

Cost per completed task is the unit that holds. It absorbs verbosity, reasoning length, retries and tool calls into one number you can compare, forecast and put in a contract.

I would stop tracking price per million tokens as a management metric. It belongs in procurement, next to the rate card, and nowhere near a budget forecast. The operational metric is cost per resolved ticket, per merged pull request, per processed claim. Ask any vendor for that number on a named workload, with the sample size and failure rate attached, as in the piece on where measurable AI return has appeared.

Where this argument is weakest

Three places, and the first undermines the obvious response to everything above.

Cutting tokens does not reliably cut the bill

The PointFive study found a 38.4% reduction in tool-output tokens producing a 6.8% increase in billed cost. Compressing context forced the agent to spend extra steps retrieving what it had just discarded. One tested tool raised cost by about half with no measurable improvement.

That result comes from a vendor with a commercial interest, on a narrow coding workload. It is still the most useful finding here, because it says the intuitive optimisation can make things worse.

Rising spend can be the correct outcome

An over-budget rate of 93% describes a forecasting failure, not necessarily a spending failure. If a token bill displaces engineering time, the bill rising is the plan working.

The weak elasticity in the OpenRouter data supports that reading. Buyers spent the price saving on capability because capability was worth more to them. I think that is mostly rational, and it makes the interesting question what changed, not what was spent.

The numbers here have real limits

The McKinsey FinOps figure rests on 75 respondents. Ramp's token data skews toward technology companies. Gartner's fivefold and Goldman's 24-fold are forecasts, not measurements. The Epoch price series ends before the current agent wave, so it prices capability rather than your workload.

Frequently asked questions

Where to start this week

Pick your single largest AI workload and answer two questions about it.

First, what does one completed task cost, including retries and reasoning tokens? Divide last month's spend for that workload by finished units of work, not by API calls. That ratio is your real price, and it is the one to trend.

Second, what share of your prompt is identical on every call? If it is large and uncached, you have a pricing fix available today that changes no model and no behaviour.

Then set a hard monthly ceiling per agent before the next rollout, not after it. Every over-budget firm in the McKinsey sample had a budget. What they lacked was a unit that scaled with the workload.

Related reading