AI model cost per task: how to measure what a workflow really costs
The short answer
Price per million tokens is a supplier's metric. Cost per finished task is yours. To get it, log four numbers per request: input tokens, output tokens, retries, and the model that served it. Then divide by tasks completed, not calls made. In the worked example below, a 10,000-conversation support workflow drops from $190 a month to about $36 with one routing rule, an 80 percent cut.
Why per-token pricing misleads
A model at $0.10 per million input tokens looks twenty times cheaper than one at $2.00. That comparison breaks the moment the cheap model needs three attempts and a retrieval step to answer what the expensive model answers once.
Agents make this worse. A chat reply might read 2,000 tokens. An agent that plans, calls four tools, reads results back and retries on failure can read 200,000. The same task, one hundred times the input.
So a 90 percent price cut and a tenfold volume increase nets out to a bigger bill. That is exactly what happened in 2026: prices fell, spend rose.
The four numbers
Log these for every request, and you can answer any cost question later.
| Metric | Where it comes from | Why it matters |
|---|---|---|
| Input tokens | API response usage field | The biggest line for agents |
| Output tokens | Same response | Usually 3 to 5 times the input price |
| Model served | Response metadata | Routing decides most of the bill |
| Retries | Your wrapper, not the API | The invisible multiplier |
If you cannot see the model that actually served a request, you cannot audit spend. Providers change routing without telling you.
Worked example
Take a support reply workflow handling 10,000 conversations a month.
On a mid-tier model at $2 per million input and $10 per million output, with 6,000 input and 700 output tokens per conversation, that is $0.019 per conversation, or about $190 a month.
Now send the same work to a small model at $0.10 and $0.50, and let it escalate 15 percent of conversations to the mid-tier model. The small tier handles 85 percent at $0.00095 each, the mid-tier handles the rest at $0.019. Total is about $36 a month.
Same outcome, roughly 80 percent less spend. No negotiation, no new vendor, one routing rule.
Where the savings actually are
Caching. Anthropic's prompt caching and similar features cut repeat context by 50 to 90 percent. If your system prompt is 4,000 tokens and you send it 10,000 times, caching it is the single largest saving available to you.
Routing. Classify the task first, then pick the model. Anything that is extraction, summarising or formatting does not need a frontier model. Anything that is judgment or multi-step reasoning does.
Trimming context. Most agents carry the entire conversation. Half of it is usually irrelevant to the current step. Retrieval beats stuffing, both for cost and for quality.
Batching. Where latency does not matter, batch endpoints are cheaper and often rate-limited less aggressively.
What to put on a dashboard
Four lines are enough: cost per completed task, share of traffic on the cheapest tier, retry rate, and cache hit rate. If cost per task rises while price per token falls, the problem is in your workflow, not your vendor.
Review weekly. Anything that moves more than 20 percent in a week has a cause you can find, usually a prompt change, a new tool, or a model swap upstream.
The uncomfortable part
Cheap tokens make waste invisible until the invoice arrives. A team that never measured tokens per task will not notice a prompt that grew from 800 to 6,000 tokens across three releases. Instrument first, optimise second.
That is also why the 2026 price war has not made AI budgets easier to forecast. The unit got cheaper and the work got bigger, and only one of those two numbers appears on a pricing page.
Read next: why token prices fell 94 percent and bills still rose, and what gets expensive when tokens are cheap.
Questions people ask
How do I calculate AI cost per task?
Take the API response for one request, multiply input tokens by the input price and output tokens by the output price, add any retries, then divide total spend by tasks completed. A support reply on a mid-tier model at $2 input and $10 output per million tokens works out at about $0.019.
Why is cost per token a bad metric on its own?
Because it ignores how many tokens a finished task actually uses. A chat reply reads about 2,000 tokens, while an agent that plans, calls tools and retries can read 200,000 for the same task.
What is the fastest way to cut AI spend?
Routing. Send extraction, summarising and formatting to a small model and escalate only the harder cases. In the worked example, routing 85 percent of a 10,000-conversation workflow to a $0.10 model took the bill from about $190 a month to about $36.