I looked at last quarter’s bill for an agent that summarises ticket queues for a forty-person engineering team. The input cost was flat against the year before. The output cost had tripled. Nothing about the workload had changed.
Two things happened to LLM pricing in the last twelve months. Input prices kept dropping — providers competed for context-window land. Output prices stopped dropping, and on some tiers went up. Reasoning models charge separately for the thinking tokens you never see. The asymmetry is now five to ten times: a token in costs ten cents per million, the same token out costs a dollar.
Most agent code was written when the two sides cost roughly the same. So the prompts ask the model to “think step by step” and “explain your reasoning” and “return the answer in JSON with a rationale field”. Each of those instructions is a tax on the most expensive part of the bill, paid every call, forever.
Runnable artifact
Output token tax calculator
Client-side onlyAt these prices, output tokens are 84% of the bill even though they are 35% of tokens.
Where the tax is hiding
Three places, in roughly descending order:
- Reasoning fields you never read. The agent fills in
rationaleso the prompt feels rigorous. Nobody reads it downstream. Delete the field. - Restated context. “Based on the user’s request to update their billing address, I will…” — the model is parroting the input back at output prices. Tell it not to.
- Verbose tool plans. “First, I will call
get_user. Then I will callupdate_address. Then…” Multi-tool reasoning loops generate output before each call. Move the planning to a singleextract_intentcall and let the executor be silent.
The diagnostic
Print the average output tokens per call from your fleet. If it’s bigger than 200, you have prose where you wanted JSON. If it’s bigger than 1,000, you have an essay generator.
Cut prompts. Tighten schemas. Stop asking models to think out loud unless you want a transcript. The bill rewards brevity now in a way it didn’t a year ago.
The strongest case against thisVerbosity can be cheaper than ambiguity
The essay treats output tokens as waste too quickly; in some agent systems, extra words are the audit trail that prevents a much more expensive support or safety failure.
- What would make me wrong
- Verbose rationale fields materially reduce escalations, bad tool calls, or debugging time enough to beat their token cost after measurement.
- Last reviewed
The strongest case against the output-token argument is that it optimizes the visible bill while underpricing the invisible one. A terse JSON answer is cheap until a human has to reconstruct why the agent took a dangerous action, why it skipped a candidate, or why it called the wrong tool. In regulated, support-heavy, or safety-sensitive workflows, a rationale field is not prose decoration. It is an evidence trail.
The essay also assumes nobody reads the verbose fields. That may be true for many internal agents, but it is not generally true. Some teams pipe rationales into review queues, QA sampling, fraud investigations, or customer-support summaries. If those fields turn a 20-minute investigation into a 2-minute one, the output-token spend is not waste. It is labor substitution.
The better version of the claim is narrower: delete verbose output that no downstream human, evaluator, or safety process consumes. Do not delete explanation merely because it is expensive. First prove it is unused.