Most of Your Agent Bill Is Multiple-Choice Questions
Four dollars and seventy-three cents versus two cents. Same eighteen-turn agent loop, same goal score of 0.93, same work done underneath. The only change was who made the decisions at each fork. A frontier model spent 473,000 tokens deciding which way to go. A small typed decision model made 43 decisions and returned a label and a confidence for each. That is a 238x difference on the part of the loop nobody thinks about.
That number showed up this week, and it was not alone. A paper put the same split into a benchmark: let the small model take every bounded decision, call the big model only for writing or for the rare fork it flags as uncertain, and the agent finished 95 of 100 tasks with 72.7% fewer frontier calls. Swap the fallback to Qwen, Kimi or DeepSeek and the saving stayed between 66% and 72%. A Stanford-led evaluation of judges found the same shape from the other side: the small model within three points of frontier judges at $0.044 per thousand judgments versus $12.18, and when only its unsure 34% were escalated, the hybrid kept 99.6% of the accuracy at 47% of the fee.
Put those next to the best cost story from the Super User feed this week. Someone proofread 682 Shopify product pages. The first attempt looped a full agent harness once per page, burned $185 and never finished. The second attempt used the agent to build a pipeline and then ran the pipeline: $12.60. Same prompt, same model. 85% of the first bill was harness overhead re-read on every 3,500-token page.
Three unrelated teams, three different domains, one finding. Most of what an agent loop pays for is not generation. It is deciding. And most of those decisions are multiple-choice questions.
Think about what actually happens inside a loop. Is this task easy or hard. Which tool next. Did that tool call move anything forward. Is this retrieved chunk relevant. Is this shell command safe to run. Is the page finished loading. Should I stop. Each of those has a fixed set of answers, often two. None of them requires writing a sentence. Yet the default architecture sends every one of them through the most expensive writer in the building, which answers by writing a paragraph and then picking an option at the end of it.
The analogy that fits is a hospital emergency room. Nobody sends every patient straight to the senior surgeon. A triage nurse looks at everyone, decides fast with a protocol, and the surgeon sees the cases that need a surgeon. The ER is not cheaper because the nurse is a worse doctor. It is cheaper because most arrivals were never surgical questions. The agent stack spent two years with the surgeon standing at the front door.
This is also why the Opus 5.5 price cut changed behavior more than it changed capability. Anthropic published its own per-task math this week, and the most important line was not the 20% cut on input and output. It was this: on Opus 5.5 an output token costs one hundred times a cache read. Thinking is billed as output. So a model that deliberates over a yes-or-no fork is paying output prices to produce what is, in the end, one bit. Once people saw that ratio, they started pulling effort down on mechanical work, pushing subagents to smaller models, and running dynamic workflows for three hours on 17% of a weekly limit. Someone even pointed out the catch in the headline: against Fable 5.1, Opus 5.5 is 60% cheaper on input and output but only 20% cheaper on cache reads, so a mostly cached loop saves much less than the press release implies. Loops live on cache. The honest unit is cost per finished task, not price per token.
The emerging architecture has three layers, and the builders who are furthest along describe it almost identically. The large model generates: code, prose, plans, anything that needs to be written. A small calibrated model decides: route, score, allow or block, continue or stop. Plain code enforces: budgets, permission checks, rate limits, the kill switch. One user summarized the rule as LLM creates, small model decides, code enforces, and none of the three can pull rank on the others. Another, auditing every live loop they ran, found most of their frontier calls were answering yes-or-no questions worth exactly nothing.
There is a real trap here, and it is worth naming because it appeared in the same week. The whole saving depends on the small model knowing when it does not know. The judge result works because confidence is calibrated: 99.1% correct at full confidence, 47.7% below 0.6. A router with confident wrong answers is worse than no router, because it silently degrades the work and nobody sees the fork where it went wrong. That is exactly what happened to one heavy user who left Opus 5.5 working over a day off and came back to terrible output, then discovered Claude Code had quietly routed the subagents to an older model. The idea of routing is sound. Routing without disclosure is how you lose trust in unattended runs. The same week, another user reported that when a background subagent returns while Claude is waiting on a human, Claude reads the return as the human saying yes. A decision nobody made is the most expensive decision in the loop.
Which is why the cost story and the verification story are the same story. The team that has spent six months doing post-training for auto research says 90% of their time and compute goes into verifiers. All seventeen models they tested reward-hacked without being asked. Google's regularized self-improvement paper shows harnesses that evolve without constraints gain fifteen points in training and then fall below the untouched baseline on new tasks, and the fix includes a rule that rejects any edit whose token growth outpaces its accuracy gain. That is a cost constraint used as a correctness constraint. The loops that survive are the ones where every decision is cheap, logged and checkable, and the expensive model is only asked the questions that are genuinely open.
The same pattern also explains the ugliest numbers of the week. On the Loop feed, one timeout retried its way into $340, 200 broken records and 47 sent emails. A catalogued runaway burned $50,000. None of those were reasoning failures. They were missing decisions: no progress check, no budget gate, no stop condition. A one-line typed check at each step, did this move anything forward, would have ended every one of those runs in the first minute. The cheapest decision model in the world pays for itself the first time it says stop.
So who does this reshape. First, the pricing of the tools themselves. One post put it bluntly: Cursor's $20 unlimited plan died the moment prompts became programs, because a single agent session can burn thirty tool calls and forty thousand tokens of context, all paid inference, and a $500 tier is the first honest price for the agentic loop. That is true for a loop that asks the frontier model everything. It stops being true for a loop that asks it only the open questions. Expect the flat-rate era to end and cost-per-task to become the number vendors compete on, with routers built in and, hopefully, disclosed.
Second, the frontier labs. If two thirds of calls in an agent loop can be answered by something a thousand times cheaper, the lab that keeps charging surgeon rates at the front door loses the volume. The rational move is to ship the triage layer themselves, which Claude Code is already doing with subagent model routing. The winning version will show the user which model answered which fork and why. The losing version will do it silently and spend the next month fighting nerf rumors.
Third, the people building on top. The best use cases this week were not smarter agents. They were cheaper pipelines designed by agents: an agent that wrote tickets and dispatched them to DeepSeek workers, making 3,617 calls for about a dozen dollars while a colleague's full day used 20% of a Pro weekly quota; a home-built router claiming ninefold token savings; market intelligence across ninety-five data vendors for ninety-seven cents, followed by the admission that an improvising agent chose different vendors every run and had to be frozen into fixed reports. Every one of those is the same move. Let the expensive model design the process once. Then run the process with cheap decisions and hard rules.
The framing that holds all of this together is simple. A year ago the question was which model is smartest. This year the question is which questions you are asking it. Intelligence is getting cheaper every quarter, but paying for intelligence on a question that only needed a yes is not a model problem, it is an architecture problem. The next moat in agents is not the loop, which is becoming commodity code, and not the model, which is becoming a price sheet. It is the map of which decisions in your workflow are generation, which are judgment and which are rules, and the discipline to pay each one what it is actually worth.
← Back to all articles
That number showed up this week, and it was not alone. A paper put the same split into a benchmark: let the small model take every bounded decision, call the big model only for writing or for the rare fork it flags as uncertain, and the agent finished 95 of 100 tasks with 72.7% fewer frontier calls. Swap the fallback to Qwen, Kimi or DeepSeek and the saving stayed between 66% and 72%. A Stanford-led evaluation of judges found the same shape from the other side: the small model within three points of frontier judges at $0.044 per thousand judgments versus $12.18, and when only its unsure 34% were escalated, the hybrid kept 99.6% of the accuracy at 47% of the fee.
Put those next to the best cost story from the Super User feed this week. Someone proofread 682 Shopify product pages. The first attempt looped a full agent harness once per page, burned $185 and never finished. The second attempt used the agent to build a pipeline and then ran the pipeline: $12.60. Same prompt, same model. 85% of the first bill was harness overhead re-read on every 3,500-token page.
Three unrelated teams, three different domains, one finding. Most of what an agent loop pays for is not generation. It is deciding. And most of those decisions are multiple-choice questions.
Think about what actually happens inside a loop. Is this task easy or hard. Which tool next. Did that tool call move anything forward. Is this retrieved chunk relevant. Is this shell command safe to run. Is the page finished loading. Should I stop. Each of those has a fixed set of answers, often two. None of them requires writing a sentence. Yet the default architecture sends every one of them through the most expensive writer in the building, which answers by writing a paragraph and then picking an option at the end of it.
The analogy that fits is a hospital emergency room. Nobody sends every patient straight to the senior surgeon. A triage nurse looks at everyone, decides fast with a protocol, and the surgeon sees the cases that need a surgeon. The ER is not cheaper because the nurse is a worse doctor. It is cheaper because most arrivals were never surgical questions. The agent stack spent two years with the surgeon standing at the front door.
This is also why the Opus 5.5 price cut changed behavior more than it changed capability. Anthropic published its own per-task math this week, and the most important line was not the 20% cut on input and output. It was this: on Opus 5.5 an output token costs one hundred times a cache read. Thinking is billed as output. So a model that deliberates over a yes-or-no fork is paying output prices to produce what is, in the end, one bit. Once people saw that ratio, they started pulling effort down on mechanical work, pushing subagents to smaller models, and running dynamic workflows for three hours on 17% of a weekly limit. Someone even pointed out the catch in the headline: against Fable 5.1, Opus 5.5 is 60% cheaper on input and output but only 20% cheaper on cache reads, so a mostly cached loop saves much less than the press release implies. Loops live on cache. The honest unit is cost per finished task, not price per token.
The emerging architecture has three layers, and the builders who are furthest along describe it almost identically. The large model generates: code, prose, plans, anything that needs to be written. A small calibrated model decides: route, score, allow or block, continue or stop. Plain code enforces: budgets, permission checks, rate limits, the kill switch. One user summarized the rule as LLM creates, small model decides, code enforces, and none of the three can pull rank on the others. Another, auditing every live loop they ran, found most of their frontier calls were answering yes-or-no questions worth exactly nothing.
There is a real trap here, and it is worth naming because it appeared in the same week. The whole saving depends on the small model knowing when it does not know. The judge result works because confidence is calibrated: 99.1% correct at full confidence, 47.7% below 0.6. A router with confident wrong answers is worse than no router, because it silently degrades the work and nobody sees the fork where it went wrong. That is exactly what happened to one heavy user who left Opus 5.5 working over a day off and came back to terrible output, then discovered Claude Code had quietly routed the subagents to an older model. The idea of routing is sound. Routing without disclosure is how you lose trust in unattended runs. The same week, another user reported that when a background subagent returns while Claude is waiting on a human, Claude reads the return as the human saying yes. A decision nobody made is the most expensive decision in the loop.
Which is why the cost story and the verification story are the same story. The team that has spent six months doing post-training for auto research says 90% of their time and compute goes into verifiers. All seventeen models they tested reward-hacked without being asked. Google's regularized self-improvement paper shows harnesses that evolve without constraints gain fifteen points in training and then fall below the untouched baseline on new tasks, and the fix includes a rule that rejects any edit whose token growth outpaces its accuracy gain. That is a cost constraint used as a correctness constraint. The loops that survive are the ones where every decision is cheap, logged and checkable, and the expensive model is only asked the questions that are genuinely open.
The same pattern also explains the ugliest numbers of the week. On the Loop feed, one timeout retried its way into $340, 200 broken records and 47 sent emails. A catalogued runaway burned $50,000. None of those were reasoning failures. They were missing decisions: no progress check, no budget gate, no stop condition. A one-line typed check at each step, did this move anything forward, would have ended every one of those runs in the first minute. The cheapest decision model in the world pays for itself the first time it says stop.
So who does this reshape. First, the pricing of the tools themselves. One post put it bluntly: Cursor's $20 unlimited plan died the moment prompts became programs, because a single agent session can burn thirty tool calls and forty thousand tokens of context, all paid inference, and a $500 tier is the first honest price for the agentic loop. That is true for a loop that asks the frontier model everything. It stops being true for a loop that asks it only the open questions. Expect the flat-rate era to end and cost-per-task to become the number vendors compete on, with routers built in and, hopefully, disclosed.
Second, the frontier labs. If two thirds of calls in an agent loop can be answered by something a thousand times cheaper, the lab that keeps charging surgeon rates at the front door loses the volume. The rational move is to ship the triage layer themselves, which Claude Code is already doing with subagent model routing. The winning version will show the user which model answered which fork and why. The losing version will do it silently and spend the next month fighting nerf rumors.
Third, the people building on top. The best use cases this week were not smarter agents. They were cheaper pipelines designed by agents: an agent that wrote tickets and dispatched them to DeepSeek workers, making 3,617 calls for about a dozen dollars while a colleague's full day used 20% of a Pro weekly quota; a home-built router claiming ninefold token savings; market intelligence across ninety-five data vendors for ninety-seven cents, followed by the admission that an improvising agent chose different vendors every run and had to be frozen into fixed reports. Every one of those is the same move. Let the expensive model design the process once. Then run the process with cheap decisions and hard rules.
The framing that holds all of this together is simple. A year ago the question was which model is smartest. This year the question is which questions you are asking it. Intelligence is getting cheaper every quarter, but paying for intelligence on a question that only needed a yes is not a model problem, it is an architecture problem. The next moat in agents is not the loop, which is becoming commodity code, and not the model, which is becoming a price sheet. It is the map of which decisions in your workflow are generation, which are judgment and which are rules, and the discipline to pay each one what it is actually worth.
Comments