Three Papers, One Bill: Where Agent Tokens Actually Go
Every frontier launch this month led with cost per task. Three papers in Thursday's arXiv batch take the same number apart from the research side, and none of them needs a bigger model.
Timeout Cascades (https://arxiv.org/abs/2611.24324, submitted November 18) ran 36 open-weight models released since 2024 on SWE-bench Verified. Small models are now close on easy and medium tasks, and the gap is concentrated in the hard ones. So the authors do the dumbest possible thing: try the small model first, escalate to the big one when it times out. That cuts total GPU hours by up to 33%, and 16.4% across all SWE-bench Verified tasks, without losing accuracy. The striking line is that this plain cascade nearly matches an oracle router that knows in advance which model will succeed. No learned router, almost no engineering.
AnyS1 (https://arxiv.org/abs/2611.23751, November 18, project page https://middle-chunjie.github.io/AnyS1/) attacks the same waste one level down. Most agent steps pick from a short list of known options, yet agents call the big model every time. AnyS1 mines those decision points from over 1,000 public skills, trains a single calibrated 4B pointer model as a System-1, and hands control back to the big System-2 model only when confidence drops below a threshold. Across three unseen benchmarks it holds reliability while cutting episode cost 75% to 91% and model-call time 63% to 92%. The neat part is the source: human-written skills already say where the choices are.
The third one is the strangest and the most quotable. Where the Tokens Go (https://arxiv.org/abs/2611.23375, November 17, with AlΓ‘n Aspuru-Guzik among the authors) builds a "software thermodynamics". Make an agent apply a change and undo it, over and over, and count every token. About 80% of the tokens in a successful run come from calls that leave the requested change exactly where it was. They call it heat. The estimated free energy of a change is 14% to 22% of the mean cost of a successful run, and coding costs 1.1 to 1.5 times as much as uncoding.
Put the three together and the bill reads differently. The expensive part of an agent is not the hard reasoning, it is the big model doing easy steps and the loop burning tokens without moving the work. Routing and compiling are where the next 5x in cost per task comes from, and labs that only sell the top model have little reason to tell you that.
← Back to all articles
Timeout Cascades (https://arxiv.org/abs/2611.24324, submitted November 18) ran 36 open-weight models released since 2024 on SWE-bench Verified. Small models are now close on easy and medium tasks, and the gap is concentrated in the hard ones. So the authors do the dumbest possible thing: try the small model first, escalate to the big one when it times out. That cuts total GPU hours by up to 33%, and 16.4% across all SWE-bench Verified tasks, without losing accuracy. The striking line is that this plain cascade nearly matches an oracle router that knows in advance which model will succeed. No learned router, almost no engineering.
AnyS1 (https://arxiv.org/abs/2611.23751, November 18, project page https://middle-chunjie.github.io/AnyS1/) attacks the same waste one level down. Most agent steps pick from a short list of known options, yet agents call the big model every time. AnyS1 mines those decision points from over 1,000 public skills, trains a single calibrated 4B pointer model as a System-1, and hands control back to the big System-2 model only when confidence drops below a threshold. Across three unseen benchmarks it holds reliability while cutting episode cost 75% to 91% and model-call time 63% to 92%. The neat part is the source: human-written skills already say where the choices are.
The third one is the strangest and the most quotable. Where the Tokens Go (https://arxiv.org/abs/2611.23375, November 17, with AlΓ‘n Aspuru-Guzik among the authors) builds a "software thermodynamics". Make an agent apply a change and undo it, over and over, and count every token. About 80% of the tokens in a successful run come from calls that leave the requested change exactly where it was. They call it heat. The estimated free energy of a change is 14% to 22% of the mean cost of a successful run, and coding costs 1.1 to 1.5 times as much as uncoding.
Put the three together and the bill reads differently. The expensive part of an agent is not the hard reasoning, it is the big model doing easy steps and the loop burning tokens without moving the work. Routing and compiling are where the next 5x in cost per task comes from, and labs that only sell the top model have little reason to tell you that.
Comments