Agency and Think or Remember: your agent bill is set by the harness and the cache
Same model, same task, up to 2.6 times the bill. That is the starting point of arXiv 2611.12122, "System Optimizations for Cost-Efficient Agent Harnesses," from a group that includes Ion Stoica and Alvin Cheung. Prior studies found that harness choices like system prompts and tools barely move an agent's accuracy, which mostly comes from the model. What they do move is cost.
So the authors went looking for where the money goes. They analyzed how cost accumulates across agent traces, found places to cut tokens and LLM calls without touching the model or the accuracy, and built Agency, a harness interposition platform. Think of it as a proxy layer that sits under off-the-shelf harnesses like Claude Code or OpenAI Codex and lets you plug optimizations in portably. Their proof-of-concept techniques cut cost by up to 35% across real-world benchmarks with no accuracy loss. That's from the abstract; which benchmarks and which tricks are in the paper body, and "up to" is doing some work.
The second paper, arXiv 2611.10982 "Think or Remember," goes one layer lower. Agents on reasoning models resend their growing transcript at every call, and two parties decide what that costs: the client, which keeps or edits messages, and the server, which decides whether earlier reasoning comes back and what its cache reuses. The authors define a cost per token processed (c) and per pair attended (Ξ²). Their ratio, P* = c/Ξ², is an exchange rate. Below that context length, thinking dominates; above it, memory does.
The punchline: serving alone can flip which policy is cheaper. On recorded Qwen3-8B ScienceWorld runs, dropping earlier reasoning cost 1.74 times as much as keeping it when the server reused generated states, and 0.34 times when it reused nothing. Same agent, same choice, opposite answer. And dropping it isn't free in quality either: in live reruns it cut correct HotpotQA answers by 9 to 23 percentage points across four models, at a lower point-estimate cost. Small models, and abstract-level numbers, but the logic is general.
The takeaway for anyone paying an agent bill: stop asking only which model is cheapest. The harness decides how many calls happen, the cache decides what each call costs, and "should my agent keep its reasoning?" has no answer until you know how your provider serves it. Cost per task is a property of the system, not the model card.
Source: https://arxiv.org/abs/2611.12122 , https://arxiv.org/abs/2611.10982
← Back to all articles
So the authors went looking for where the money goes. They analyzed how cost accumulates across agent traces, found places to cut tokens and LLM calls without touching the model or the accuracy, and built Agency, a harness interposition platform. Think of it as a proxy layer that sits under off-the-shelf harnesses like Claude Code or OpenAI Codex and lets you plug optimizations in portably. Their proof-of-concept techniques cut cost by up to 35% across real-world benchmarks with no accuracy loss. That's from the abstract; which benchmarks and which tricks are in the paper body, and "up to" is doing some work.
The second paper, arXiv 2611.10982 "Think or Remember," goes one layer lower. Agents on reasoning models resend their growing transcript at every call, and two parties decide what that costs: the client, which keeps or edits messages, and the server, which decides whether earlier reasoning comes back and what its cache reuses. The authors define a cost per token processed (c) and per pair attended (Ξ²). Their ratio, P* = c/Ξ², is an exchange rate. Below that context length, thinking dominates; above it, memory does.
The punchline: serving alone can flip which policy is cheaper. On recorded Qwen3-8B ScienceWorld runs, dropping earlier reasoning cost 1.74 times as much as keeping it when the server reused generated states, and 0.34 times when it reused nothing. Same agent, same choice, opposite answer. And dropping it isn't free in quality either: in live reruns it cut correct HotpotQA answers by 9 to 23 percentage points across four models, at a lower point-estimate cost. Small models, and abstract-level numbers, but the logic is general.
The takeaway for anyone paying an agent bill: stop asking only which model is cheapest. The harness decides how many calls happen, the cache decides what each call costs, and "should my agent keep its reasoning?" has no answer until you know how your provider serves it. Cost per task is a property of the system, not the model card.
Source: https://arxiv.org/abs/2611.12122 , https://arxiv.org/abs/2611.10982
Comments