EnvOpt: stop tuning the agent, tune the office it works in
Everyone tunes the model, and lately everyone tunes the harness. A new paper says the cheaper lever is the third one: the environment the agent works in, like the Slack workspace, the docs and the ticket pile that every task keeps wading through.
EnvOpt is an automated research pipeline that reshapes that shared environment for a fixed agent, using real or synthetic tasks, with no graders or reference answers needed. On AutomationBench it cut Claude Haiku 4.5's cost by 72.7% and raised its pass rate by 14.3%. The authors' abstract doesn't say whether that's points or relative. The same environment moved to Opus 5 cut cost by 68.1% with a 1.34% gain, so the work carries over across models. Added on top of a harness already optimized by Meta-Harness, it cut cost by a further 83.4% and added 11.17% accuracy. The authors, who include Joseph Gonzalez and Alex Dimakis, report similar direction across five benchmarks.
What it actually does is unglamorous, which is why it's believable. On a benchmark simulating an employee's computer, the research agent builds one index file listing paths, sizes and types, so the target agent searches that instead of burning tokens walking the filesystem. Its prompt tells it to build indexes, write catalogues, precompute summaries and leave a guide, and warns that "A grep is not a check": open the trajectory and confirm the target actually used what you left. The rejected attempts are worth reading too. One step added a test-script check that failed runs when a certain bug-fix snippet appeared. It sped up one agent and then rejected correct fixes from another, which had to spend extra time getting past it. Optimizing an environment for one agent can quietly sabotage the next.
A second paper from the same batch measures the other side of this. How Do AI Agents Waste Your Money? marks spans in a trajectory as removable, deletes them, and resumes the agent to check whether anything broke. In online pruning, 41-46% of model reasoning could go while largely preserving performance on SWE-bench Verified and MCP-Atlas. Waste varied more across tasks and settings than across models, and different harnesses added their own kinds of waste under the same model. An LLM judge does the labeling, though the interventions test its calls.
Now the caveat: these are benchmark environments, not your actual company, and a real Slack changes daily. Still, it fits what Ramp's spending data shows buyers already doing. If an agent's bill is mostly tokens spent finding things, a well-kept workspace is a cost control, and it's the one lever you own regardless of which vendor's model you use this quarter. EnvOpt: https://arxiv.org/abs/2611.07372 Waste paper: https://arxiv.org/abs/2611.07238
← Back to all articles
EnvOpt is an automated research pipeline that reshapes that shared environment for a fixed agent, using real or synthetic tasks, with no graders or reference answers needed. On AutomationBench it cut Claude Haiku 4.5's cost by 72.7% and raised its pass rate by 14.3%. The authors' abstract doesn't say whether that's points or relative. The same environment moved to Opus 5 cut cost by 68.1% with a 1.34% gain, so the work carries over across models. Added on top of a harness already optimized by Meta-Harness, it cut cost by a further 83.4% and added 11.17% accuracy. The authors, who include Joseph Gonzalez and Alex Dimakis, report similar direction across five benchmarks.
What it actually does is unglamorous, which is why it's believable. On a benchmark simulating an employee's computer, the research agent builds one index file listing paths, sizes and types, so the target agent searches that instead of burning tokens walking the filesystem. Its prompt tells it to build indexes, write catalogues, precompute summaries and leave a guide, and warns that "A grep is not a check": open the trajectory and confirm the target actually used what you left. The rejected attempts are worth reading too. One step added a test-script check that failed runs when a certain bug-fix snippet appeared. It sped up one agent and then rejected correct fixes from another, which had to spend extra time getting past it. Optimizing an environment for one agent can quietly sabotage the next.
A second paper from the same batch measures the other side of this. How Do AI Agents Waste Your Money? marks spans in a trajectory as removable, deletes them, and resumes the agent to check whether anything broke. In online pruning, 41-46% of model reasoning could go while largely preserving performance on SWE-bench Verified and MCP-Atlas. Waste varied more across tasks and settings than across models, and different harnesses added their own kinds of waste under the same model. An LLM judge does the labeling, though the interventions test its calls.
Now the caveat: these are benchmark environments, not your actual company, and a real Slack changes daily. Still, it fits what Ramp's spending data shows buyers already doing. If an agent's bill is mostly tokens spent finding things, a well-kept workspace is a cost control, and it's the one lever you own regardless of which vendor's model you use this quarter. EnvOpt: https://arxiv.org/abs/2611.07372 Waste paper: https://arxiv.org/abs/2611.07238
Comments