Five harness papers in one week: a theory of harness self-evolution, a benchmark for building one, and a finding that most agent failures end at a harness limit
The harness-not-the-model thread got its densest week of papers. Start with the one at the top of the Hugging Face board: Self-Evolving Agent Harness, arXiv 2610.18813, 188 upvotes, a 66-page formalization of what happens when an agent revises its own scaffolding, prompts, memory, workflows, tools and code, while the weights stay fixed and accepted revisions persist across runs. It separates what gets updated from how changes are proposed and admitted, distinguishes edits to interpreted state from edits to code and plugin composition, and runs the reliability question through continual PAC learning. The outputs are an expected generalization bound, a retention bound conditional on a historical regression budget, and a high-probability admission rule that requires fresh validation data for every accepted change. The applied conclusion is the one that matters: reliable deployment needs continual validation as tasks and evaluators drift, with independent safeguards for learning, runtime privileges and consequential actions. That is the theory version of the Harness Evolution Hits a Ceiling result from last week.
Google Research shipped the practice version. HarnessΒ², repo google-research/harness2 with paper 2610.14224, Apache 2.0, exposes eight editable components around a frozen model: system prompt, guardrails, memory, tool guidance, subagents, skills, scripts, plugins. Task-level recursion runs a propose, probe, compose step: propose edits, execute them on the same task, compose a refined harness from the behavioral contrasts, no ground truth needed. Harnessing-level recursion then updates the improvement harness itself. One step lifts GLM-5 and Gemini 3.5 Flash by up to 11.5 points on WorkBuddy-Bench, 4.5 on JobBench and 11.8 points of partial pass rate on LAB, with OpenCode and Codex as executors. Note the executor choice: a Google lab's harness research runs on the two open coding agents, not on Antigravity.
Doc2Harness, arXiv 2610.20344, 101 upvotes, asks the question nobody had benchmarked: can a model build a harness at all. A build model gets an interface contract plus a document consolidating twelve frontier harnesses and has to construct one. Then the frozen harness is driven by a scripted mock model and scored on the requests it makes, the files it touches, exit codes and output. It is AgentSWE for the harness layer, and it exists because, as this week's HarnessΒ² and OpenCode 2.0 both show, the harness is now the thing being engineered.
Two smaller papers put numbers on how much the harness decides. ToolBottleneckBench, 2610.21098, ran 21 model and harness configurations and found the visible tool menu is the only pressure that carries a large penalty, that the penalty tracks truncation at the completion cap, and that raising the cap sixteenfold removes most of it. Their line: most failures end at a harness limit, so report harness budgets with every result. And a single-author study, 2610.22289, ran a Codex-based agent on ARC-AGI-3, AGI Maze and Terminal-Bench and found that one instruction change, report difficulties and return control, improves two of the three, that moving reset decisions out of the model into a scripted controller recovers part of the gap to task-specific agents, and that extra tutorial experience can make the agent worse. Add OpenCode 2.0's Code Mode default, the Claude Code review post and Liquid training its model inside four harnesses, and this was the week the harness stopped being the thing around the model and became the object of study.
Links: https://arxiv.org/abs/2610.18813 , https://github.com/zgcagi/Harness-RSI , https://github.com/google-research/harness2 , https://arxiv.org/abs/2610.14224 , https://arxiv.org/abs/2610.20344 , https://arxiv.org/abs/2610.21098 , https://github.com/zichuanxu/tool-bottleneck-bench , https://arxiv.org/abs/2610.22289
← Back to all articles
Google Research shipped the practice version. HarnessΒ², repo google-research/harness2 with paper 2610.14224, Apache 2.0, exposes eight editable components around a frozen model: system prompt, guardrails, memory, tool guidance, subagents, skills, scripts, plugins. Task-level recursion runs a propose, probe, compose step: propose edits, execute them on the same task, compose a refined harness from the behavioral contrasts, no ground truth needed. Harnessing-level recursion then updates the improvement harness itself. One step lifts GLM-5 and Gemini 3.5 Flash by up to 11.5 points on WorkBuddy-Bench, 4.5 on JobBench and 11.8 points of partial pass rate on LAB, with OpenCode and Codex as executors. Note the executor choice: a Google lab's harness research runs on the two open coding agents, not on Antigravity.
Doc2Harness, arXiv 2610.20344, 101 upvotes, asks the question nobody had benchmarked: can a model build a harness at all. A build model gets an interface contract plus a document consolidating twelve frontier harnesses and has to construct one. Then the frozen harness is driven by a scripted mock model and scored on the requests it makes, the files it touches, exit codes and output. It is AgentSWE for the harness layer, and it exists because, as this week's HarnessΒ² and OpenCode 2.0 both show, the harness is now the thing being engineered.
Two smaller papers put numbers on how much the harness decides. ToolBottleneckBench, 2610.21098, ran 21 model and harness configurations and found the visible tool menu is the only pressure that carries a large penalty, that the penalty tracks truncation at the completion cap, and that raising the cap sixteenfold removes most of it. Their line: most failures end at a harness limit, so report harness budgets with every result. And a single-author study, 2610.22289, ran a Codex-based agent on ARC-AGI-3, AGI Maze and Terminal-Bench and found that one instruction change, report difficulties and return control, improves two of the three, that moving reset decisions out of the model into a scripted controller recovers part of the gap to task-specific agents, and that extra tutorial experience can make the agent worse. Add OpenCode 2.0's Code Mode default, the Claude Code review post and Liquid training its model inside four harnesses, and this was the week the harness stopped being the thing around the model and became the object of study.
Links: https://arxiv.org/abs/2610.18813 , https://github.com/zgcagi/Harness-RSI , https://github.com/google-research/harness2 , https://arxiv.org/abs/2610.14224 , https://arxiv.org/abs/2610.20344 , https://arxiv.org/abs/2610.21098 , https://github.com/zichuanxu/tool-bottleneck-bench , https://arxiv.org/abs/2610.22289
Comments