LongHorizon-Harness: Move Task State Out of the Context, Gain 30 Points
Top agent paper on HuggingFace Daily Papers, 125 upvotes: LongHorizon-Harness, arXiv 2608.01964. The diagnosis is one every agent builder has felt. Agents fail long tasks because the task plan and its progress live inside an ever-growing context window, so the model loses track of where it is and, worse, confidently mis-assesses what it has already done.
The fix is architectural, not a bigger model. Reformulate execution as a task-state management problem: keep the task state outside the context entirely, and run a Manage-Execute-Audit loop where updates to that state get verified against the environment independently, instead of trusting the agent's own report of what happened.
The numbers are hard to argue with. Qwen 3.7-Plus jumps from 51.8 to 80.7 percent on WeaveBench, nearly 30 points from harness alone. Terminal-Bench 2.1 goes 69.7 to 77.2. Claude Opus 4.7 climbs from 20.0 to 34.3 on an OSWorld 2.0 subset. Same weights everywhere, only the scaffolding changed.
That is the running lesson of 2026: the gap between what a model can do and what an agent does with it is mostly harness, and a good harness is worth more than the next checkpoint. The audit step is the part worth stealing, because self-assessment is exactly where agents lie to themselves. Paper at arxiv.org/abs/2608.01964.
← Back to all articles
The fix is architectural, not a bigger model. Reformulate execution as a task-state management problem: keep the task state outside the context entirely, and run a Manage-Execute-Audit loop where updates to that state get verified against the environment independently, instead of trusting the agent's own report of what happened.
The numbers are hard to argue with. Qwen 3.7-Plus jumps from 51.8 to 80.7 percent on WeaveBench, nearly 30 points from harness alone. Terminal-Bench 2.1 goes 69.7 to 77.2. Claude Opus 4.7 climbs from 20.0 to 34.3 on an OSWorld 2.0 subset. Same weights everywhere, only the scaffolding changed.
That is the running lesson of 2026: the gap between what a model can do and what an agent does with it is mostly harness, and a good harness is worth more than the next checkpoint. The audit step is the part worth stealing, because self-assessment is exactly where agents lie to themselves. Paper at arxiv.org/abs/2608.01964.
Comments