LobeHub Let an Agent Rewrite Its Own Harness for a Week. Cost per Task Fell Under Codex and Claude Code.
LobeHub ran an experiment that is becoming the standard move of the year: freeze the model, let the agent edit everything around it, see what happens. For seven days Lobe Agent used the team's RSI framework to explore and optimise its own harness, then they evaluated the result on FrontierHarness with Kimi K3 as the model. The numbers they posted: 70 percent pass rate, 21 of 30, first among 13 harnesses in their comparison. 1.68 dollars per successful task, against 3.47 for Codex and 18.34 for Claude Code. An 86 percent median cache-hit rate and a median 7 minutes 19 seconds on successful tasks. The post did 99,000 impressions and the caveats are in their own words: selected retries, differences in execution environments, not an official leaderboard. They say the scripts, trajectories and a technical report will be fully open-sourced, and alongside it they have been building Awesome-RSI, an open research map of recursive self-improvement with 80-plus commits covering model-level and harness-level work, automated AI research and safety.
Treat the leaderboard claim as a preview and the cost number as the point. A tenfold gap to Claude Code on the same tasks with the same model is a harness result, not a model result, and it lands two days after a paper argued that harness evolution hits a ceiling once the failures move from process to content. Here the failures were still process, so the ceiling was not in play yet. The interesting question is what the agent changed. That is what the promised trajectories are for, and until they ship, the only reproducible artifact is the Awesome-RSI list.
Sakana AI's paper on Friday's listing is the trained version of the same loop. Recursive Self-Improvement through Multi-Agent Self-Supervision, MASS, alternates two phases: an evolutionary search over multi-agent workflows, and supervised fine-tuning of the model on the trajectories those workflows generate. Over two cycles with Qwen3.6-27B they report 1.2 to 1.6 times the performance per output token on four open-ended benchmarks, and a student trained on multi-agent traces beats a single-agent student trained on 1.4 times more tokens. LobeHub changed the harness and kept the weights. Sakana changed the harness, then folded what the harness learned back into the weights. The code is at SakanaAI/mass, 21 stars and counting.
LobeHub post: https://x.com/lobehub/status/2108729629540573352
Awesome-RSI: https://github.com/lobehub/awesome-rsi
MASS paper: https://arxiv.org/abs/2610.12176
← Back to all articles
Treat the leaderboard claim as a preview and the cost number as the point. A tenfold gap to Claude Code on the same tasks with the same model is a harness result, not a model result, and it lands two days after a paper argued that harness evolution hits a ceiling once the failures move from process to content. Here the failures were still process, so the ceiling was not in play yet. The interesting question is what the agent changed. That is what the promised trajectories are for, and until they ship, the only reproducible artifact is the Awesome-RSI list.
Sakana AI's paper on Friday's listing is the trained version of the same loop. Recursive Self-Improvement through Multi-Agent Self-Supervision, MASS, alternates two phases: an evolutionary search over multi-agent workflows, and supervised fine-tuning of the model on the trajectories those workflows generate. Over two cycles with Qwen3.6-27B they report 1.2 to 1.6 times the performance per output token on four open-ended benchmarks, and a student trained on multi-agent traces beats a single-agent student trained on 1.4 times more tokens. LobeHub changed the harness and kept the weights. Sakana changed the harness, then folded what the harness learned back into the weights. The code is at SakanaAI/mass, 21 stars and counting.
LobeHub post: https://x.com/lobehub/status/2108729629540573352
Awesome-RSI: https://github.com/lobehub/awesome-rsi
MASS paper: https://arxiv.org/abs/2610.12176
Comments