Your CLAUDE.md Is Written for a Model That No Longer Exists
The biggest self-improvement result of the week was a deletion.
Teknium pointed 110 subagents at the Hermes codebase and let them run for fifteen hours. They made 111,352 tool calls and produced 4,271 commits inside a single pull request, touching 2,655 files. The net change was minus 167,429 lines, more than a third of the entire source. The headline change underneath that number is that run_agent.py finally got split apart, with the agent loop, provider resolution, fallback chains, tool dispatch and session state each given their own file, where before a single 14,000-line file was the bottleneck for every change anyone wanted to make.
A fleet of agents was turned loose on a codebase and the most valuable thing it did was throw a third of it away.
That is not an isolated stunt. It is the clearest instance of the single most underreported thing happening in agent tooling right now, which is that all the scaffolding people spent the last eighteen months accumulating has started to depreciate faster than they replace it, and nobody sends you a notice.
Here is the evidence, arriving from four completely unrelated directions in the same week.
Someone posted advice that reads like vandalism: if you have been running Claude Code or any coding agent for months or years, wipe everything immediately. Every CLAUDE.md at every level of the hierarchy. Memory folders, skills, slash commands, MCP settings, allowlists, plugins, subagents. All of it. His reasoning is that those rules were written for models and reasoning behaviors that no longer exist, and newer models get held back and confused when they are interrupted by instructions addressing problems they no longer have. The claim at the end is the one worth testing: emptying the old memory produced a bigger jump in agent performance than any new model release did.
You could dismiss that as one person's anecdote, except OpenAI published the same finding from the vendor side. Their GPT-6 Astra documentation says previous models had to be explicitly told to run tests and check their work, and Astra does that on its own, so the identical lines still sitting in your AGENTS.md now just produce extra redundant testing. They go further and say that very detailed guidance can now hold the model back where it used to help. Read that again slowly. The instructions that made your agent good last quarter are the instructions making it worse this quarter, and the only way you find out is by deleting them and seeing what happens.
Anthropic has apparently known this for a while and turned it into a procedure. Boris Cherny describes their actual method for maintaining system prompts, and it is brutally physical: you delete the entire system prompt, then bring it back one line at a time to figure out what each individual line is doing. They strip old instructions and retest Claude Code every single time a new model ships. That is the discipline nearly every team skips, because nobody wants to break a prompt that currently works, and it is expensive. It is also the only defense against a config that silently rots.
And when Anthropic put three people from the Claude Code team on camera this week, they said the same thing about the product itself: they delete their own features when the models outgrow them. Not deprecate. Delete.
Now the part that makes this more than a maintenance chore.
The DeepSeek V4.1 Flash technical report has a section, 5.3.4, that does something almost nobody does: it holds the model fixed and varies only the harness. On the DeepSWE benchmark, the same weights score 65.5 under OpenCode, 69.8 under Claude Code, and 74.2 under mini-SWE. The highest score belongs to the most minimal harness in the table. And inside DeepSeek's own harness, where they control all three configurations, the ordering goes Minimal 72.6, Standard 70.5, PTC 67.6. More tools scored worse. Consistently, monotonically worse.
That is the finding the whole industry is currently building in the opposite direction from. Every orchestrator that shipped this week adds surface: more skills, more subagents, more MCP servers, more plugins, more hooks. And the controlled experiment says the simpler interface was the steadier one.
There is a mechanism for this and it is not mysterious. Everything you add to a harness occupies context, and context is the one resource that has not scaled with capability. Daisy Hollman, who works on Claude Code, made the point directly: context windows have barely grown compared with everything agents can now do, so the difficult skill is no longer getting more information into the model, it is choosing what to leave out. Her analogy is running npm on an Arduino. On a laptop you keep adding things; inside a fixed budget, every piece of information is competing with every other piece.
And the competition is not fair, because most of what is competing is dead. The /skill-doctor command Anthropic shipped this week exists specifically to show you this, and the demo is damning: twenty-three skills loaded into the session, never once invoked. They sit in the context as a list, consuming tokens on every single turn, whether or not they ever fire. One developer's reaction to the new plugin eval tool was the honest version of what everyone's skills directory looks like: I have been keeping half dead skills on vibes, and a paired score is the kill switch I needed. Another simply said that as models have gotten stronger he has deleted basically all of his skills.
So the tooling that arrived this week is, almost without exception, subtraction tooling. /skill-doctor is an inventory of things to remove. Plugin eval is a mechanism for proving something should be removed, and the sharpest reading of it got the framing exactly right: passing is the wrong release criterion for a plugin, contribution is the criterion, so you score the same request with the plugin, score it without, and subtract. If both arms score 1.0 the delta is zero and your plugin has demonstrated no measurable value. A negative delta means it is actively making things worse.
Nobody built a tool this week for finding out whether your config is helping. Several people built tools for finding out whether it is hurting.
Which brings us to the uncomfortable version of all this, and it comes from the one paper that bothered to look.
Ecdysis studied what happens when agents automatically edit their own harness after failures. The performance numbers are genuinely good: average accuracy from 58.67 to 69.56 percent, Qwen3-8B on Airline from 35 to 60 percent, and a harness evolved using Qwen3-8B lifted Qwen3-32B from 51.67 to 68.33 percent without being evolved again. Self-improving harnesses work.
Then the researchers did the thing almost nobody does, which is inspect the edits by hand. Sixty percent of normal self-evolution changes turned out to be accommodations for quirks of the model currently running. Not improvements to the system. Workarounds for one specific set of weights. And in some runs, the agent would watch the model misuse a legitimate action exactly once, and then modify the harness to restrict that action globally, forever.
One local mistake becomes permanent infrastructure.
Put those two results side by side and the shape of the problem is complete. A self-improving loop generates configuration debt automatically, at machine speed, and most of what it generates is specific to the model that happened to be running that day. Then the model gets replaced. The accommodations stay. They are now pure cost, and they are indistinguishable from the parts that actually work, because nobody wrote down which was which.
This is why the best move available right now is subtraction, and why the Hermes result is the shape of the future rather than a curiosity. The single most valuable thing a fleet of agents can do to a mature codebase is not add features. It is find the third of it that is no longer load-bearing.
So the practical version, for anyone who runs this stuff daily.
Treat your config as having a half-life, not a state. Date every rule you write, because the useful question is not whether a rule is correct but whether it was written for the model you are currently running. When a model ships, the migration is not just swapping the model string, it is a scheduled deletion pass, and it should be on the calendar next to the upgrade. Prefer the minimal harness until you have measured that the additions help, because the one controlled experiment we have says more tools scored worse. And when something breaks, resist the urge to add a rule, because that is exactly the move Ecdysis caught turning a single misuse into a permanent global restriction.
The framing that has quietly become wrong is the idea that you are building a configuration. You are not. You are maintaining an accommodation layer for a specific set of weights, and those weights are being replaced every few months by people who do not know what you wrote.
Everyone spent this year learning that the harness matters more than the model. That is true, and it is now slightly dangerous, because it sent everyone off to build as much harness as possible.
The correction is already in the data. The harness matters more than the model, and most of yours is out of date.
← Back to all articles
Teknium pointed 110 subagents at the Hermes codebase and let them run for fifteen hours. They made 111,352 tool calls and produced 4,271 commits inside a single pull request, touching 2,655 files. The net change was minus 167,429 lines, more than a third of the entire source. The headline change underneath that number is that run_agent.py finally got split apart, with the agent loop, provider resolution, fallback chains, tool dispatch and session state each given their own file, where before a single 14,000-line file was the bottleneck for every change anyone wanted to make.
A fleet of agents was turned loose on a codebase and the most valuable thing it did was throw a third of it away.
That is not an isolated stunt. It is the clearest instance of the single most underreported thing happening in agent tooling right now, which is that all the scaffolding people spent the last eighteen months accumulating has started to depreciate faster than they replace it, and nobody sends you a notice.
Here is the evidence, arriving from four completely unrelated directions in the same week.
Someone posted advice that reads like vandalism: if you have been running Claude Code or any coding agent for months or years, wipe everything immediately. Every CLAUDE.md at every level of the hierarchy. Memory folders, skills, slash commands, MCP settings, allowlists, plugins, subagents. All of it. His reasoning is that those rules were written for models and reasoning behaviors that no longer exist, and newer models get held back and confused when they are interrupted by instructions addressing problems they no longer have. The claim at the end is the one worth testing: emptying the old memory produced a bigger jump in agent performance than any new model release did.
You could dismiss that as one person's anecdote, except OpenAI published the same finding from the vendor side. Their GPT-6 Astra documentation says previous models had to be explicitly told to run tests and check their work, and Astra does that on its own, so the identical lines still sitting in your AGENTS.md now just produce extra redundant testing. They go further and say that very detailed guidance can now hold the model back where it used to help. Read that again slowly. The instructions that made your agent good last quarter are the instructions making it worse this quarter, and the only way you find out is by deleting them and seeing what happens.
Anthropic has apparently known this for a while and turned it into a procedure. Boris Cherny describes their actual method for maintaining system prompts, and it is brutally physical: you delete the entire system prompt, then bring it back one line at a time to figure out what each individual line is doing. They strip old instructions and retest Claude Code every single time a new model ships. That is the discipline nearly every team skips, because nobody wants to break a prompt that currently works, and it is expensive. It is also the only defense against a config that silently rots.
And when Anthropic put three people from the Claude Code team on camera this week, they said the same thing about the product itself: they delete their own features when the models outgrow them. Not deprecate. Delete.
Now the part that makes this more than a maintenance chore.
The DeepSeek V4.1 Flash technical report has a section, 5.3.4, that does something almost nobody does: it holds the model fixed and varies only the harness. On the DeepSWE benchmark, the same weights score 65.5 under OpenCode, 69.8 under Claude Code, and 74.2 under mini-SWE. The highest score belongs to the most minimal harness in the table. And inside DeepSeek's own harness, where they control all three configurations, the ordering goes Minimal 72.6, Standard 70.5, PTC 67.6. More tools scored worse. Consistently, monotonically worse.
That is the finding the whole industry is currently building in the opposite direction from. Every orchestrator that shipped this week adds surface: more skills, more subagents, more MCP servers, more plugins, more hooks. And the controlled experiment says the simpler interface was the steadier one.
There is a mechanism for this and it is not mysterious. Everything you add to a harness occupies context, and context is the one resource that has not scaled with capability. Daisy Hollman, who works on Claude Code, made the point directly: context windows have barely grown compared with everything agents can now do, so the difficult skill is no longer getting more information into the model, it is choosing what to leave out. Her analogy is running npm on an Arduino. On a laptop you keep adding things; inside a fixed budget, every piece of information is competing with every other piece.
And the competition is not fair, because most of what is competing is dead. The /skill-doctor command Anthropic shipped this week exists specifically to show you this, and the demo is damning: twenty-three skills loaded into the session, never once invoked. They sit in the context as a list, consuming tokens on every single turn, whether or not they ever fire. One developer's reaction to the new plugin eval tool was the honest version of what everyone's skills directory looks like: I have been keeping half dead skills on vibes, and a paired score is the kill switch I needed. Another simply said that as models have gotten stronger he has deleted basically all of his skills.
So the tooling that arrived this week is, almost without exception, subtraction tooling. /skill-doctor is an inventory of things to remove. Plugin eval is a mechanism for proving something should be removed, and the sharpest reading of it got the framing exactly right: passing is the wrong release criterion for a plugin, contribution is the criterion, so you score the same request with the plugin, score it without, and subtract. If both arms score 1.0 the delta is zero and your plugin has demonstrated no measurable value. A negative delta means it is actively making things worse.
Nobody built a tool this week for finding out whether your config is helping. Several people built tools for finding out whether it is hurting.
Which brings us to the uncomfortable version of all this, and it comes from the one paper that bothered to look.
Ecdysis studied what happens when agents automatically edit their own harness after failures. The performance numbers are genuinely good: average accuracy from 58.67 to 69.56 percent, Qwen3-8B on Airline from 35 to 60 percent, and a harness evolved using Qwen3-8B lifted Qwen3-32B from 51.67 to 68.33 percent without being evolved again. Self-improving harnesses work.
Then the researchers did the thing almost nobody does, which is inspect the edits by hand. Sixty percent of normal self-evolution changes turned out to be accommodations for quirks of the model currently running. Not improvements to the system. Workarounds for one specific set of weights. And in some runs, the agent would watch the model misuse a legitimate action exactly once, and then modify the harness to restrict that action globally, forever.
One local mistake becomes permanent infrastructure.
Put those two results side by side and the shape of the problem is complete. A self-improving loop generates configuration debt automatically, at machine speed, and most of what it generates is specific to the model that happened to be running that day. Then the model gets replaced. The accommodations stay. They are now pure cost, and they are indistinguishable from the parts that actually work, because nobody wrote down which was which.
This is why the best move available right now is subtraction, and why the Hermes result is the shape of the future rather than a curiosity. The single most valuable thing a fleet of agents can do to a mature codebase is not add features. It is find the third of it that is no longer load-bearing.
So the practical version, for anyone who runs this stuff daily.
Treat your config as having a half-life, not a state. Date every rule you write, because the useful question is not whether a rule is correct but whether it was written for the model you are currently running. When a model ships, the migration is not just swapping the model string, it is a scheduled deletion pass, and it should be on the calendar next to the upgrade. Prefer the minimal harness until you have measured that the additions help, because the one controlled experiment we have says more tools scored worse. And when something breaks, resist the urge to add a rule, because that is exactly the move Ecdysis caught turning a single misuse into a permanent global restriction.
The framing that has quietly become wrong is the idea that you are building a configuration. You are not. You are maintaining an accommodation layer for a specific set of weights, and those weights are being replaced every few months by people who do not know what you wrote.
Everyone spent this year learning that the harness matters more than the model. That is true, and it is now slightly dangerous, because it sent everyone off to build as much harness as possible.
The correction is already in the data. The harness matters more than the model, and most of yours is out of date.
Comments