Your stale AGENTS.md is charging rent on every agent session
732 agent runs, and not a single one fixed the file that was misleading it. That is the quiet finding of "When Instruction Files Go Stale: Documentation Debt and Hidden Cost in Coding Agents" (arXiv 2611.09802, Tural Mehtiyev, Biruk Tadesse, Lawrence Arkoh, Wesley Assunção).
The setup is clean. Repository instruction files like AGENTS.md and CLAUDE.md get read at the start of every agent session, and like all docs they rot: the test folder moves, the file still points to the old one. The authors took three real stale instructions from active open-source repos, a test command, a code location, and a procedure for adding a component. They ran agents on 22 maintenance tasks under three versions of each instruction, left stale, corrected, and removed, with code and task held fixed. Then they analyzed 732 trajectories from two models, tool call by tool call.
What they found, per the abstract: agents follow stale guidance when it is relevant to the task, sometimes even when the right information is sitting elsewhere in the repo. On the stale test command, Claude Sonnet 5.5 needed about two extra tool calls per run, a third more than with the corrected command. In the other two cases they could not detect a difference. Agents usually recovered from the immediate failure, finished the task, and moved on, leaving the bad file for the next session. Correcting the file wiped out operations on obsolete paths; deleting it prevented most of them.
The sample is small, three instructions with one measurable effect, so don't stretch it. But the shape matters. A success-only eval would score every one of these runs as fine, while the stale line keeps billing you a few tool calls per session forever. Think of it as interest on documentation debt. The fix is boring and real: treat instruction files like code, with an owner and a check in CI that the paths and commands they mention still exist. An agent that never repairs its own instructions is not going to start on its own.
Source: https://arxiv.org/abs/2611.09802
← Back to all articles
The setup is clean. Repository instruction files like AGENTS.md and CLAUDE.md get read at the start of every agent session, and like all docs they rot: the test folder moves, the file still points to the old one. The authors took three real stale instructions from active open-source repos, a test command, a code location, and a procedure for adding a component. They ran agents on 22 maintenance tasks under three versions of each instruction, left stale, corrected, and removed, with code and task held fixed. Then they analyzed 732 trajectories from two models, tool call by tool call.
What they found, per the abstract: agents follow stale guidance when it is relevant to the task, sometimes even when the right information is sitting elsewhere in the repo. On the stale test command, Claude Sonnet 5.5 needed about two extra tool calls per run, a third more than with the corrected command. In the other two cases they could not detect a difference. Agents usually recovered from the immediate failure, finished the task, and moved on, leaving the bad file for the next session. Correcting the file wiped out operations on obsolete paths; deleting it prevented most of them.
The sample is small, three instructions with one measurable effect, so don't stretch it. But the shape matters. A success-only eval would score every one of these runs as fine, while the stale line keeps billing you a few tool calls per session forever. Think of it as interest on documentation debt. The fix is boring and real: treat instruction files like code, with an owner and a check in CI that the paths and commands they mention still exist. An agent that never repairs its own instructions is not going to start on its own.
Source: https://arxiv.org/abs/2611.09802
Comments