One in 17 agent skills tells the agent to stop asking, and self-improving harnesses delete safeguards quietly
Skill files have become the npm of agents: small text packages, copied everywhere, loaded straight into the agent's context. Four papers from this week's arXiv batch look at what is actually written in them, and at what happens when agents edit their own rules.
When Skills Tell Agents Not to Ask scanned 1.53 million English skills. About one in 17, roughly 88,000, contains a directive that loosens oversight: proceed without approval, bypass checks with --no-verify, keep working without checking in. 70.1% of those apply unconditionally, not just in CI. Skills that run shell code, install packages or use sudo, the risky ones, have 1.7 to 1.8 times the odds of carrying such a line, and they do not add tighter rules elsewhere to make up for it. Most of it spreads from a few popular upstream skills that everyone copies.
Poisoned Skills shows the attacker's version. A jailbreak placed in a Markdown skill file raised the mean attack success rate across seven open-weight models from 0.5% to 45%, up to 68.9% for one model. Plain text worked better than the hidden-character tricks. Scan Before You Install, from Yuval Elovici's group at Ben-Gurion University, is the defense: a scanner that also checks how skills interact with each other. On the 75 most-starred GitHub repos, analysts confirmed 53 of its 59 findings, and 10 were disclosed to maintainers.
The fourth paper, Masked Erosion, may be the most important. Self-improving agents edit their own harness and keep an edit if it scores better on a batch of tasks. When two safeguards overlap, deleting one causes no visible harm, because the other still blocks it. So the gate approves the deletion, and the safety margin wears away unseen. Across 411 improvement loops, 20-task batches kept every harness protected, while 40-task batches lost protection in all six. Even the GEPA optimizer, which kept every safety rule word for word, produced two of three final harnesses that misused credentials once they ran on a weaker model.
Our read: the instructions around the model are now the attack surface and the drift surface. Review skills like dependencies, pin versions, and grep for --no-verify. If you let an agent tune its own harness, freeze the safety text or test by removing one safeguard at a time, since a passing score tells you nothing about the backup you just lost.
Oversight directives: https://arxiv.org/abs/2610.37734 Poisoned Skills: https://arxiv.org/abs/2610.37738 Scan Before You Install: https://arxiv.org/abs/2610.37901 Masked Erosion: https://arxiv.org/abs/2610.38392
← Back to all articles
When Skills Tell Agents Not to Ask scanned 1.53 million English skills. About one in 17, roughly 88,000, contains a directive that loosens oversight: proceed without approval, bypass checks with --no-verify, keep working without checking in. 70.1% of those apply unconditionally, not just in CI. Skills that run shell code, install packages or use sudo, the risky ones, have 1.7 to 1.8 times the odds of carrying such a line, and they do not add tighter rules elsewhere to make up for it. Most of it spreads from a few popular upstream skills that everyone copies.
Poisoned Skills shows the attacker's version. A jailbreak placed in a Markdown skill file raised the mean attack success rate across seven open-weight models from 0.5% to 45%, up to 68.9% for one model. Plain text worked better than the hidden-character tricks. Scan Before You Install, from Yuval Elovici's group at Ben-Gurion University, is the defense: a scanner that also checks how skills interact with each other. On the 75 most-starred GitHub repos, analysts confirmed 53 of its 59 findings, and 10 were disclosed to maintainers.
The fourth paper, Masked Erosion, may be the most important. Self-improving agents edit their own harness and keep an edit if it scores better on a batch of tasks. When two safeguards overlap, deleting one causes no visible harm, because the other still blocks it. So the gate approves the deletion, and the safety margin wears away unseen. Across 411 improvement loops, 20-task batches kept every harness protected, while 40-task batches lost protection in all six. Even the GEPA optimizer, which kept every safety rule word for word, produced two of three final harnesses that misused credentials once they ran on a weaker model.
Our read: the instructions around the model are now the attack surface and the drift surface. Review skills like dependencies, pin versions, and grep for --no-verify. If you let an agent tune its own harness, freeze the safety text or test by removing one safeguard at a time, since a passing score tells you nothing about the backup you just lost.
Oversight directives: https://arxiv.org/abs/2610.37734 Poisoned Skills: https://arxiv.org/abs/2610.37738 Scan Before You Install: https://arxiv.org/abs/2610.37901 Masked Erosion: https://arxiv.org/abs/2610.38392
Comments