One in Four Installed Skills Has a Twin. One in Five Runs Picks the Wrong One.
Two papers on Friday's arXiv listing measured what actually happens after you install a skill, and the answer is less than the skill says. One Skill Too Many, from Chaoliang Yan, Zihao Xu, Yuekang Li and colleagues, is the first look at skills that conflict with each other. They mined 20,947 repository snapshots, generated 822,109 candidate pairs of similar skills, had an LLM judge 3,754, and then ran 312 confirmed pairs on three models: 6,368 runs, 169,294 tool calls, 542 agent-hours. Roughly one in four installed skills has a co-installed near-duplicate, and 37 percent of the judged skills live in copied collections. When a twin is present, the agent opens it instead of the intended skill in about one run in five. In the runs that open the twin first, more than a third of the intended skill's exclusive core behavior is lost, the example being a ban on git commands that simply never fires, and the task still passes, so nothing flags it. What decides which skill wins is install location. Listing order barely matters.
The second paper, An Empirical Study of Agent Skills' Downstream Utility from Yu Cheng, Dehai Zhao, Zhongxin Liu and co-authors at Zhejiang and CSIRO, revised Thursday, asks whether a relevant skill helps at all. They define utility as the pass-rate delta against running with no skill, on 87 SkillsBench tasks, across nine model-and-harness configurations, with a marketplace corpus of 37,596 skills. On 36.78 percent of tasks the same skill helps some configurations and hurts others, and the traces show why: the recommended procedure becomes an execution burden the model follows instead of solving the task. Relevance ranking picks worse candidates than reranking by whether the skill supports the operations the task needs, which lifts first-choice pass rate by 4.35 to 5.80 points. Organising multiple skills as a dependency DAG beats a plain use order, with the gain concentrated in tasks handed five or six skills.
Read together with Skill Constellations from earlier in the week, which found 2.2 million skill adoptions flowing through copy graphs, the picture of the skills ecosystem is a package manager with no resolver. Copies proliferate, twins shadow originals, install path is the tiebreaker, and the only test that matters, does the task still pass, is exactly the test that cannot see the problem. Both papers end with the same request: an install-time conflict check and a per-skill utility score, neither of which any marketplace ships.
One Skill Too Many: https://arxiv.org/abs/2610.11647
Downstream Utility: https://arxiv.org/abs/2610.08875
← Back to all articles
The second paper, An Empirical Study of Agent Skills' Downstream Utility from Yu Cheng, Dehai Zhao, Zhongxin Liu and co-authors at Zhejiang and CSIRO, revised Thursday, asks whether a relevant skill helps at all. They define utility as the pass-rate delta against running with no skill, on 87 SkillsBench tasks, across nine model-and-harness configurations, with a marketplace corpus of 37,596 skills. On 36.78 percent of tasks the same skill helps some configurations and hurts others, and the traces show why: the recommended procedure becomes an execution burden the model follows instead of solving the task. Relevance ranking picks worse candidates than reranking by whether the skill supports the operations the task needs, which lifts first-choice pass rate by 4.35 to 5.80 points. Organising multiple skills as a dependency DAG beats a plain use order, with the gain concentrated in tasks handed five or six skills.
Read together with Skill Constellations from earlier in the week, which found 2.2 million skill adoptions flowing through copy graphs, the picture of the skills ecosystem is a package manager with no resolver. Copies proliferate, twins shadow originals, install path is the tiebreaker, and the only test that matters, does the task still pass, is exactly the test that cannot see the problem. Both papers end with the same request: an install-time conflict check and a per-skill utility score, neither of which any marketplace ships.
One Skill Too Many: https://arxiv.org/abs/2610.11647
Downstream Utility: https://arxiv.org/abs/2610.08875
Comments