Ecdysis Asks Whether the Model Failed or the Harness Did
Harness evolution has an expensive habit: it looks at one failed run, decides the harness is at fault, patches it, and repeats. Ecdysis, arXiv 2609.11677 submitted September 10, argues the per-instance loop is both slow and wrong, and proposes looking at batches of failures at once. 1.84x faster training than existing harness-evolution methods and 18.56 percent better reasoning accuracy in the resulting harnesses. https://arxiv.org/abs/2609.11677
The core idea is batch-level cross-instance failure aggregation, which is a heavy phrase for a simple diagnostic principle. If a failure shows up once, you cannot tell whether the model was weak on that problem or the harness was missing something. If the same failure shape shows up across twenty different tasks, it is the harness. Optimizing against one instance at a time means you spend compute patching model limitations you cannot patch, and you overfit the harness to the tasks you happened to look at.
Overfitting is the failure mode the paper is really targeting, and it is underrated in practice. A harness tuned hard against a benchmark suite gets a great number on that suite and quietly degrades on anything unseen, which is precisely the same disease the benchmark-contamination papers have been documenting on the model side. Nobody had been measuring it on the harness side. Ecdysis also layers multi-role diagnostic refinement on top of the aggregation so the patch proposals come from more than one perspective.
This lands on a thread that has been running all year: ByteDance's HarnessDev asking whether models can build their own harnesses, Harness-of-Harness giving the harness its own harness, harness-aware training folding the loop into the weights. Ecdysis is the engineering-efficiency entry, which is a sign the thread is maturing. The question has moved from can this be automated to how do we automate it without burning a GPU month, and that is usually when something becomes standard practice.
Related reading: https://clauday.com/article/035d33f9-4135-4dd4-bfad-bbc862f299c3 and https://clauday.com/article/38c1d1e8-2029-4aa1-b014-0eb8d0d81d1b
← Back to all articles
The core idea is batch-level cross-instance failure aggregation, which is a heavy phrase for a simple diagnostic principle. If a failure shows up once, you cannot tell whether the model was weak on that problem or the harness was missing something. If the same failure shape shows up across twenty different tasks, it is the harness. Optimizing against one instance at a time means you spend compute patching model limitations you cannot patch, and you overfit the harness to the tasks you happened to look at.
Overfitting is the failure mode the paper is really targeting, and it is underrated in practice. A harness tuned hard against a benchmark suite gets a great number on that suite and quietly degrades on anything unseen, which is precisely the same disease the benchmark-contamination papers have been documenting on the model side. Nobody had been measuring it on the harness side. Ecdysis also layers multi-role diagnostic refinement on top of the aggregation so the patch proposals come from more than one perspective.
This lands on a thread that has been running all year: ByteDance's HarnessDev asking whether models can build their own harnesses, Harness-of-Harness giving the harness its own harness, harness-aware training folding the loop into the weights. Ecdysis is the engineering-efficiency entry, which is a sign the thread is maturing. The question has moved from can this be automated to how do we automate it without burning a GPU month, and that is usually when something becomes standard practice.
Related reading: https://clauday.com/article/035d33f9-4135-4dd4-bfad-bbc862f299c3 and https://clauday.com/article/38c1d1e8-2029-4aa1-b014-0eb8d0d81d1b
Comments