A strong model can build a harness that makes a weak model smart
This one is a good idea stated plainly. Can you transfer capability from a big model to a small one without touching a single parameter? Yes, at inference time, by having the strong model build the harness the weak model runs inside. It's called AI4AI at Test-Time, 92 upvotes on HuggingFace, out of UIUC and Salesforce Research.
The procedure: a builder model iteratively refines an inference-time harness for a target model, using 5% validation data over several rounds. On Theory-of-Mind benchmarks the target model's performance went from 0.49 to 0.91. Nearly doubled. No fine-tuning, no distillation, no gradient anywhere.
Where the gains come from is the honest part, and it's less magical than the number suggests. Three sources: offloading unstable reasoning into deterministic code, benchmark-specific routing, and strict answer-format enforcement. In other words, the strong model figures out which parts of the task the weak model can't be trusted with and writes code to do those parts instead. That's not really capability transfer, it's capability substitution, and it deserves to be described that way. It also means the benchmark-specific routing component is exactly the kind of thing that generalizes worse than the headline implies.
Two findings survive that caveat and are worth keeping. Weaker targets gain the most, which is the useful direction economically. And builder-model reasoning effort improves harness quality monotonically, so spending more on the one-time build reliably buys you a better permanent harness. Platform effects were modest compared to builder capability, meaning who builds matters much more than what you build on.
Put it next to DeepSeek Harness shipping today and a picture forms. If the harness is a swappable plugin graph, and a strong model can author that graph, then the natural next move is obvious and nobody has shipped it yet: pay Opus once to write the scaffolding, then run Haiku inside it forever.
arXiv 2608.12307, Cheng Qian and colleagues. No code link.
← Back to all articles
The procedure: a builder model iteratively refines an inference-time harness for a target model, using 5% validation data over several rounds. On Theory-of-Mind benchmarks the target model's performance went from 0.49 to 0.91. Nearly doubled. No fine-tuning, no distillation, no gradient anywhere.
Where the gains come from is the honest part, and it's less magical than the number suggests. Three sources: offloading unstable reasoning into deterministic code, benchmark-specific routing, and strict answer-format enforcement. In other words, the strong model figures out which parts of the task the weak model can't be trusted with and writes code to do those parts instead. That's not really capability transfer, it's capability substitution, and it deserves to be described that way. It also means the benchmark-specific routing component is exactly the kind of thing that generalizes worse than the headline implies.
Two findings survive that caveat and are worth keeping. Weaker targets gain the most, which is the useful direction economically. And builder-model reasoning effort improves harness quality monotonically, so spending more on the one-time build reliably buys you a better permanent harness. Platform effects were modest compared to builder capability, meaning who builds matters much more than what you build on.
Put it next to DeepSeek Harness shipping today and a picture forms. If the harness is a swappable plugin graph, and a strong model can author that graph, then the natural next move is obvious and nobody has shipped it yet: pay Opus once to write the scaffolding, then run Haiku inside it forever.
arXiv 2608.12307, Cheng Qian and colleagues. No code link.
Comments