Tree-Ising fixes the model the agent fixed, for 90 cents and zero tokens
89.7 million tokens versus zero. Last week MIT's Weight-Space Programming paper had GPT-6 Astra read the weights of Qwen3.5-0.8B and write a repair for its reasoning loops, using 158 agent turns, 89.7M tokens and 50 forward-pass requests. Tree-Ising (https://arxiv.org/abs/2611.14210, code at https://github.com/TheQuantumGOAT/Tree-Ising, Salloum and Ali, top of Hugging Face's Nov 11 board) does the same job with no language model in the loop at all.
The trick is to treat the repair as a math problem instead of a reasoning problem. Forward statistics define an objective: weaken the loop direction, strengthen the concluding direction, keep the layer's output unchanged on productive tokens. The allowed edits are only sign flips and exponent shifts inside the model's own BF16 weights. The objective splits exactly into one small Ising problem per output row, which a GPU solves with simulated annealing or simulated bifurcation.
The results on the same two layers the agent edited: 54.6% on MATH-500 versus 51.7% for the agent's edit, with the base model at 7.9% as reported. The bifurcation solver lands at 55.1%, statistically the same. Both beat the agent across five math benchmarks, significantly on MATH-500 and GSM8K after multiple-comparison correction. Tree-Ising touches 13 to 17% fewer weights, flips 2.2 to 2.3 times fewer bits, needs four forward passes, and the annealing run costs about $0.90 of GPU time. On Qwen3.5-2B, with layers chosen from the base model alone, greedy GSM8K goes from 21.6% to 69.4%. There is even an eight-variable subproblem run on an IBM Heron quantum chip, which is a demo, not the point.
Deletion Tuning (https://arxiv.org/abs/2611.10866, dated Nov 7) pushes in the same direction from training: zeroing fewer than 0.05% of weights matched full fine-tuning across RLVR, SFT and on-policy distillation, with far less forgetting.
The lesson is not that the MIT agent was useless. It found where to cut. Once the problem was located and stated, a solver did the cut cheaper and better. That is a recurring pattern worth stealing: let the expensive agent discover the objective, then hand it to something dumb, fast and exact.
← Back to all articles
The trick is to treat the repair as a math problem instead of a reasoning problem. Forward statistics define an objective: weaken the loop direction, strengthen the concluding direction, keep the layer's output unchanged on productive tokens. The allowed edits are only sign flips and exponent shifts inside the model's own BF16 weights. The objective splits exactly into one small Ising problem per output row, which a GPU solves with simulated annealing or simulated bifurcation.
The results on the same two layers the agent edited: 54.6% on MATH-500 versus 51.7% for the agent's edit, with the base model at 7.9% as reported. The bifurcation solver lands at 55.1%, statistically the same. Both beat the agent across five math benchmarks, significantly on MATH-500 and GSM8K after multiple-comparison correction. Tree-Ising touches 13 to 17% fewer weights, flips 2.2 to 2.3 times fewer bits, needs four forward passes, and the annealing run costs about $0.90 of GPU time. On Qwen3.5-2B, with layers chosen from the base model alone, greedy GSM8K goes from 21.6% to 69.4%. There is even an eight-variable subproblem run on an IBM Heron quantum chip, which is a demo, not the point.
Deletion Tuning (https://arxiv.org/abs/2611.10866, dated Nov 7) pushes in the same direction from training: zeroing fewer than 0.05% of weights matched full fine-tuning across RLVR, SFT and on-policy distillation, with far less forgetting.
The lesson is not that the MIT agent was useless. It found where to cut. Once the problem was located and stated, a solver did the cut cheaper and better. That is a recurring pattern worth stealing: let the expensive agent discover the objective, then hand it to something dumb, fast and exact.
Comments