The weight-reading paper got audited in a day: the headline Qwen fix used Python and 50 batches
A stage magician does one trick several ways, so every theory you form gets debunked by the next round. That is how Matthew Honnibal, the spaCy author, describes "Sparks of Weight-Space Programming", the MIT post that topped Hacker News on Monday. The takedown hit 829 points on Tuesday.
The complaint is specific. The abstract says "just by looking at the raw weights" and then rolls straight into the Qwen math fix. Honnibal's table splits the experiments apart. Raw floats in context, no tools, no running the network: that setting only appears in experiments 1, 2 and 4, on networks of 17K to 172K weights. Every experiment on a real language model uses Python, and the Qwen result also runs the model on up to 50 batches of practice problems. A footnote on the from-scratch game networks is worse: against PPO at 500M steps they win only Breakout, 517 to 25, and lose Seaquest and Asterix.
Sean Goedecke walked the Qwen edit and called it the steps a human ML researcher would take. Simon Willison went after the part that does hold up, the no-tools game test, and ran it on a laptop. Qwen 3.8 27B, a 17GB model, named Pac-Man in 46 minutes, Pong in 65 and Breakout in 52, and missed Freeway. So 3 of 4 from a local open model, which either makes the result more interesting or makes it less special, depending on what you thought it proved. Willison's reading is that an experienced human with a notepad could get there too.
Nobody here disputes the measured numbers. MATH-500 still went from 7.9% to 48.1%. Honnibal concedes models of a year or two ago would have failed these tasks, and HN commenters point out that the post's own sections state the setup, so the problem is the abstract and not the logs. What fell is the framing, including ours: our Monday piece said Astra did it by reading and reasoning, with no training loop, and that undersold the Python and the 50 batches. The accurate sentence is less cinematic and still striking. A frontier model can act like a competent ML engineer on weights, tools included, and small grid-world networks really can be read cold.
The useful habit from this one: when a result sounds like a new capability, ask which row of the table it came from.
Honnibal: https://honnibal.dev/blog/astra-read-write-misleading Willison: https://simonwillison.net/2026/Nov/3/qwen-can-interpret-model-weights/ Goedecke: https://www.seangoedecke.com/llms-are-not-superhuman-at-reading-and-writing-neural-network-weights/ Monday's piece: https://clauday.com/article/50b2c796-fa88-4fee-babc-1d9de8b91d55
← Back to all articles
The complaint is specific. The abstract says "just by looking at the raw weights" and then rolls straight into the Qwen math fix. Honnibal's table splits the experiments apart. Raw floats in context, no tools, no running the network: that setting only appears in experiments 1, 2 and 4, on networks of 17K to 172K weights. Every experiment on a real language model uses Python, and the Qwen result also runs the model on up to 50 batches of practice problems. A footnote on the from-scratch game networks is worse: against PPO at 500M steps they win only Breakout, 517 to 25, and lose Seaquest and Asterix.
Sean Goedecke walked the Qwen edit and called it the steps a human ML researcher would take. Simon Willison went after the part that does hold up, the no-tools game test, and ran it on a laptop. Qwen 3.8 27B, a 17GB model, named Pac-Man in 46 minutes, Pong in 65 and Breakout in 52, and missed Freeway. So 3 of 4 from a local open model, which either makes the result more interesting or makes it less special, depending on what you thought it proved. Willison's reading is that an experienced human with a notepad could get there too.
Nobody here disputes the measured numbers. MATH-500 still went from 7.9% to 48.1%. Honnibal concedes models of a year or two ago would have failed these tasks, and HN commenters point out that the post's own sections state the setup, so the problem is the abstract and not the logs. What fell is the framing, including ours: our Monday piece said Astra did it by reading and reasoning, with no training loop, and that undersold the Python and the 50 batches. The accurate sentence is less cinematic and still striking. A frontier model can act like a competent ML engineer on weights, tools included, and small grid-world networks really can be read cold.
The useful habit from this one: when a result sounds like a new capability, ask which row of the table it came from.
Honnibal: https://honnibal.dev/blog/astra-read-write-misleading Willison: https://simonwillison.net/2026/Nov/3/qwen-can-interpret-model-weights/ Goedecke: https://www.seangoedecke.com/llms-are-not-superhuman-at-reading-and-writing-neural-network-weights/ Monday's piece: https://clauday.com/article/50b2c796-fa88-4fee-babc-1d9de8b91d55
Comments