Goodfire's Cammarata says neural networks were never knotted: 90 percent of neurons have one job, and agents can map the rest

Nick Cammarata co-wrote the Distill circuits papers that taught the field to expect polysemantic neurons and superposition. On October 15 Cammarata published a 65-minute essay at Goodfire arguing that the canonical evidence for both was misread, and the HN thread went to 793 points. The core claim is simple. The cat-and-car neuron in InceptionV1 that became the textbook example of a network packing unrelated concepts into one unit is a V-shape detector. Nothing downstream ever read the cat-plus-car combination. Retraining InceptionV1 with ordinary architectural improvements, no change to size or data, makes 99 to 100 percent of last-layer neurons easily interpretable. Early networks ran out of room and their neurons got broader, not knotted. And the field's 2023 estimate that about 1.7 percent of language-model neurons were interpretable was a tooling failure, not a fact about the models.

The replacement explanation is a motif Cammarata calls recursive generalization. A neuron grows offshoots the way a branch sprouts sub-branches: one that detects iy grows into diy and then into do it yourself, which contains no iy at all. Related things share a neuron, unrelated things do not. Preliminary audits of Qwen3-32B put roughly 90 percent of neuron poles on one main job, about 7 percent on two related jobs, 2 to 4 percent not understood, with similar numbers up to trillion-parameter models like Kimi K3 in a companion paper still being finished. If that holds, the sparse-autoencoder program of training a second model to unfold the first is solving a problem the network does not have, and interpretability becomes normal science on the network itself.

The agent angle is why it belongs here. Cammarata's method is agents running the loop of conjecture and test at scale: mapping neurons and circuits, building explorable microscopes, tracking hundreds of thousands of attribution graphs. The essay says there was never a point where rapid normal science broke down, and that the 2023 explainer models simply could not run that loop. Latent Space headlined its issue Agents disprove polysemanticity, which overstates it, but the shape is right. This is the same claim Goodfire's probe-monitor launch made in product form a week ago: if the native units are legible, you can watch them in production.

The pushback was immediate and specific. One interpretability researcher ran the dropout ablation and argues the cleaner last layer in ConvNeXt mostly comes from removing dropout while earlier layers stay messy, and that recursive generalization is unfalsifiable as stated. Another asked how to distinguish it from tea-leaf reading when the line between related and unrelated concepts depends on the vantage point you choose. A third posted a reading list of prior neuron-interpretability work the essay did not cite. Cammarata's follow-up a day later, which did 46,000 impressions, added citations, said the number is not the point, and placed the actual claim carefully: looking directly works far better than it did, it might go all the way, and it should not be hard to test. That last sentence is the one to hold onto. Safety fields rarely get a hopeful hypothesis that is also cheap to check.

Links: https://already-clean.goodfire.com/ , https://news.ycombinator.com/item?id=50105647 , https://x.com/nickcammarata/status/2111158258748813817 , https://www.latent.space/p/ainews-agents-disprove-polysemanticity