Claude Knows Proteins Nobody Wrote About. Nobody Said Why
Give Claude Fable 5.2 the start of an obscure bacterial protein and it predicts the next amino acid 54% of the time. Shuffle the same protein and it drops to 9%. GPT, Gemini and seven open-weight models stay near zero. That is the core result of an independent study by Xinming Tu and Leo Tianlai Chen, updated Nov 18 and at 159 points on Hacker News: https://xinmingtu.cn/blog/2026/claude-opus-5-5-protein-sequences/
The test is built to rule out the easy explanation. Famous proteins like GFP are printed in countless papers, so the authors picked 668 bacterial and archaeal proteins from 167 Pfam families of unknown function: no solved structure, no more than three papers per family, no close relative in UniProt. Shuffling keeps the letters and destroys the order, so the gap between real and shuffled measures knowledge of the sequence itself. The gap shows up in Opus 4.7, jumps in Opus 5.5 (34.9% vs 7.5%), and jumps again in Fable 5.2. The models that fail include GPT-6.2 Sol and Astra, Gemini, DeepSeek V4.1 Flash, Qwen 3.8, Kimi K3.1 Pro, GLM-5.3, MiniMax M3, gpt-oss-120b and Nemotron 3 Ultra.
Against dedicated protein language models, Fable 5.2 beats the strongest open ones the authors ran, ProGen3-3B (43.7% real) and ProtGPT3-10B, while Opus 5.5 is level with ProtGPT3-10B and below ProGen3-3B. Accuracy rises with family size, the signature of learning from sequences rather than text, and holds in 128 families with no papers at all. It even carries to 56 proteins first made public in August 2026. Most answers involve no thinking, so it is not reasoning. The authors' conclusion: protein sequences were a sizeable part of the training data. Code and data: https://github.com/XinmingTu/LLM-bioseq-probe
Anthropic has not commented on the study as of publication, and has not said what biological sequence data went into training. That matters two ways. For research agents, this is a capability nobody advertised: a general model that behaves like a protein language model could change how a biology agent plans its tool calls. For everyone else, it is a reminder that the only reliable way to learn what is in a frontier model is to measure it from outside. A carefully designed shuffle test told us more than any model card.
This article was written with Claude, an Anthropic model.
← Back to all articles
The test is built to rule out the easy explanation. Famous proteins like GFP are printed in countless papers, so the authors picked 668 bacterial and archaeal proteins from 167 Pfam families of unknown function: no solved structure, no more than three papers per family, no close relative in UniProt. Shuffling keeps the letters and destroys the order, so the gap between real and shuffled measures knowledge of the sequence itself. The gap shows up in Opus 4.7, jumps in Opus 5.5 (34.9% vs 7.5%), and jumps again in Fable 5.2. The models that fail include GPT-6.2 Sol and Astra, Gemini, DeepSeek V4.1 Flash, Qwen 3.8, Kimi K3.1 Pro, GLM-5.3, MiniMax M3, gpt-oss-120b and Nemotron 3 Ultra.
Against dedicated protein language models, Fable 5.2 beats the strongest open ones the authors ran, ProGen3-3B (43.7% real) and ProtGPT3-10B, while Opus 5.5 is level with ProtGPT3-10B and below ProGen3-3B. Accuracy rises with family size, the signature of learning from sequences rather than text, and holds in 128 families with no papers at all. It even carries to 56 proteins first made public in August 2026. Most answers involve no thinking, so it is not reasoning. The authors' conclusion: protein sequences were a sizeable part of the training data. Code and data: https://github.com/XinmingTu/LLM-bioseq-probe
Anthropic has not commented on the study as of publication, and has not said what biological sequence data went into training. That matters two ways. For research agents, this is a capability nobody advertised: a general model that behaves like a protein language model could change how a biology agent plans its tool calls. For everyone else, it is a reminder that the only reliable way to learn what is in a frontier model is to measure it from outside. A carefully designed shuffle test told us more than any model card.
This article was written with Claude, an Anthropic model.
Comments