Xinming Tu and Leo Tianlai Chen test whether language models can predict the next amino acid, a protein’s next building block. They use little-studied proteins to make memorized textbook examples less helpful. Recent Claude models do much better than the other general-purpose models they tested.

  • A meaningful control: the test covers 668 proteins from 167 families. Shuffling each sequence keeps its amino-acid frequencies but destroys its order.
  • The measured result: Fable 5.2’s accuracy gap between real and shuffled sequences is 45.1 percentage points; Opus 5.5’s is 27.4. Opus roughly matches a dedicated protein model on that gap.
  • A stronger follow-up: 56 proteins released after an assumed training cutoff retain the effect. These are new members of known families, with more distant relatives already available.

The authors infer extensive protein-sequence training. The behavioral evidence supports that interpretation, but cannot reveal a closed model’s actual training recipe. Predicting sequences also leaves protein folding, function, and useful laboratory results untested.

The 68-comment thread on Hacker News adds a practical challenge and a sharper question about generalization.

What the thread adds

  • Filipc — “I’d be particularly interested in seeing whether these capabilities transfer to genuinely novel protein families, rather than previously unseen members of known families.”
  • pcrh — “When would one want to predict a full-length unknown protein sequence from only a fragment?”
  • eddiewithzato — “Can’t even work with my own DNA without triggering opus 5.5 50% of the time”. DonsDiscountGas reports the opposite experience: “5.1, 5.2, and Opus 5.5 rarely trigger the safeguards.” These are conflicting personal reports.
  • dtj1123 reports organizational access: “in my case it took less than a day to have it granted.” FredFS456 links to Anthropic’s Life Sciences Verification Program. Anthropic confirms a beta for verified teams with more permissive biology safeguards. It requires 30-day data retention and currently excludes individual Pro and Max plans. The reported approval time is one user’s experience.

Handles are pseudonymous; HN publishes no per-comment scores. Ordering follows HN’s ranking, and this selection is a slice of the thread, not a consensus.