Rikiya Takehi, Jayden Teoh, Alex Zhang and Omar Khattab ask a frontier language model to read and write other networks’ weights. These numerical parameters determine what a network computes. Their research post calls this weight-space programming: translating a goal into direct parameter edits, with gradient-based training forbidden.

A small repair with a large measured effect

Their strongest practical example uses GPT-6 Astra to repair Qwen3.5-0.8B. The small model often repeats its reasoning until it exhausts its answer budget. Astra inspects the weights in Python and tests edits on practice math problems, with a budget of 50 batched evaluation calls.

It removes a direction associated with repetition from two feed-forward blocks, changing 131,072 weights, or a reported 0.016% of the model. Under the same sampling settings, the authors report:

  • MATH-500: math accuracy rises from 7.9% to 48.1%.
  • GSM8K: grade-school math accuracy rises from 13.8% to 67.1%.
  • IFEval: instruction-following accuracy rises from 21.0% to 62.3%.
  • Unfinished answers: the share of MATH-500 responses reaching the 32,768-token limit falls from 93.0% to 33.1%.

The released checkpoint’s model card reports a separate rerun scoring 51.0% on MATH-500. That run covers all 500 problems with four samples each. Its base-model comparison is only a 25-problem smoke test, so it is partial corroboration rather than a full paired reproduction.

Caveat: the gain largely comes from letting existing reasoning finish. Astra could not meaningfully improve a separate Gemma model that rarely looped.

Reading weights and constructing strategies

The other experiments explore a broader range of tasks:

  • Recognizing behavior: Astra identifies four games from policy weights supplied as text. It also locates copying circuits in small transformers with their input and output layers removed.
  • Repairing control: after seeing a failed car trajectory, it writes a narrow override into the policy. The edited car succeeds in all 1,024 tested episodes.
  • Building networks: one-shot weight assignments beat policies trained for 100 million steps in three MinAtar games. Against 500-million-step policies, only the Breakout network wins.
  • A boundary on interpretation: identifying a sycophantic language model fails from weights alone. It succeeds after the researchers allow attention maps from test inputs.

Some small-network experiments forbid tools and execution entirely. The larger Qwen and Go experiments allow Python and behavioral feedback; one evaluation call can contain hundreds of examples or games.

The logs and code make targeted model repair worth investigating. The evidence supports diagnosing particular failures and compiling explicit strategies into networks. General replacements for training, arbitrary new capabilities and reliable audits from weights alone remain open questions.

Daily coverage

This story also leads the November 2 Afternoon roundup.