Liang Yue and Eric A. Vance test what language models inherit from the graders used to train them. A grader’s doubt about a question category can become a broad refusal rule, even when the model still knows the answers.

Across 814 training runs, the main students are Qwen2.5-7B and Llama-3.1-8B. They answer short factual questions or abstain, with rewards supplied by answer keys, model judges or controlled grading rules.

  • A strict Qwen grader rejects eleven of twelve correct training answers about former-state capitals. Llama then refuses 98% of the 187 unseen former-state capitals it previously knew.
  • In a separate intervention, changing only twelve verdicts moves refusal from 5% to 90%. The grader’s overall acceptance threshold matches the answer key’s, separating category bias from blanket strictness.
  • Blocking refusal recovers correct answers on 94% of the refused capitals in the headline experiment. Much of the damage concerns willingness to answer.
  • Aggregate accuracy and refusal rates can hide which categories were lost. Five judges voting together retain a blind spot when their mistakes are shared.

Caveat: These author-run experiments concern constrained factual-answer training. Without the penalty keeping students close to their original models, answers change much more; the paper’s open-ended follow-up could not test the same mechanism.

The released code and run outputs support auditing rejections by category before training and refusals afterward. Trained adapters are absent from the release.

Daily coverage