Liang Yue and Eric A. Vance test what language models inherit from the graders used to train them. A grader’s doubt about a question category can become a broad refusal rule, even when the model still knows the answers.
Across 814 training runs, the main students are Qwen2.5-7B and Llama-3.1-8B. They answer short factual questions or abstain, with rewards supplied by answer keys, model judges or controlled grading rules.
- A strict Qwen grader rejects eleven of twelve correct training answers about former-state capitals. Llama then refuses 98% of the 187 unseen former-state capitals it previously knew.
- In a separate intervention, changing only twelve verdicts moves refusal from 5% to 90%. The grader’s overall acceptance threshold matches the answer key’s, separating category bias from blanket strictness.
- Blocking refusal recovers correct answers on 94% of the refused capitals in the headline experiment. Much of the damage concerns willingness to answer.
- Aggregate accuracy and refusal rates can hide which categories were lost. Five judges voting together retain a blind spot when their mistakes are shared.
Caveat: These author-run experiments concern constrained factual-answer training. Without the penalty keeping students close to their original models, answers change much more; the paper’s open-ended follow-up could not test the same mechanism.
The released code and run outputs support auditing rejections by category before training and refusals afterward. Trained adapters are absent from the release.