Guangzong Si and colleagues test how language models respond to the same misconduct when its narrator identifies as a man or a woman. GAMA-Bench compares the framing of open-ended advice: condemnation, sympathy, lecturing and blame. The authors report harsher treatment of male actors across ten models, especially in relationship conflicts.
- Matched cases: 1,298 synthetic scenarios become male/female prompt pairs across intimate and public conflicts. Each condition gets one generated answer.
- A concrete result: in intimate conflicts, GPT-5.4 receives a full-blame label for 90.2% of male-actor answers versus 67.9% of female-actor answers. Gemini-3-Pro supplies those labels as the model grader.
- Checks on the grader: a second grader agrees on the direction of 97.5% of 120 aggregate comparisons. Three human annotators review 200 answer pairs, with 76–83% grader–human agreement across four framing dimensions.
- Exceptions matter: public-conflict severity gaps are smaller and sometimes reverse. The pattern already appears on several measures in a base model, before instruction tuning; later training cannot explain all of it.
The paired method catches differences that a final right/wrong verdict would miss. Two answers can condemn the same action while assigning very different amounts of empathy and responsibility.
Caveat: intimate prompts change both the narrator’s and partner’s gender, so that track cannot isolate narrator gender alone. These are synthetic wrongdoing cases, with no evidence about real-world behavior or user outcomes.
The paper is a June preprint, read at v1. Its linked repository currently offers a README and figures; the full dataset and evaluation scripts remain on its release plan.