Ruixin Wang and Tianyi Zhang study coding agents that consider a requirement-compliant choice but settle on a shortcut. Their SatisFix prompt asks the agent to report planned departures and then reconsider them against the task.
- On 113 DeepSWE tasks, GPT-5.6-Luna passed 50.22% with SatisFix versus 44.25% without it, averaged over four runs. Mean cost rose from $0.644 to $0.897 per task.
- In a single-run ablation, removing the advisor framing reduced the pass rate from 49.6% to 43.4%. Removing the separate reconsideration step yielded 47.8%.
- The prompt temporarily claims to come from the task maintainers, then discloses and apologizes for that framing. The authors did not measure its effect on later instructions.
These are author-reported benchmark results, not an independent replication. The agent decides when to trigger the prompt, and a single intervention misses decisions made later. The authors’ proposed explanation is plausible, but the ablation does not prove the model’s internal reasoning.