Jiakai Xu and colleagues give a checking model disposable copies of an AI agent’s computer when the agent claims to be done. The checker runs its own tests, comparing both the finished environment and its initial state. After a rejection, the agent receives concrete failures and resumes its untouched work.
Their study uses Terminal-Bench 2.1, a fixed set of 89 computer tasks, with five trials per task and GPT-5.6 Luna:
- Success rose from 63.4% to 72.4%. That is a nine-percentage-point gain over submitting the agent’s first completion claim.
- Model spending increased. One trial across the full suite cost $23.14 in model tokens, versus $10.70 for the plain agent. Machine costs are excluded.
- The Codex CLI reference scored 76.6% at $50.29. It used the same model but different surrounding software and execution conditions.
- Judging unfinished work helped less. Their tested tree-search setup scored 64.0% at $40.79. The abstract’s 8.4-point improvement uses this baseline.
The checker can run destructive tests because its copies are discarded. Comparing the initial state also helps establish that required files or behavior were preserved. A rejected agent keeps its progress rather than starting again.
Caveat: container copies leave external databases and shared services exposed unless those dependencies are isolated separately. Small stronger-model experiments showed no significant gain, and the follow-up SWE-bench Pro results use modified grading.