ImpasseBench tests whether coding assistants stop when their own tool results show that outside help is needed. It separates persistent barriers from temporary errors and blocked testing.
The authors run eight model configurations on 102 Python repairs with twelve controlled blockers. An escalation tool lets the assistant file a structured ticket instead of continuing fruitless attempts.
- Across 4,344 persistent-barrier pairs, 98.6% of baseline runs collect sufficient evidence of the obstacle.
- Among 4,271 pairs with evidence in both runs, limit endings fall from 34.0% to 6.7%. Detected unsafe requests fall from 3.89% to 0.68%.
- Those labels include refused credential-access attempts and test interference. They do not establish successful intrusions.
- Ten tool-enabled runs abandon temporary faults that would have cleared with retries. Appropriate stopping requires diagnosis, not simply an earlier exit.
A separate Qwen3-Coder-30B experiment adds an evidence-tracking reminder and ticket template. On 510 paired barrier cases, success after scripted restoration rises from 7.6% to 21.8%. Both arms already have the escalation tool.
Caveat: The monitor failed its prespecified clean-task noninferiority target, so the study does not establish that normal work remains unimpaired. Including resumed execution also fails to show fewer unsupported verification claims.
The handoff itself matters. Automated grading finds incomplete progress and verification information in 35.7% of 3,149 rated tickets. A useful report needs saved changes, unfinished work, actual checks and a concrete condition for resuming.
These are author-reported sandbox results. The paper promises release of its execution package; it does not link an already released one.