Researchers at AMD and NTU compare 36 downloadable coding models, then simulate a simple fallback policy. Try a small model first; restart on a larger model if the patch fails acceptance tests or exceeds its time budget.
Their survey uses all 500 SWE-bench Verified tasks, a fixed set of repository issues for comparing models. The fallback simulation uses only the 466 tasks with complete records for both GLM-4.7-Flash and GLM-4.7. The larger model starts from the original repository rather than inheriting partial work.
- At a 120-minute first-stage limit, calculated GPU-hour cost falls 16.4%. Task success is 73.4%, against 71.2% for the large model alone.
- Mean sequential turnaround rises from 41.6 to 84.5 minutes. A 30-minute first-stage limit still takes 65.5 minutes on average.
- A simulated dispatcher with perfect task-difficulty labels saves 17.9%, only 1.5 percentage points more than the simple fallback.
- The apparent success gain comes from single runs. The authors’ suggestion that the small model sometimes avoids over-engineering remains tentative.
The policy consults benchmark acceptance tests before deciding whether to escalate. A production system needs an equally reliable check that the first attempt really succeeded.
Caveat: These policies combine recorded runs rather than operate live. Their GPU-hour accounting charges every concurrent task as if it exclusively occupied the allocated GPUs, so it is neither physical cluster consumption nor a provider bill.
For background issue queues, a timed fallback is a useful baseline to measure before training a dispatcher. Test actual utilization, acceptance quality and waiting time together.