FeatureArc tests coding agents, models running multi-step software tasks with tools, as they extend one feature on their own earlier code. Each new request inherits any unresolved mistakes.
The authors evaluate ten agents on 73 reviewed tasks with 263 stages across four languages. Requirements and interface contracts are explicit. Hidden acceptance feedback stays withheld between stages.
- First-stage success averages 62.7%; complete-feature success on final code is 28.5%. Claude Opus 5.5 leads with 83.6% and 49.3%, respectively.
- Earlier defects appear in 72.9% of failed later stages. Agents leave earlier errors unnoticed in 94.9% of stages following a failure.
- Of 273 runs passing the last extension’s tests, 69 still fail the complete-feature check.
- Passing self-tests offer limited assurance: only 52.7% of stages ending green pass the hidden new-behavior tests.
Maintain independent acceptance checks for the whole evolving feature. An agent can write tests that agree with its own mistaken reading of a specification.
Caveat: These are author-reported autonomous runs without intermediate acceptance feedback; the results do not estimate success with human review between changes. Tencent’s authors also use the evaluation to develop Hunyuan, and cross-agent rankings combine model and wrapper differences.