3D GameDev Arena asks coding assistants to build browser games from empty workspaces. Its 108 tasks span 14 genres, with written requirements and reference images. Evaluation separates working mechanics, recorded interaction and what human players prefer.

Requirements are frozen before any game is inspected. Tests are then adapted to each implementation without changing its rules. Separate recordings support model judgments of presentation and control feedback; blind human comparisons measure preference across whole playing sessions.

  • GPT-6.1 Sol in Codex leads functional scoring at 64.7 out of 100. GPT-6 Astra and Kimi K3 score 59.4 and 59.8.
  • Astra’s fitted human-preference score is 69.24 against Kimi’s 25.86. These estimate wins against an average competing configuration, not requirements satisfied.
  • Only 37% of recorded moments are reached naturally; 63% use state assistance. Such recordings show an experience in that state, not that normal play reaches it.
  • Natural-only playability scores remain close to the overall scores, so assistance cannot explain the entire pattern. Short model-scored clips still differ from uninterrupted human play.

The preferred configurations spend more debugging effort on full builds and runtime checks. The paper associates that behavior with better games, but each model appears in only one surrounding software setup. Testing policy, model and tools were not independently varied.

Caveat: These results compare complete model-and-software configurations, not isolated model quality or a causal effect of live testing. Preference vote counts and clustered uncertainty are not supplied alongside the headline scores.

The practical evaluation pattern is to keep mechanics, visible feedback and human preference as separate gates. Assisted states help diagnose failures when clearly labeled; they cannot certify a reachable gameplay loop.

Daily coverage