Elad Rosenheim (Principal PM at Netlify) ran the same “build a coffee shop website” prompt through 11 different AI models — Claude Opus 5, Sonnet 5, GPT 5.6 Sol, GPT 5.6 Terra, Gemini 3.6 Flash, Gemini 3.1 Pro, Kimi K3, Kimi K2.7 Code, GLM 5.2, DeepSeek V4 Pro, and DeepSeek V4 Flash 0731 — and compared the visual quality against credit cost. The results are sobering for anyone buying into the “just use the best model” narrative.
- 200x cost spread: Claude Opus 5 averaged 519 credits/run (one outlier: 1,055 credits); DeepSeek V4 Flash 0731 averaged 2.4 credits/run
- Quality ≠ price: GPT 5.6 Terra (39 credits) produced more coherent results than DeepSeek V4 Pro (37 credits); Gemini 3.6 Flash (103 credits) crushed Gemini 3.1 Pro (53 credits) despite only 2x the cost
- Opus outliers are a pattern: One of three Opus runs used 4x more credits than its peers without producing a 4x-better result — this happens frequently, not rarely
- Value sweet spot: DeepSeek V4 Flash 0731 occasionally produces results competitive with mid-tier closed models at 2.4 credits average
- GLM 5.2 is text-only: Unlike Kimi, it cannot accept image inputs for design inspiration
- Image generation is still weak: All models produce mediocre vector graphics; DeepSeek V4 Pro generated a broken image link
The article’s real contribution is framing the choice as a strategic one: do you want a turned-key Opus site that pre-plans everything, or a cheaper model with an iterative human-in-the-loop workflow? The answer depends on your budget, your quality bar, and how much you enjoy steering the output. Netlify open-sourced their AXIS evaluation framework to help teams make this call systematically — a useful complement to the “Don’t be a meat proxy” argument that the human still needs to judge what’s worth shipping.