Thibault Jaigu at Requesty tests whether Jev can choose a coding model from the developer’s first message. The question asks what kind of change is needed, especially whether existing behavior must change consistently across several places.

Keep the question separate from the model map

Jev sees the request, a short codebase description and named change categories. Application policy maps those categories to models:

  • Additions, refactors and single-place behavior changes go to Haiku.
  • Coordinated changes across handlers or streaming paths go to Opus.
  • Low-confidence answers, errors and slow decisions fall back to the configured default model.
  • The chosen model stays pinned throughout the session.

These labels describe work. They let code control model selection without asking Jev to name a model directly.

Separate replay savings from live results

Requesty reports 120 of 129 tasks solved at 72% of the Opus-only cost, versus Opus solving 119. This is an offline replay: real Jev decisions select among previously measured agent outcomes and costs.

On the 43 locked tasks, the short policy solves 40, against Opus’s 41. Requesty treats the small difference as noise, citing variation in its own model reruns.

A related live difficulty gate solves 80 of 90 tasks, versus Opus’s 81, at 87% of the cost. That gate uses a different question. The final change-category policy was checked end to end in staging, rather than receiving this same live task comparison.

More elaborate prompts can memorize the test

  • A short paragraph of codebase context preserved the study’s best result.
  • Longer guides and a full package map added cost or lost tasks.
  • An outcome-informed prompt search looked strong on training tasks but solved 32 of 43 locked tasks, versus the short policy’s 40.
  • Generic routing questions failed to beat the strongest-model control on SWE-bench Verified; DeepSWE results also showed limited transfer.

All counts and cost ratios above are Requesty’s reported results, not independently reproduced performance. The study uses one company’s codebase and a policy developed on Jev; other decision models may need different prompts.

Routing earns its place when held-out completed tasks justify the whole-session cost. A valid category and a confidence score alone cannot establish that a cheaper model will finish the job.

Verification: reviewed the complete article, policy example, result tables and limitations. No Requesty deployment, Jev inference or agent benchmark was run.