1LittleCoder tests Jev as a router between DeepSeek V4 Flash and Kimi K2.6 on Nebius, against GPT-5.1 with medium reasoning. The useful finding is not the savings headline alone: the accompanying repository includes fixed-model baselines that change what the experiment establishes.

I read the complete X article and inspected the experiment repository at 56b4c3b847892bcf2641290c0822f20c75c21c39, including routing, API accounting, grading, and committed result summaries. I did not run the benchmark or reproduce its API results. This is source inspection, not a security audit.

The full workload loses some quality

The author’s recorded results cover 30 tasks with three attempts per arm:

  • Jev-routed: 83/90 passes, or 92.2%, versus GPT-5.1’s 87/90, or 96.7%.
  • Fixed Flash: 84/90 passes; fixed Kimi passes 90/90.
  • Estimated API cost: the author reports at least 78.0% savings for routing against GPT-5.1, including routing charges and conservative allowances for unknown-billed failures.
  • Median end-to-end latency: the committed summary records 2.174 seconds routed versus 1.829 seconds for GPT-5.1. The cheaper path is not the faster path here.

These are author-recorded figures, not independently reproduced. The task-clustered confidence interval does not establish the experiment’s proposed quality margin; the summary explicitly marks noninferiority as unsupported.

The attractive subset does not establish a router benefit

The article’s 98.4% success and 73.9% cost reduction come from a post-hoc 21-task subset excluding classification tasks. Its committed artifact labels that selection exploratory.

  • Routed: 62/63 passes, with estimated cost bounded above at $0.01239.
  • GPT-5.1: 63/63 passes, estimated cost $0.04752.
  • Fixed Flash: 63/63 passes, estimated cost $0.00263—cheaper than routing and without its failed attempt.

All figures are the author’s, not reproduced here. The article attributes the routed failure to an API timeout; operational failures still count against success. This supports trying a cheaper generator, not assuming that Jev adds value over simply using Flash.

What the routing code actually does

  • Pin jev-1.13.0; ask about multistep reasoning, algorithmic demands, and generator tier.
  • Escalate on complexity signals, a strong-tier choice, low routing confidence, or a routing failure.
  • Keep the final generation separate from Jev’s typed decision.
  • Price token usage at frozen rates; these are estimated API charges, not total operating costs.

The author identifies with Nebius and ends with a Nebius credit offer. The inspectable code and less-flattering baselines make the piece useful despite that promotional context. Typed output is not a correct decision, and cheaper generation is not evidence that a router is necessary.