Oxylabs compares Jev with language models on support-ticket and FDA recall classification. The review adds a small comparison with other decision models. Its labeled recall test provides more evidence than the accompanying product demo.

All performance and quality figures below are Oxylabs’ reports, without independent reproduction. The description discloses an AI-generated avatar and says the research dates to mid-October.

4:07 — Separate repeated answers from varied examples

  • The support-ticket example asks about issue type, department, priority, frustration and refund intent.
  • Each model receives the same ticket 10 times. That tests repeatability on one author-labeled example, with limited evidence about general accuracy.
  • Oxylabs reports Jev answering in roughly a quarter-second, versus 3–4.5 seconds for the language models.
  • Prompt caching is disabled, and competing models must return probabilities for every option. Those settings shape the cost comparison.

6:12 — Jev ties Gemini on the harder task

  • Oxylabs samples 100 FDA-labeled recalls, supplying product descriptions, recall reasons, distribution patterns and quantities.
  • Models predict recall class and whether the product is food, a drug or a device.
  • The presenter reports 78% accuracy for both Jev and Gemini, against 70% for Haiku. The narration leaves the combined scoring denominator unclear.
  • Oxylabs reports Haiku costing 7.2 times and Gemini 12.5 times as much as Jev on this test.

A typed answer makes the result easy to consume. It still needs to match the correct recall label.

6:57 — Validate the confidence gate on another sample

  • Oxylabs reports every answer at or above 0.90 confidence being correct in this sample. Jev reaches that threshold on approximately half its answers.
  • The video description links no raw predictions, dataset snapshot or held-out threshold test. The reported clean subset leaves future error rates uncertain.
  • Check a proposed cutoff on separate labeled cases before allowing consequential automatic actions.

8:49 — Cheaper decision models also pass the small comparison

  • Oxylabs samples 36 questions from JevBench’s stated 72-question collection, balancing the task kinds.
  • The presenter reports Jev getting every sampled task right while ranking 14th of 18 by cost, ordered cheapest first.
  • Respan’s Span-02 and Perplexity’s Decider also achieve reported perfect accuracy on this sample. A small public question set cannot settle the broader ranking.

“It’s just a quick look at available options.” — Oxylabs, 9:47

The intervening company-news demo shows collection, article cleanup and bounded questions working together. Its selected articles and company-outlook judgments lack an independent relevance or prediction-quality check.

Caveat: the API comparisons were reviewed from the complete transcript, not rerun; TypeSafe separately labels image and Box features experimental and unsuitable for production assurances.