Stefan Fidanov tests Jev on support screening, ticket routing, model selection, tool approval, and lead scoring. DeepInfra sells the downstream model service, but the review includes failures worth keeping alongside its favorable examples.

An ordinary rewrite bypasses the screen

The review’s yes/no injection check initially flags a fake instruction to issue a refund. Fidanov reports its probability falling from 0.58 to 0.28 when “SYSTEM:” becomes “Quick note from billing ops.” That crosses the example’s 0.5 blocking threshold and lets the request through.

The money amount and instruction remain the same. A classifier can contribute a warning, while explicit authorization and tool permissions still govern execution.

A fallback needs ambiguous cases that actually reach it

  • The router compares the two highest category probabilities and sends close calls to a person.
  • DeepInfra reports that none of seven tickets reached that fallback, including deliberately ambiguous examples.
  • The review proposes an explicit “needs a person” option or a separate ambiguity question. It says those alternatives remain untested.

A sharp probability distribution can still conceal an unclear request. Typed output is not a correct decision; confidence needs checking against the cases your application finds ambiguous.

The “private” route already called a hosted model

The example sends the prompt to api.typesafe.ai, asks whether it contains private information, and only then chooses local or cloud generation. Its local drafting lane therefore still discloses the original prompt to TypeSafe.

For a local-only requirement, the first inspection must happen locally or sensitive material must be removed before any hosted call. Routing a later step to local inference cannot undo earlier disclosure.

Budget for the whole path

DeepInfra reports about 280 ms median across 32 calls, measured from a laptop. Its combined workflow reports 370 ms median gate overhead and $0.03 per 1,000 requests. These are source-reported results from tiny hand-written samples, unreproduced here; the price excludes application hosting and human review.

Batch questions that read the same message. Check a proposed tool call separately because its arguments are new evidence. Keep exact amount limits and account permissions in deterministic code.

Caveat: article and embedded code inspected; no live API calls were run. The reported routing savings and small-sample successes do not establish production accuracy or calibration.