eepstein201/jev-mcp exposes local decision tools through the Model Context Protocol. Despite its name, the default inference path uses Kev and Qwen, not a hosted TypeSafe Jev call. Its substantial implementation is worth reading for the cascade, threshold reports and failures—not the README’s “ultimate local safety gate” framing.

I inspected the routing and provider code, calibration tool, prompt-selection helper and evaluation harness. I did not run the MCP server, local models or benchmark. All model measurements below are the maintainer’s reports, not independently reproduced results; this is not a security audit.

Route individual questions, not just requests

  • Small inputs go to Kev first; questions whose largest probability is below 0.85 escalate to Qwen.
  • Inputs estimated above 4,000 tokens bypass the fast tier; the estimate is character-based, not a tokenizer count.
  • Missing fast answers also escalate. If the smart tier omits a question, an existing fast answer can survive the merge; only questions with no usable distribution cause an error.

Those are inspected policy settings, not evidence that every accepted answer is safe. Typed output is not a correct decision, and a confident fast-model mistake need not escalate.

The useful results are smaller than the pitch

The maintainer reports 89.8% accuracy and 44 ms warm latency for Kev, versus 95.5% and 662 ms for Qwen, on an 88-row set. Most rows were LLM-authored, loading and warm-up were excluded, and the figures measure each engine rather than the complete cascade.

More instructive are the reported failures: confidence-only prompt selection chose the worst phrasing on a small labeled check; a local fine-tune did not improve held-out classification. Neither finding establishes how the system will perform on production traffic.

Fit thresholds without inventing certainty

The calibration tool uses the cascade and supports binary logistic probability adjustment. With enough examples in both classes, its report uses cross-validation; with fewer, it explicitly reports fitting and scoring the same rows.

  • Saving calibration is opt-in because the exported temperature changes every smart-engine question.
  • The saved temperature uses the fitted slope, not the full logistic model’s intercept; do not assume it reproduces the report’s adjusted probabilities.
  • The report’s “Safe” or “Optimal” labels describe observed sample behavior, not a future false-positive guarantee.

The README’s stronger claims—eliminating hallucinations, impenetrable XML boundaries, and guaranteed zero false positives—are not established by this evidence. Treat sanitization and prompt delimiters as defenses to test, and keep deterministic permissions and human review around consequential actions.