Sydney Runkle and Hunter Lovell use a synthetic e-discovery pipeline to show a sensible division of labor: Jev classifies each page once, LangGraph routes the result in code, Claude rewrites only pages that need redaction, and uncertain privilege calls pause for an attorney.

The graph, not the model, owns the workflow

  • One typed request asks whether a page is responsive, contains personal information, or may be privileged.
  • Plain Python maps those results to produce, withhold, redact, or attorney_review.
  • LangGraph fans pages out concurrently, checkpoints state, and resumes human-review interrupts without repeating completed decisions.
  • The classifier is injected behind one interface, so the same graph can use Jev, the open SemIf model, or a structured-output LLM.

Three implementation lessons

  • A Score result is an expected value across rubric levels, not an integer label; equality checks can silently route the wrong pages.
  • Confidence measures concentration of the returned distribution, not correctness. The first policy escalated five of six pages because it treated low concentration as failure even when the score was decisive.
  • Confidence formulas are provider-specific. The authors changed the LLM adapter’s formula and routing agreement moved from four of six pages to five without changing a threshold.

What the small comparison establishes

  • On the six synthetic pages, the authors report classifier time per page of 0.34 seconds for Jev, 0.49 seconds for SemIf 4B, and 3.80 seconds for Claude Sonnet 5.
  • That makes Jev roughly 11× faster than Sonnet in this tiny recorded run—not a general 200× claim.
  • Engines agreed on four or five of six routes; the disagreements were on deliberately ambiguous pages.
  • Different network paths, a synthetic corpus, six observations, and provider-specific confidence calculations prevent a broad quality or latency conclusion. I did not reproduce the live calls.

What I verified

  • I inspected the linked gist’s graph, classifier adapters, synthetic corpus, comparison script, and offline tests.
  • In a clean uv environment, its test suite completed with 14 passing tests and one beta-API warning.
  • The tests exercise every routing branch, interrupt/resume behavior, and a real TypeSafeClassifier over mock transport. They do not verify Jev’s live accuracy or the reported timing.

The production boundary

  • Preserve the original document after classification; a label should select the next step, not replace its evidence.
  • Let a generative model handle open-ended transformation only where needed.
  • Keep legal authorization and final privilege decisions outside the classifier.
  • Tune thresholds against labeled application traffic for each engine. Typed probabilities are useful inputs, not proof that a route is correct.

“Code keeps the structure and handles exact computation, and a model sits only at the branches that need semantic judgment.”

The durable pattern is narrower than “build agents with Jev”: put bounded judgment behind a swappable interface, keep control flow visible, and escalate uncertainty without surrendering authority to the score.