Jev

A browser-agent postmortem shows why batched questions, evidence budgets and deterministic replay matter more than a classifier-only speed headline.

An SO-101 robot-pet prototype separates message interpretation from allowed performances, with local state, reviewed transitions and explicit hardware limits.

An inspectable Claude Code skill turns verbatim directives into Jev questions, with offline tests and a reproduced edge case in its development gate.

Bardia Pourvakil shares concrete failures in batching, citation support and test-quality judgments, plus a reusable design guide from GC AI’s hackathon.

Johnnys.News explains an inspectable embedding-based decision demo, with cached options, browser inference and unusually explicit limits on its probabilities.

Pusu’s browser-agent experiment exposes premature DONE, adds deterministic checks, and publishes a safety probe whose rules-only result we reproduced.

Ves Stoyanov reports six classification experiments where editable rules compete with fine-tuning, and uncertainty sampling misses confidently wrong routing policy.

Unsloth documents serving local Laya through /v1/systemone, with explicit warnings about confidence thresholds and trimmed choice descriptions.

Sina Booeshaghi compares Jev and OpenAI on personal research relevance and RELISH, with a useful distinction between ranking scores and calibrated probabilities.

A Telegram moderation implementation uses Jev to classify messages and link spam fragments, while keeping failures open and evaluating self-hosted alternatives in shadow mode.

A sponsored but useful head-to-head finds where Jev's cheap typed decisions hold up, and where an encoded shell command defeats a model-only safety check.

Stephen Solka tests Jev against political outreach mail, mines difficult counterexamples, then builds a tiny local classifier; his held-out results and threshold caveats are the story.

An email-security company's synthetic hard-case benchmark finds decision APIs promising, but deployment thresholds, class tradeoffs, and serving costs complicate the leaderboard.

An A/B test of 973 coding-agent threads found much lower median LLM cost, while its merged-PR metric could not establish either a quality gain or equivalence.

Four measured search experiments show better ranking, a relevance-versus-diversity tradeoff, a confident query-filter failure, and promising document splitting.

A local Jev-compatible endpoint reads answer-token logits from Qwen rather than training a new classifier; its published results also show why confidence gates need testing.

Mateus Fonsek compares three decision-model APIs across Portuguese agent routing, injection screening, and answer grading—with cases, code, and response logs available to inspect.

An inspectable roguelike puts Jev behind legal choices and a rule fallback; its own paired runs find better style-matching rewards but less variety.

A macOS desktop pet routes bounded pose choices to hosted Jev or a native Open-Jev scorer, while voice uses separate models; the code is inspectable but parity tests are opt-in.

Li and Wagle extract typed choices from ordinary LLM token probabilities; their experiments show when narrow fine-tuning helps—and when it damages other decisions.

A reproducible decider-2b deployment on Snowflake shows public-benchmark accuracy, GPU scaling, and the idle-capacity bill behind impressive per-decision prices.

SREGym’s Jev-only diagnosis pipeline passed 80 of 105 judged incident reports; its missed faults expose the limits of evidence selection and single-component answers.

Byren Cheema’s reproducible chess test found fast, cheap, always-legal Jev moves—but almost no wins against opponents trying to win.

Veris put Jev in a Pipecat voice agent's turn gate: fewer caller interruptions across simulated calls, but slower replies and virtually unchanged task success.

Vals AI's two paired evaluations show Jev tying frontier models on filing-based claim checks but ranking last on a small LegalBench slice; a tuned error threshold also missed its target.

A Claude Code mod uses Jev to choose effort, models, and context pruning; its cache-switch math and measured-versus-estimated savings are more interesting than its headline savings.

Lee and Berg-Kirkpatrick compare one-pass answer probabilities with Jev across classification, agent monitoring, and triage; task mix and hosting matter more than the headline tie.

On Cisco's content-safety evaluation, a policy-trained classifier beats zero-shot Jev at strict false-positive budgets; the reference labels and thresholds limit deployment claims.

Harness compares its production API-inventory pipeline with a single LLM and Jev on a human-checked endpoint sample; the cost story is strong, but misses still matter.

A TREC passage study finds Jev less aligned with human relevance grades than GPT-6 Luna, but roughly tied on search-system rankings at lower reported cost.

A browser-tool proxy offers Jev bounded element choices; its author reports slightly cheaper but slower runs than direct Playwright, with raw benchmark results unavailable.

Kevin Liu's agent testbed makes Jev choose from visible browser actions and records the entire decision—but its Jev contract tests use a fake provider, not a measured live Jev run.

Brian Zhang built a traceable Jev question atlas with option-order checks and human comparisons, but keeps the results private and explicitly disclaims benchmark status.

A local-first agent-memory project uses Jev as an optional, bounded retrieval advisor while source checks, user approval, and writes remain outside the model.

University of Bonn researchers publish Jev's raw responses on 37 datasets, finding strong zero-shot routing and calibrated choices—but important binary-threshold, language, and benchmark-contamination caveats.

Sebastian Raschka places Jev in the history of text classifiers, reports an IMDb test, sketches a reusable choice-scoring head, and separates real calibration work from guesses about TypeSafe's proprietary training.

Sean Goedecke proposes using Jev as a low-friction first version of a classifier, collecting real decisions, then distilling a proven workflow into a smaller task-specific model.

GCA Americas’ workshop puts Jev or an open DiffusionGemma endpoint in a deterministic game loop while Gemini handles slower vision and prose work.

DoubtBench scores 7,455 decisions against individual HelpSteer2 votes. Jev leads the tested models on accuracy, speed, and recorded cost, but its uncertainty barely tracks human disagreement.

JevMDB turns movie metadata into per-criterion Jev scores, publishes usage and feedback plumbing, and includes a local mock—but the ranking has no labeled quality evaluation yet.

Metis wraps Jev judgments in explicit Python steps for issue triage and pull-request screening, with uncertainty exits and state checks before GitHub writes—but no accuracy benchmark yet.

Cloudflare releases 27B and 9B Jev-compatible models with weights, inference code, vision support, and broad self-run benchmarks—but the leaderboard still needs independent reproduction.

Amplitude tested Jev on 64 synthetic product-analytics charts: perfect recall and very low cost came with 55.8% precision, making downstream verification part of the economics.

Open-Jev releases a 27B Qwen-based decision model, training code, adapters, probability head, and audited artifacts—while keeping sealed-evaluation and safety failures visible.

A reproducible zh-TW evaluation finds that a local Jev-compatible model can route useful cases—but option count, paraphrasing, prompt injection, and confident errors make it a poor sole gate for consequential actions.

A large Spanish-language Jev research notebook tests how prompts, answer sets, wording, confidence, and hidden policy shape typed decisions—and keeps the failed interpretations.

Marcus Rox built a 75-case, artifact-backed benchmark for Jev alternatives, with raw answers, cost and latency records, automated model discovery, and unusually explicit limits.

AI Profs tested Jev and Claude Opus 4.8 on 18 document-heavy business cases; Jev was far faster and cheaper, but the author-written references and repeated runs sharply limit the accuracy claim.

TeXposit put Jev into assistant preflight, answer checking, and model routing; its small tests show large latency gains, useful false-positive controls, and no substitute for real evaluation.

Glean’s internal tests found Jev stronger than its LLM router, weaker than a fine-tuned classifier and production search ranker, and more repeatable—but not proven more accurate—as a citation judge.