TypeSafe’s Jev is being sold as the next frontier of language models: a “decision model” that returns a probability for each option you hand it, in one pass, instead of writing an answer in prose. Duarte O.Carmo at NobodyWho responds with a parody that doubles as a proof — 25 lines of Python, a 0.6B model on a laptop, and the same kind of numbers coming out.

What the 25 lines do

  • Load a small open model (Qwen3-0.6B, via llama-cpp-python) locally. No API call, nothing leaves the machine.
  • Build a multiple-choice prompt: an email to classify, then A. Legitimate / B. Spam / C. Phishing, with an empty assistant turn left open for a one-token answer.
  • Every language model already produces a score for every word it could say next. The script reads the scores for the first token of “A”, “B” and “C” and turns those three numbers into percentages — no sampling, no generated text, one pass.
  • The output for the phishing example: Legitimate 3.1%, Spam 8.4%, Phishing 88.5%.

What the post is arguing

  • The headline properties — fast, local, returns probabilities rather than prose, changeable by editing a prompt instead of retraining a classifier — are not evidence of a new paradigm. They fall out of looking at the numbers a model was already producing and ignoring everything else.
  • The post is precise about what it does not claim: no API call, no synthetic data, no training run, no reinforcement learning to calibrate the probabilities, and no invoking of the “System One decision model” label. That omission list is the argument.
  • At the bottom it is labelled a parody, with pointers to fuller open implementations (OpenJev, openjev-sglang). Several readers took that as a tell: the piece makes a serious point, sells a product at the end, and settles the question with a joke rather than a measurement.

The useful takeaway is not “Jev is fake” — the post never quite says that. It is that a mechanism and a product are different things, and the difference here is calibration and reliability, which a 25-line demo cannot establish and a headline accuracy number cannot either. If the trick is public, the argument moves to whether the training data that makes the probabilities trustworthy is defensible — and whether anyone outside the company can check it.

What the thread adds

The 100-comment thread on Hacker News

  • dTal — the hardest-won detail, from someone who says they built this exact thing last year. The multiple-choice tokens “should sum to 95% or more, ideally 99%, or the LLM is not following instructions properly”; run it several times with the options permuted across the letters and average, because models “are surprisingly biased towards picking ‘A’”; and frame the prompt as if it were question 12, not question 1.
  • sigmoid10 — where the method gets shaky: on a chat model trained to write prose, the letter tokens “might get diluted in whatever else it wanted to say,” so the model can be partway into a sentence like “A phishing attempt seems likely…” and top-score a label it isn’t choosing. They prefer structured outputs with whole-word options for that reason.
  • antirez — a free accuracy trick aimed at how the model reads the prompt: put the options before the text being classified, because then the model can spend its tokens building state aimed at the actual task, and repeat the question twice for the same effect.
  • armcat — the counter-case to dTal: token probabilities are usable on local models but “more or less broken since GPT-4o” on frontier ones, and in their testing you get better agreement with real confidence by simply asking the model to state it.
  • nautilus50 — the mainstream version of this, for anyone who just wants it working: llama.cpp takes a BNF-style grammar file that constrains generation to your options.
  • petercooper — the same idea extended past text: Qwen 3.5 0.8B doing image classification through structured JSON output, roughly 250–500 ms per call on a Mac, “good accuracy (depending on task)”.
  • xg15 — the tell in the reactions rather than the code: the post dismisses the calibrated-probability pitch with “only 99% correctness! Borderline unusable!” while treating the demo’s own bare numbers as self-evidently meaningful.

The question the thread kept asking

Two things the post never answers, raised repeatedly and in different words.

First, what actually separates the 25 lines from the product. bruhhhhhh asks directly whether this is just a task-specific small model, “possible and faster with a classic BERT model,” and whether the pitch is a task-specific smaller model — garciasn answers that BERT needs a large labelled dataset and fine-tuning, while this approach is zero- or few-shot, summing it up as “an optimized, task-specific small model with the flexible understanding of a traditional LLM.” no-name-here supplies the missing columns: nothing on latency or compute against Jev, nothing on error rate, and no evidence it always emits a parseable format — a point onion2k sharpens by noting that “fast” was never stated relative to anything, with davide countering that prompt processing dominates both paths anyway, so it cannot be much slower. The companion post on whether OpenAI can replicate Jev spends its length on the same gap: whether the probabilities hold up.

Second, whether the post is an argument or a funnel. jorisw calls it “highly suspect of content marketing” that “ends with referring to a product, and saying ’this is a parody post,’ after pretending to make a serious point.” ramon156 files it under the genre of “OneDrive in 10 lines of code (SFTP) — while technically correct, it’s not the same thing.” dhsysusbsjsi takes the other side, sympathising with the Jev team having to explain “why the 100 smaller things done better than this add up to a much better product,” and jpnc answers that the verb is wrong: “Replace ’explain’ with ‘sell’. Don’t forget that it’s a gold rush.”

A note on reading comments as evidence: HN handles are pseudonymous and the site publishes no per-comment scores, so the ordering here is HN’s own ranking, not a vote. This is a slice of the thread. Where commenters describe how Jev or the logprob method works, that is their claim, not the article’s — and dTal and armcat disagree with each other on whether the numbers can be trusted at all.