Omar’s Gutsy is a genuinely open Jev-style stack: Apache-2.0 weights, a 775 MB Q8 GGUF, and a Python runtime that exposes typed yes/no, choice, and score decisions locally through llama.cpp. It is a fine-tuned Qwen3.5-0.8B model—not a new architecture—and its documentation is notably explicit about training overlap and failure modes.

What is open

  • The model reads next-token logits for option letters rather than generating text.
  • Training used roughly 90,000 questions, option shuffling, and cross-entropy plus Brier loss over soft labels.
  • A dedicated Z slot estimates “none of these” for choice and score questions.
  • The runtime implements Jev-style endpoints, temperature scaling, full probability distributions, margins, and concentration-based confidence.
  • State and question prompts are split so the llama.cpp sequence state can be snapshotted and restored across several questions.

The runtime is more than an API shim

  • A self-check compares cached and fully re-encoded paths; caching disables itself if state restoration disagrees.
  • Repeated states use an LRU of sequence snapshots, including Qwen3.5’s recurrent-layer state—not merely a text-response cache.
  • The model handles at most 16 options per pass. Lists up to 255 use a recursive shortlist, so the long-list probability is a multi-call construction rather than one direct model decision.
  • Tests cover cache equivalence, context limits, calibration, rejection, schema compatibility, and long-list behavior under synthetic fixed preferences.

The author’s reported results

  • Gutsy v0.3 scored 167/231 (0.723) on public JevBench items; the repository reports Jev at 0.866 and decision-4b at 0.883 on that set.
  • In 700-item side-by-side runs, Omar reports Gutsy/Jev accuracy of 0.850/0.906 on BoolQ, 0.879/0.883 on SNLI, and 0.587/0.863 on CommonsenseQA.
  • Those are self-run results. Some evaluation families are in-domain, other-system numbers may use different harnesses or samples, and no official sealed JevBench result exists yet.
  • I inspected the model card, prompt construction, cache engine, shortlist implementation, and tests at commit 7e7be17f7283540f41f67340103fa2122dd4ca71; I did not download the 775 MB weights or reproduce inference and benchmark claims.

Honest limits

  • The model card reports near-chance performance on dates, arithmetic, and new multi-step policies.
  • It scored 0 of 7 on JevBench’s ambiguity family: sometimes abstaining unnecessarily and sometimes committing when it should abstain.
  • CPU latency on the author’s hard-tier machine was a reported 4.2-second median and 23-second p95 for long states—not Jev-like hosted latency.
  • RAGTruth and WinoGrande training splits are in the training data; Financial PhraseBank is close to a training source.
  • Long-list shortlisting can choose correctly while spreading probability too thin, so its confidence is not directly comparable to a one-pass 255-option decision.

Why it matters

  • Gutsy makes the Jev interface inspectable and local without pretending to match Jev across the board.
  • The valuable unit is not merely “typed JSON.” It is the combination of constrained readout, explicit rejection, reproducible prompt construction, calibration metadata, and code-owned action thresholds.
  • For production use, every threshold still needs labeled application data. A probability distribution can be well-formed and still point to the wrong decision.

“Gate risky actions on thresholds you measure on your own labeled data.”

Gutsy is strongest as a concrete baseline: small enough for ordinary CPU deployment, open enough to study and change, and candid enough that its reported 72.3% public JevBench score is useful evidence rather than a replacement for evaluation.