Ebad Sayed builds a Jev-style decision model from a pretrained Qwen reader. His tutorial explains the input layout, attention rules and training loop, then shows the model failing questions outside its narrow training mix. This is an inferred design, not a reproduction of TypeSafe’s proprietary model.

Make the choices part of the input

A small answer head compares each supplied option with a representation of the question. The output grows with the menu, so adding a label requires no new output weights.

  • Borrow existing special tokens to mark the state, question, options and decision position.
  • Sanitize caller text so literal marker strings cannot create structural tokens.
  • Use an attention mask, which controls what tokens can read, to keep options separate.
  • Give each option the same starting position to reduce order dependence.
  • Read yes/no answers as probabilities and ordered scores as probability-weighted levels.

The author’s current results still show small order-related numerical changes at reduced precision, including occasional answer flips. Structural isolation deserves tests at the precision actually used.

A perfect template score can hide a brittle model

Sayed reports 97.3% accuracy on five-option banking questions and 100% on templated delivery-delay questions. Those results come from his local model, not hosted Jev.

On harder delay wording, he reports accuracy falling to 70%. Before probability rescaling, 26.7% of all answers are wrong despite a top probability of at least 0.90. Better training examples raise hard-delay accuracy to 94% in his saved results.

Changing the data improves the targeted skill. It leaves the broader question of general judgment open.

Check the unfamiliar questions

The article’s improved-data model gets 11 of 22 hand-labeled answers right across tool safety, spam, language identification and other examples. Recounting its saved answers reproduced that tally. One saved decision selects “run it” for a destructive command that the author labeled “block it.”

This is a small diagnostic set. Its value is showing how cleanly typed answers can coexist with plainly mistaken judgments.

An API-shaped replacement has semantic differences

The current project adds broader training recipes and a TypeSafe-shaped endpoint. Inspect the adapter before treating it as interchangeable:

  • Choice descriptions are accepted but ignored; only option names reach the model.
  • Its confidence is the largest option probability, which differs from TypeSafe’s documented Choice statistic.
  • The encoder truncates the state to its configured token limit.
  • Trained checkpoints are absent from the repository and must be built locally.

Verification: read the full article; inspected the README, encoder, mask, answer head, model and API adapter. Offline checks exercised adapter formatting and recounted saved answers. No training, trained-model inference, hosted Jev comparison or timing reproduction was performed.