Sivasuthan’s Fovea-4B is an open image decision model with downloadable weights and a local System One-compatible server. It answers typed questions about documents, charts, photographs and JSON evidence.

The serving implementation reveals an important uncertainty tradeoff: an internal unknown answer disappears from returned Choice and Score distributions.

Unknown can become a confident allowed answer

The model scores the requested alternatives plus an unknown candidate. For Choice and Score, the response code removes unknown and renormalizes the remaining probabilities.

  • An offline synthetic-logit fixture gave unknown probability 0.9999999979.
  • The actual response function still returned the fixture’s allow label with probability 0.9999546 and confidence 0.9999092.
  • The equivalent Noul, or yes/no probability, stayed near 0.5 because that path splits unknown mass evenly.

These are tests of the response transformation using constructed scores. They are not predictions obtained from a real image. Choice confidence measures separation among the surviving alternatives while discarding whether the whole set was unsuitable.

A failed question still gets a label

  • The per-question exception path returns uniform probabilities with zero confidence.
  • For Choice, it also fills the answer with the first supplied label.
  • A mocked inference failure exercised that path and returned the same valid-looking answer envelope.

Callers need explicit failure handling and an abstention policy. Acting on the label alone can turn a failed classification into the first allowed action.

One pass applies per question

The server loops through questions and processes each separately. An offline runner fixture observed three calls for three questions.

A compatible multi-question API therefore does not establish shared-state inference in one pass. Measure complete requests before assuming extra questions are nearly free.

Read the evaluation as an author-run comparison

The author reports 80.6 Intelligence and 0.035 expected calibration error, a measure of probability mismatch, on 2,360 multiple-choice questions. These results were not reproduced here.

  • Intelligence is the source’s chance-adjusted score; the card also calls the same number raw accuracy, which needs clarification.
  • The author’s reported lead over H2O-Lightning is 0.7 points, with a 95% interval from −0.7 to +2.0. That interval leaves the ordering unresolved.
  • This is a public-data evaluation, separate from ImageJevBench’s sealed leaderboard. The comparison’s open model named Jev-Vision is distinct from TypeSafe’s hosted Jev.

Verification: six offline checks passed against source at 7708ff86bb36640478b1583d864124a9a5cb77cf. No weights were loaded, vision benchmark run or security audit performed.

Typed output is not a correct decision. Preserve unknown evidence and make failure visible before wiring the returned label into an action.