TypeSafe’s confidence documentation explains how Jev compresses a probability distribution into a single number. The number describes the model’s own distribution; it does not establish that its answer is right.
Choice: compare the leading option with an even split
For a Choice with a fixed number of options, confidence is a transformation of the highest probability. It adds no independent evidence to that top probability.
- TypeSafe’s formula subtracts the probability of an even split from the leading probability, then scales the result to the range from zero to one.
- The same confidence can correspond to different leading probabilities when the number of choices changes.
- The formula ignores how the remaining probability is divided. Inspect the full distribution or the gap to the runner-up when that matters.
Score: account for distance between levels
For ordered Score levels, TypeSafe measures probability-weighted distance from the most likely level. Probability on a neighboring level reduces confidence less than probability on a distant one.
A score’s position and its confidence can still obscure a split between adjacent levels. If the decision depends on one specific level, inspect that level’s probability rather than the summary alone.
Select thresholds using observed outcomes
A Noul, Jev’s yes/no primitive, returns a probability of true and no separate confidence field. TypeSafe suggests an intermediate band for uncertain cases and different thresholds for actions with different consequences.
The documentation also describes a choice probability as a chance of correctness. Treat that as a model estimate, not a measured error rate. Calibration must be checked on labeled examples from the actual workload before using it to automate consequential decisions. Typed output is not proof of a correct decision.
Caveat: This summarizes TypeSafe’s documented formulas and recommendations. No Jev calls or calibration tests were run for this capture.