Snowflake’s Matt Harris and Josh Reini compare Jev with open decision models, general language models and a managed SQL classifier. Their selection guide starts with the mistakes a workflow can tolerate, then compares serving paths and costs.

Count missed records and unnecessary reviews separately

The authors report these results on a synthetic sensitive-information test. These measurements were not independently reproduced for this capture.

  • The test contains 3,000 items, split across table cells, narrative passages and JSON records. Three-quarters contain sensitive information.
  • Jev catches 98.80% of sensitive items, while flagging 17.87% of clean items. Its exact-label accuracy is 94.27%.
  • Claude Opus 5 catches 99.96%, with 0.80% false positives. The stronger model reduces unnecessary reviews on this workload.
  • Decider-2B catches 60.89% of sensitive items despite performing well on the separate banking-intent task.

A routing model’s success on customer-support categories provides little assurance about its suitability for sensitive-data screening. Typed output guarantees an answer’s shape; decision quality still needs its own test.

Similar accuracy can hide different training exposure

The expanded BANKING77 comparison found no statistically significant differences among the strongest candidates. Laya was significantly worse than each of the other five.

  • Decider-2B had learned BANKING77’s training split. The evaluated messages came from its held-out test split.
  • Those results support testing it on similar intents, with a fresh check before using it in another domain.
  • The separate sentiment task had no reference labels. Agreement between models there measures consistency, rather than independent accuracy.

Measure the whole serving path

Snowflake reports Jev API calls taking approximately 0.2 seconds, while single-row SQL calls through the tested Python user-defined function took approximately 3.8 seconds. The authors attribute the difference to starting the wrapper for each call.

Batching changes the comparison. On the harder banking task, the expanded Jev run completed in 5.59 seconds, the fastest reported result. The article warns that batch time and interactive response time answer different questions.

Keep the cost denominators aligned

The authors report $0.112 in gateway charges for 4,729 Jev calls, consistent with the gateway’s usage page. The surrounding warehouse and SQL wrapper add costs outside that figure.

Compare API charges, serving capacity, startup, idle time and human review separately. The earlier Snowflake deployment capture covers the GPU-capacity side of that bill.

Verification: read the full article and checked its source page. No Snowflake deployment, model calls or performance reproduction was performed. The sensitive-data test is synthetic and supplies no certified compliance guarantee.