Joel Grus’s Jeviris is a runnable experiment for classifying Iris flowers with Jev itself. It calls OpenRouter’s Decisions API with measurements in state and a typed species choice. The code rejects a response from jev-router, which routes requests to other models instead of making Jev’s species decision.

Keep labels out of the question

  • A seeded, class-stratified split holds out 45 flowers and uses 105 as examples. All prompt treatments score the same held-out rows.
  • zero_shot sends measurements alone; few_shot adds three labelled examples per species; full_train adds every training example to the request. These are prompt examples, not model training.
  • Results include the request, true label, predicted species, returned probabilities, resolved model, latency, reported usage and failures. A summary records accuracy and a confusion matrix for each treatment.

A testable experiment, not a reported win

The README gives no actual Jev accuracy results. The full default run would make 135 billable requests. Network errors and rate limits get limited retries; failed requests count against accuracy. The experiment does not compare Jev with a simple Iris baseline or calibrate its returned probabilities.

Verification: inspected the implementation and tests, ran all six offline mocked-HTTP tests successfully, and ran --dry-run to inspect the actual request shape. I made no paid Jev calls, reproduced no model accuracy or latency figures, and performed no security audit. A typed species name can still be the wrong species.