LiteLLM’s beta auto router can use Jev to choose a model tier for a request. Its documentation describes what the classifier sees and what happens when the call fails, both important when adding a second provider to an existing workflow.

Classify, then route

  • A Jev Choice question maps the request to a configured tier. That tier selects a model or model pool for the completion.
  • On supported versions, the classifier receives up to three prior user turns by default. Set classifier_context_window_size: 0 to leave prior turns out; the current request and extracted system text can still be sent.
  • The TypeSafe endpoint receives that context even when another provider serves the completion. Review data permissions before enabling the classifier.

Plan for failure and measure the full path

  • Timeouts, malformed answers and other classification failures use a configured fallback. The default is a heuristic; an operator can choose the default model instead.
  • A process-local circuit breaker opens on recognized timeouts and allows a later recovery probe. A timed-out call may still have reached the provider or incurred charges.
  • Classifier usage is logged separately from the completion. LiteLLM says reported routing savings deduct the classifier cost; avoid counting it twice.
  • Test routing can incur classifier charges even without a completion. Pin the model version and evaluate tier choices, quality and end-to-end costs on representative requests.

A valid typed choice is not evidence that the selected model is adequate. The documentation was inspected; the router and its performance claims were not independently tested.