LLMxRay’s October 12 release adds a Live page for observing other applications’ Ollama traffic. Instead of testing a model only inside its own dashboard, it can sit between your existing coding assistant or chat frontend and Ollama, the server running your local models. This is useful when an agent feels slow and you need to distinguish oversized prompts from poor cache reuse or a model running partly on the CPU.

  • Setup: With Ollama running and a model available, launch npx llmxray@0.7.2. Change your client’s Ollama address to http://localhost:11435, then open the dashboard’s Live page. Continue uses apiBase; Open WebUI uses its Ollama API URL. An OpenAI SDK client uses http://localhost:11435/v1.
  • What it shows: Per-turn prompt tokens, cached tokens, processing and generation speeds, and tool calls. A detail view breaks the request into instructions, tool definitions, history, and the latest message; it compares consecutive prompts to locate the first changed text that may disrupt cache reuse.
  • What is genuinely new: Earlier pages observed requests generated inside LLMxRay. Version 0.7.2 adds the forwarding port and per-client summaries for traffic from other tools, including estimates of time a better-cached prompt could save.
  • Access and cost: Free, Apache-2.0 source, with npm and Docker distribution. There is no required hosted account or subscription; local inference still consumes your own hardware and electricity. Cache figures require Ollama 0.33.3 or newer.
  • Retention: The guide says counts and timings stay in memory by default. Saving prompt text is opt-in and writes daily files under ~/.llmxray/observe/; switching it off does not delete those files.

The limitation is measurement fidelity, not just compatibility. Prompt sections are apportioned by character count, not tokenized individually, and projected cache savings are estimates rather than replayed experiments. OpenAI-compatible streaming clients must request stream_options: { include_usage: true } to expose usage counts; otherwise cache figures show “not reported.” The proxy does not add that option or optimize your requests for you.

This is a release-and-documentation capture, not a hands-on test or independent confirmation of the author’s performance claims. For an Ollama-based workflow it is a concrete debugging aid; it is not evidence of improved answer quality or a replacement for observability across hosted model providers.

Live setup and limitations