Jacob Edgar-Anderson at Sayora tests a concrete alternative to chunk-based retrieval for an AI receptionist: give Jev short descriptions of a business’s pages, ask which page answers the caller, and include “none of these.” The interesting result is not just better ranking—it is refusing to supply irrelevant evidence.

All measurements below are Sayora’s reported results, not independently reproduced. The questions and most relevance judgments were model-generated; this was not a live-call trial.

Pick a page, not a related passage

  • Four customer sites contributed 22–232 pages after removing property listings and repetitive town landing pages.
  • Haiku wrote page descriptions; Opus separately generated 168 questions from actual page text and judged pooled candidate pages.
  • Jev received the caller’s words plus page options; ordinary code handed the first two pages to the answering model, or abstained.
  • The production baseline used vector similarity: finding text that is semantically close, not necessarily text that answers the question.

The useful result is the “none” option

Sayora reports that Jev placed an answering page first on 88% of answerable questions, versus 70% for production chunk retrieval and 73% for embeddings over the same descriptions.

  • Across all questions, neither handed-over page answered the question in 3% of Jev cases versus 33% for production retrieval.
  • Jev rejected 38 of 40 questions the sites could not answer; production retrieval rejected 9 of 40.
  • Sweeping the production similarity threshold still left a reported 15% wrong-page handoff rate at its best overall setting.
  • GPT-6 Luna’s first-page accuracy was statistically indistinguishable from Jev’s; Sayora reports slower requests and more incorrect handoffs for Luna.

These are retrieval outcomes, not measured final spoken-answer accuracy. The article’s “answered right” metric counts useful page handoff or correct abstention—not whether a generated response was correct.

The description can erase the answer

A plumbing page mentioned water softeners, but its short description did not. Jev confidently chose “none”; chunk retrieval found the actual answer. Key cutting and EV chargers produced similar failures.

  • Maintain descriptions when pages change, and preserve specific services rather than just broad categories.
  • Embeddings can still shortlist larger sites before Jev makes the final choice.
  • Sayora’s reported Jev median was 0.28 seconds, versus 0.16 seconds for production retrieval: better selection was not a faster lookup.

Promising pilot, not a universal replacement

The study used synthetic questions, one model judge, one run from a UK laptop, and only five hand-checked judgments. It did not compare against a strong hybrid-search or reranking system. A larger site received a partial latency probe, not a completed accuracy evaluation.

Sayora plans real-transcript and live-call tests and would retain embeddings as a fallback. Its reported about $0.45 covers Jev requests in the experiment—not description generation, crawling, final answer generation, or total operating cost.

Typed output is not a correct decision. Here, the useful engineering pattern is an explicit abstention path plus a maintained evidence catalog; neither protects you from information that never reaches the picker.