Qdrant’s Andrei Cristea and Chadha Sridi tested Jev at four points in a search pipeline. Their best finding isn’t that a fast decision model wins everywhere; it’s that each decision needs its own success measure. The team published experiment notebooks, but I have not rerun them.

Reranking and repetition

  • On 100 NFCorpus queries, asking Jev to score each retrieved document improved top-ten ranking relative to Qdrant’s initial retrieval. Repeatedly asking it to pick the winner improved the score only slightly, at ten requests rather than one.
  • On 240 product searches, Jev reranking raised Qdrant’s relevance metric from 0.683 to 0.769—but left near-duplicates and narrowed the variety of product classes. Combining its scores with a diversity method kept relevance above the original search while reducing repetition. These are Qdrant’s measurements, not reproduced here.

A confident filter can hide the answer

  • For product categories, Jev’s scores drove a rule: filter when the category score exceeded 0.9, boost from 0.5 to 0.9, otherwise leave the search alone. Qdrant says this worked best among its tested setups on 300 long, generated queries; it added roughly 0.5–2 seconds per query.
  • On an illustrative short query, “apple”, Jev scored Food & Beverage at 0.91. The filter then hid MacBooks, trees and decals. This was an example without relevance labels, not a benchmark result. A well-formed probability did not make the interpretation correct.

Where document splitting helped

  • Jev judged where topics changed between sentences before indexing papers for search. On QASPER, Qdrant reports better retrieval of marked evidence than fixed-size, recursive and embedding-based splitters at matched average chunk size. Its reported relative gain in evidence-overlap score was 13.0%; I did not reproduce the experiment. Better evidence retrieval is not yet proof of better final answers.

Qdrant’s practical takeaway: choose the pipeline decision worth improving, keep irreversible filters conservative, and test against real short queries—not only long benchmark prompts. For related work, see Jev as a search judge.