justindobbs builds a small code-search tool with an optional Jev reranker: a second pass that scores the files a local search shortlisted. The implementation is concrete enough to inspect, run offline, and find a reporting mistake.

Narrow the candidate list before sending code

  • Default search uses word matching, expanded synonyms, and boosts for matching definitions and file paths.
  • --live sends a local shortlist to Jev; --live-all scores every eligible file through the hosted adapter.
  • --smart-fallback asks Jev only about candidates the local thresholds classify as uncertain.
  • The Jev request asks whether a file directly contains the implementation or definition needed to answer the question.

The adapter sends the query, file path, and an opening excerpt through OpenRouter. Simple line-based redaction removes likely secrets, but cannot guarantee private code stays private.

A local fallback can still say Jev

The scorer uses local search when its API key is missing or a request fails. The surrounding search loop still labels those scores jev.

An offline reproduction supplied an explicitly empty key. It returned exactly the local search’s files and scores, reported zero API tokens, and labelled the results Jev. A provider label alone therefore cannot establish which model made a decision.

Fallbacks should record both the requested provider and the provider that actually answered. Otherwise a plausible local result can quietly become evidence for the wrong system.

Keep the benchmark’s scope visible

The repository’s published tables report local-search results. They do not demonstrate Jev’s added value.

  • Local execution passed 77 tests, including mocked API behaviour.
  • Re-running the project’s 28 self-repository queries recovered every expected file within the first five results.
  • The same run averaged 29% precision at five: most returned slots were outside the hand-labelled expected set.
  • The other benchmark suites were skipped because their repository checkouts were absent.

These are a smoke test on the project’s own questions and labels. The scorer includes many hand-written rules, so broader performance needs fresh repositories and held-out questions.

Typed output is not a correct decision. Likewise, a relevance score crossing a threshold does not prove the retrieved file answers the question.

Caveat: implementation, offline tests, local search, and fallback provenance were exercised; live Jev quality, timing, cost, and security were not evaluated.