Radu, a search engineer at Vespa, on how to actually choose and tune embedding models for search and RAG — hosted by Hamel Husain. ~22 minutes.

Why not just take the leaderboard

  • Most people pick embeddings one of two ways: the top of a benchmark leaderboard, or the model from whatever provider they already use — leaving a lot on the table
  • Cost, latency, robustness to future model changes, and your actual use case aren’t reflected in a single ranking
  • MTEB is a starting point, not an answer: find the subtask closest to your use case, restrict by model size, and watch for small models that punch above their weight (this talk covers dense embedders — one vector per chunk)

What MTEB doesn’t tell you

  • Quantization depends on where you run: on CPU, int8 keeps most of the precision at a fraction of the cost and runs much faster; on GPU, an int8 model runs at its native rate and is actually slower — run FP16 instead, basically the same result for significantly less
  • Vector precision: storing vectors as bf16 instead of float32 is a no-brainer (no measurable loss); bit vectors (packed to int8) do cost quality — but in hybrid search the gap shrinks and they become viable
  • Matryoshka dimensions: some models are trained so the earliest dimensions matter most, so you can just cut the vector — 2048 → 1024 dims cost nothing, 512 a little, 256 more; and 2048-dim bit vectors run a quarter the size of bf16 at 512 dims
  • Query latency comes from two places: how fast the embedder turns the query into a vector, and the distance math — normalized vectors let you use dot product instead of cosine, bit vectors unlock Hamming distance (fastest, and CPU-optimized in modern search engines)
  • Constraints MTEB never covers: multilingual needs, long-context support

Tune on your own data with Vespa Embed

  • Open-source fine-tuning tool (UI over sentence-transformers, models from Hugging Face): feed pairs (query → document) or triplets when you have negatives; it auto-splits a validation set
  • MNR (multiple negatives ranking) treats other documents in the batch as negatives; symmetric and JSD variants check whether a batch-picked document is actually a negative
  • Start with defaults; if you have labeled negatives, try triplets — his e-commerce results were similar either way, but real hard negatives (from search logs: positives rank on top, negatives below) are worth mining
  • No clean labels? Use an LLM as a judge: have it rank query→document pairs on a 0/1/2 scale, give it examples (the most important part), and run micro-batches of 8-10 docs so it doesn’t assume listwise context
  • Fine-tune embeddings far more readily than LLMs — it’s a constrained problem: cheap, quick, and consistently a big NDCG jump

The cheap-embedder escape hatch: re-rankers

  • Two-phase search: an intentionally “great but not greatest” cheap embedder does first-phase ranking over millions of docs, then a more expensive re-ranker (float vectors, cross-encoders, late interaction) scores only the top N
  • Store cheap bit vectors (with HNSW) in memory for the first phase and keep float vectors on disk for the re-rank — memory is expensive, disks aren’t
  • Late-interaction re-ranking (MaxSim) is literally: per-token dot products, keep the max per patch, sum across the document

Do’s and don’ts from real experiments

  • Don’t train on title + description as a proxy for queries — proxies lie; people’s real queries look nothing like product copy. Train on real queries and relevance judgments
  • NDCG has a blind spot: change your relevance function and newly-surfaced documents score zero until they’re rated — if you don’t re-rate everything in the metric, your NDCG looks like it never improves
  • Pick the metric by the problem: e-commerce ranking → NDCG/ERR (top results matter); RAG → precision matters more, because junk retrieved = hallucination, and recall is capped by context size
  • Don’t over-optimize: with ~2M documents most of these knobs don’t matter yet — start with defaults that lose little, then measure

“The embedding is a much more constrained problem — and it’s a lot easier to fine-tune. I always find a really good benefit from doing it. And it’s not that complicated.” — Hamel Husain