Aamir Shakir of Mixedbread presents a 16-minute retrieval talk on AI Engineer, with a financial-document search demo and a customer case study.

An agent can spend its budget finding the evidence

  • Financial questions span PDFs, earnings decks, interviews, audio, and tables whose meaning depends on their layout.
  • Keyword search can return many mentions while leaving the agent to work out which period, company, or metric actually answers the question.
  • Repeated searches and document reads consume tokens; adding more subagents can multiply that work.

Where the retrieval pipeline loses information

  • RAG, answering from documents retrieved at question time, often chains parsing, OCR, chunking, embeddings, a vector database, and reranking.
  • OCR converts page images to text; errors in tables and diagrams can discard relationships before search begins.
  • Chunking splits documents into pieces, potentially separating a value from the heading or context that explains it.
  • One embedding, a numerical representation used for matching, can blur fine differences between similar financial reports.

Search the page as well as the text

  • Mixedbread uses late interaction: multiple contextualized embeddings represent a document and a query before their details are matched.
  • Visual page representations preserve information from figures and layout; downstream agents can receive screenshots or extracted text.
  • The Intel demo retrieves a page for a foundry-segment question and supports source regions through bounding boxes.
  • A more qualitative profitability question invokes agentic search, a model making several searches to gather supporting evidence.

Read the customer result at its actual scope

  • Shakir reports an unnamed hedge fund’s human-graded answer accuracy rose from 36% to 98% after changing its retrieval stack.
  • He also reports roughly threefold lower LLM spend as the agent spends fewer tokens finding and reading evidence.
  • These are vendor-reported customer results; the talk supplies no question count, evaluation protocol, or independent reproduction.
  • The onstage demo suffers network delays; Shakir’s usual sub-200-millisecond search claim is separate from the observed demonstration.

“The real real bottleneck is actually access to your data and to your knowledge.” — Aamir Shakir, 8:35