Vinoo Ganesh, Kepler’s CEO and a former Citadel business-engineering leader, describes source-backed financial answers in this 18-minute AI Engineer talk.
Separate three ways numbers go wrong
- Retrieval can select the wrong document or the wrong source for the firm’s question.
- Extraction can select the wrong number inside the right document, such as the wrong reporting year.
- Arithmetic can fail when the model generates an answer instead of calling calculation code.
Make facts addressable
- Kepler preprocesses documents into structured facts with identifiers before answering questions.
- The model plans which facts to use; code resolves their identifiers and performs calculations.
- Clickable number chips expose the underlying evidence to the reader.
- A final verification step checks the output and supports verified, corrected, or refused results.
Search broadly within enforced access rules
- Combine approximate-neighbor search with BM25 keyword retrieval, then fuse rankings and apply a cross-encoder reranker.
- Ganesh describes up to 200 initial candidates, narrowed to roughly 60 before the next ranking stage.
- Enforce tenant identity and document access on direct reads as well as search.
- Handle stale data and unpopulated indexes explicitly, alongside cross-firm data-leak risks.
Verify the meaning as well as the lookup
- A valid identifier proves that a stored fact exists; its extraction, date, units, and relevance still need checking.
- Ganesh later acknowledges that models can still generate unsupported numbers and need post-processing.
- His production slide describes fewer refusals, without a narrated rate, sample size, or independent accuracy comparison.
- Ask vendors to expose calculation sources and explain exactly what happens when a number has no supporting fact.
“You can’t just tack verification on at the end.” — Vinoo Ganesh, 1:18