Bridgewater’s Aaron Linsky and turbopuffer CTO Nikhil Benesch explain permission-aware retrieval in this 18-minute AI Engineer talk.
Search the documents this user may see
- Bridgewater’s Vault combines about 100 million documents from external research and internal writing, meetings and reports.
- Each document can carry user or group permissions. Licensing can restrict external research within the same organization.
- An AI analyst needs those access rules carried into retrieval, the process of finding evidence for an answer.
Two simple filtering strategies have different weaknesses
- Search, then filter: retrieve similar documents and discard unauthorized ones; relevant allowed documents may never enter the candidate set.
- Filter, then scan: find all allowed documents first, then compare them with the query; broad access can leave millions to scan.
- The first approach can lose relevant results. The second can become too slow at a large scale.
Combine page-level permissions with a clustered index
- Vector search represents meaning numerically and groups nearby documents to narrow the search.
- turbopuffer’s “native filtering” skips clusters containing no allowed documents, then searches promising clusters using the permission filters.
- Access to one document in a cluster does not grant access to its neighbors; document-level restrictions still apply.
- Benesch reports roughly 120 milliseconds at the 90th percentile across Bridgewater’s namespaces, covering hundreds of millions of vectors.
Keep the rest of the security model explicit
- Applications still need to translate their user and group rules correctly into search filters.
- Benesch acknowledges that nested groups introduce complications and leaves their handling largely unexplained.
- Caveat: the latency is a customer/vendor report, with no independent recall study or end-to-end security audit presented.
“The trick is to combine pre- and post-filtering.” — Nikhil Benesch, 15:15–15:17