Steve Miller's Blog

A digital workshop for systems-minded tech commentary.

Bridgewater’s 100 Million Document Vault and Its Permission Problem: Building Enterprise AI Search That Respects Who’s Allowed to See What

Three professionals with laptops discuss work at a conference table in a modern glass-walled office.

Written by

in

Most enterprise AI pilots work fine until the corpus gets big and the access rules get messy. Then the search quietly breaks. Bridgewater Associates has now described, in public, how that happened to its own document platform. It is the clearest case study I have seen of why permission-aware RAG enterprise AI search is a retrieval-engineering problem first and a model problem second. If your team is wiring a chatbot into SharePoint, email, or a research archive, this is the part you cannot skip.

What Bridgewater Says It Built

At AI Engineer NYC 2026, Bridgewater’s Sam Green, team lead for Applied AI, described the Vault as “our repository of external and internal unstructured data.” By his account it holds “more than 100 million documents,” with thousands more added every day from 135 external sources, plus internal data from inside the firm. Each document is enriched with tags like country, concept, market, and the time period it discusses (session transcript).

Aaron Linsky, who runs product engineering at Bridgewater, listed five dimensions the team had to get right: recall, precision, filtering, entitlements, and scale. He said production traces showed “almost always there was some condition,” such as a date, a source, or a specific asset. That is where the team “ran into our first major performance issue.” He also put numbers on the two audiences. Humans generate about 100 queries an hour across roughly 300 users. Agents easily run “100-plus queries a second,” and agent swarms can reach thousands.

Why Permissions Break Vector Search

In a separate joint session with turbopuffer CTO Nikhil Benesch (“Can Your AI Analyst Keep a Secret?”), Linsky said that when Bridgewater first built this “without turbopuffer, we thought it worked fine.” Then, as documents piled into the Vault, “it fell over tremendously.” Benesch explained why. Internally, “every email, every meeting note, every report might have a completely different set of permissions attached to it.”

Benesch described two classic approaches, and each one fails differently:

  • Post-filtering: grab the nearest vectors first, then throw away the ones the user cannot see. It is fast, but a user with narrow access can end up with few or no results, even when a relevant document they are allowed to read exists.
  • Pre-filtering: narrow to everything the user may see, then compare vectors. Recall is good. But his example of a user with access to 50 million of 100 million documents means a huge sequential scan, and the vector index stops helping.

turbopuffer’s answer, which Benesch calls native filtering, combines the two. The index tracks which vector clusters contain at least one document a given user can access. It skips the clusters with none, then scans inside the nearest eligible ones. Benesch said clusters currently hold about 200 documents and that the size is still being tuned. Linsky summarized the outcome in his keynote: fast like post-filtering, “but with close to the same recall as pre-filtering.”

About Those Headline Numbers

You will see “450 million vectors” and “120 milliseconds” attached to this story. Those figures come from Benesch, the vendor’s CTO, not from a Bridgewater statement. In the joint talk he said that across 450 million vectors, Bridgewater’s 100 million documents “turn into 400 million vectors,” and that turbopuffer gets “120 millisecond searches” at P90 across those namespaces. Note that 400 million and 450 million appear in the same sentence. Treat both the vector count and the latency as turbopuffer’s reported results, not an independent benchmark. Benesch acknowledged that nested groups are a limitation where “things get tricky.”

How the Permissions Are Actually Wired

This is the part security teams should read twice. turbopuffer’s own permissions documentation says permissions “currently have to be implemented at the user-level with filters.” The product “doesn’t provide built-in mechanisms for row/document-level RBAC.” The recommended pattern is to store the user IDs or group IDs with read access on each document. At query time, you pull the caller’s identity and groups from your own auth layer and pass them in as a filter.

The docs flag a subtle failure mode, too. An empty permission array should mean no access. To make a document public, write an explicit flag like is_public: true rather than inferring openness from an empty list. That design choice is safer, and the docs say it is faster.

The lesson goes beyond any one vendor. The vector store enforces whatever filter you hand it. The truth about who may see what still lives in your identity system and your ingestion pipeline.

My Recommendations for Permission-Aware RAG in Enterprise AI Search

These are my opinions, not Bridgewater or turbopuffer guidance.

  1. Make entitlements part of ingestion. Copy ACLs, groups, and source licensing onto each chunk when you index it, and keep them in sync when permissions change. A stale ACL is a data leak waiting for a prompt.
  2. Default-deny. Follow the explicit public-flag pattern. Never let a missing or empty permission field mean “everyone.”
  3. Resolve identity at query time, server-side. The model should never pick the filter. Your backend attaches the caller’s user and group IDs after authentication.
  4. Benchmark with permissions turned on. Bridgewater built about 200 hand-labeled examples in under a week and measured precision at 10 rising from about 60% to the 90–95% range. Run your test set as different users, including narrow-access ones, or post-filtering failures will hide from you.
  5. Plan for agent load. Size for machine query rates, not human ones. Bridgewater’s own gap between human and agent traffic is, in Linsky’s words, “a few orders of magnitude.”
  6. Flatten or pre-compute nested groups. Even the vendor says nested groups get “tricky.” Expand memberships in your pipeline instead of hoping the search engine resolves them.

Here is a hypothetical to test your design. An analyst in a restricted deal team asks the assistant about a target company. The best-matching memo belongs to a different team. With naive post-filtering, the analyst gets nothing useful. With a missing ACL, they get a memo they should never see. Both outcomes are failures, and only one of them shows up in a latency dashboard.

The Bottom Line

Bridgewater’s account shows that permission-aware retrieval decides whether enterprise AI search is usable and safe at scale. The impressive 450-million-vector, 120 ms figures are turbopuffer’s numbers. The durable lesson is architectural: carry entitlements on every document, filter at query time from a trusted identity source, and test recall as real users with real restrictions.

Comments

One response to “Bridgewater’s 100 Million Document Vault and Its Permission Problem: Building Enterprise AI Search That Respects Who’s Allowed to See What”

  1. Jimmy Avatar

    The query-rate asymmetry is the detail most RAG writeups skip: humans poll at a hundred queries an hour while agent swarms run hundreds per second, and every one of those queries still has to cross the entitlement check before it can touch a document. That is why permission-aware retrieval is a retrieval-engineering problem first – the entitlement filter has to sit in the hot path, not as a post-hoc scrubbing step that catches leaks after the fact. The Bridgewater trace finding that production queries almost always carry some extra condition matches what anyone running agent workloads sees: an agent rarely asks the open question a human would, it asks the scoped one, and scoped queries are exactly where a naive vector index hides its permission gaps.

Leave a Reply

Your email address will not be published. Required fields are marked *