37 min read

Build a Vector Index and Row-Level Security Stops Meaning What It Meant

Approximate nearest-neighbor retrieval moves tenant enforcement from query time to index-build time. Why HNSW graphs cross tenant boundaries, and how to falsify isolation with a cross-tenant probe.

Build a Vector Index and Row-Level Security Stops Meaning What It Meant

A smaller tenant's query could search graphs containing a much larger tenant's vectors. That sentence, from Abnormal AI's engineering account of running vector search at billion scale, describes a working system with a correct permission filter. The filter returned the right answer. The search had already looked at the wrong one.

This is the inversion that approximate nearest-neighbor retrieval introduces. Row-level security, as most teams implement and review it, is a query-time construct: a predicate evaluated against rows, bounding what a read can touch. An HNSW graph is built at index time over whatever vectors share a shard, and traversal happens before any predicate is applied. Permission enforcement has quietly moved from query time to index-build time, and most permission models were never built for that move.

The evidence below comes from one production deployment, described by Jeremy Ryu, Software Engineer on the Model Platform team at Abnormal AI, in "Building vector search at billion scale" (4 November 2026). It is a single vendor, a single deployment, with no third-party reproduction. The generalization to your stack is this article's argument, not the vendor's conclusion.

A Graph Index Is Not Row-Addressable

Row-level security works because rows are addressable. A predicate on tenant_id can be pushed into the storage layer, evaluated against an index of values, and used to bound the read before any data page is fetched. The guarantee is structural: the engine knows which rows exist and which of them the caller may see, and it never materializes the rest.

An HNSW graph has no such structure. It is a navigable proximity structure — layers of neighbor links built by inserting vectors one at a time and connecting each to its nearest existing neighbors. There is no column of tenant identifiers to index, no sorted order to binary-search, and no way to ask the graph "give me the nearest vectors belonging to tenant A" without walking it. The graph is constructed from whatever vectors occupy the shard's segments, and the shard's segments are determined by the write path, not the query path.

Abnormal AI's post states the mechanism directly: "OpenSearch's approximate kNN search uses Hierarchical Navigable Small World (HNSW) graphs that can consume substantial native memory. These graphs are built over all of a shard's segments, even when a query restricts results to one tenant. A smaller tenant's query could therefore search graphs containing a much larger tenant's vectors."

The causal chain is short. Vectors from multiple tenants land in the same shard because the shard layout was chosen for ingestion throughput, not for tenancy. The graph is built over that shard's segments at index time. A query from any tenant traverses that graph, reaching candidate vectors from every tenant represented in it. The WHERE tenant_id = ? clause then filters the candidate set. The predicate still filters the output. It no longer describes what the search touched.

Where the permission model assumed query time

Most teams treat the filter as the enforcement point because, in a B-tree or a row store, the predicate genuinely bounds the read. The mental model transfers without friction: the database evaluates the policy, the policy restricts the rows, the caller sees only permitted rows. A permission review inspects the policy configuration, confirms the predicate is present on every query path, and passes.

In an approximate index, the candidate set is produced by graph traversal and the filter is applied to that set afterward. The policy is still correct. The review still passes. What changed is when it takes effect — and the review has no instrument that would detect the change, because it inspects configuration rather than traversal. OpenSearch's own documentation describes several filtering modes with materially different behavior, from post-filtering applied after the approximate search to efficient filtering that restricts candidates during traversal (OpenSearch filtering documentation). Which mode your query uses is a configuration detail. Whether other tenants' vectors are in the graph at all is a layout fact, and no filter setting changes it.

The Failure Mode Is Memory, Not Leakage

The Abnormal AI account does not describe a leak. It describes an outage. During testing, native memory pressure "triggered OpenSearch's kNN circuit breaker. Data nodes also stopped responding to cluster-manager heartbeats and were removed from the cluster, blocking ingestion." The author states they "did not identify customer-visible impact from these failures."

The causal chain runs through memory, not through disclosure. A graph spanning multiple tenants consumes native memory proportional to everything in the shard, not to what any one tenant queries. A large tenant's vectors raise the memory floor for a small tenant's searches, because the small tenant's query traverses the same graph. When the aggregate exceeds the circuit breaker's limit, the breaker trips and search degrades; when nodes stop answering heartbeats, they leave the cluster and writes are blocked. The first symptom is availability, which is why the isolation question surfaces late — and often not at all.

This is the part of the story most teams will recognize. Shards grow, searches hit memory limits, the cluster starts rejecting writes. The remediation path is familiar: add memory, add shards, cap concurrency. Service returns. What does not return is an answer to the question the incident never asked.

Why the breaker trips before anyone audits the filter

Capacity incidents get escalated and fixed. The fix is usually more memory per node or fewer concurrent searches, both of which restore service without explaining why the graph was that large. Nothing in the incident review asks which tenants' vectors shared a graph, because the incident was framed as a resource problem and resource problems have resource solutions.

The isolation defect stays in place, now with more headroom above it. The next capacity incident arrives later and larger. Each remediation buys time without changing the topology that produced the pressure. Abnormal AI's own mitigations, examined next, are instructive precisely because the effective ones changed topology rather than adding capacity.

The Mitigations That Worked Were Layout Changes

Abnormal AI lists four mitigations. Their shape is the tell.

Partitioned tenant routing distributes large tenants' data across multiple shards. This changes who shares a graph. It is the only one of the four that touches isolation, and it does so by changing topology, not policy.

Capping concurrent search requests per service instance changes resource pressure. It bounds how much memory concurrent traversals can demand, which keeps the breaker from tripping. It does not change which vectors a traversal reaches.

Memory-optimized search, which accesses vector index files through memory mapping and the operating system's page cache, changes resource pressure again. It reduces reliance on loading entire graphs into native memory, though cache misses introduce disk I/O.

Compressed candidate retrieval followed by full-precision rescoring changes ranking quality. Candidates are retrieved using compressed vector representations, then rescored with full-precision vectors.

Three of the four are resource and ranking fixes. One is a layout fix. If tenant isolation were a policy property, you would expect the remediation list to include policy changes — tightened predicates, an additional enforcement layer, a validation gate. It does not, because policy was never the failing component.

Rescoring cannot recover what retrieval missed

The post supplies the sentence that should end the debate about post-filtering: "Rescoring improves candidate ranking but cannot recover messages missed during approximate retrieval."

Read it as a statement about the retrieval pipeline. If a candidate never entered the set, no downstream stage can rank it, filter it, or audit it. Post-filtering shrinks what the caller sees. It does not change what the search traversed, and it cannot restore a candidate that approximate retrieval skipped. The same logic applies to tenant isolation: a filter applied after traversal reduces exposure but does not undo the traversal. The graph was walked. The vectors were compared. The boundary was crossed inside the index, and the filter is downstream of that fact.

What Isolation Costs

The opposing case deserves a fair hearing, because the recommendation is not free.

Partitioned routing gives up cross-tenant recall. The shared store existed in the first place because similarity search across a large corpus is the point — finding emails similar to known threats across a customer's history, in Abnormal AI's case, and finding similar items across a catalog in most others. Partition by tenant and you lose the ability to ask corpus-wide questions without a fan-out that reassembles results across shards. For some workloads that is a real capability loss, not a rounding error.

It also adds operational complexity. A tenant that outgrows its shard boundary needs re-partitioning, and re-partitioning a billion-vector index is not a maintenance-window task. It means rebuilding graphs, which means re-embedding or re-inserting at scale, which means a migration plan with its own capacity ceiling. Over-isolating degrades the very thing vector search is for.

Then there is the cost of proving the recommendation. Verifying isolation requires a probe harness that most teams do not have. The probe has to run against production-shaped data — synthetic tenants with realistic vector distributions and realistic size skew — which means someone has to own those synthetic tenants and keep them current as the real corpus evolves. That is ongoing work with no feature attached to it, which is exactly the kind of work that gets deferred.

State the cost honestly: partitioned routing trades recall and operational simplicity for isolation, and proving isolation trades engineering time for evidence. Teams should decide with both costs on the table, not with only the risk on one side.

Falsify Isolation Before You Trust It

For security leads, data platform leads, and the engineers who own retrieval services, the recommendation is narrow and testable: before wiring a vector index into multi-tenant queries, falsify isolation with a cross-tenant probe rather than reading the permission documentation. The documentation describes policy. The probe describes traversal.

The two-week action is concrete. In a test environment, take tenant A's query and plant a known string belonging to tenant B as a candidate. Change no policy. Check whether B's content appears anywhere in the candidate set, even if a later filter removes it from the response. If it appears, the graph crossed the boundary and the filter is compensating, not enforcing. In the same exercise, record the memory threshold at which the circuit breaker trips and the shard size at which that happens, and treat that threshold as a capacity ceiling rather than a tuning parameter to raise.

This matters because the failure is invisible to the controls most organizations already run. The permission review passes, because it inspects policy configuration and the policy is correct. The exposure sits inside the index, where no policy document describes it. After an incident, the logs cannot show who retrieved whose vectors, because the retrieval happened during graph traversal and the filter ran afterward — the record shows what was returned, not what was touched.

Where permissions, detail rows, and index layout share one set of table and partition definitions, the tenant boundary can be expressed once at the storage layer instead of being compensated for repeatedly at the retrieval layer. That is the architectural property Singdata builds toward, and it comes with an honest caveat: it only holds if the layout comes first, and changing layout means rebuilding the index. Any vendor performance or cost figure cited in that context should be treated as vendor-stated, not independently verified.

The landing is the same for every team. Isolation in a vector store is a property of shard layout, routing, and index topology. Verify it there, with a probe, before you trust a filter to do work it was never positioned to do.

FAQ

Can't a metadata filter after the search fix tenant isolation?

No. The filter decides what the caller receives, not what the graph traversal touched. Once other tenants' vectors are in the graph, the candidate set has already crossed the boundary. The filter reduces exposure — fewer foreign results reach the response — but it does not restore isolation, because the search still compared the query against vectors it should never have reached.

Why did the cluster fall over instead of simply returning slow results?

Because the binding constraint was native memory, not query latency. Per the Abnormal AI account, memory pressure triggered OpenSearch's kNN circuit breaker, and data nodes stopped responding to cluster-manager heartbeats and were removed from the cluster, which blocked ingestion. Degradation surfaced as availability and write failure, not as slow search. The author states they did not identify customer-visible impact from these failures.

What should a recall and isolation SLO actually contain?

Two separate commitments, measured differently. Recall states the fraction of true nearest neighbors returned within the tenant's own data. Isolation states that no candidate from another tenant enters the candidate set at any stage — verified by probe against production-shaped data, not by policy review — and should be paired with a recorded memory ceiling at which the circuit breaker trips, so that capacity headroom is a documented limit rather than an assumption.

Previous posts

1. Medallion Architecture: What Changes Between Bronze, Silver, and Gold? — https://singdata.com/trending/medallion-bronze-silver-gold

2. How Much Data Preparation Is Hidden in Your BI Reports? — https://singdata.com/trending/hidden-data-prep-in-bi-reports

3. How to Migrate from ETL to ELT Without Breaking Reporting or Agentic AI Freshness — https://singdata.com/trending/2026-11-04-elt-file-first-ingestion