58 min read

Enterprise Vector Search Without a Separate Vector Database

A decision framework for enterprise vector search without a separate vector database: how native hybrid retrieval, filtering and row-level permissions behave in governed platforms, when splitting genuinely wins, and an acceptance checklist to test before you commit.

Enterprise Vector Search Without a Separate Vector Database

The assumption that enterprise retrieval-augmented generation needs a dedicated vector database is an architectural reflex, not an engineering requirement. It became a reflex for a good reason, and that reason has largely expired: the platforms that already hold your documents now ship vector, keyword and hybrid retrieval as native features. Once retrieval is available where the data lives, the question changes. It is no longer whether a separate store performs better on a benchmark, but what a second copy of your corpus costs you in governance terms.

That cost is not primarily storage. Every duplicated vector copy is a second security model that must be reconciled with the source system, a second freshness problem that must be monitored, and a second failure domain that must be failed over. When a document's permissions change in the system of record, the copy does not learn about it unless something tells it. When the embedding model is upgraded, the copy must be rebuilt. When the copy is unavailable, retrieval fails even though the source data is healthy.

This article is a decision framework, not a product pitch. It examines why the separate-store default took hold, what duplication actually costs, how hybrid retrieval and filtering behave inside governed platforms, and the narrow conditions under which splitting genuinely wins. It closes with an acceptance checklist you can run against your own corpus before committing either way.

The Separate Vector Store Is a Default, Not a Decision

The conventional wisdom formed when it was correct. General-purpose data platforms did not offer approximate nearest-neighbor indexes, so any team that wanted semantic retrieval had to stand up a specialized engine. Best-of-breed thinking did the rest: if retrieval is a distinct capability, it deserves a distinct system, owned by a distinct team, with its own scaling curve and its own release cadence.

That premise has shifted. The major governed platforms now ship retrieval natively. Databricks AI Search combines vector similarity with BM25 keyword search and hybrid ranking, and its indexes appear in and are governed by Unity Catalog. Snowflake's Cortex Search provides a hybrid vector-and-keyword engine created directly over a source query with CREATE CORTEX SEARCH SERVICE. BigQuery vector search supports index-backed and brute-force similarity over tables already in the warehouse. And PostgreSQL, which many enterprises already run, gains vector search through pgvector.

The question is therefore no longer whether the platform can do retrieval. In most cases it can. The question is what a second copy of the data costs in governance terms — and whether the retrieval quality difference, if any, is large enough to justify paying that cost. That reframing matters because the two costs are not symmetric. Retrieval quality differences are measurable and usually small; governance gaps are structural and tend to surface during audits, incidents or permission reviews rather than during a benchmark.

The Real Cost of a Duplicated Vector Copy Is Governance, Not Storage

Storage for embeddings is cheap and getting cheaper. The expense that compounds is the second permission model. A separate vector store does not inherit your source system's access controls; it has its own, and someone must keep the two in agreement as documents are added, reclassified, moved between departments and deleted.

The OWASP RAG Security Cheat Sheet treats this as the most common compliance failure in enterprise RAG deployments. Its two illustrative failure patterns are worth reading literally: a classified document is chunked and stored in a shared vector store without per-chunk access control metadata, so an unauthorized user's query retrieves a chunk of it; and document-level permissions are checked at ingestion but not at retrieval time, so later permission changes are ignored. OWASP's recommended control is access control metadata on every vector chunk plus enforcement before content reaches the model — not enforcement by the language model.

Sync pipelines add a second class of failure. Every pipeline is a component that can lag, retry, partially fail or silently drop a deletion. OWASP explicitly warns against assuming that removing a document from the ingestion source removes it from the RAG pipeline; deletion must be explicitly propagated. A duplicated store also adds failover surface: one more system to monitor, page on and rehearse recovery for. And it creates an audit gap, because retrieval decisions now happen outside the system of record that your lineage and access logs describe.

Native retrieval is not automatically free of these subtleties, and buyers should verify rather than assume. Snowflake documents that a Cortex Search service queried with owner's rights can return results from rows the querying user's role cannot read in the source table — for example, when the source carries row-level masking policies. That is a permission-model property you must design around, not a reason to prefer a separate store. The point is that "governed" is a claim to test in every architecture, and testing it in one system is materially easier than testing it across two.

Hybrid Retrieval Is One Workflow, and Filtering Is the Hard Part

Hybrid retrieval composes two ranked lists — one from keyword scoring such as BM25, one from vector similarity — into a single result. Reciprocal rank fusion is the common mechanism. Cormack, Clarke and Büttcher (SIGIR 2009) introduced RRF as a simple, untrained combiner: each document's score is the sum over result lists of 1/(k + rank), with k fixed at 60 in their experiments. They reported that RRF outperformed Condorcet Fuse, CombMNZ and the best individual system by roughly 4% to 5% on average across their pilot and TREC experiments; in a separate LETOR 3 experiment, RRF bettered all individual rankings, the best by about 4%, while CombMNZ edged RRF slightly.

The appeal in practice is that ranks are comparable even when scores are not. As OpenSearch's documentation puts it, a BM25 score of 12.4 and a vector similarity of 0.87 measure different quantities on different scales, but a rank of 1 identifies the top result in both lists. The trade-off is documented too: OpenSearch reports that across benchmarks on six BEIR datasets, RRF averaged 3.86% lower NDCG@10 than its score-based hybrid pipeline, with comparable latency and coordinating-node CPU. That is a published benchmark on those datasets, not a universal result — but it is a useful prior that the RRF-versus-score-based gap was a few percent in that comparison, not an order of magnitude.

Filtering is where the architecture decision is actually made. Approximate indexes search a graph or a set of lists; a filter restricts which rows are eligible. Whether the filter is applied before or after the index scan changes both recall and cost, and the behavior differs by engine and by how you write the query.

Azure AI Search makes the distinction explicit in its vector filter documentation. preFilter is documented as very high recall with no risk of false negatives and is the recommended default, especially for selective filters or small k. postFilter has medium-to-high recall that decreases with filter selectivity and a moderate risk of missing matches per shard. strictPostFilter has the lowest recall, decreasing most quickly with selectivity, and can return zero results for selective filters or small k.

Elasticsearch draws the same line at the query-syntax level. In its filtered kNN documentation, a filter inside the knn clause is applied during the approximate search so that k matching documents are returned; a filter placed as a sibling of the knn query in a bool clause is applied after the vector search and can return far fewer than k hits, or none, even when the index holds plenty of matching documents.

pgvector states the consequence plainly in its README: with approximate indexes, filtering is applied after the index is scanned. If a condition matches 10% of rows, with HNSW and the default hnsw.ef_search of 40, only about 4 rows will match on average. Iterative index scans address this by scanning more of the index when needed.

This is why filtering, not embedding, is the acceptance test that matters. Row-level and metadata filters are exactly where a separate store diverges from the platform's own permission model — and where a native implementation can either inherit the platform's semantics or quietly fail to.

What to Keep in One System and What to Split

The decision reduces to three questions, asked in order. First, is the corpus already in a governed platform that offers native retrieval? Second, must retrieval honor existing row- and column-level permissions at query time? Third, does the platform's sync cadence meet your freshness requirement? If the answers are yes, yes and yes, the default should be to keep retrieval where the data lives. Splitting requires a named constraint that survives measurement, not a general preference for specialized tools.

Dimension

Native retrieval in the governed platform

Separate vector store

Latency

Bounded by platform serving tier; verify with your own query mix

Tunable, but adds a network hop and a second serving tier to size

Recall

Depends on index type and filter mode; test pre- vs post-filter

Depends on index type and filter mode; same test applies

Scaling

Constrained by documented platform limits (endpoints, QPS, index size)

Scales independently, at the cost of a second capacity plan

Filtering

Can inherit platform permission semantics if the engine pre-filters

Must be reimplemented and kept in sync with source ACLs

Freshness

Governed by platform sync; lag is documented and observable

Governed by your pipeline; lag is yours to monitor and page on

Governance

One access model, one lineage graph, one audit trail

Two of each, reconciled by process

The table is a prompt for measurement, not a verdict. The scaling row is where splitting most often earns its place, and it should be quantified rather than assumed. Snowflake, for instance, documents that a single Cortex Search service supports up to 20 queries per second and 140 QPS across all services in an account before you must contact support to raise throughput. If your peak is 15 QPS, that limit is irrelevant; if it is 400 QPS with a tight latency SLA, it is a real constraint and you should say so explicitly in the design document.

Split when, and only when, one of these is true and measured: sustained query throughput beyond the platform's documented or negotiated ceiling; an index type or quantization scheme the platform does not expose and that your recall targets require; an embedding model the platform cannot host and whose quality difference you have demonstrated on your own judgment set; or an organizational mandate that a dedicated search team own retrieval as a separate service with its own SLOs. Each of these is falsifiable. "We prefer best-of-breed" is not.

Where Native Retrieval Breaks: Failure Modes and an Acceptance Checklist

Native retrieval fails in predictable ways, and most of them are configuration and governance failures rather than engine limitations.

Silent under-return under selective filters. A post-filter that matches 2% of the corpus can return three results when you asked for ten, and the application may render those three as if they were the best available. Elasticsearch and Azure both document this behavior; the fix is pre-filtering or iterative scanning, and the test is a filter with known selectivity and a known expected result count.

Permission drift. Source ACLs change; the index does not. OWASP's guidance is to carry access control metadata on every chunk and enforce it at retrieval time, and to propagate deletions explicitly rather than assuming they follow the source.

Embedding-model changes that invalidate the index. Swapping models without re-embedding leaves queries and documents in different vector spaces. OWASP recommends logging the embedding model version per document and flagging documents whose relative positions shift significantly after re-embedding.

Assuming "governed" means row-level security at query time. Databricks documents this limitation directly: AI Search does not support row- and column-level permissions, and you can implement your own application-level ACLs using the filter API. That is a workable design, but it is application-level enforcement, and it belongs in your threat model rather than in an assumption.

Before committing, run this checklist against a representative corpus and a judgment set of queries with known relevant documents:

  1. Filter selectivity. Query with filters matching roughly 50%, 10% and 1% of the corpus. Confirm you receive k results where k results exist, and record latency at each selectivity.

  2. Permission-change propagation. Revoke a user's access to a document in the source, then query as that user. Measure how long until the document stops appearing, and confirm the answer is zero for your compliance requirement.

  3. Deletion propagation. Delete a source document and confirm it disappears from retrieval results within the same window.

  4. Freshness lag. Insert or update documents and measure end-to-end time to retrievability against your stated requirement.

  5. Recall against a judgment set. Compute recall@k for hybrid, keyword-only and vector-only configurations on queries you have labeled. Compare against any candidate separate store using the same set.

  6. Fail-closed behavior. Simulate an index or embedding-service failure and confirm the application denies retrieval rather than returning unfiltered results.

  7. Model-change rehearsal. Re-embed a sample with the intended new model and measure ranking movement before scheduling a full rebuild.

The APAC Multi-Cloud and Residency Dimension

Data-residency requirements change the calculus because a separate vector store is another jurisdiction to reason about. Three locations matter independently: where the embeddings are stored, where the embedding model runs at ingestion and query time, and whether inference requests may cross regional or cloud boundaries. A design that keeps documents in-country but sends text to a model endpoint in another region has not satisfied a residency requirement, even if the vector index itself sits locally.

Platform-native retrieval inherits the platform's regional availability, which can be an argument for consolidation — but only if you verify the specific region and model. Snowflake publishes per-model, per-region availability for Cortex Search, and its cross-region inference documentation makes routing an explicit parameter: ANY_REGION allows processing in any supported region across any cloud provider; regional values such as AWS_APJ keep requests within designated cloud provider regions and geography; and DISABLED restricts processing to the account's home region for strict sovereignty requirements. Snowflake also recommends testing your specific use case with cross-region inference enabled rather than assuming behavior.

Residency documentation varies by provider, and no universal APAC rule applies. Treat these as questions to resolve in writing with each candidate platform: Which regions offer the embedding model you intend to use, and is that list current? Is cross-region inference on by default, and what is the exact parameter that disables it? Where are embeddings stored at rest, and are they treated as derived data or as a copy subject to the same residency rules as the source? What is the deletion path for embeddings when a source record is removed? If a required model is unavailable in your region, is the fallback a different model, a different region, or a separate store — and which of those is acceptable to your regulator?

The Strongest Case for Splitting — and Why It Usually Still Loses

The best argument for a dedicated vector store is not that it retrieves better. It is that it can be engineered, scaled and owned independently of everything else. Three versions of that argument are genuinely strong.

First, peak throughput and latency SLAs. A shared platform serves analytics, ETL and retrieval from the same governance and capacity envelope. If retrieval must sustain thousands of queries per second at a fixed p99, a system designed for that workload — with its own autoscaling, replica management and capacity isolation — can be the honest answer. Databricks markets its AI Search as serverless and scaling to thousands of queries per second, which illustrates that native options are improving here, but the general point stands: verify against your own peak, not a vendor's headline.

Second, index algorithms and quantization the platform does not expose. If your recall targets depend on a specific HNSW parameterization, a particular quantization scheme, or a specialized index type, and the platform does not let you set it, that is a concrete gap. pgvector, for example, documents quantization as the route to scaling large vector volumes, and the specific options available differ by engine.

Third, organizational separation. A search or relevance team that owns retrieval as a product, with its own evaluation harness, release cadence and on-call rotation, may legitimately run its own service. That is a real operating model, not a technical failing.

The response is not to dismiss these. It is to price them. Each can often be met inside the platform: serverless and storage-optimized endpoints address capacity, quantization and index tuning are increasingly exposed as configuration, and a platform team can own retrieval as a governed internal service with its own SLOs. Splitting genuinely wins when the constraint is measured and the platform cannot meet it after reasonable configuration — and when the team accepting the second governance model has the staffing to run it. Splitting loses when it is chosen to avoid a conversation about limits that turn out not to bind.

Conclusion: Govern the Retrieval, Not the Index

Default to retrieval inside the governed platform that owns the data. Split only against a named, measured constraint — throughput beyond a documented ceiling, an index or model capability the platform genuinely lacks, or an explicit organizational mandate — and write that constraint down before you write the architecture diagram.

Whichever path you choose, validate filtering and permission propagation before committing. The failure modes that matter are not ranking differences of a few percent; they are a revoked user retrieving a document they should not see, a deleted record that lives on in an index, and a filter that silently returns three results when the application expects ten. Those are governance failures, and they are cheaper to prevent in one system than to reconcile across two.

Run the acceptance checklist against your own corpus and judgment set. If you are evaluating consolidated platforms, Singdata's lakehouse is one candidate where AI functions and vector search sit alongside data processing in a single platform, which makes it a reasonable inclusion in a shortlist — evaluated on the same checklist, with the same skepticism, as any other option.

FAQ

Is a separate vector database ever required?

Sometimes, but rarely by default. The defensible cases are measured throughput or latency beyond what the governed platform can provide, an index type or embedding model the platform cannot host, or an organizational mandate for a separately owned retrieval service. If none of those applies after testing, a separate store adds a governance burden without a corresponding benefit.

How do I test whether native filtering honors my row-level permissions?

Create two users with different access to the same document. Query as the restricted user and confirm the document does not appear. Then revoke access for a user who could previously see it, and measure the time until it stops appearing. Repeat for deletion. Record both windows against your compliance requirement — step 2 and step 3 of the checklist.

Does hybrid search inside a warehouse match a dedicated engine on recall?

It depends on the engine, the fusion method and your data. OpenSearch's published benchmark found RRF averaged 3.86% lower NDCG@10 than score-based fusion across six BEIR datasets, which suggests fusion choice moves relevance by a few percent rather than dramatically. The only reliable answer is recall@k on your own labeled queries, run against both candidates.

What happens to my index when I change embedding models?

Queries and documents end up in different vector spaces unless everything is re-embedded. Log the model version per document, re-embed a sample first, measure ranking movement, and only then schedule a full rebuild. OWASP recommends flagging documents whose relative positions shift significantly after re-embedding.

How do I handle data residency if my platform's region lacks a model?

Resolve it explicitly rather than by default. Check the provider's per-region model availability, confirm whether cross-region inference is enabled and how to disable it, and decide in advance whether the acceptable fallback is a different model, a different region, or a separate store. Get the answer in writing from the provider for your specific region.

See Also

Exploring AI-Driven Layered Structures in Global Supply Chains

Transforming Loyalty Programs with Engaging Gamification Strategies

Understanding OLAP Cubes: Their Importance and Functionality

Comprehending the Fundamentals of Cloud Data Structures

An In-Depth Exploration of Big Data Architecture Elements