The Limits of LLM-Generated SQL

An LLM writes SQL the way a strong analyst writes a first draft: fast, mostly right, and confidently wrong in ways that are hard to see. That is not a defect you can prompt your way out of. When people say LLM-generated SQL does not work at enterprise scale, they usually describe a model problem. It is closer to an architecture problem. The model is being asked to hold authority it cannot hold, over a search space nobody has bounded.

The distinction matters because it determines where you spend your next engineering quarter. If the limit is model intelligence, you wait, upgrade, and hope the next release clears the bar. If the limit is unbounded authorization, you can fix it this quarter — by moving the constraints out of the prompt and into the layers around the model.

Here is the position this piece argues: a language model is a competent SQL drafter and a structurally unreliable SQL authority. The failure is not that it writes bad SQL. It is that nothing in the model knows which of five similarly named tables is canonical, what your company means by an active customer, or whether the person asking is allowed to see the rows. Those are not intelligence problems. They are governance problems wearing a machine-learning costume.

Where text-to-SQL accuracy actually goes

Public benchmarks have made the field look nearly solved for years. Execution accuracy above 90% on tidy academic datasets has been the headline since 2023, and those numbers get quoted in procurement decks as if they describe the reader's warehouse. They do not, and the research is unusually blunt about how far off they are.

The clearest evidence comes from the BEAVER enterprise benchmark (2024), built from real private warehouse query logs rather than curated academic schemas. Its authors report that end-to-end execution accuracy on enterprise data is, on average, 64.4% lower than Spider, 27.6% lower than BIRD, and 13.3% lower than Fiben. The critical detail is why: that enterprise data was never on the public internet, so a model cannot lean on patterns it memorized during training.

The gap reproduces when you hold the model constant and change only the schema. Spider 2.0's enterprise results (Lei et al., ICLR 2025) report that a code-agent framework built on a leading reasoning model solved 21.3% of tasks, against 91.2% for the same framework on the academic Spider 1.0 set. Same model. Same framework. Different database. The schemas in the enterprise set average roughly 800 columns.

None of these numbers mean the model cannot write SQL. They mean accuracy is a function of the structure you hand it, and enterprise warehouses hand it the worst case.

Why scale breaks generation

Four mechanisms compound, and only the first is about the model's raw capability.

The search space stops being a space and becomes a maze. With twelve tables and a documented foreign key graph, a model can reason over the plausible join paths. With two thousand tables, several of which are near-duplicates maintained by different teams for different eras of the business, the join path is no longer inferable from names. The ICDE 2024 evaluation of text-to-SQL failures on real-world data puts a number on this: roughly half a percentage point of execution accuracy lost for every additional ten columns. That is not noise. It is a slope, and it points down.

Ambiguity has no in-model resolution. Ask for revenue last quarter and the model must decide: gross or net, recognized or booked, inclusive of refunds, which fiscal calendar, which entity. A plausible choice gets made. It may even match a definition someone recorded in a wiki two years ago. Nothing in the model can prefer your definition over its own because nothing in the model knows your definition exists. This is where LLM-generated SQL fails most quietly — the query runs, returns a number, and the number is defensible in isolation and wrong in context.

Hallucinated objects survive until they hit a catalog. A model that does not have the live schema will invent a column named for what it should be called, not what it is called. Those queries have to fail loudly to be safe, and they only fail loudly if something binding-aware sits between generation and execution.

Non-determinism turns an error into a distribution. The same question against the same schema can produce different SQL on different runs, because sampling and prompt context both vary. A single successful demonstration therefore proves almost nothing about production behavior. This is the mechanism that makes per-query accuracy the wrong acceptance criterion: you are not shipping a query, you are shipping a process that will be invoked thousands of times by people who will not verify the output.

Natural language to SQL in the enterprise: constraints by enforcement location

The useful question is not which constraints to add. It is where each constraint can actually be enforced, because a constraint that lives in a prompt is a request, and a request can be ignored by the very component you are trying to control.

⚠️ Warning: Prompt-level guardrails are not access control. Instructions like "only query approved tables" are honored by a model that has no schema-wide privileges and bypassed by anything that gets a foothold in the retrieved context. If your only defense against unauthorized queries is a sentence in a system prompt, you do not have that defense.

Governed definitions, held in a semantic layer

The first constraint is meaning, and it has to be declared rather than inferred. A semantic layer for text-to-SQL is a governed description of your business concepts — measures, dimensions, entities, and the relationships between them — that the model reads instead of guessing. Revenue becomes a named object with a definition, an owner, and a grain. Active customer stops being a judgment call.

The 2026 GROUND paper on governed semantic definitions reports its strongest results when generation is constrained to approve definitions and every query is validated before execution, rather than supplied as loose context. The paper's framing is worth internalizing: a governed definition is not a hint to the model, it is a contract the query must satisfy. That shift from context to contract is what makes the constraint enforceable.

Live schema and value grounding

The second constraint is binding the model's vocabulary to reality. Grounding means the model retrieves the relevant subset of the live catalog — tables, columns, and ideally the literal values in low-cardinality filter columns — instead of reasoning from remembered conventions. Retrieving only the tables a user is authorized to touch also shrinks the search space and removes irrelevant join paths from consideration. Static schema dumps in a prompt decay; catalogs change, and a prompt pinned at build time starts lying the week a column is renamed.

Deterministic validation gates

The third constraint is procedural and needs no model at all. Generated SQL should pass an ordered set of cheap-to-expensive checks before anything touches the warehouse:

1. Parse the statement against your target dialect to confirm it is syntactically real.

2. Bind every table and column reference against the live catalog, and reject unknown identifiers.

3. Check policy — allowed objects, required predicates such as tenant or time filters, prohibited statements.

4. Dry-run the plan to estimate cost and catch obvious problems without executing.

5. Execute only what passed every gate, under a role that cannot do anything else.

The ordering matters because it fails fast and cheap. A parse check costs milliseconds; a full-table scan against a petabyte costs considerably more and shows up on an invoice.

Permissions and cost, enforced by the engine

Here the research delivers the most useful correction to how most teams talk about this problem. Semantic correctness does not produce access control. GROUND's finding that semantic correctness alone leaves row-level security violated reports that a condition with exact metric definitions but no access policy still leaked data, with row-level security violated on 35–47% of questions — while the fully governed condition, which enforces filters and row-level security in the execution path, reported zero violations across every model tested. The lesson generalizes past any single system: knowing what a metric means and being permitted to see the underlying rows are separate guarantees, and the second one belongs in the database, not in the generated query.

That is also how the enforcement-location question resolves itself. Row-level security, tenant scoping, masked columns, byte and runtime caps, and mandatory partition filters are all engine concerns. A unified SQL engine that runs the same semantics across batch and interactive workloads is where those guarantees become properties of the platform rather than expectations of the prompt — which is also what makes AI Gateway-style model routing and per-call usage visibility tractable, since model selection and cost accounting are infrastructure decisions, not instructions to a language model.

An evaluation harness that grades the system

The last constraint is measurement, and it has to score the pipeline rather than the prompt. A useful harness combines a set of golden questions with their approved answers, adversarial cases for schema drift and sensitive-column access, and policy checks that fail independently of numeric correctness. Two metrics, tracked separately, tell you whether you are ready to widen access: answer accuracy, and policy compliance. A system can look excellent on aggregates while quietly leaking rows.

What to delegate to the model and what to encode

The workable rule is the one that survives contact with a budget conversation: let the model propose, and let deterministic machinery dispose.

Delegate to the model the parts where being approximately right and fast is genuinely valuable — drafting transformations, scaffolding a migration from a legacy dialect, explaining an unfamiliar schema, generating candidate joins for a human to approve, writing documentation from catalog metadata. In each case a human or an automated check catches the error before it becomes a number on a dashboard.

Encode, do not generate, the parts where a plausible mistake is indistinguishable from a correct answer at the point of consumption — certified metric definitions, join paths between canonical entities, tenant and row-level filters, cost limits, and anything that determines who can see which rows. It is worth noticing what this list has in common. Every item on it is a decision someone in the organization has already made. The trick is that the decision exists in a document, a ticket, or a person's memory, and the model has no access to any of it unless you give the decisions a machine-readable home. This is exactly what a Semantic View is for: it puts business semantics between raw physical tables and the questions asked of them, so that a definition is stated once and every consumer — a BI tool, an analyst, or a model — reads the same one.

The constrained pattern, when it is wired end to end, looks unremarkable: a user asks a question, the system maps intent onto governed metrics and dimensions, generates SQL grounded in those definitions and the live catalog, validates the statement before execution, runs it under enforced permissions, and returns a result that can be traced back to the definition it used. Products that offer natural-language analytics grounded in governed metrics follow this shape precisely because the alternative — free-form generation with a retrieval hint — has no enforcement point.

The obvious objection

A fair counterargument: models are improving, context windows are growing, and reasoning-heavy agents are already better at multi-table work than the frameworks in that 91.2%-to-21.3% comparison. Why build an elaborate constraint layer around a capability that will be stronger in eighteen months?

The premise is correct, and it is worth conceding cleanly. Enterprise text-to-SQL accuracy will rise. It has risen before and the slope of improvement is real, not imagined.

The conclusion does not follow, for three reasons. First, the enterprise catalog grows at the same time — the ICDE slope is about schema width, and most large organizations are adding columns faster than models are adding capability per token. Second, the residual gap in GROUND's results is not a capability gap at all. It is a governance gap, and a model that understands metrics perfectly while holding no concept of who is asking will leak exactly as reliably next year as it does now. Capability improvements do not relocate the enforcement point. Third, and least comfortably: as generated queries get better, they get harder to audit. A query that fails obviously costs you a support ticket. A query that returns a defensible wrong number costs you a quarterly business review.

This is the part that does not improve on its own, no matter how good the model gets.

The practical consequence is a change of framing rather than a change of tools. Stop asking whether LLM-generated SQL is accurate enough to trust. Start asking which decisions you have delegated to a probabilistic system that were never decisions about prediction in the first place. Definitions, permissions, and cost limits were deterministic before a model touched them. There is no reason they should be probabilistic now.

For teams ready to move from argument to sequencing, an enterprise rollout sequence for natural-language analytics covers how to stage governed access, validation gates, and evaluation before opening the interface to a broader audience. The starting point is not a better prompt. Pick one metric that has caused an argument in the last quarter, write its definition down in a place both humans and machines can read, and wire enforcement to that definition instead of to the SQL text.