The Value of Formal Verification Is the Precondition, Not the Proof
A Lean formalization of the Stacks Project surfaced missing hypotheses and overstated claims, not faulty reasoning. The transferable lesson for CDOs: write metric preconditions before you compute.
The Stacks Project began in 2005 as Johan de Jong's attempt to write down the foundations of algebraic geometry in full detail. Two decades later it is a roughly 7,500-page open-source reference, built and cross-reviewed by hundreds of mathematicians, with an editorial discipline that requires every theorem to carry a proof, every assumption to be stated explicitly, and every step to point back to the tags it depends on. It is about as heavily reviewed as mathematical text gets.
When Axiom released a Lean formalization of the whole corpus on 4 November 2026, the first machine pass did not report that the proofs were wrong. It reported something narrower and more uncomfortable. The system, in Axiom's words, "surfaced dozens of statements in the Stacks Project that required local refinement, including missing hypotheses, edge cases in positive characteristic, and claims that did not hold in the full generality stated".
Read those three categories again. None of them is a reasoning error. A missing hypothesis is a sentence that left something out. An edge case in positive characteristic is a scope the statement never excluded. A claim that fails at full generality is a claim written more broadly than its proof supports. In each case the argument, where it existed, was fine. What was incomplete was the statement of what the argument was about.
The deliverable of formalization, then, is not primarily a proof. It is an explicit list of assumptions that must be written down before a proof can even be attempted. That artifact — a complete, checkable statement of preconditions — is precisely what enterprise data lacks, and it is the reason this mathematical result belongs in a conversation about metric definitions.
Missing hypotheses are invisible to human review because readers supply them silently
Human readers repair texts as they read them. A mathematician working through a lemma in algebraic geometry supplies the standing assumptions from the surrounding chapter, from the conventions of the subfield, from what the objects in play are normally taken to be. The repair happens below the level of conscious attention, which is exactly why it never shows up as a defect. Nothing fails, so nothing gets reported, so nothing gets fixed.
A machine has no repair mechanism. Lean's kernel, as Axiom describes it, "guarantees that every cited result is applied only when its exact hypotheses are satisfied." There is no silent accommodation available. An unstated assumption becomes a hard stop at the point of application, and the stop is attributable to a specific statement.
This is why the finding is notable given the corpus rather than despite it. The Stacks Project already demanded that assumptions be stated explicitly; that was the editorial principle from the beginning. What the formalization exposed is that explicit-statement discipline, applied by humans, still leaves gaps — because the reviewers doing the reviewing were also supplying the missing pieces silently, and a gap that everyone fills in identically is invisible to everyone.
Keep the scope of the claim tight. "Dozens" is Axiom's own wording, no full list was published, and the finding describes the Stacks Project corpus specifically. It is not a rate, not a proportion of the 7,500 pages, and not a statement about mathematics in general. The point is not that the text was sloppy. The point is that silent reader repair is structurally invisible to the people performing it, no matter how careful they are.
A counterexample is a better deliverable than a correction
A correction tells you what is true. A counterexample tells you where the original stopped being true — and that second piece of information is what you actually need, because it tells you whether your own use of the result was ever inside the valid region.
Axiom reports that rather than treating failed proof attempts as obstacles, its agents "diagnosed the underlying mathematical obstructions and constructed explicit counterexamples to prove where the original statements broke down." Where a false supporting lemma opened a gap in the proof of a deeper theorem, the agents found alternative routes through the literature to establish the main result under its original hypotheses. For local oversights, they identified the exact missing hypotheses and formalized the corrected theorems alongside their counterexamples.
The reasoning for why the counterexample is the stronger artifact is ours, not Axiom's. Three properties make it durable. First, it is human-checkable in a way that a failed proof search is not: a reader can verify a single counterexample in minutes, whereas "the proof did not go through" conveys almost nothing without the search trace. Second, it survives restatement — if someone later rewrites the theorem, the counterexample still marks the boundary. Third, it converts a vague worry into a named precondition, because to exhibit the counterexample you must say precisely which condition was absent.
For a data team, the analogue is the query that returns a number nobody can defend. The correction — patching the SQL so the number changes — resolves the incident. The counterexample — the specific account, order or date range where the metric's stated definition and its computed value diverge — resolves the class of incidents, because it names the boundary. Keep the counterexample, not just the patch.
Faithfulness of the statement outranks validity of the proof
Axiom states the ordering directly: ensuring that each formal statement faithfully captures the mathematical meaning of the original text is "even more important than verifying its proof". The mechanism is simple. A proof can be entirely valid about the wrong object. A trivialized statement — one that has been quietly weakened until it is easy — is the easiest thing in the world to prove, and proving it proves nothing.
Axiom also notes that in a tightly interconnected dependency graph, "an unfaithful or trivialized formalization is quickly exposed." That is the enforcement mechanism: faithfulness is not checked statement by statement in isolation but by requiring every definition and lemma to interface successfully with the downstream theorems that depend on it. A definition that has been hollowed out will not support the results built on top of it.
A metric definition has the same two-layer structure. The first layer is what the metric claims to measure — which entities count, over which time window, with which exclusions. The second is whether the SQL computes that. Disputes between finance, sales and operations almost never originate in the second layer. The query returns what it returns; everyone can see the number. The argument is about the first layer, and it recurs because the first layer was never written down completely enough to be argued about precisely.
State the caveat plainly, because it limits how far the analogy travels. Mathematical text has formal semantics; enterprise data definitions generally do not. A metric definition cannot be mechanically checked the way a theorem can, because there is no kernel that knows what "active customer" means. The analogy is structural, not exact. What transfers is the ordering — statement before proof, definition before computation — not the machinery.
The dependency graph is what makes checking cheap
Axiom draws its own contrast with the work it released three days earlier. On 27 October 2026 it published a Lean formalization of the Classification of Finite Simple Groups, which is organized around a single landmark theorem. The Stacks Project is a different kind of object: a foundational library whose value lies in thousands of interlocking definitions, constructions and lemmas designed to be reused across countless future proofs.
That difference explains why this defect class surfaced here. The formalization spans some 16 million lines of code and is described by Axiom as "the largest integrated, reusable library of formalized mathematics ever built," assembled by "coordinated swarms of AI agents" under a hierarchical system with specialized responsibilities and strict coordination protocols. Agents worked in parallel across the dependency graph, translating statements, searching Mathlib for existing abstractions, constructing proofs and resolving prerequisites.
The causal chain is worth spelling out. A definition that many downstream results depend on gets exercised constantly. An unstated hypothesis inside it will eventually fail somewhere far away, in a theorem that has nothing obvious to do with the original statement — and because the failure is attributable to a specific application, the gap gets located. A definition that nobody depends on is never exercised, so its gaps stay invisible indefinitely. Checking is cheap exactly where the dependency count is high, because the checking has already been done by everyone downstream.
The enterprise inference follows, and it is an inference rather than a published finding. A metric that feeds forty dashboards and three agent workflows is under constant implicit test; every consumer that reads it slightly differently is a probe. A metric nobody queries is not tested at all, and its definitional gaps will surface the first time someone builds something on it. That is an argument for prioritizing precondition work by dependency count — how many things break if this definition is wrong — rather than by how recently the metric was touched or how visible its owner is.
The cost is real, and it is political rather than technical
The case against is strong and should be stated at full strength. Formal methods are expensive, slow, and require expertise that is genuinely scarce. The vast majority of business logic does not need machine-checked proof and never will. A team that spends a quarter writing preconditions ships nothing during that quarter, and the opportunity cost lands on the same engineers who are already the binding constraint. The Stacks Project formalization was only possible because the Lean and Mathlib communities had spent years building the underlying library first; nothing here recommends that a data team replicate that effort.
Concede all of it. The recommendation in this article is narrower than formal verification, and its cost falls somewhere different from where the objection points.
Writing preconditions forces decisions that used to be deferred. Someone must choose what "active customer" means — logged in within thirty days, or transacted within ninety? Someone must decide whether refunds net out of revenue, whether the cutoff is midnight UTC or midnight local, whether a cancelled order that shipped counts. Those choices were previously absorbed silently by whoever wrote the query, and the ambiguity was invisible because it had a single author. Written down, the ambiguity becomes an explicit decision with a name attached, and the business will contest it, because different functions want different answers and each has a legitimate case.
That is the real expense: not compute, not tooling, not the price of a proof assistant. It is that a definitional question previously settled by default becomes a decision someone has to own and defend. This is not free, and the political cost is the main reason the work gets skipped — not ignorance of the technique. A team that understands the value perfectly well may still decline to open the argument, and that is a rational choice under many deadlines.
What CDOs and metric-layer owners should change
The recommendation is a substitution of the quality gate. For CDOs and the owners of semantic and metric layers: treat "does each metric definition state its preconditions" as the acceptance criterion, rather than "does it pass the tests."
The substitution matters because tests only check what was written down. A validation rule set built on an incomplete definition will keep passing indefinitely — it is checking the SQL against the definition, and the SQL matches the definition. Meanwhile the same dispute recurs on the business side, and it ends, every time, with "the definitions differ," because there was never a written statement complete enough to settle it. Passing tests and recurring disputes are not in tension; they are the expected outcome when the gate is in the wrong place.
The two-week action is deliberately small. Take the ten most-used metrics — by dependency count, not by recency — and add three preconditions to each:
Time granularity and cutoff convention. Calendar month or fiscal month? Rolling window to query time? Midnight UTC or midnight local, and which local?
Included and excluded entity scope. Test orders, internal accounts, cancelled or refunded transactions, duplicates from multi-system ingestion.
Treatment of nulls and late-arriving data. Does a null dimension drop the row or land in an "unknown" bucket? How long does the metric stay open to restatement after the period closes?
Then mark the ones you cannot fill in. Those marks are the deliverable. Each one is a decision that must be made before any SQL is written, and the list of marks is a more useful artifact than a green test suite, because it names the open questions instead of hiding them behind passing checks.
One governance note on how to publish the result. De Jong's 21 October 2026 announcement asked that AI-generated comments and pull requests not be submitted to the upstream Stacks Project for the time being, and Axiom keeps its formalization, counterexamples and refinements in its own repository in respect of that boundary. The refinement is published as a separate artifact rather than pushed into a canonical text the formalizers do not solely own. That is a reasonable model for a shared metric layer: propose refinements with counterexamples attached, in a place where the owning team can review them on their own terms, rather than editing definitions in place.
Where definitions and detail rows sit in one engine, as they do in Singdata's lakehouse, whether a metric holds within a given scope can be verified alongside the data rather than exported to a second system to compare. That is a mechanical convenience, not a substitute for the written precondition — the definition still has to say what it means before anything can check it.
FAQ
Why do preconditions matter more than tests?
Tests validate what was written; preconditions determine whether what was written is the right claim. The Stacks Project formalization is instructive precisely because the defects clustered in the second category — missing hypotheses, unexcluded edge cases, claims stated more generally than they hold — rather than in faulty reasoning. A test suite built on an incomplete definition will pass forever while the underlying dispute remains unresolved.
Isn't formal verification overkill for business metrics?
Yes, in the sense that machine-checked proof is not the transferable part. Nobody needs a kernel to verify that a revenue query sums a column. What transfers is the discipline of writing the statement completely before computing it, and the ordering that puts statement faithfulness ahead of proof validity. The extension of Axiom's three defect classes to enterprise metric definitions — missing granularity, missing filters, missing time-zone and cutoff conventions — is this article's inference, supported by argument rather than by a published study.
What is the smallest useful version of a data contract?
The three preconditions from the action above, written into the definition text itself: time granularity and cutoff convention, included and excluded entity scope, and treatment of nulls and late-arriving data. Items you cannot fill in should be marked as open decisions rather than left blank — the mark is the signal that a business choice is outstanding, and it is more honest than a default that nobody has agreed to.
Previous posts
1. Medallion Architecture: What Changes Between Bronze, Silver, and Gold? — https://singdata.com/trending/medallion-bronze-silver-gold
2. How Much Data Preparation Is Hidden in Your BI Reports? — https://singdata.com/trending/hidden-data-prep-in-bi-reports
3. How to Migrate from ETL to ELT Without Breaking Reporting or Agentic AI Freshness — https://singdata.com/trending/2026-11-04-elt-file-first-ingestion