The Data Floor
Nobody knows what's actually in their own data until they need to separate it. And most companies are training AI on what they think they have, not on what they actually have.
A shared belief, until someone asks
I led, on the technology side, the separation of a global company into two independent businesses. Unified systems supporting three very different businesses had to become two separate sets, under a legal deadline, without interrupting a single day of sales.
What that operation teaches isn't technical. It's that a company only discovers what it actually has when someone asks, field by field, whose data it is, where it came from, who depends on it, and what happens if it disappears. Before that, the data inventory is a shared belief: something everyone believes because nobody verified it.
Most organizations are training or feeding AI systems on exactly that kind of belief.
The error that doesn't warn you
There's a specific way for this to go wrong that almost never comes up in board conversations, and it's more subtle than a data leak.
Researchers describe the phenomenon as model collapse. Companies use AI to generate data that trains other AI, which makes sense: it works around scarce real data, cuts labeling costs, avoids copyright disputes over third-party content.
And here's the part that usually gets told wrong. The problem isn't using synthetic data. It's letting it replace real data, instead of accumulating alongside it. The most recent academic literature on the subject, including the work that analytically proved the difference between the two scenarios, is clear on this distinction: when synthetic data replaces real data across training generations, model error diverges without bound. When synthetic data accumulates alongside real data, instead of taking its place, the error has a finite ceiling.
Without that ceiling, the system doesn't fail all at once. It degrades slowly, in a way that's hard to detect, producing worse and worse answers with exactly the same confidence as always.
A system that's wrong with confidence is more dangerous than one that answers "I don't know."
The yardstick I didn't invent for AI
Whenever I needed to decide something big, including moving countries with my family, the first question was never the expected return. It was what the worst-case scenario looks like, and what happens to me if it materializes.
If the worst case is recoverable, the risk is acceptable even with a high probability of happening. If it's irreversible, a low probability saves nobody.
Applied to data, this yardstick reorders priorities in a way most governance discussions don't. A leak is serious, and it's reversible to some degree: the company notifies, contains, remediates, responds. There's a protocol for it, and it works because the problem is visible the moment it happens.
Silent degradation of a training base is a different kind of thing. By the day someone notices, there's no known prior state to go back to, because nobody knows exactly when the contamination started, or which decisions made along the way were affected by a model that was already getting it wrong.
The data governance question, then, isn't whether the data is correct today. It's: would you notice if it stopped being correct? How long would it take? Compared against what?
What the question reveals when you ask it in practice
Taking these three questions to an AI system already in production, what's instructive isn't the answer to each one in isolation. It's where the conversation gets stuck.
Where does the data feeding this system come from? Almost always, someone knows, or knows who to ask.
What proportion of that data is synthetic, or generated by another model? Almost never does anyone know, and the question is usually the first time anyone in the room realizes nobody's measuring it.
How would the company detect degradation, if it were happening right now? The most common answer, when there is one, is "users would complain." If detection depends on someone complaining, the company doesn't have data governance. It has luck, and luck isn't a control you can document.
What this puts on your desk
Pick an AI system already in production at your company and put the three questions in writing to whoever operates it. It's not an academic exercise: it's the difference between knowing where the floor is and finding out when it's already given way.
The system that needs this question most isn't the newest one. It's the one that's been running the longest without anyone asking anything, because that's exactly the one where degradation has had the most time to accumulate without leaving a trace.
Where this piece's yardstick comes from
The question "would you notice if the data stopped being correct?" is a direct application of the criterion that organizes the whole series: what matters in a decision isn't the probability of error, it's the reach of that error added to the control left to detect and undo it.
A data leak has medium reach and high control: the company sees the problem, and the response tools exist and are well known. Silent degradation of a training base has growing reach and shrinking control: the longer it goes undetected, the less recoverable the prior state becomes, because decisions have already been made on top of the error.
It's the same imbalance that shows up on any floor of the building when the control that remains falls below what the effect's reach demands. The rule linking the two variables, the four reach bands, and the ceiling for each one are published, with dates, on the method page.
The distinction between synthetic data that accumulates and synthetic data that replaces the real thing, including the result on error divergence in the two scenarios, is established in the academic literature on model collapse, including the work of Gerstgrasser et al., "Is Model Collapse Inevitable? Breaking the Curse of Recursion by Accumulating Real and Synthetic Data" (2024), and discussed critically in "Position: Model Collapse Does Not Mean What You Think" (2025). The corporate separation scene comes from the author's professional experience, on the technology side of the operation.