The Builders' Floor
Every AI agent sales pitch leads with accuracy: ninety-five percent, ninety-eight, ninety-nine. It's the wrong metric. Risk isn't the probability of error. It's probability multiplied by reach.
The metric everyone knows how to produce
Every AI agent sales pitch opens with accuracy. Ninety-five percent, ninety-eight, ninety-nine. It's the metric engineering knows how to produce and the buyer knows how to compare. And it answers the wrong question.
Risk isn't the probability of error. It's probability multiplied by reach.
The same model, two different risks
An agent with ninety-nine percent accuracy that drafts memos is excellent: the worst possible mistake is a bad paragraph, which someone fixes before it ships. The same agent, with the same accuracy, executing financial reversals, changing vendor records, or publishing under the company's name, is a completely different decision. The difference isn't in the model. It's in what you let the model reach.
Reach is a design choice. Accuracy is a vendor characteristic. One of the two is under your control, and it isn't the one that shows up in the proposal.
The new attack surface isn't in the model
The distinction stopped being theoretical the moment agents got tools. MCP servers, the interface through which an agent accesses external systems and tools, have already accumulated disclosed vulnerabilities since the protocol has existed, starting from essentially zero. The new risk surface doesn't live in the language model. It lives in the plumbing that gives the model its reach.
That's why the right order for the conversation is different. Before asking how often the system gets it right, define how far it can go. The yardstick isn't how much you trust it. It's what happens when it gets it wrong, and how much it costs to undo.
An action that undoes itself in minutes can support high autonomy, even with middling accuracy. An action that takes weeks to undo always requires real approval. An action that can't be undone shouldn't be delegated, no matter how good the model looks in the demo.
Delegating without trust, long before agents existed
I learned to delegate this way before any agent existed. I've always worked surrounded by people technically better than me, which I consider an obligation for anyone who leads, not a concession. It worked for a reason that isn't trust: it was clear, to me and to them, which decisions came back to my desk, and why.
Autonomy without that explicit boundary isn't delegation. It's omission dressed up as modernity.
Most of the supervision out there is theater
Most of the human-supervision controls I see in production are theater. There's a screen, an approve button, a log of who clicked. And there's a person receiving hundreds of requests a day, with no real context to evaluate any of them, whose refusal would delay a process the whole organization is demanding run fast.
Supervision only exists when refusing is genuinely possible. If refusing has a prohibitive cost, you don't have human supervision. You have a signature — and a signature is for assigning blame after the fact, not for preventing damage before it.
What this puts on your desk
The question that separates who has control from who has hope isn't whether the company uses AI. It's: which action does a system already execute today, on its own, that the company can't undo within a week?
In one answer, someone has already decided, in writing, how far that action can go. In the other, the limit was never formalized: autonomy grew by omission, not by decision. The first is design. The second is luck that hasn't run out yet.
Where this piece's yardstick comes from
The question "what happens when it gets it wrong, and how much does it cost to undo" isn't personal intuition. It's the central half of the same criterion that organizes the rest of this series.
The first axis is the reach of the error: a draft, an internal effect, an operational effect, and an effect that crosses the company's front door can't live under the same permission, because the cost of undoing each one is a different order of magnitude. The second axis is the control left on the human side, which ranges from examining case by case down to only noticing the effect after the fact.
The rule linking the two fits in one line: the control that remains has to be greater than or equal to the irreversibility of what was delegated. High accuracy doesn't replace that math. It only changes how often the error happens, never how big it is.
The four reach bands, the ceiling for each one, and the questions that make the yardstick bite are published, with dates, on the method page.
The delegation practice described comes from the author's professional experience, predating the existence of AI agents. The accumulation of disclosed vulnerabilities in MCP servers since the protocol's launch is public knowledge in the industry; no specific figure is cited here because, at the time of publication, there is no consolidated count verifiable against a primary source.