Generation Is Cheap, Verification Is Not

Generation Is Cheap, Verification Is Not

"We ship 10x faster now." I hear versions of this constantly, and every time I want to ask the same follow-up: faster to where? Because shipping was almost never the constraint. Understanding what you shipped, and trusting that it's correct, that was the constraint. And the software factory makes it worse, not better.

This is the contrarian core of how I think about AI in engineering, so let me defend it properly.

The bottleneck moved, and it moved to the expensive side

Picture the pipeline as two costs: the cost to generate a change, and the cost to verify it, to actually convince yourself it's correct, safe, and doesn't quietly break something three systems over.

For decades those costs were roughly coupled. Writing the code was slow, so the volume of code you had to review was naturally rate-limited by the volume you could produce. Generation gated verification for free.

The factory severs that coupling. Generation cost falls off a cliff. Verification cost does not; verifying a change requires understanding the system it lands in, tracing its blast radius, and reasoning about failure modes, and none of that got cheaper because a model wrote the diff. So the ratio inverts. You can now generate far faster than you can verify, and everything you generate but haven't truly verified is unreviewed risk sitting in your codebase.

That's the whole thesis: the binding constraint of the factory era is verification, and "shipping faster" is often just accumulating that risk at higher speed.

Volume is not value, it's surface area

There's a seductive assumption hiding under the speed metrics: that more code, faster, is progress. It isn't necessarily. Every line you produce is also surface area you have to maintain, secure, operate, and eventually debug at 2 a.m. Generating more code faster can increase net liability. A feature you didn't need, implemented flawlessly and shipped in an hour, made your system worse.

The factory is very good at producing volume. Volume is the thing it's least able to tell you is valuable. That judgment (is this even worth building) stays firmly human, and it's the cheapest place to save yourself enormous downstream verification cost. The change you don't generate needs no review.

Three ways the verification gap actually bites

This isn't abstract. Here's where I've watched it go wrong, or expect to.

Erosion of understanding. When no human deeply understands a generated system, debugging and incident response slow to a crawl. "Nobody knows how this works" isn't a documentation gap; it's an operational time bomb. The factory can produce a system faster than any human can build a mental model of it, and the day you need that mental model is the day it's on fire.

Homogenized mistakes at scale. A careful human makes a subtle mistake in one service. A factory makes the same subtle mistake in three hundred services simultaneously, because it's applying the same flawed pattern uniformly. Cross-cutting concerns (auth, error handling, data handling) are exactly where this is most dangerous and hardest to walk back.

Spec debt. Output quality is capped by input clarity. Vague requirements used to fail slowly, one hand-written module at a time, with a human noticing the ambiguity mid-implementation. Now they fail fast and wide. The bottleneck moves upstream to knowing what to build, and a machine will build the wrong thing very efficiently if you let it.

The guardrail is the review

So what do you do? You can't manually review your way to safety at factory volume; the whole problem is that review capacity is the ceiling. The answer is to make verification scale the same way generation did: automate it into the substrate.

This is where it connects to platform work. The guardrail is the review, performed continuously and uniformly instead of by a human reading every diff:

• Policy-as-code and secure-by-default templates, so whole classes of mistake can't be generated in the first place.
• Automated security scanning and quality gates in the pipeline, so verification runs at machine speed alongside generation.
• Progressive delivery, SLOs, and error budgets, so the things you couldn't verify up front are caught fast and contained when they slip through.
• Blast-radius limits, so a homogenized mistake can't take down everything it touched at once.

The high-leverage human work shifts accordingly. It's no longer reviewing every line; it's designing the controls that review every line for you. That's a fundamentally different, and more valuable, kind of engineering than the one the factory automated away.

The question to actually optimize

If I could get one metric to replace "how fast do we ship," it would be some honest measure of how fast can we verify what we ship: the gap between your generation rate and your trustworthy-review rate. Close that gap and speed is pure upside. Ignore it and every efficiency gain is just risk, compounding, arriving faster.

Generation is cheap now. Act like the expensive thing is the thing that's scarce, because it is.

Sources & further reading

• AI Code Generation: Reliability & the Verification Bottleneck gathers the verification-bottleneck reporting, the constrained-versus-open reliability research, and DORA 2024's throughput-versus-stability data.

Third in a series on the industrialization of software engineering. Previous: Your Platform Is the Factory Floor. Series opener: The Software Factory Hollows the Middle.