The diff nobody wrote and everybody approved

There is an incident pattern becoming common enough that I want to name it before everyone has independently discovered it through pain: the production bug that nobody caught in review because nobody actually wrote the code, so nobody had the context to catch it. The model wrote the diff. The model wrote the tests. The tests passed. Three engineers read through it, found it plausible, approved it. Six weeks later you’re at 0300 staring at a data corruption bug that makes perfect sense in retrospect and makes no sense from the diff alone.

This is not an “AI is bad” argument. I’ll get to what it actually is.

The invariant that lives in a head, not in a file

Code has two kinds of constraints. The first kind is enforced: the type system checks it, the test suite covers it, the assert fires if it’s violated. The second kind is known: the original author understood it, it’s in the oral history of the team, maybe it’s in a comment somewhere, maybe it’s just the answer to “why does it work that way” that somebody has to explain to every new person who touches the file.

The LLM reads the code. It does not attend standups. It has no memory of the production incident three years ago that explains why the initialization order in this subsystem is exactly backwards from what you’d expect, documented nowhere because the person who fixed it figured it was obvious from the fix. It doesn’t know that you never call flush() with the write lock held because of a footnote in an upstream library’s changelog from 2019 that one of your senior engineers happened to catch at 2am before a deploy.

When a human writes the diff, they carry the oral history — imperfectly, but they carry some of it, and the review is where the person who knows more asks “did you consider the case where…” When the model writes the diff, nobody is carrying the history. The reviewer sees code that looks correct. The invariant that isn’t in the code isn’t covered in the test suite. It gets merged.

What the failure mode actually looks like

It looks like correctness. Syntactically valid, passes the compiler, passes type checking, passes the test suite. Correct for every input the test suite exercises. Wrong for an input nobody thought to document, for a reason obvious to anyone who was in the incident two years ago.

Common shapes I keep hearing about:

Resource cleanup that works on the happy path, leaks on a specific error path the model didn’t see covered in tests. The original author knew about the error path because they wrote the error handling. The model inferred the structure from context and got 90% of it right.

Retry logic that handles transient errors correctly but ignores a per-upstream backoff constraint that’s enforced nowhere in code — it’s in the runbook, or it was once, or it’s in one person’s memory of a slack thread with an upstream team four years ago.

Cache invalidation that’s correct for single-region and silently stale in multi-region, on a code path that had never seen multi-region traffic in the test suite because the original author wrote it when the service was single-region and it seemed like an obviously safe assumption.

Locking that’s correct for the documented operation order and deadlocks on an interleaving the original author had mentally excluded as unreachable — and it was unreachable, until someone added a new caller six months later.

The model does not know what it does not know. Neither does the reviewer if they’re reviewing model-generated code the way they’d review a senior engineer’s output — which is the social dynamic now, because it’s the speed the team has committed to. Nobody wants to be the one blocking the merge queue because they held the AI diff to a higher standard.

The actual problem

This is a code ownership problem, not an AI problem.

The same failure mode exists with any code added by someone who doesn’t understand the codebase’s invariants. We used to call it “contractor code” and have feelings about it. The feelings were correct. The AI is a very fast contractor with extremely good pattern-matching and no judgment about which patterns don’t apply here.

The fix is not “stop using the model.” The fix is that the reviewer’s job changed, and most shops haven’t noticed. When the model generates the diff, the reviewer is now the author — the one carrying the oral history, the one accountable for behavioral correctness, not just syntactic plausibility. That’s a harder job than reviewing code from a colleague you trust. It requires understanding the code at the level you’d need to have written it yourself.

Shops treating AI-generated diffs as “faster human code that needs normal review” are accumulating technical debt denominated in unwritten invariants. It doesn’t show up in velocity metrics. It shows up in incident reports. You’ll recognize it by the 0300 call and the phrase “it was obvious in retrospect.”

The oral history problem is why The Practice of System and Network Administration dedicates real space to runbooks and documentation culture — advice that reads as naive until the third time you’re debugging something whose fix lived only in the head of someone who left the company. Same principle, different medium: write down the invariant, or the next person (human or model) will get it wrong. And models, unlike junior engineers, never ask why.

For the broader theory of why complex systems accumulate invisible constraints — the correctness that lives in what you don’t do — The Art of Unix Programming is the best treatment I’ve found. Not for “how to fix AI slop” specifically, but for the point that well-engineered software encodes a lot of decisions that don’t survive the diff.

Related: the supply chain typosquatting post from a few days ago makes the same underlying point — the default configuration of a widely-used tool is not your security posture, and “the tool did it” is not accountability. Neither is “the model wrote it.”