The reviewer is not your test suite

The reviewer is not your test suite

On 5 August 2026, two threads went up in two communities that do not read each other. In r/ExperiencedDevs: "What do you do when a developer submits AI generated code they clearly don't understand?" It took about 335 upvotes and 284 comments (https://old.reddit.com/r/ExperiencedDevs/comments/1vg0cx8/what_do_you_do_when_a_developer_submits_ai/, read 14 August 2026; Reddit fuzzes scores, so repeat reads returned 335 and 336). The same day, in r/dataengineering: "How do you deal with AI Slop PRs from team members?", about 130 upvotes and 100 comments (https://old.reddit.com/r/dataengineering/comments/1vg6k87/how_do_you_deal_with_ai_slop_prs_from_team_members/).

Two different stacks, two different job titles, one question. Neither thread is really asking how to detect machine-written code. Both are asking who is supposed to read it.

The size of the thing, stated honestly

We ran a sweep across eleven subreddits and this theme was the only one that grew while the corpus around it shrank. The comparable window was 16 to 30 July against 31 July to 13 August 2026: the whole sample fell from 1,401 posts to 1,257, and titles about AI slop in pull requests went from 4 to 10.

Four posts. That is the base period, and a four-post base cannot carry a headline number, so we are not going to give you one. Divide 10 by 4 and you get a figure that would look authoritative in a deck and would move by a third if two posts had landed on the other side of a date. The direction is real and the magnitude is fragile, and saying so costs us the better-sounding sentence. That distinction is the entire subject of this article, so it would be strange to break it in the first section.

Generation scaled. Review did not.

The asymmetry is the whole mechanism. Writing a five-hundred-line change used to cost roughly what reading a five-hundred-line change costs. That coupling is gone on the writing side and completely intact on the reading side. A reviewer still reads at reading speed, still holds the same amount of the system in their head, and still runs out of attention at the same point in the afternoon.

So the cost did not vanish. It moved. What the threads are describing is a transfer: an author's five minutes becomes a reviewer's forty, and the author is not the one who notices.

One commenter in the dataengineering thread puts the mechanism in a single clause. Models "can generate code much faster than you can review it manually, effectively pushing more of the work onto the reviewer(s)" (u/RepulsiveBicycle7713, same thread, a comment sitting at one point, which is worth saying given how carefully we are counting everything else here). That is a diagnosis rather than a complaint. The work did not disappear. It changed owner, and nobody consented to the transfer.

The most useful thing in either thread comes from that same dataengineering discussion, and it is what one person decided to do about it. u/sib_n, at 17 points when we read it, is not summarising a consensus and is not venting. He is quoting the sentence he is adding to his team's written procedure: "I'm putting somewhere in the documented PR process: 'Do not shift the responsibility of reviewing your LLM prompt outputs onto the reviewer.'" The inner sentence is proposed policy language, and read as policy it is precise about its target. It does not prohibit the tool. It prohibits the transfer.

Which is exactly what Rust did, in a thread posted the same day, and it is the thing nearly everyone is currently getting wrong about it. Rust adopted an LLM policy on 5 August 2026 and a large number of people have since written that Rust banned AI contributions. Rust did not. The policy is at https://blog.rust-lang.org/inside-rust/2026/08/05/rust-langrust-is-adopting-an-llm-policy/, it is short, and it says the opposite: "Pre-arranged, non-critical, high-quality, well-tested, and well-reviewed code changes that are originally created by an LLM are allowed, with disclosure." The governing line is about role, not permission: use LLMs "to answer questions, analyze, distill, refine, check, suggest, review. But not to create."

The clause worth stealing is the third one: "No one is required to review LLM PRs unless they volunteer." That is not a quality rule. It is a rationing rule. Rust looked at reviewer attention, correctly identified it as the scarce resource in the project, and gave the people holding it the right to decline. The discussion drew about 614 upvotes and 133 comments (https://old.reddit.com/r/programming/comments/1vg555b/the_rust_programming_language_is_adopting_a_new/). The clause to take to your own team is the one about volunteering.

Two very different scales, one day apart, the same move: name reviewer attention as the scarce resource and stop treating it as free.

"Review it more carefully" is not available

The obvious answer is that reviewers should simply read harder. There is a measurement that suggests they will not, and it is worth reading with its caveats attached.

On 5 August 2026, the authors of an agent-permissions browser game published results from 40,000 plays and 409,000 approve/deny decisions, at 66.3 percent mean accuracy (https://scalex.dev/blog/ai-agent-permissions-stats/). Their own caveat travels with the number, and we are quoting it verbatim because it should: "Big caveat: this was a short game, where approximately 34% of the commands players saw were threats. In our day-to-day work these threats (fortunately, so far) appear rarely. Players were under artificial time pressure." It is a vendor post about a game with an artificial base rate. Do not headline the accuracy figure.

The load-bearing finding is elsewhere in the same post and does not depend on the base rate. The single most-missed command was npm run analyze, approved 64.7 percent of the time, while the history log displayed directly above the approval prompt showed the payload piping output to a remote curl. The evidence was on screen. The authors' conclusion is that the log "may not be read closely". Miss rates split hard by category: 11.7 percent for obviously destructive commands, 35.0 percent for scope violations such as reading credentials.

That is the review problem in one number. People catch the change that looks dangerous and miss the change that looks boring, and machine-generated code is overwhelmingly the second kind. It compiles. It has tests. It is plausible everywhere and wrong in one place.

The next obvious answer is to put a model in the reviewer's seat, and "Why Software Factories Fail" disposes of that in a sentence: "If a model could reliably tell good code from bad, it might have written the good version to begin with" (https://github.com/humanlayer/advanced-context-engineering-for-coding-agents/blob/main/wsff.md, 23 July 2026). The same piece names why nothing self-corrects: "there is no penalty for eroding codebase maintainability." The person who pays is not in the room, and often not yet hired.

What we do instead, including the part that does not work

Our position is that reviewer attention is a budget, and the job of engineering process is to spend as little of it as possible on things a machine can decide, so that what remains is spent on judgement.

Concretely, we stopped letting controls describe their own coverage. The recurring defect in our own codebase was a control that stated what it covered in one file, did its work in another, and had nothing binding the two, so the statement drifted and nothing ever failed. The fix is a shared coverage-assertion primitive with a one-line invariant: the population minus the covered set must equal the exemption keys exactly, in both directions. A new item that escapes a control changes the computed set and fails the build. Exemptions have to carry a written reason over a minimum length, and there is a pattern that rejects reasons that say nothing, including "n/a", "tbd", "legacy" and "pre-existing".

The second piece is a control-canary harness. Every registered control is fed a committed known-bad it must flag, for the right reason, and a known-good it must not flag. It has four verdicts, and the interesting one is OVERZEALOUS, because a control that flags everything consumes exactly as much reviewer attention as one that flags nothing. It exists because an earlier readiness review found ten safety controls producing green or silent signals while providing no protection, with one root cause recorded in the harness itself: "no control was ever fed a known-bad input to confirm it fires. They were written, wired, and trusted."

And the honest limit, quoted from the same harness: "There is no branch protection on this repository today (deliberate), so this harness cannot block anything." It makes rot visible fast. It cannot stop a merge. We would rather print that sentence than let you infer a gate that is not there.

The line

A test suite is a claim you can rerun on demand, that costs the same every time, and that fails in a way nobody has to be persuaded by. A reviewer is a person with a finite budget of attention who gets tired, and who is being asked to do the one thing the measurement above says people reliably fail at: read closely enough to catch what looked fine.

Every process that treats human review as the place where correctness gets established is spending the scarcest thing it has on the task it is worst at. Rust's policy is honest about this and lets reviewers walk away. The threads on 5 August are the same recognition arriving without a policy attached. The reviewer is not your test suite, and the fastest way to find out is to hand them something no one can explain.