Review Is a Dial, Not a Switch: How Much AI Code Should You Actually Read?

A pull request lands. Twelve files, 400 lines, all AI-written. The tests are green. CI is a wall of check marks. You skim the diff, it reads clean—well-named functions, tidy comments, no obvious smell. You approve it.

Three weeks later it's a production incident. The bug was an off-by-one in a boundary check. And the test? The AI wrote that too—it asserted the wrong value with total confidence. The pipeline wasn't verifying the code. It was agreeing with it.

I've been chewing on this exact failure mode all week, because two threads blew up on X and pulled the whole "should you even read AI code anymore" argument back into the open.

Theo (@theo) argued that you should be reviewing a much smaller percentage of your code than you did five years ago—and asked how much better models need to get before we stop reading generated code line by line. Matt Pocock (@mattpocockuk) reframed it: this isn't a yes/no question, it's a 1–7 scale, and where do you sit?

I dug into the current (2026) data to ground my take instead of just reacting. Here's where I land—and the policy I'm setting for my team.

---

The reframe that actually helps: it's a dial

Matt's framing is the most useful thing in either thread, so start there. Stop asking "should I read the code?" It's the wrong question. The right one is: how much scrutiny does this change deserve?—and you answer it per change, not once for your whole career.

His scale runs from "read every line of every diff" all the way to "let the model handle everything, no human review":

Once you see review as a dial, the old debate dissolves. You're not a "read everything" person or a "trust the process" person. You're someone who sets the dial to match the blast radius of what's in front of you. A typo fix in a docs page and a change to your payment authorization path are not the same decision, and pretending they sit at the same level is how you either waste your life or ship a breach.

---

Reconciling Theo and Matt

Theo's instinct isn't wrong—it's just aimed at the average. As models get better, the average setting on the dial genuinely can drift up: less line-by-line reading on the boring, well-tested, low-stakes 80%.

But averages hide the tail. The riskiest ~10% of your changes need more scrutiny than ever, not less—because AI now produces plausible, clean-looking code for those paths too, which earns it less suspicion exactly when it should earn more.

Both things are true at once. "Review less on average" and "review the dangerous stuff harder" aren't in tension—they're the same policy seen from two ends. The mistake is turning Theo's directional point into a blanket rule.

---

Why "just review less" isn't a safe default yet

I wanted to know whether the data supports "models are good enough now, ease off." As of mid-2026, it doesn't—not as a default.

• As many as 92% of AI-generated codebases carried at least one critical vulnerability in one set of assessments run January–April 2026 (Sherlock Forensics 2026 report; see also SQ Magazine's 2026 roundup).
• CVEs linked to code with AI-generated components climbed sharply through early 2026 — roughly 6 → 15 → 35 from January to March (CSA research note). (CVEs aren't tagged human-vs-AI, so treat the exact counts as directional.)
• In a May 2026 SmartBear survey of 273 engineering leaders, 70% said application quality has already degraded as AI accelerated their development (Shift Asia summary).
• The 2026 DORA analysis names the thing directly—a "verification tax": the time you save generating code gets re-spent auditing it, and about 30% of developers report little to no trust in AI output (DORA: Balancing AI tensions).

Speed went up. Quality, on average, went down. "The models are good enough to stop reading" just isn't where the evidence sits right now.

---

The one that matters most: green pipelines lie

Here's the single most important technical point, and it's the direct rebuttal to "just let the tests catch it."

When the same model writes both the code and the tests, the tests inherit the same blind spots as the code. If the implementation has an off-by-one error, the AI-written test will happily assert the wrong value. The test becomes a mirror of the bug, not a check on it. Coverage looks great, CI is green, and you've shipped a defect with a gold star on it. People have started calling this tautological testing—the test proves the code does what the code does, which is not the same as proving the code is right (BSWEN case study, Mar 2026).

The fix isn't "write more tests." It's about who defines correct:

A human owns the test specification—what "correct" means—even when the AI writes the test code.

Write the test's intent from the user story or the spec before you look at the AI's implementation, so the test is a source of truth instead of an echo. Simon Willison's bar is the cleanest version of this discipline I've seen: "I won't commit any code to my repository if I couldn't explain exactly what it does to somebody else." If an AI wrote every line but you reviewed, tested, and genuinely understood it—that's not vibe coding, that's just software development.

---

The hidden cost: review concentrates, and skills erode

Two second-order effects the "review less" crowd tends to skip.

The load doesn't vanish—it moves onto your seniors. As AI volume grows, the only people who can safely approve high-context, security-sensitive changes are your most experienced engineers. So the review pile lands on them. The future where nobody reads code is, ironically, a present where your best people read more—because they're the only ones who catch what AI gets subtly wrong.

Leaning on AI during skill formation weakens the muscle you'll need for exactly this. Cognitive-offloading research keeps pointing the same way: juniors who lean hard on AI while they're still forming judgment build weaker independent debugging and reasoning. That's a compounding trap—if juniors never develop deep review skill, the pool of people who can do high-stakes review shrinks right when the volume of code needing it explodes. The 2026 DORA guidance is blunt about the countermove: deliberately pair juniors with seniors on AI-generated architectural decisions, and keep some hand-coding for complex components so the foundational understanding actually gets built (DORA 2026).

For my juniors specifically, I want deliberate no-AI reps on real problems. Not because AI is bad—because the struggle is where the judgment gets built.

---

What the best practitioners are converging on

I cross-checked my gut against what people I actually trust are saying in 2026, and there's real convergence:

• Simon Willison treats the LLM as an over-confident pair programmer—it writes bugs with the same conviction it writes correct code, and won't warn you. His rule of thumb: review, test, understand, and don't ship what you can't explain (agentic-engineering notes).
• Addy Osmani ("Code Review in the Age of AI") lands on testing as the differentiator between real engineering and vibing—AI drafts features well but stumbles on logic, security, and edge cases, so a human-owned test suite is what lets an agent iterate safely (post, and his 2026 workflow).
• DORA 2026 tells leaders to keep DORA as a floor but extend it with AI attribution and code-durability metrics, and to write explicit policies for how AI is used—so you can tell the teams genuinely improving from the ones generating impressive-looking numbers (InfoQ on the report).

My own synthesis: the defensible version of Theo's point is narrower than the tweet. Shift review away from nitpicking syntax toward intent, architecture fit, and security boundaries, with human-authored tests defining correctness — absolutely. "Review less overall" — not yet. The mechanical parts get faster; the judgment part has to get deeper.

---

The policy I'm setting: risk-tiered, traffic-light review

Map every change to a tier, set the dial to match. This is now common enough that there's literally a security product—Guardrail's "Traffic Light for Code & AI," launched May 2026—built around the same red/amber/green framing (VentureBeat).

• 🔴 Red — auth, payments, crypto, data handling, infra, public APIs. Dial 1–2. Read it closely. Human-authored tests. Mandatory senior + security review, plus a threat model. Compiling and passing tests is not sufficient—SQL injection, XSS, and hardcoded secrets all pass functional tests happily (2026 review standards).
• 🟡 Amber — normal feature work. Dial 2–3. Scan the whole diff, actually review the important logic, and make sure you understand the why of the PR even if you don't read every line.
• 🟢 Green — boilerplate, scaffolding, internal tooling, throwaway scripts, tests-with-human-specs. Dial 4–5. Spot-check and lean on the automated gates.

Regardless of tier, make the machine do the mechanical part: static analysis, dependency/secret scanning, and SAST run as hard merge blocks, not suggestions. This matters more now because AI quietly pulls in extra dependencies and expands your attack surface without telling you. Keep PRs small so review stays humanly possible. And treat every AI output as a first draft—review for architecture fit ("does this respect the decisions we already made?"), not just "does it work in isolation."

---

The bottom line

Don't ask "should I read the code?" Ask "what could this change break, and does my level of scrutiny match that?"

Turn the dial up for boring, safe, well-tested code—that's where Theo is right and your time is better spent elsewhere. Turn it down hard for anything that touches security, money, data, or infra—that's where the tail lives, and where clean-looking AI code is most dangerous precisely because it looks fine.

And whatever you do: never let an AI grade its own homework.

---

References

The threads that started it

1. Theo (@theo) — "you should be reviewing a much smaller % of your code…"
2. Matt Pocock (@mattpocockuk) — "it's not binary, it's a 1–7 scale, where are you?"

2026 data on AI code quality & security 3. Sherlock Forensics — 2026 AI Code Security Report 4. Cloud Security Alliance — Vibe Coding's Security Debt: The AI-Generated CVE Surge (2026) 5. SQ Magazine — AI Coding Security Vulnerability Statistics 2026 6. Shift Asia — Does AI-Generated Code Reduce the Need for Testing, or Demand More? (SmartBear May 2026 survey)

Testing, verification & the "tautological testing" trap 7. BSWEN — Why AI-Generated Tests Have Dangerous Blind Spots (Mar 2026) 8. metacto — Code Review for AI-Generated Code: 2026 Standards

Frameworks & practitioner guidance 9. DORA — Balancing AI Tensions (2026) 10. InfoQ — New DORA Report: Strong Engineering Foundations Drive AI ROI (May 2026) 11. Guardrail — "Traffic Light for Code & AI" launch (VentureBeat, May 2026) 12. Simon Willison — agentic engineering notes 13. Addy Osmani — Code Review in the Age of AI 14. Addy Osmani — My LLM Coding Workflow Going Into 2026

One honest caveat: several of the figures above — the survey percentages, the vulnerability rates, and especially the CVE counts (which are inherently fuzzy, since CVEs aren't labeled human-vs-AI) — reached me through secondary write-ups rather than the primary reports. The direction is consistent across independent sources, but if you're going to quote a specific number externally, pull the original first.