Last quarter I watched a junior developer merge 340 lines of AI-generated code into production. It compiled. The tests passed. It shipped on a Friday. By Monday morning we discovered the assistant had "helpfully" replaced a parameterized database query with string concatenation because that pattern appeared more often in its training data. One SQL injection later, we were rotating credentials and writing an incident report.
Here is the uncomfortable statistic that should change how your team works: a 2023 Stanford study found that developers with access to AI coding assistants wrote code that was less secure, yet were more confident it was correct. GitHub's own research suggests around 46% of code on the platform is now AI-assisted in some form. The tools are extraordinary at velocity. They are indifferent to whether the thing they just produced is safe, correct, or maintainable.
This guide walks through exactly how to review AI-generated code before it hits your main branch. Not the vague "be careful" advice, but a repeatable checklist, a worked example with real numbers, a comparison of review approaches, and the specific failure patterns these models produce. Whether you use Copilot, Cursor, Claude, or a local model, the discipline is the same.
Key Takeaways
- Treat every AI diff as a pull request from a stranger who is fast, confident, and occasionally wrong in dangerous ways.
- Read the code, don't just run it. Passing tests prove behavior, not correctness or security.
- Watch for six recurring failure classes: injection, hallucinated APIs, silent dependency additions, broken error handling, licensing leaks, and outdated patterns.
- Automate the boring 80% with linters, SAST, and dependency scanners so your human attention goes to logic and intent.
- Never let AI touch auth, crypto, payments, or file access without line-by-line human review and a second reviewer.
- Keep a paper trail: annotate which commits were AI-generated so you can audit them later.
Why AI-Generated Code Needs a Different Review Standard
Human-written code carries context. When a colleague writes a function, they usually understand why it exists, what it connects to, and what will break if it changes. AI-generated code has none of that memory. It produces the statistically most likely next token given your prompt, which is not the same as the correct solution to your problem.
This creates three distinct risks that ordinary code review isn't calibrated for:
- Confident wrongness. The code looks idiomatic and reads cleanly, which lowers reviewer skepticism precisely when it should be highest.
- Plausible hallucination. Models invent function names, config options, and library methods that do not exist but sound entirely reasonable.
- Scale. A developer can now generate more code in an afternoon than a reviewer can carefully read in a week.
The mindset shift is simple. You are not reviewing your teammate's work. You are reviewing a contribution from an anonymous contractor who has read a billion repositories, cannot explain their reasoning, and will never learn from the bug you just caught. That framing keeps your guard up. The same principle applies when you vet AI agents before giving them access to your data: capability is not the same as trustworthiness.
The Six Failure Patterns You Must Look For
After reviewing thousands of AI-generated diffs, the mistakes cluster into predictable categories. Memorize these and your review speed doubles.
1. Injection and unsafe input handling
Models love string concatenation for SQL, shell commands, and HTML. Look for any user input that flows into a query, a system call, or the DOM without parameterization or escaping. This is the single highest-severity pattern.
2. Hallucinated APIs and dependencies
Search for methods, flags, or packages you don't recognize. If the model imports fast-json-parse and you've never heard of it, verify it exists on the registry, check its download count, and confirm it isn't a typosquat. Malicious actors publish packages named after common hallucinations specifically to catch this.
3. Silent dependency bloat
AI happily adds a 2MB library to do something the standard library handles in four lines. Every new dependency is new attack surface and a future maintenance burden.
4. Broken or swallowed error handling
Watch for empty catch blocks, ignored return values, and generic except: pass patterns. The code "works" in the happy path and fails silently everywhere else.
5. Licensing and copied code
Models sometimes reproduce substantial chunks of GPL or otherwise licensed code verbatim. If a function looks suspiciously complete and specific, search a distinctive line to check its origin.
6. Outdated or deprecated patterns
Training data skews toward older code. You'll see deprecated crypto (MD5, SHA-1 for passwords), abandoned framework idioms, and superseded APIs presented as current best practice.
A Worked Example: Reviewing a 60-Line Auth Function
Let's make this concrete. Suppose you asked an assistant to "write a login endpoint that checks a user's password and returns a session token." It gives you 60 lines. Here's how a proper review plays out, step by step.
- Read the intent first (2 minutes). Before any line-by-line reading, ask: does this function do only what I asked? In our example it also wrote a
logUserData()call that ships the email to an analytics endpoint. Scope creep. Cut it. - Check the crypto (3 minutes). The code hashes the password with
md5(). That's an instant reject. Passwords need bcrypt, scrypt, or Argon2 with a proper work factor. This one flaw alone would have exposed every account in a breach. - Trace the inputs (3 minutes). The SQL lookup uses
"SELECT * FROM users WHERE email = '" + email + "'". Classic injection. Rewrite as a parameterized query. - Inspect the token generation (2 minutes). The session token is
Math.random().toString(36). That is not cryptographically secure. Replace withcrypto.randomBytes(32). - Verify the dependencies (2 minutes). It imported a package called
jwt-simple-authwith 41 weekly downloads and no updates in three years. Rejected in favor of a maintained, widely-audited library. - Run the automated tools (1 minute of your time, more of theirs). A SAST scanner flags the injection and the weak randomness automatically, confirming your manual read.
Total human time: about 13 minutes. In those 13 minutes you caught four separate issues, any one of which could have caused a serious breach. The code compiled and would have passed a naive smoke test the whole time. That gap between "it runs" and "it's safe" is the entire reason this discipline matters.
Automated vs Manual vs Hybrid Review: Which Approach Wins
You cannot manually read everything, and you cannot automate away judgment. The right answer is a layered approach, but it helps to see the tradeoffs clearly.
| Approach | Speed | Catches logic bugs | Catches security flaws | Cost | Best for |
|---|---|---|---|---|---|
| Manual line-by-line | Slow | Excellent | Good (if reviewer is skilled) | High (human time) | Auth, crypto, payments |
| Linters + formatters | Instant | Poor | Weak | Low | Style, obvious errors |
| SAST scanners | Fast | Weak | Excellent | Medium | Injection, secrets, unsafe calls |
| AI reviewing AI | Fast | Moderate | Moderate | Low | First-pass triage only |
| Hybrid (all layered) | Moderate | Excellent | Excellent | Medium | Everything shipping to prod |
The hybrid model wins in every serious environment. Let the machines burn through the mechanical checks so your finite human attention is spent where machines are worst: understanding whether the code actually solves the right problem.
A Repeatable Review Workflow You Can Adopt Today
Here is the exact sequence I use. It takes minutes per pull request once it becomes habit, and it front-loads the cheap checks before the expensive ones.
- Isolate the diff. Never review AI code mixed with human changes in one commit. Keep AI-generated commits separate and label them (a simple
[ai]prefix works). This gives you an audit trail later. - Run formatters and linters. These fix noise so it doesn't distract you during the real review.
- Run a SAST scanner and a dependency audit. Tools like Semgrep, Bandit, or
npm auditcatch the mechanical security issues before a human looks. - Verify every new dependency by hand. Check it exists, check its maintenance status, check its download count. A five-second search prevents supply-chain disasters.
- Read for intent. Does the code do only what was asked, and nothing more? Hunt for scope creep, hidden network calls, and telemetry.
- Trace the dangerous flows. Follow every user input to where it lands. Follow every output to where it goes. Anything touching auth, files, or the network gets extra scrutiny.
- Test the unhappy path. Feed it empty strings, negative numbers, unicode, and oversized inputs. AI code is optimized for the demo case.
- Require a second human for high-risk code. Anything involving credentials, payments, or data deletion needs two sets of eyes. No exceptions.
If your project is a WordPress site or plugin, this workflow overlaps heavily with hardening practices. The same instinct that makes you suspicious of an AI-generated file-upload handler should make you audit any security plugin before trusting it and Cover image: The Torch Graduate circuit board (bottom) by Chris Whytehead, licensed under BY-SA 3.0 via Openverse.
