How to Vet AI-Generated Code Before Shipping

There's a moment every developer using Copilot, Cursor, or Claude Code has felt: you accept a 40-line suggestion, it works on the first run, and a small voice asks whether you actually understand what you just shipped. That voice is right to be nervous. In a 2024 study of code generated by GitHub Copilot, Stanford researchers found that developers with AI assistance wrote code that was less secure while being more confident it was correct. The overconfidence is the dangerous part.

AI-native development is no longer a fringe workflow. Surveys from GitHub and Stack Overflow put AI tool adoption among professional developers north of 75%. The productivity gains are real. But the review discipline hasn't caught up. Most teams still treat AI output the way they'd treat a trusted senior engineer's pull request, when they should be treating it like a fast, tireless junior who occasionally hallucinates an API that doesn't exist and copies a SQL injection straight out of a 2011 Stack Overflow answer.

This guide walks through exactly how to vet AI-generated code before it reaches production: the categories of failure to look for, a repeatable review checklist, a worked example, and a comparison of the tools that help. By the end you'll have a process you can run on every AI-assisted commit without slowing your team to a crawl.

Key Takeaways

Never trust, always verify. AI code is a draft, not a decision. Treat every suggestion as an unreviewed pull request from an anonymous contributor.

Hallucinated dependencies are a live attack vector. "Slopsquatting" packages target names AI models invent. Verify every import exists and is legitimate.

Security bugs cluster in predictable places: input handling, authentication, cryptography, and SQL. Audit these first.

Automate the boring 80%. Linters, SAST scanners, and dependency auditors catch the obvious problems so humans can focus on logic and intent.

Understand before you merge. If you can't explain what a block does line by line, you cannot own it in an incident at 3 a.m.

Why AI-Generated Code Needs a Different Review Standard

Human code and AI code fail in different ways, and that changes how you review them. A human engineer who writes a bug usually has a mental model that's slightly wrong. An AI model has no mental model at all. It produces the statistically likely next token based on patterns in its training data, which includes a decade of insecure examples, deprecated APIs, and confidently wrong answers.

The practical consequences show up in three recurring patterns:

Plausible but nonexistent APIs. The model invents a method like user.validateEmailStrict() that reads perfectly but doesn't exist in your library version.
Correct-looking, insecure defaults. String-concatenated SQL, disabled TLS verification, hardcoded secrets, and weak hashing appear because they're common in training data.
Confident licensing violations. AI can reproduce GPL-licensed code verbatim into your MIT project without any warning.

We covered the specific security failure modes in depth in 7 AI Coding Assistant Mistakes That Introduce Security Bugs, and it's worth reading alongside this piece. The short version: the tool is optimized to make you feel done, not to make you safe.

The Five Categories You Must Vet Every Time

Not all lines deserve equal scrutiny. When you're short on time, focus your attention on the five categories where AI most often introduces real risk.

1. Dependencies and imports

This is the single highest-leverage check. AI models hallucinate package names, and attackers have noticed. The practice known as slopsquatting involves registering malicious packages under names that models commonly invent. In one 2024 analysis, roughly 20% of package names suggested by popular models did not exist on the registry. Some of those were later registered by bad actors.

For every import in AI-generated code, confirm the package exists on the official registry, has a plausible download count and maintenance history, and matches the exact name you intended.

2. Input handling and injection

Any code that touches user input, query strings, file paths, or shell commands is a prime candidate for injection flaws. Look specifically for parameterized queries, allow-list validation, and output encoding. AI loves to concatenate.

3. Authentication and authorization

Watch for missing permission checks, JWTs verified without signature validation, session tokens that never expire, and role checks applied on the client but not the server.

4. Cryptography and secrets

Flag MD5/SHA1 for passwords, hardcoded keys, Math.random() used for tokens, and any TLS verification that's been disabled "to make it work."

5. Error handling and resource cleanup

AI often produces happy-path code. Check that files, connections, and locks are released, that errors don't leak stack traces to users, and that failures fail closed rather than open.

A Worked Example: Vetting a Login Function

Let's make this concrete. Say you ask your assistant to "write a Node.js function to log a user in against a Postgres database," and it hands you this:

const q = "SELECT * FROM users WHERE email = '" + email + "' AND password = '" + md5(password) + "'"; const rows = await db.query(q); if (rows.length) return jwt.sign({user: email}, "secret123");

It runs. It returns a token. A junior might ship it. Here's what a proper vet turns up in under two minutes:

SQL injection (critical). The query concatenates email directly. An input of ' OR '1'='1 logs in as the first user. Fix: parameterized query with $1/$2 placeholders.
Broken password hashing (critical). md5 is unsalted and trivially reversible with rainbow tables. Fix: bcrypt or argon2 with a per-user salt and a work factor of at least 12.
Hardcoded JWT secret (high). "secret123" is in source control and forgeable. Fix: load from an environment variable or secrets manager, minimum 256 bits of entropy.
No token expiry (medium). The JWT never expires. Fix: set expiresIn to something like 15m with a refresh flow.
Username enumeration (low). Different responses for "no such user" versus "wrong password" leak information. Fix: constant-time generic error.

Before: five vulnerabilities, two of them critical, in a nine-line function that passed its manual smoke test. After: a parameterized, bcrypt-backed, environment-configured, expiring-token version that took about seven minutes to rewrite. That seven minutes is the entire value proposition of vetting.

A Repeatable Checklist to Vet AI-Generated Code

Turn the categories above into a workflow you run on every AI-assisted change. This is the process I use daily.

Read it before you run it. Understand each line's purpose. If a block is opaque, ask the AI to explain it, then verify the explanation against the docs.
Verify every dependency. Confirm each package name against the official registry. Check publish date, weekly downloads, and repository link. Reject anything you can't confirm.
Diff against your conventions. Does it match your error handling, logging, and naming standards? AI ignores your house style.
Run static analysis. Pipe it through a SAST tool (Semgrep, CodeQL, Snyk Code) and your linter before human review.
Trace the untrusted input. Follow every piece of user-controlled data from entry to sink. Validate at the boundary.
Check secrets and config. Scan for hardcoded credentials with a tool like Gitleaks. AI pastes example keys constantly.
Write or generate tests. Include an adversarial case: empty input, oversized input, injection payloads, and the null case.
Confirm licensing. If a block looks copied verbatim from somewhere, it might be. Our guide to auditing open source licenses in your software stack covers how to catch this.

Most of these can be wired into a pre-commit hook or CI pipeline so they run automatically. The human steps that remain, reading and tracing, are where your judgment earns its keep.

Tools That Help You Vet AI Code: A Comparison

No single tool covers everything. A practical setup layers a linter, a SAST scanner, a dependency auditor, and a secrets scanner. Here's how the common categories compare on what matters.

Tool type	Example	Catches	Speed	False positive rate
SAST scanner	Semgrep, CodeQL	Injection, auth flaws, dangerous patterns	Medium	Medium
Dependency auditor	Snyk, npm audit, Dependabot	Known CVEs, malicious packages	Fast	Low
Secrets scanner	Gitleaks, TruffleHog	Hardcoded keys, tokens, passwords	Fast	Low
Linter	ESLint, Ruff, golangci-lint	Style, unused vars, minor bugs	Very fast	Low
Human review	You	Logic errors, intent, business rules	Slow	Depends on you

The pattern to notice: automated tools are fast and cheap at catching known problems, but only human review catches logic that's wrong for your specific business. AI can write a perfectly secure function that does the wrong thing. If you're assembling a toolkit, browsing the AI tools category and the SDKs on LionScripts is a reasonable place to find scanners and utilities that slot into an existing pipeline.