AI-Generated Code Passes Your Review. Here's Why It Fails in Production.

How to test AI-generated code: AI code fails differently from human code. It looks clean, passes lint and code review, then breaks at boundary values, integration points, and API contracts. The fix is risk-based testing — boundary value analysis, contract tests, integration tests, and adversarial inputs — instead of trusting review or AI-written unit tests.

That’s the short answer. Here’s why it took the industry two years to learn it.

The pull request that looked perfect

Picture a pull request every QA engineer has now seen some version of. Fully typed. Sensible names. Docstrings on every function. Unit tests included — passing, green, tidy. The reviewer reads it twice, finds nothing to say except “nice, ship it,” and merges.

Three weeks later, month-end billing runs. Every customer whose subscription started on the 31st gets billed on the wrong date — some twice, some not at all. The bug was one line, and we’ll look at it below. Nobody caught it, because there was nothing visibly wrong to catch.

That is the uncomfortable truth about testing AI-generated code in 2026: it is dangerous precisely because it looks good. The code isn’t sloppy. It’s confident. And your review instincts — trained for years on human mistakes — have no alarm that rings for confident code.

Why code review misses AI bugs

Code review is, at its core, a plausibility check. A reviewer reads the diff and asks: does this look like correct code? Naming, structure, obvious logic, style. That works on human code because human bugs correlate with visible sloppiness — the typo sits next to the copy-paste error, and messy code invites a closer look.

AI is optimized to produce plausible code. That’s what a language model is: a machine for maximizing plausibility. So the one signal reviewers rely on — “this looks right” — is exactly the signal AI code always emits, whether it’s correct or not. The reviewer’s instincts were trained on human mistakes; AI doesn’t make those mistakes. It makes confident, invisible ones.

And this is no longer a niche concern. Research from Tricentis estimates that over 40% of the code written in the past year was AI-generated. Between GitHub Copilot in the IDE, agents opening whole pull requests, and “vibe coding” — shipping AI output on trust because it ran once — this isn’t a corner of your codebase. Statistically, it is your codebase.

So where does confident-but-wrong code actually fail? In my experience, in the same five places, over and over.

The 5 failure patterns of AI-generated code

1. Boundary and off-by-one failures

Here’s the billing bug from the intro:

// Returns the next monthly billing date
function nextBillingDate(current) {
  const next = new Date(current);
  next.setMonth(next.getMonth() + 1);
  return next;
}

Clean, readable, commented. Now feed it January 31. JavaScript’s setMonth tries to produce February 31, which doesn’t exist — so it silently rolls over to March 2 or 3. The customer’s February bill vanishes; March gets two. No exception, no log line, no failing test — because the AI’s own tests used the 15th.

The same pattern shows up everywhere the input has edges: pagination code that divides by a page size of zero, discount logic that mishandles exactly 100%, loops that skip empty lists, quantities that go negative. AI handles the middle of every range beautifully and the edges poorly — which is exactly backwards from where bugs live.

2. Hallucinated or outdated API usage

const response = await fetch(url, {
  timeout: 5000,  // fetch has no "timeout" option
  retries: 3      // ...or a "retries" option
});

This compiles. It runs. It even works in the demo. But native fetch silently ignores unknown options — so there is no timeout and there are no retries. The resilience this code appears to add does not exist. The model blended the option names from axios, got no error back, and moved on.

The nastier variant is version drift: the AI calls a method that exists in the latest release of your library but not in the version your project pins, or one whose default behavior changed between versions. When it fails at build time you got lucky. When it doesn’t, you find out in production.

3. Missing integration context

public void transferPoints(long fromId, long toId, int points) {
    accountRepo.deduct(fromId, points);
    accountRepo.credit(toId, points);
}

Two clean lines — and no transaction boundary. If credit throws after deduct succeeds, the points evaporate. Every human on the team knows writes like this go through the @Transactional service layer, and that outbound calls use the shared retry wrapper, and that this endpoint sits behind the tenant-scoping filter. The AI knows none of it. It generates code for a codebase, not your codebase — auth flows, retry policies, idempotency rules, and transaction boundaries are exactly the context a prompt rarely carries.

4. Happy-path-only error handling

try {
  const receipt = await paymentGateway.charge(order);
  return receipt;
} catch (error) {
  console.error('Payment failed:', error);
  return null;
}

Look closely: this code has error handling — that’s what makes it pass review. But catching an exception, logging it, and returning null isn’t handling anything; it’s converting a loud failure into a quiet one. The caller never checks for null, the order flow continues, and somewhere downstream an unpaid order is marked complete. AI code is full of these ceremonial catch blocks: the shape of robustness with none of the substance.

5. Silent contract drift

// Old response: { "total": "1299.00", "currency": "INR" }
res.json({ total: order.total, currency: order.currency });
// After the AI "cleanup", total serializes as a number: 1299

Asked to refactor an endpoint, the model helpfully “fixes” what it sees as inconsistencies — a string that should “obviously” be a number, a user_id renamed to userId. It has no idea that a mobile app released two years ago parses total as a string, because nothing in the code says consumers exist. The endpoint’s tests pass. The mobile app crashes. This is the failure mode contract testing was invented for — and AI refactors have made it a weekly event. When you suspect drift, diffing a before/after response with a JSON diff tool makes the change jump out in seconds (pretty-print both sides in the JSON formatter first if they arrive minified).

The testing playbook: what actually catches AI bugs

Notice what all five patterns have in common: none are visible in the diff. They’re visible in the behavior, at the edges. So the playbook is risk-based testing aimed exactly there.

1. Boundary value analysis and equivalence partitioning — first, always. For every input the AI-generated code touches, test the edges: 0, 1, the maximum, one past the maximum, the empty list, the last day of the month, exactly-100%. This is the single highest-yield technique against AI code because it attacks the exact place AI is weakest. If BVA is new to you (or your team needs a refresher), our boundary value analysis and equivalence partitioning guide covers the method with worked examples — and the free Boundary Value Generator produces the test values for you.

2. Contract tests on every API the code touches. Snapshot the response schema — field names, types, nullability — and fail the build when it drifts. This is the only reliable defense against pattern 5, because the consumers an AI doesn’t know about are also the consumers a human reviewer forgot.

3. Integration tests over unit tests. AI code usually arrives with unit tests, and they usually pass — the function is rarely wrong in isolation. It’s wrong in context: the missing transaction, the ignored auth filter, the fetch options that don’t exist. Only a test that exercises the real wiring can see any of that.

4. Negative and adversarial inputs. Feed it garbage on purpose: the payment that declines, the API that times out, the malformed JSON, the duplicate submit. Pattern 4 — ceremonial error handling — only reveals itself when something actually fails, and nothing fails on the happy path the AI optimized for.

Don’t trust AI’s own tests

One rule deserves its own section: the unit tests that arrive with AI-generated code are not evidence.

They test what the AI thinks the code does — and the AI’s understanding of the code is the same understanding that produced the bug. Same model, same assumptions, same blind spots. nextBillingDate shipped with tests, remember: they used the 15th. The tests and the bug were written by the same author in the same moment of confidence. That’s not verification; it’s circular reasoning with a green checkmark.

This is the human’s new job description. AI can generate code and even generate tests, but independent test design — deciding what could break, from outside the author’s assumptions — is now the highest-value thing a QA engineer does. I say this as someone who happily puts AI inside test frameworks: my Playwright + local LLM failure explainer uses a model to triage failures — but the test cases it triages were designed independently of the code they exercise. That separation is the entire point.

Try it yourself: break nextBillingDate in two minutes

Take the billing function from pattern 1 and design its edge cases the systematic way:

1. Open the Boundary Value Generator.
2. Describe the input: a day-of-month field, valid range 1–31.
3. Generate. You’ll get the boundary set — 0, 1, 2, 30, 31, 32 — plus invalid classes (negatives, non-numeric), derived from BVA and equivalence partitioning rules.
4. Now apply the date-specific twist the tool just surfaced: run the function for Jan 29, 30, 31 with a one-month jump. The 31st fails immediately — the case every AI-written test missed, found in under two minutes of deliberate test design.

That’s the whole argument of this post in miniature: the bug was always one boundary value away, and no amount of reading the code would have found it.

Frequently asked questions

Is AI-generated code reliable?

It is reliably plausible, which is not the same as reliably correct. AI-generated code rarely contains the surface mistakes reviewers catch — typos, bad naming, obvious logic holes — but it disproportionately fails at boundary values, error handling, integration points, and API contracts. Its reliability depends on how it is tested, not on how clean it looks. Treat it like a contribution from a fast, confident developer who has never seen your production traffic.

Should you test AI-generated code differently from human-written code?

Yes — shift effort from plausibility review to risk-based testing. Human review catches human-style mistakes; AI doesn’t make those. Prioritize boundary value analysis and equivalence partitioning on every input, contract tests on every API response it touches, integration tests over unit tests, and adversarial negative inputs. And never accept the AI’s bundled unit tests as sufficient evidence — they share the code’s blind spots.

Can AI review AI-generated code?

It helps, but it cannot be the only gate. An AI reviewer shares training-distribution blind spots with the AI author, so it tends to confirm plausible-looking code rather than challenge its assumptions about your library versions, auth flow, or consumers. Use AI review as an extra layer for surface issues, and keep independent, human-designed tests — especially boundary and contract tests — as the deciding check.

What is boundary value analysis and why does it catch AI bugs?

Boundary value analysis is a test design technique that targets the edges of input ranges — 0, 1, the maximum, one past the maximum, empty collections, the last day of the month — because defects cluster at edges. It is unusually effective against AI-generated code, which typically handles the middle of every range well and the edges poorly. Pairing it with equivalence partitioning gives broad coverage from a small number of high-value test cases.

The reframe

Here’s the conclusion worth carrying into your next sprint planning:

AI didn’t reduce QA’s job. It moved it — from finding sloppy code to distrusting clean code.

The skills that matter most now are the old, unglamorous ones: boundary value analysis, contract testing, negative testing, integration thinking. Not because AI failed, but because AI succeeded at everything else. It writes the plausible 90% faster than any human ever has — which makes the humans who can attack the remaining 10% more valuable, not less.

Clean is not correct. Plausible is not proven. Test the edges.