AI-generated test code can be a useful accelerator, especially when a team is trying to cover common user flows quickly. The risk is that generated tests often look finished while still encoding brittle selectors, weak assertions, unsafe waits, or data assumptions that fail the moment the UI changes. If you are using AI-assisted testing to draft Playwright, Selenium, or Cypress tests, the review step matters more than the generation step.

The goal is not to reject AI-written tests by default. The goal is to review AI-generated test code with the same rigor you would apply to production code, plus a few automation-specific checks that catch the hidden failure modes. A test suite fails silently when it is too trusting, too coupled to the UI, or too ambiguous about what it is actually verifying.

A test that passes for the wrong reason is often worse than a test that fails, because it trains the team to trust bad signals.

This article gives you a practical review workflow for QA engineers, SDETs, and tech leads. It focuses on the areas where AI-assisted testing commonly needs human correction, locators, assertions, data handling, synchronization, and maintainability. The examples use Playwright because its modern API makes these patterns easy to see, but the review principles apply to Selenium and Cypress as well. For Playwright reference material, see the official docs.

What AI-generated test code usually gets right, and what it gets wrong

AI tools are often decent at producing the outline of a test. They can infer a user journey from a prompt, create the basic structure of a spec, and wire up common actions such as navigation, clicking, and form filling. That makes them helpful for bootstrapping coverage.

The problems start when the tool makes assumptions that are invisible at a glance:

  • It chooses selectors that are easy to write, not stable over time
  • It asserts on text that is incidental rather than meaningful
  • It introduces arbitrary sleeps or fragile timing logic
  • It hardcodes test data that collides with shared environments
  • It duplicates logic across tests instead of abstracting reusable flows
  • It misses negative cases, edge conditions, and postconditions

In other words, AI can create a test that is syntactically correct but operationally weak. Reviewing the code means checking that the test is not only runnable, but also resilient, isolated, and diagnostic when it fails.

Review AI-generated test code in layers

A good review workflow is layered. Do not try to evaluate everything at once.

1. Check the intent first

Before reading selectors or assertions, ask what the test is supposed to prove. A human reviewer should be able to answer these questions:

  • What user behavior is being verified?
  • What is the expected business outcome?
  • What failure would this test detect that another test would not?
  • Is the test redundant with existing coverage?

If the test name says one thing and the code checks something else, stop there. For example, a test named user can reset password should not only confirm that a page opens. It should verify that the reset request succeeds, the right confirmation message appears, and the new credentials are usable if that is part of the intended scope.

2. Review the selectors and locators

Selectors are one of the most common places where AI-generated code becomes brittle. A model may choose whatever is visible in the DOM, even if it is not a stable contract.

Look for these patterns:

  • XPath based on long DOM paths
  • CSS selectors tied to layout classes like .grid > div:nth-child(2)
  • Text selectors for labels that may change with copy edits
  • Locators depending on auto-generated IDs

Prefer selectors that reflect stable semantics. In Playwright, that often means roles, labels, test IDs, and accessible names.

import { test, expect } from '@playwright/test';

test('user signs in', async ({ page }) => {
  await page.goto('/login');
  await page.getByLabel('Email').fill('user@example.com');
  await page.getByLabel('Password').fill('secret123');
  await page.getByRole('button', { name: 'Sign in' }).click();
  await expect(page.getByRole('heading', { name: 'Dashboard' })).toBeVisible();
});

This is better than selecting by fragile structure because the locator reflects user-visible meaning. When reviewing generated code, ask whether the locator would survive a redesign that preserves the same behavior.

A useful rule of thumb, if the selector would break when a designer changes a wrapper div, it is probably too brittle.

3. Verify that assertions are meaningful, not decorative

AI-generated tests often contain assertions that are technically valid but not strong enough to catch regressions. A common pattern is asserting only that a page or element exists, when the real behavior needs a stronger signal.

Weak assertions include:

  • Checking that a page loaded, but not that the correct data loaded
  • Checking that a toast appears, but not that the operation succeeded in the backend
  • Checking that a button exists, but not that it is enabled or behaves correctly after click
  • Checking a URL change without verifying the page content after navigation

Strong assertions should map to the purpose of the test. If the test verifies checkout completion, assert on order confirmation details, not just that the URL changed. If it verifies form validation, assert on the exact validation message and the invalid field state.

typescript

await expect(page.getByText('Payment failed')).toBeVisible();
await expect(page.getByLabel('Card number')).toHaveAttribute('aria-invalid', 'true');

That is more diagnostic than simply checking for an error banner. When a test fails, the assertion should help the reader understand whether the UI, business rule, or data setup was wrong.

Use a code review checklist tailored to test automation

A standard application-code review checklist is not enough for AI-assisted testing. Add questions that focus on the peculiar failure modes of test code.

Locator checklist

  • Does the test use accessible roles, labels, or test IDs where appropriate?
  • Is the selector stable across UI refactors?
  • Does the test avoid deep CSS or XPath chains?
  • Is the locator scoped narrowly enough to avoid matching the wrong element?

Assertion checklist

  • Does each major action end with a verifiable outcome?
  • Are assertions specific enough to catch regressions?
  • Does the test check both happy-path behavior and visible side effects?
  • Are error messages, status codes, or data states asserted when needed?

Data checklist

  • Is the test data isolated from other tests?
  • Can the test run repeatedly without colliding with prior runs?
  • Does the test clean up its own state if it creates data?
  • Are timestamps, random values, or unique identifiers handled safely?

Stability checklist

  • Does the test use built-in waiting mechanisms instead of arbitrary sleeps?
  • Are async operations awaited correctly?
  • Could the test race with rendering, network calls, or background jobs?
  • Does it rely on timing assumptions from a fast local machine?

Maintainability checklist

  • Is the test easy to read without a prompt history?
  • Are repeated steps factored into helpers or fixtures?
  • Is there a clear reason for each wait, assertion, and helper?
  • Will another engineer know how to update it after the UI changes?

Watch for bad waits, they are a common AI mistake

One of the easiest ways for generated test code to become flaky is by using sleeps to cover up uncertain timing. AI systems frequently insert fixed delays because they are easy to synthesize. In test automation, fixed delays are usually a smell.

typescript // Fragile, avoid this pattern

await page.waitForTimeout(2000);
await page.getByRole('button', { name: 'Save' }).click();

The right approach is to wait for the specific condition that makes the next step safe.

typescript

await expect(page.getByRole('button', { name: 'Save' })).toBeEnabled();
await page.getByRole('button', { name: 'Save' }).click();
await expect(page.getByText('Saved successfully')).toBeVisible();

Review any generated test that waits on time instead of state. Ask what the code is really waiting for. If the answer is “for everything to settle,” the test is probably masking a synchronization problem.

Validate data handling like you would in production code

AI-generated tests often work on the assumption that the environment will always contain the same data. That is risky. Test data issues can produce false failures, shared-state collisions, and hidden dependencies between tests.

During review, look for these problems:

  • Hardcoded usernames or emails reused across parallel runs
  • Mutable shared records that previous tests can alter
  • Date-sensitive values that depend on local time zones
  • Random data that is generated but never tracked or cleaned up

A better pattern is to generate unique inputs per test and make the cleanup strategy explicit. For example, if a test creates a record through an API or UI, capture the identifier and remove it after the test or let the environment auto-expire it.

typescript

const email = `qa-${Date.now()}@example.com`;
await page.getByLabel('Email').fill(email);

That example is simple, but in a real suite you would also consider uniqueness under parallel execution and traceability when the test fails. If a generated test uses unique data, reviewers should confirm that the uniqueness strategy is sufficient for the CI concurrency level.

Check whether the test is truly isolated

A common quality issue in AI-generated test code is hidden dependence on a previous test run. Isolation problems are especially damaging because they create order-dependent suites that pass locally and fail in CI.

A reviewer should ask:

  • Does this test require seeded state, and if so, where does it come from?
  • Can the test run by itself?
  • Can it run in parallel with the rest of the suite?
  • Does it assume a user already exists, a cart already contains items, or a record already has a specific status?

If the test depends on upstream setup, encapsulate the setup in fixtures or API helpers rather than burying it in the middle of the UI flow. That makes the dependency visible and easier to maintain.

Isolation is not a luxury. It is what lets CI tell you the truth.

Review the structure, not just the behavior

AI-generated code can be verbose while still being poorly structured. When tests grow past a few steps, structure starts to matter because it affects readability, reuse, and failure analysis.

Look for these patterns:

  • Repeated login steps in every test
  • Long monolithic tests that do too much
  • Helper functions that hide important assertions
  • Missing comments where a business rule is non-obvious

A maintainable test suite separates reusable setup from test-specific verification. For Playwright, fixtures can help keep common flows clean. For Selenium or Cypress, the same idea applies through helper functions or page objects, as long as the abstraction does not swallow the meaning of the test.

Good structure makes reviews faster because the reader can see where setup ends and verification begins.

Human reviewers should look for negative and edge cases AI often skips

AI-generated test code is typically biased toward happy paths. That is understandable, but incomplete. A production-quality suite needs negative coverage too.

Ask whether the generated test includes, or should be extended with:

  • Invalid input validation
  • Permission and role-based access checks
  • Empty states
  • Rate-limited or failed API responses
  • Expired sessions and token refresh behavior
  • Boundary values, such as maximum input lengths

If the AI produced only the happy path, do not treat that as complete coverage. Treat it as a draft of the primary scenario, then add cases that reflect actual failure modes in the product.

Compare the test to the real product contract

A strong review technique is to compare the test code against the contract the app actually exposes, not just what the UI shows at the moment. That means checking:

  • API response expectations, when the UI depends on them
  • Accessibility semantics for visible controls
  • Form validation rules from the product spec
  • State transitions the backend must support

For example, if the app returns a 201 for resource creation, the test should probably assert that the resulting record appears with the expected ID or state, not just that a success message was displayed. UI text is useful, but it should not be the only evidence.

This is where a code review checklist helps. It turns vague doubts into concrete questions about behavior, contracts, and observability.

A practical review workflow for teams

If your team is reviewing a lot of AI-generated test code, use a repeatable workflow instead of ad hoc opinions.

Step 1: Review the prompt and the intended scenario

If possible, keep the prompt alongside the generated test. You want to know what the system was asked to produce, because weak output often reflects a weak prompt. If the prompt was ambiguous, fix the prompt template so the next generation is better.

Step 2: Read the test name and assertions before the body

This is a fast way to catch mismatches. If the title promises one thing and the assertions cover another, rewrite the test before worrying about implementation details.

Step 3: Audit locators and waits

These are the first two technical sources of flakiness. Replace brittle selectors and arbitrary sleeps immediately.

Step 4: Validate data and environment assumptions

Check for shared state, hardcoded credentials, and dependencies on seed data. If the test cannot run cleanly in CI, it is not ready.

Step 5: Assess failure diagnostics

When the test fails, will the output tell you why? Good tests fail with enough context to debug quickly. Bad tests fail with a generic timeout and no clue where the real problem is.

Step 6: Decide whether the test is production-worthy or draft-only

Some generated tests are fine as exploratory scaffolding. Others are ready after minor fixes. A few should be discarded because they encode the wrong behavior. Be explicit about the category.

Example review of AI-generated Playwright code

Here is a deliberately weak generated test, followed by a stronger revision.

import { test, expect } from '@playwright/test';

test('can submit contact form', async ({ page }) => {
  await page.goto('/contact');
  await page.fill('input', 'Jane');
  await page.fill('input', 'jane@example.com');
  await page.click('button');
  await page.waitForTimeout(3000);
  await expect(page.locator('text=Success')).toBeVisible();
});

Problems in this version:

  • It uses ambiguous selectors, multiple input elements and a generic button
  • It does not reveal which field receives which value
  • It waits on time instead of state
  • The assertion is too broad to diagnose failures well
  • It does not verify that the form submission contained the expected payload

A more reviewable version looks like this:

import { test, expect } from '@playwright/test';

test('can submit contact form', async ({ page }) => {
  await page.goto('/contact');
  await page.getByLabel('Name').fill('Jane Doe');
  await page.getByLabel('Email').fill('jane@example.com');
  await page.getByLabel('Message').fill('Please contact me about pricing.');
  await page.getByRole('button', { name: 'Send message' }).click();

await expect(page.getByRole(‘status’)).toHaveText(/message sent/i); await expect(page.getByText(‘We will respond within 2 business days’)).toBeVisible(); });

This version is easier to review because the intent is visible in the code itself. It still may need environment-specific adjustments, but it is far closer to something maintainable.

What to automate in the review process

You do not need to rely only on manual review. Some checks can be automated in CI to catch the most common mistakes before code review even begins.

Useful automation ideas include:

  • Lint rules that flag waitForTimeout or equivalent sleep calls
  • Static checks that discourage brittle selectors
  • Type checks for page object and fixture interfaces
  • CI jobs that run a subset of tests in parallel to expose isolation issues
  • Review templates that require test intent, data strategy, and cleanup notes

This ties into broader practices in test automation, software testing, and continuous integration. The more you can move predictable failure modes into automation, the more your human reviewers can focus on behavior and maintainability.

A lightweight checklist you can reuse in pull requests

Here is a concise review checklist you can drop into a PR template or internal guide:

  • Does the test name match the behavior it verifies?
  • Are locators stable, semantic, and narrow enough?
  • Are there any arbitrary sleeps or timing hacks?
  • Do assertions verify a real business outcome?
  • Is the test isolated from other runs and parallel execution?
  • Is test data unique, traceable, and cleaned up appropriately?
  • Does the test fail with a useful diagnostic message?
  • Is the structure readable enough for a future maintainer?
  • Are negative cases needed but missing?
  • Would this test still make sense after a minor UI refactor?

If a generated test cannot answer most of these questions positively, it should not merge yet.

When to rewrite instead of editing

Sometimes the fastest path is to discard the generated test and rewrite it manually. That is especially true when the code has the wrong abstraction level, or when the generated structure is so noisy that it would take longer to repair than to replace.

Rewrite when:

  • The locator strategy is fundamentally brittle
  • The test mixes multiple unrelated user journeys
  • The assertion model is too weak to be trustworthy
  • The code depends heavily on arbitrary waits
  • The generated helpers obscure the actual behavior under test

Edit when the core flow is correct, but the details need tightening. A good human reviewer should be able to tell the difference quickly.

The bottom line

To review AI-generated test code well, treat it like any other code artifact that can hide defects behind surface-level correctness. Focus on the things that make test automation reliable in practice, stable locators, meaningful assertions, proper synchronization, isolated data, and readable structure. AI-assisted testing can save time, but only if humans keep ownership of quality.

A strong review workflow does not ask, “Did the model produce something that runs?” It asks, “Will this test keep telling the truth after the next UI change, the next refactor, and the next CI run?” That is the standard that keeps a suite useful instead of noisy.