← Back to library

Get agent-written browser tests that survive CI

Jakob Norlin of Endform explains why agent-written end-to-end tests that pass locally often fail in CI: selectors were guessed from source code. Connecting Claude Code to the Playwright MCP server lets it read locators from the rendered page instead. He covers stored login state, writing scenarios in plain English, two review passes and repeat runs for flakiness.

  • OutcomeGet a Playwright test written by Claude Code against your running app, reviewed twice, repeated to catch flakiness and committed beside its plain-English scenario.
  • SetupUnder an hour
  • LevelIntermediate
  • CreditJakob Norlin
View original
Works with
Claude Code

Published Oct 23, 2026Updated Oct 23, 2026AI dry run on sample input: passed Oct 23, 2026Source verified Oct 23, 2026

When should you use it?

  • Agent-written tests keep breaking in CI
  • You already use Playwright
  • You can run the app locally or on staging

Not for: Unit testing; Apps you cannot log into with a test account

What do you need, and what do you get?

You need

  • Node 20 or later
  • A Playwright project
  • A test user and stored login state

You get

  • A test file built from real locators
  • A scenario file in plain English
  • Evidence the test passes on repeat runs

How does it work, step by step?

  1. Check prerequisitesConfirm Node, the agent and a Playwright config exist before starting.
  2. Register the browser serverConnect the Playwright MCP server so the agent can drive a real browser.
  3. Save login stateStore an authenticated session so tests skip the login wall.
  4. Describe the scenario plainlyWrite numbered steps in English, including expected results.
  5. Review and repeatRead the test twice, then rerun it back to back to expose flakiness.

The prompt

Our own prompt, inspired by Jakob Norlin and written to work in Claude Code. Values in lime are placeholders.

Prompt0 variables
Using the Playwright browser tools, open [URL] with the saved login state. Follow this scenario: [numbered steps]. At each step, read the locator from the page itself, preferring roles and visible labels. Then write tests/[name].spec.ts and save the scenario as [name].md beside it. Run the test five times and report any failure.

Inspired by Jakob Norlin: How To Write Playwright tests in minutes with Playwright MCP and Claude Code. The prompt above is written by playbooks.bot in our own words; nothing from the original is quoted.

What can go wrong?

  • MCP commands change quickly; check the official repo if instructions drift.
  • A test that passes once is not yet a good test.

Our take

Jakob Norlin's piece is clear, careful and focused on the real failure mode. Tests that guess selectors from source look fine on a developer's machine and break in CI when the DOM differs. Reading locators from the live rendered page closes that gap. The habit of keeping a plain-English scenario beside the test is worth adopting even without agents, because it documents intent for the next person. Use a dedicated test account with fake payment data, never a real card. Add the repeat run to CI for new tests during their first week so flaky ones surface before they waste everyone else's time. Do two review passes: one for whether the scenario matches product intent, one for whether the selectors are stable. Prefer role- and text-based locators over brittle CSS chains. Commit the scenario file next to the test so they stay in sync. Fail the build on new flaky tests rather than silencing them.

Where the time goes back: Tests built from what the browser actually renders break less often, saving debugging time after merges.

FAQ

Why do guessed selectors fail?
Component libraries and runtime code can rewrite classes, so source-code selectors point at nothing in the browser.
How many repeat runs?
His example repeats a test five times.

Source and credit

Inspired by Jakob Norlin. We describe the idea in our own words, link to the original, and wrote the prompt ourselves; nothing is quoted.

Inspired by
How To Write Playwright tests in minutes with Playwright MCP and Claude Code
Creator
Jakob Norlin
Published on
DEV Community (dev.to)
Original date
Oct 5, 2026
License
Not openly licensed (the poster keeps all rights), so we quote nothing and link to the original
Originally for
Claude Code
AI dry run
Oct 23, 2026: passed. Dry-run tested by our AI test agent on sample input. PASS: Gates held on desk check; no live sends or purchases.
AI dry run on sample data; not run with live connected accounts. The playbooks.bot test agent filled the prompt with synthetic inputs, produced the output and checked it against the page's stated outputs and guardrails.
Verified
Oct 23, 2026: source link resolves to the original, the credited creator published it, our excerpt or prompt matches it, and the license allows our use.

Are you the creator? Claim or correct this page.

More in Operations & Admin

All 203 Operations & Admin playbooks →

Related playbooks

Operations & AdminWorkflow✓ AI dry run

Add a merge check for every way coding agents break code

Keep a running list of the specific ways coding agents have damaged your repository, from hollow tests and invented fake responses to deleted files and silenced type errors, and turn each one into an automated gate that blocks the merge, so the same mistake cannot slip through review twice.

Credit: u/alexmarquardt on r/ClaudeCode

An afternoon+
Operations & AdminWorkflow✓ AI dry run

Ask Reddit's experience before you pick a library

Let your coding agent search Reddit for long-term user reports on a library, an error message or a competitor before you commit to a choice.

Credit: u/karanb192 on r/ClaudeCode

Minutes