Test Stability

Introduction

Test Stability is a practical Playwright skill for engineers who want failures to explain themselves. Instead of memorizing syntax, learn the production reason behind it: what risk it reduces, what evidence it gives, and how it changes the way a team ships software.

Purpose of this lesson

Story: A release blocker was fixed in 12 minutes because the trace showed a 500 response before the UI timeout. In this lesson, test stability combines locators, data, waits, and environment control. That is the difference between a test that merely runs and a test that helps a team decide.

Understanding the topic

Why this exists: Test stability combines locators, data, waits, and environment control. Observability turns test failure from a screenshot mystery into a timeline of browser, network, console, and DOM evidence.

• Real problem solved: Test Stability reduces ambiguity when browser behavior, data, timing, or infrastructure changes.
• Production use: CI triage, flake analysis, performance checks, release gates, and team debugging rituals.
• Beginner misuse: Rerunning tests repeatedly instead of reading the artifact once.
• Elite SDET move: Start with trace, classify the failure, then change the smallest thing that addresses root cause.

Visual explanation

Mental model:

Failure
  ↓
Trace timeline
  ├─ DOM snapshot
  ├─ Network
  ├─ Console
  └─ Action log
  ↓
Root cause

Informative example

A practical example for Test Stability:

// playwright.config.ts
use: {
  trace: "on-first-retry",
  screenshot: "only-on-failure",
  video: "retain-on-failure",
} // test-stability

Execution workflow

Real-world use

In real teams, Test Stability matters because product code changes every week. The test must still tell a useful story: what user behavior was protected, what state was expected, what evidence was captured, and whether the failure belongs to the app, the test, the data, or the environment.

Best practices

• Keep trace on first retry or on failure in CI.
• Attach console and network evidence for critical flows.
• Track failure categories over time.
• Treat flake rate as a product quality metric.

Common mistakes

• Saving videos but not traces.
• Not uploading artifacts from CI.
• Changing waits without understanding the failed timeline.

Debugging tips

• Open the trace first. Check the action timeline, DOM snapshot, network calls, console errors, and screenshot before changing code.
• Classify the failure: selector, timeout, app bug, test data, network mock, authentication, browser difference, or CI resource pressure.
• Prefer user-facing locators. If a selector is tied to DOM layout instead of user intent, it will age badly.
• Record evidence on every retry. A passing retry without evidence hides the real flake.
• Reproduce with the same browser project, viewport, storage state, baseURL, worker count, and environment variables.

Advanced interview questions

Summary

Test Stability is valuable when it makes browser automation faster, clearer, and easier to debug under real product change.