Flaky Tests

Introduction

Flaky Tests is a practical Selenium skill for SDETs responsible for making large Selenium suites trusted again. Instead of learning it as a command, learn the release problem it solves, the failure it prevents, and the framework decision behind it.

Purpose of this lesson

Story: A banking team had 900 UI tests and 80 random failures every night. They did not need more tests; they needed stable locators, explicit waits, isolated data, and failure triage dashboards. In this lesson, flaky tests are trust killers and must be treated as production defects. The goal is to make Selenium feel like engineering, not record-and-playback automation.

Understanding the topic

Why this exists: Flaky tests are trust killers and must be treated as production defects. Stability is an engineering practice: reliable locators, waits, data, environments, parallel safety, and failure classification.

• Real problem solved: Flaky Tests reduces manual regression cost, flaky feedback, or debugging ambiguity.
• Production use: flaky-test reduction, large regression suites, release gates, parallel runs, and framework health dashboards.
• Beginner misuse: Calling all failures flaky without measuring cause.
• Expert move: Create a flake budget and fix root causes by category instead of adding retries blindly.

Visual explanation

Mental model:

Flake
  ↓
category
  ├─ locator
  ├─ wait
  ├─ data
  ├─ environment
  └─ app defect
  ↓
systemic fix

Informative example

A practical Java + Selenium example for Flaky Tests:

// Stability signal: classify failure before retrying
enum FailureType { LOCATOR, WAIT, DATA, ENVIRONMENT, APP_DEFECT }
FailureType type = classifier.classify(result, "flaky-tests");

Execution workflow

Real-world use

In real teams, Flaky Tests becomes valuable when it gives a trustworthy signal under product change. The lesson is not “how to use one API”; it is how to design a check that survives browser differences, frontend re-renders, test data changes, and CI timing pressure.

Best practices

• Track failures by category.
• Use isolated test data.
• Keep suites risk-based and lean.
• Review slow tests and repeated retries weekly.

Common mistakes

• Adding sleeps instead of readiness checks.
• Keeping obsolete tests forever.
• Parallelizing tests with shared state.

Debugging tips

• Reproduce locally with the same browser, driver, environment, account, and test data.
• Check whether the failure is locator, wait, data, browser, network, or application behavior.
• Inspect DOM at failure time. Dynamic frontends often replace elements after your reference is captured.
• Replace Thread.sleep with a condition: visible, clickable, URL changed, network complete, or text updated.
• Save evidence automatically so nobody debugs from memory.

Advanced interview questions

Summary

Flaky Tests becomes valuable when it protects a real release, shortens feedback, and stays maintainable as the product changes.