Flaky Test Fixer

npx skills add WST2002/claude-code-skills --skill flaky-test-fixer

Overview

A flaky test has a cause that can be found: timing, shared state, order, environment or randomness. Reproduce the flake reliably, identify which of those it is, and fix the cause. Retries and longer timeouts hide the problem; they do not fix it.

Method

1. Reproduce the flake. Run the test many times, alone and with the full suite:
   • Jest/Vitest: run in a loop, or --runInBand vs parallel.
   • pytest: pytest -x --count=50 (pytest-repeat) and random order with pytest-randomly.
   • Playwright: npx playwright test <file> --repeat-each=20 --workers=4.
   • JUnit: @RepeatedTest(50) on the test, or run the suite in a shell loop. Record the failure rate and the exact error.
2. Classify using the table below. Compare passing and failing runs: logs, timing, order, environment.
3. Confirm the cause with a deliberate change that makes it fail every time (shuffle order, add a delay, slow the network, change the time zone).
4. Fix the cause, then rerun the loop until there are zero failures across many runs.
5. Remove any retries, sleeps or skips added as workarounds.

Causes and fixes

Cause	Signs	Fix
Timing / async	Passes when slow, fails when fast; waitForTimeout, missing await	Await the real condition (event, response, visible element); add missing awaits
Shared state	Fails only with other tests; order matters	Fresh data per test, reset singletons, mocks and module state in beforeEach, isolate database (transaction rollback or unique data)
Test order	Fails with random ordering	Remove dependence on data created by earlier tests
Time and dates	Fails near midnight, month end, DST, other time zones	Freeze or inject the clock; use UTC; avoid new Date() in assertions
Randomness	Different inputs per run	Seed the generator or use fixed fixtures
Concurrency	Parallel workers clash on ports, files, accounts	Unique ports, temp directories, accounts per worker
External services	Network calls to real third parties	Mock third parties; use local containers for owned dependencies
Environment	Only in CI	Compare versions, CPU speed, locale, time zone, screen size, env vars
Resource leaks	Fails late in long runs	Close servers, connections, timers, listeners in afterEach

Output

Test: <name and file>
Failure rate: <before> → <after> over <N> runs
Cause: <category and the specific line or condition>
Fix: <change made>

Common mistakes

• Adding retries or increasing timeouts and calling it fixed.
• Running the test once after the fix.
• Fixing the test when the flake is a real race condition in product code.
• Quarantining tests with no owner or follow-up date.