𝗗𝗲𝘁𝗲𝗿𝗺𝗶𝗻𝗶𝘀𝘁𝗶𝗰: 𝗦𝗮𝗺𝗲 𝗜𝗻𝗽𝘂𝘁, 𝗦𝗮𝗺𝗲 𝗢𝘂𝘁𝗽𝘂𝘁

𝗗𝗲𝘁𝗲𝗿𝗺𝗶𝗻𝗶𝘀𝘁𝗶𝗰: 𝗦𝗮𝗺𝗲 𝗜𝗻𝗽𝘂𝘁, 𝗦𝗮𝗺𝗲 𝗢𝘂𝘁𝗽𝘂𝘁

You run a test. It is green. You run it again. It is red. You changed nothing.

This is a flaky test. It is not deterministic.

Deterministic means same input always gives same output.

Think of a vending machine. Press B4. You get a cola. You do this a thousand times. You get a cola a thousand times.

Think of a slot machine. Pull the lever. You get a different result each time.

Some code changes. A function using the current time changes every millisecond.

Other code stays the same. A function to double a number is stable. Double 5 is 10 today. Double 5 is 10 tomorrow.

Flaky tests rely on unpredictable things:

• Current time or date.
• Random values.
• API or database order.
• Leftover data from other tests.

The fix is simple. Stop depending on unpredictable things.

• Use fixed dates in tests.
• Use ORDER BY for database results.
• Keep randomness out of your tests.

Find your flakiest test. Find the unpredictable input. Fix it.

Source: https://dev.to/edriso/deterministic-same-question-same-answer-every-time-59km