Intermittent Failures

Intermittent Failures

Intermittent test failures are one of the hardest kinds of bug to find. Before you
can fix a bug, you need to know why it is happening, and if the bug manifests itself
at seemingly random intervals, this can be especially difficult. Intermittent failures
can happen for a lot of reasons, typically due to time or from tests affecting other tests.

We usually advise running your tests in a random order. The goal of this is to
make it easy to tell when tests are being impacted by other tests. If your tests
aren’t cleaning up after themselves, then they may cause failures in other tests,
intermittently depending on the order the tests happen to be run in. When this
happens, the best way to start diagnosing is to rerun the tests using the `seed` of
the failing test run.

Youmayhavenoticedthatyourtestsoutputsomethinglike 'Randomized with seed 30205'
at the end of each test run. You can use that seed to rerun the tests in the same
“randomized” order: `rspec --seed 30205` If you want to narrow down the number
of examples that are run, you can use RSpec bisect https://relishapp.com/rspec/rspec-core/v/3-3/docs/command-line/bisect 
`rspec -seed 30205 --bisect,
which runs the tests in different combinations to hone in on the one that is causing problems.

(https://github.com/thoughtbot/climate_control)