testing.md

Here are some thoughts on testing, this example is probably too simple and doesn't fully articulate the issue, but hopefully you get the idea. Something to discuss.

Define upfront

let(:car)           { F(:car) }
let(:fast_car)      { F(:car, fast: true) }
let(:red_car)       { F(:car, color: :red) }
let(:red_fast_car)  { F(:car, fast: true, color: :red) }


it 'is not great'
  expect(car).not_to be_great

it 'is great if a driver exists'
  expect(red_fast_car).to be_great

it 'is not great if red'
  expect(red_car).not_to be_great

it 'is not great if fast'
  expect(fast_car).not_to be_great
  
it 'is great if red and fast'
  expect(red_fast_car).to be_great

In here we create upfront cases for each thing we want to test

What is wrong with this?

• As we have more things to test we need to create more cases e.g. big_red_fast_car, when there are more contributing variables we need to be sure that our factories account for all permutations
• As we change the rules e.g. change how we define great. Then we need to revise all our setup and make sure it is correct for all the cases
• By looking at a test it is not clear why a car is great, it it because it is red, fast or something else hidden in the factory?

Setup in each test

it 'is not great'
  car = F(:car) 
  expect(car).not_to be_great

it 'is not great if red'
  car = F(:car, color: :red) 
  expect(red_car).not_to be_great

it 'is not great if fast'
  car = F(:car, fast: true) 
  expect(fast_car).not_to be_great
  
it 'is great if red and fast'
  car = F(:car, fast: true, color: :red)
  expect(red_fast_car).to be_great

What is wrong with this?

• A lot of setup is repeated (specially with complex tests)
• When a lot of setup is repeated is very easy to miss a key ingredient in a test, e.g. I might miss creating an important record or attribute that changes the behaviour of the system
• Do we know straightaway why the car is great? This is hard to know when there is a lot of setup. By looking at the last test I don't if the car is great, because it is red or fast or something else.

Base case + mutations

let(:car)           { F(:car) }

it 'is not great'
  expect(car).not_to be_great

it 'is not great if red'
  car.update_attribute(:color, :red)
  expect(car).not_to be_great

it 'is not great if fast'
  car.update_attribute(:fast, true)
  expect(car).not_to be_great
  
it 'is great if red and fast'
  car.update_attribute(:color, :red)
  car.update_attribute(:fast, true)
  expect(car).to be_great

• We start we a base case and assert a known state
• Then in each test we change something small and assert
• By looking at each test we can see what are we changing to make the system gives us what we want

The ‘there is a lot of setup’ problem is something to be paying attention to - it’s the tests telling us that there might be a poor design decision here. Listening to the pain of the tests is a big part of test driven development - it’s one of the best sources of feedback for test driven design.

The base case + mutations style suffers from the same problems as the upfront in that the setup isn't localised, and keeping track of it becomes harder the more set up there is, more gets pushed up out of the example in ways that may affect other tests. At that point, it's the listen-to-the-test-pain issue again.
Mutation like that also doesn’t work in cases where the interface doesn’t allow the thing under test to be mutated after construction, and testing it that style encourages allowing a wider interface solely for the purpose of the test.