Variant Testing

README.md

I wrote an example of something I've been thinking about a lot. I have no idea if it's something or nothing, but I am compelled to share out of excitement that it actually runs.

I want to be able to specify tests, and specify "variants", and run the tests with a specified list of variants. The variants in my head are stuff like mock out the Foo service, use the real Foo service, login with an admin user, etc.

Testing.main do |t|
  t.context "Widget#price" do |c|
    c.let(:widget) do
      Widget.new(id: rand(1000))
    end

    c.variant "Use actual price service" do
      PriceService.prices[widget.id] = 20
    end
    c.variant "Mock out price service" do
      allow(PriceService).to receive(:get) { 20 }
    end

    c.test "price should be 20" do
      expect(widget.price).to eq(20)
    end

    c.test "price is cached" do
      expect do
        PriceService.prices[widget.id] = 999
      end.not_to change { widget.price }
    end
  end
end

# Widget fetches its price from the price service,
# with its ID as the key
class Widget
  include FromHash
  attr_accessor :id

  def price
    PriceService.get(id)
  end
end

# PriceService plays the role of a remote service call here
# Instead of making an actual HTTP call, it looks up the price
# in the prices hash
class PriceService
  class << self
    def get(id)
      prices.fetch(id)
    end
    fattr(:prices) { {} }
  end
end

✔ ~/code/liveramp/variant_testing
$ ./variant_testing spec.rb --variants all

Widget#price
  Use actual price service
    price should be 20
    price is cached (FAILED - 1)
  Mock out price service
    price should be 20
    price is cached

Failures:

  1) Widget#price Use actual price service price is cached
     Failure/Error: DEFAULT_FAILURE_NOTIFIER = lambda { |failure, _opts| raise failure }
       expected `widget.price` not to have changed, but did change from 20 to 999
     # spec.rb:42:in `block (3 levels) in <top (required)>'

Finished in 0.0476 seconds (files took 0.2165 seconds to load)
4 examples, 1 failure

Failed examples:

rspec main.rb[1:1:2] # Widget#price Use actual price service price is cached

✔ ~/code/liveramp/variant_testing
$ ./variant_testing spec.rb --variants "Mock out price service"

Widget#price
  Mock out price service
    price should be 20
    price is cached

Finished in 0.01415 seconds (files took 0.19709 seconds to load)
2 examples, 0 failures