Skip to content

Instantly share code, notes, and snippets.

@Janiczek
Last active May 6, 2022
Embed
What would you like to do?
elm-test 2.0.0 upgrade guide

elm-explorations/test 2.0.0 upgrade guide

Table of Contents:

  1. Fuzzer overhaul
  2. Expect.true and Expect.false
  3. Test.Runner.Failure.format removal
  4. Test.Html.Event additions
  5. (low-level) Test.Runner changes

1. Fuzzer overhaul

There are many changes in the Fuzz module:

  • many helpers added to achieve common patterns in an optimized way
  • Fuzz.andThen added 🎉
  • Shrink and Fuzz.custom is gone 🔥 (all shrinking simplifying is now automatic!)
  • behaviour of some fuzzers has changed (eg. Fuzz.float), trying out more edge cases and simplifying to nicer values

Here are the details:


  • Removed: module Shrink
  • Removed: Fuzz.custom : Generator a -> Shrinker a -> Fuzzer a

There are now no more Shrinkers! All simplification is automatic. This is thanks to the integrated shrinking approach popularized by the Hypothesis library.

With Fuzz.custom gone and with Fuzz.andThen added, it should now be always possible to mirror your Random.Generators with Fuzzers. Take a look at the Random to Fuzz translation section for examples and more help.


  • ✳️ Added: Fuzz.examples : Int -> Fuzzer a -> List a

This function lets you test out your fuzzers—say, in REPL—during development. It will use a hardcoded Random seed and generate a few example values from the fuzzer:

Fuzz.examples 10 Fuzz.int
--> [1,68,54,37,111,-32,0,-30,7,36] : List Int

Fuzz.examples 5 Fuzz.int
--> [1,68,54,37,111] : List Int

Fuzz.examples 5 (Fuzz.intRange -10 10)
--> [-3,6,-1,-6,3] : List Int

  • ✍️ Changed:
-Fuzz.tuple : ( Fuzzer a, Fuzzer b ) -> Fuzzer ( a, b )
+Fuzz.pair : Fuzzer a -> Fuzzer b -> Fuzzer ( a, b )
  • ✍️ Changed:
-Fuzz.tuple3 : ( Fuzzer a, Fuzzer b, Fuzzer c ) -> Fuzzer ( a, b, c )
+Fuzz.triple : Fuzzer a -> Fuzzer b -> Fuzzer c -> Fuzzer ( a, b, c )

The pair and triple-creating functions have changed their API, and should now be nicer to use.


  • ✳️ Added: Fuzz.andThen : (a -> Fuzzer b) -> Fuzzer a -> Fuzzer b
  • ✳️ Added: Fuzz.filter : (a -> Bool) -> Fuzzer a -> Fuzzer a
  • ✳️ Added: Fuzz.sequence : List (Fuzzer a) -> Fuzzer (List a)
  • ✳️ Added: Fuzz.traverse : (a -> Fuzzer b) -> List a -> Fuzzer (List b)
  • ✳️ Added: Fuzz.map6
  • ✳️ Added: Fuzz.map7
  • ✳️ Added: Fuzz.map8

The star of this section is undoubtedly Fuzz.andThen. This function missing is a big part of why people have been reaching for Fuzz.custom and defining their fuzzers via Random.Generators, where Random.andThen is available.

With Fuzz.andThen now being available, you have total freedom in how you'll define your fuzzers. (But also, if some of the helpers like Fuzz.listOfLengthBetween do what you want already, we advise you to use them instead of Fuzz.andThen, as they've been tuned to work better with the simplifying algorithm.)

Fuzz.filter allows you to reject/keep values based on a predicate, but note it will always be better to generate good values instead of generating a mixture of good and bad values and filtering the bad ones out. The Fuzz.filter function will bail out after 15 consecutive rejections!


  • ✳️ Added: Fuzz.oneOfValues : List a -> Fuzzer a
  • ✳️ Added: Fuzz.frequencyValues : List ( Float, a ) -> Fuzzer a
  • ✳️ Added: Fuzz.shuffledList : List a -> Fuzzer (List a)

Helpers for common patterns.

-- BEFORE
Fuzz.oneOf (List.map Fuzz.constant [ 1, 5, 42, 999, 1000 ])

-- AFTER
Fuzz.oneOfValues [ 1, 5, 42, 999, 1000 ]

  • ✳️ Added: Fuzz.asciiChar : Fuzzer Char
  • ✳️ Added: Fuzz.asciiString : Fuzzer String
  • ✳️ Added: Fuzz.asciiStringOfLength : Int -> Fuzzer String
  • ✳️ Added: Fuzz.asciiStringOfLengthBetween : Int -> Int -> Fuzzer String
  • ✳️ Added: Fuzz.stringOfLength : Int -> Fuzzer String
  • ✳️ Added: Fuzz.stringOfLengthBetween : Int -> Int -> Fuzzer String

Various Char/String helpers. Note that the Fuzz.char and Fuzz.string* functions will happily give you emoji and other various Unicode characters. The new Fuzz.ascii* functions will only give you printable ASCII characters (meaning, characters in the 32..126 or 0x20..0x7E range).


  • ✳️ Added: Fuzz.intAtLeast : Int -> Fuzzer Int
  • ✳️ Added: Fuzz.intAtMost : Int -> Fuzzer Int
  • ✳️ Added: Fuzz.uniformInt : Int -> Fuzzer Int

Integer fuzzers prefer generating smaller values by default. We've introduced functions Fuzz.intAtLeast and Fuzz.intAtMost which use Random.minInt and Random.maxInt as their limit values, removing the need for you to import the Random module.

The Fuzz.uniformInt n fuzzer generates integers in range 0..n inclusive, without any bias towards smaller values.

Fuzz.examples 10 (Fuzz.intRange 0 2000)
--> [2,136,108,74,222,65,0,61,14,72] : List Int

Fuzz.examples 10 (Fuzz.uniformInt 2000)
--> [1414,491,962,1824,1068,255,1838,1655,1403,847]

  • ✳️ Added: Fuzz.floatAtLeast : Float -> Fuzzer Float
  • ✳️ Added: Fuzz.floatAtMost : Float -> Fuzzer Float
  • ✳️ Added: Fuzz.niceFloat : Fuzzer Float

Floats have received a lot of care in this release. All float fuzzers will now prefer simplifying to nice values:

  • integers over fractions
  • positive over negative
  • simpler fractions over complex fractions

Meaning, you can expect a lot more of 1.5 and a lot less of 1.0000000000014 🎉

Unfortunately there was no easy way to keep this behaviour for Fuzz.floatRange and certain intervals inside Fuzz.floatAtLeast and Fuzz.floatAtMost, so it's preferred to keep using Fuzz.float if you don't need to constrain the float range.

Fuzzers will also try edge cases like NaN, Infinity and -Infinity. If you don't want these generated, there's Fuzz.niceFloat that skips them.


  • ✳️ Added: Fuzz.listOfLength : Int -> Fuzzer a -> Fuzzer (List a)
  • ✳️ Added: Fuzz.listOfLengthBetween : Int -> Int -> Fuzzer a -> Fuzzer (List a)

Common helpers for list of given length.

-- OK
Fuzz.intRange 0 8
  |> Fuzz.andThen (\length -> Fuzz.listOfLength length item)

-- BETTER
Fuzz.listOfLengthBetween 0 8 item

  • ✳️ Added: Fuzz.weightedBool : Float -> Fuzzer Bool
  • ✳️ Added: Fuzz.lazy : (() -> Fuzzer a) -> Fuzzer a
  • ✳️⚠️ Added: Fuzz.fromGenerator : Generator a -> Fuzzer a

Miscellaneous helpers.

Fuzz.weightedBool behaves like a biased coin: it will give you True with the probability p you give it:

Fuzz.examples 10 (Fuzz.weightedBool 0.8)
--> [True,True,True,True,False,True,True,False,True,True] : List Bool

Fuzz.lazy is helpful when defining recursive fuzzers for your ASTs. If you'll need it you'll know!

Fuzz.fromGenerator is an escape hatch. Please don't use ⚠️. It allows you to use a Random.Generator as a Fuzzer, but the simplification process will be effectively turned off: it will have no idea what's a simpler value. It will always result in better results if you create proper fuzzers instead.

Back to top ⬆️

2. Expect.true and Expect.false

  • Removed: Expect.true : String -> Bool -> Expectation
  • Removed: Expect.false : String -> Bool -> Expectation

The Expect.true and Expect.false functions were a solution that was too easy to reach for while better solutions were possible.

If you're certain checking a Boolean really is what you want, you can still achieve the same with your current error messages via a combination of Expect.equal and Expect.onFail.

-- BEFORE
yourBoolean
  |> Expect.true "Your custom failure message"

-- AFTER
yourBoolean
  |> Expect.equal True 
  |> Expect.onFail "Your custom failure message" 

3. Test.Runner.Failure.format removal

  • Removed: Test.Runner.Failure.format : String -> Reason -> String

This function has been deprecated for a long time (1.0.01.2.2) with the following message:

DEPRECATED. In the future, test runners should implement versions of this that make sense for their own environments.

4. Test.Html.Event additions

We've added new Expectations for Html tests that allow you to check whether an event handler will stop propagation or prevent default behaviour of the event or not.

  • ✳️ Added: Test.Html.Event.expectNotPreventDefault : Event msg -> Expectation
  • ✳️ Added: Test.Html.Event.expectNotStopPropagation : Event msg -> Expectation
  • ✳️ Added: Test.Html.Event.expectPreventDefault : Event msg -> Expectation
  • ✳️ Added: Test.Html.Event.expectStopPropagation : Event msg -> Expectation

5. (low-level) Test.Runner changes

We've made the internal naming change from shrink to simplify, and this touched the low-level helpers inside Test.Runner. Due to reimplementation of the underlying simplification process the API of the two functions has changed somewhat too, but all the intended usecases should still be doable. Let us know if you have some special needs!

  • ✍️ Renamed:
-Test.Runner.Shrinkable a
+Test.Runner.Simplifiable a
  • ✍️ Changed:
-Test.Runner.shrink : Bool -> Shrinkable a -> Maybe ( a, Shrinkable a )
+Test.Runner.simplify : (a -> Expectation) -> ( a, Simplifiable a ) -> Maybe ( a, Simplifiable a )
  • ✍️ Changed:
-Test.Runner.fuzz : Fuzzer a -> Result String (Generator ( a, Shrinkable a ))
+Test.Runner.fuzz : Fuzzer a -> Generator (Result String ( a, Simplifiable a ))

Random to Fuzz translation

With the advent of Fuzz.andThen, you can now translate any Random.Generator a into a Fuzzer a.

That's good, because we've also removed Fuzz.custom and Shrinkers, so you'll likely need to translate your generators! (There's a Fuzz.fromGenerator escape hatch but it will not shrink nicely at all so you really really REALLY shouldn't use it.)

Here's a table showing how:

Generator Fuzzer
Random.int 5 10 Fuzz.intRange 5 10
Random.float 0.5 50 Fuzz.floatRange 0.5 50
Random.uniform g1 [g2, g3] Fuzz.oneOfValues [f1, f2, f3]
Random.weighted (p1,g1) [(p2,g2), (p3,g3)] Fuzz.frequencyValues [(p1,f1), (p2,f2), (p3,f3)]
Random.list 5 g1 Fuzz.listOfLength 5 f1
--- ---
Random.constant Fuzz.constant
Random.pair Fuzz.pair
Random.lazy Fuzz.lazy
Random.andThen fn g1 Fuzz.andThen
Random.map, Random.map2..map5 Fuzz.map, Fuzz.map2..map5

Here's a worked example from the Janiczek/transform test suite:

fuzz-translation

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment