[RFC] Elm Inline Tests

elm-inline-tests.md

An idea: elm inline tests

Everybody's second favourite programming language to be developed in the last decade has the following to say about unit tests:

The purpose of unit tests is to test each unit of code in isolation from the rest of the code to quickly pinpoint where code is and isn’t working as expected. You’ll put unit tests in the src directory in each file with the code that they’re testing.

To allow elm developers to put their unit tests in the same file as the source code, this document proposes a special type of comment with the syntax {-test- ... -} in elm source files. These comments are test blocks, wrapping code that should only be run when testing. The elm compiler ignores code in test blocks (as it ignores everything in comments) but an elm test runner could extract tests and supporting code from such test blocks and run them.

The format of elm inline tests might look something like this. Note that this is a legal elm file which the compiler will happily accept.

module Complex exposing (Complex, complex)

import Internal.Complex
import Parser exposing ((|.), (|=), Parser)

{-test-util

import Expect exposing (Expectation)
import Fuzz exposing (Fuzzer)
import Test exposing (describe, fuzz, fuzz2, fuzz3, test)

-}

type Complex =
    Complex

{-| Construct a complex number from real and imaginary parts.
-}
complex : Float -> Float -> Complex
complex =
    Debug.todo "snip"

{-test-

testComplex : Test.Test
testComplex =
    [ describe "Build"
        [ describe "complex"
            [ -- snip 
            ]
        ]
    ]

-}

Implemention

The elm test runner would extract and run tests in the following stages.

1. Duplicate source directory of an elm project into elm-stuff/generated-code/elm-explorations/test/inline/.

2. Parse each elm file:

   1. Uncomment all -test-util comments.
   2. Uncomment all -test- comments and validate/take note of the single Test.Test contained within each comment.
   3. Add import Test.Test as ElmTestRunnerImplTest
   4. Add a snipet along the lines of

   amalgamatedTestsForMODULENAME : ElmTestRunnerImplTest.Test
   amalgamatedTestsForMODULENAME =
       ElmTestRunnerImplTest.describe "MODULENAME"
           [ firstTestFoundInStep2ii
           , secondTestFoundInStep2ii
           -- etc ...
           ]

   to the end of the elm file. 5. Edit the module's exposing to expose amalgamatedTestsForMODULENAME.

3. Create a "main" elm file which imports all the tests exposed in step 5 and defines a Test.Runner.Node. The existing node test runner infrastructure can then run all unit tests.

Transpiled file

Following the steps above, the generated file should look something like the following.

module Complex exposing (Complex, complex, amalgamatedTestsForComplex)

import Test as ElmTestRunnerImplTest

import Internal.Complex
import Parser exposing ((|.), (|=), Parser)

{-test-util -}

import Expect exposing (Expectation)
import Fuzz exposing (Fuzzer)
import Test exposing (describe, fuzz, fuzz2, fuzz3, test)

{- -}

type alias Complex =
    Complex


{-| Construct a complex number from real and imaginary parts.
-}
complex : Float -> Float -> Complex
complex =
    Debug.todo "snip"

{-test- -}

testComplex : Test.Test
testComplex =
    [ describe "Build"
        [ describe "complex"
            [ -- snip 
            ]
        ]

{- -}

{-| Inserted by the test runner.
-}
amalgamatedTestsForComplex : ElmTestRunnerImplTest.Test
amalgamatedTestsForMODULENAME =
    ElmTestRunnerImplTest.describe "MODULENAME"
        [ testComplex
        ]

Downsides

Special comments are not as good as first class syntax, a better version of this proposal would add a new syntax (as rust does using #[cfg(test)] and #[test]). However, such syntax would require compiler support whereas this proposal can be implemented purely by the test runner.

Editors will give no syntax highlighting for test code (as it is commented out).