public
Last active

Cabal file with test-suite block and explanation in a USAGE.md file.

  • Download Gist
ExitCodeTestsExample.hs
Haskell
1 2 3 4 5 6 7 8 9 10 11 12 13 14
module Main where
 
import Test.QuickCheck (quickCheck)
import Your.Module (encrypt, decrypt)
 
prop_reverseReverse :: [Char] -> Bool
prop_reverseReverse s = (reverse . reverse) s == s
 
prop_encryptDecrypt :: [Char] -> Bool
prop_encryptDecrypt s = (encrypt . decrypt) s == s
 
main = do
quickCheck prop_encryptDecrypt
quickCheck prop_reverseReverse
QCTestModule.hs
Haskell
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24
module QCTestModule where
 
import Test.Framework
import Test.Framework.Providers.HUnit
import Test.Framework.Providers.QuickCheck
import Your.Module (encrypt, decrypt)
 
prop_reverseReverse :: [Char] -> Bool
prop_reverseReverse s = (reverse . reverse) s == s
 
prop_encryptDecrypt :: [Char] -> Bool
prop_encryptDecrypt s = (encrypt . decrypt) s == s
 
tests =
[ test "prop_reverseReverse" prop_reverseReverse
, test "prop_encryptDecrypt" prop_encryptDecrypt
, ...
]
 
-- or have a testsuite function that is exported
testSuite = testGroup "Group name"
[ testProperty "desc1" prop_reverseReverse
, testProperty "desc2" prop_encryptDecrypt
]
README.md
Markdown

Integrating Test Suites with Cabal Haskell Projects

Basics

Add a Test-Suite block to your project Cabal file like below:

Test-Suite projectname-testsuite
  Type:                     exitcode-stdio-1.0
  Main-is:                  TestSuite.hs
  Build-depends:            base, ....your test framework deps here...

OR:

-- This appears to be contested on ML and in comments below - I have NOT tested this method!
Test-Suite awesomesauce-testsuite
  Type:                 detailed-1.0
  Test-Module:          Awesomesauce.Tests
  Build-depends:        base, QuickCheck, HUnit 

Note: you might need to run: cabal configure --enable-tests and cabal build again.

Now you should be able to just run: cabal test and it should integrate.

Other options

There are two built in test suite Cabal interfaces:

  • exitcode-stdio-1.0
  • detailed-1.0

The first test suite interface built into Cabal, exitcode-stdio-1.0, depends on the block specifying a Main-is: attribute which is the path of a "main" Haskell file (i.e. a runnable file with a main defined).

The second test suite interface built into Cabal, detailed-1.0, depends on the test-module attribute inside the Test-Suite Cabal block being set to the name of the module to run. This module should export a function named tests with type signature: tests :: [Test]. The Test type can be found in module Distribution.TestSuite. It has two wrapper functions in this module: pure and impure.

Written an internal test library or framework?

Any makers of test frameworks will need to (at a minimum) define instances of the TestOptions and ImpureTestable classes (also exported from Distribution.TestSuite). If the test framework also has a pure runner, you could also define PureTestable class instances for your framework.

QuickCheck & HUnit Support

You will find cabal packages with name prefixes of cabal-test-XXX if your favorite open source Haskell test library or framework (e.g. QuickCheck, HUnit) is supported already.

Test & Have Fun!

awesomesauce.cabal
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29
-- Say you have a Cabal-ized Haskell project called 'awesomesauce'
Name: awesomesauce
Version: 0.1
Synopsis: The awesomesauce
Description: Awesomesauceness for Haskell (as if it needs more?)
License: BSD3
License-file: LICENSE
Author: Susan Potter
Maintainer: me@susanpotter.net
Copyright: 2011
Category: Development
Build-type: Simple
Extra-source-files: README.md
Cabal-version: >=1.2
 
Test-Suite awesomesauce-testsuite
Type: exitcode-stdio-1.0
Main-is: ExitCodeTestsExample.hs
Build-depends: base, QuickCheck, HUnit
 
-- OR
-- This appears to be contested - I have NOT tested this method!
Test-Suite awesomesauce-testsuite
Type: detailed-1.0
Test-Module: QCTestModule
Build-depends: base, QuickCheck, HUnit, test-framework
 
 
-- Then your Executable or Library Cabal sections/blocks under here.

Thanks for this, but the 'exitcode' variation won't work: cabal will report pass even though quickCheck reports a failure.

Also is detailed-1.0 even supported yet? Confused.

@jberryman Which version of QuickCheck are you using? With type: exitcode-stdio-1.0 our test suites run correctly (the exit code when a failure exists is 1 and when all properties pass the process exit code is 0). I admit I have never used type: detailed-1.0. It was added to the README.md since the Cabal docs had this as an option for the type in the test-suite block.

The versions (where the exitcode-stdio-1.0 method above works) are:

  • GHC 7.0.4
  • Cabal library 1.10.2.0
  • cabal-install 0.10.2
  • QuickCheck 2.4.1.1

The only difference I have is I am using cabal v1.10.1.0, but it doesn't seem to be related to cabal.

I have a test module that looks like:

prop_simple_creation :: [Char] -> Property
prop_simple_creation a = length a < 5

main = do
     quickCheck prop_simple_creation

When I run this directly (from the cabal-built binary in the dist directory) it prints this:

*** Failed! Falsifiable (after 8 tests and 7 shrinks):    
"aaaaa"

but returns status 0. Very weird.

Please sign in to comment on this gist.

Something went wrong with that request. Please try again.