Created

Embed URL

HTTPS clone URL

SSH clone URL

You can clone with HTTPS or SSH.

Download Gist

Cabal file with test-suite block and explanation in a USAGE.md file.

View ExitCodeTestsExample.hs
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
module Main where
 
import Test.QuickCheck (quickCheck)
import Your.Module (encrypt, decrypt)
 
prop_reverseReverse :: [Char] -> Bool
prop_reverseReverse s = (reverse . reverse) s == s
 
prop_encryptDecrypt :: [Char] -> Bool
prop_encryptDecrypt s = (encrypt . decrypt) s == s
 
main = do
quickCheck prop_encryptDecrypt
quickCheck prop_reverseReverse
 
View ExitCodeTestsExample.hs
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24
module QCTestModule where
 
import Test.Framework
import Test.Framework.Providers.HUnit
import Test.Framework.Providers.QuickCheck
import Your.Module (encrypt, decrypt)
 
prop_reverseReverse :: [Char] -> Bool
prop_reverseReverse s = (reverse . reverse) s == s
 
prop_encryptDecrypt :: [Char] -> Bool
prop_encryptDecrypt s = (encrypt . decrypt) s == s
 
tests =
[ test "prop_reverseReverse" prop_reverseReverse
, test "prop_encryptDecrypt" prop_encryptDecrypt
, ...
]
 
-- or have a testsuite function that is exported
testSuite = testGroup "Group name"
[ testProperty "desc1" prop_reverseReverse
, testProperty "desc2" prop_encryptDecrypt
]
View ExitCodeTestsExample.hs

Integrating Test Suites with Cabal Haskell Projects

Basics

Add a Test-Suite block to your project Cabal file like below:

Test-Suite projectname-testsuite
  Type:                     exitcode-stdio-1.0
  Main-is:                  TestSuite.hs
  Build-depends:            base, ....your test framework deps here...

OR:

-- This appears to be contested on ML and in comments below - I have NOT tested this method!
Test-Suite awesomesauce-testsuite
  Type:                 detailed-1.0
  Test-Module:          Awesomesauce.Tests
  Build-depends:        base, QuickCheck, HUnit 

Note: you might need to run: cabal configure --enable-tests and cabal build again.

Now you should be able to just run: cabal test and it should integrate.

Other options

There are two built in test suite Cabal interfaces:

  • exitcode-stdio-1.0
  • detailed-1.0

The first test suite interface built into Cabal, exitcode-stdio-1.0, depends on the block specifying a Main-is: attribute which is the path of a "main" Haskell file (i.e. a runnable file with a main defined).

The second test suite interface built into Cabal, detailed-1.0, depends on the test-module attribute inside the Test-Suite Cabal block being set to the name of the module to run. This module should export a function named tests with type signature: tests :: [Test]. The Test type can be found in module Distribution.TestSuite. It has two wrapper functions in this module: pure and impure.

Written an internal test library or framework?

Any makers of test frameworks will need to (at a minimum) define instances of the TestOptions and ImpureTestable classes (also exported from Distribution.TestSuite). If the test framework also has a pure runner, you could also define PureTestable class instances for your framework.

QuickCheck & HUnit Support

You will find cabal packages with name prefixes of cabal-test-XXX if your favorite open source Haskell test library or framework (e.g. QuickCheck, HUnit) is supported already.

Test & Have Fun!

View ExitCodeTestsExample.hs
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29
-- Say you have a Cabal-ized Haskell project called 'awesomesauce'
Name: awesomesauce
Version: 0.1
Synopsis: The awesomesauce
Description: Awesomesauceness for Haskell (as if it needs more?)
License: BSD3
License-file: LICENSE
Author: Susan Potter
Maintainer: me@susanpotter.net
Copyright: 2011
Category: Development
Build-type: Simple
Extra-source-files: README.md
Cabal-version: >=1.2
 
Test-Suite awesomesauce-testsuite
Type: exitcode-stdio-1.0
Main-is: ExitCodeTestsExample.hs
Build-depends: base, QuickCheck, HUnit
 
-- OR
-- This appears to be contested - I have NOT tested this method!
Test-Suite awesomesauce-testsuite
Type: detailed-1.0
Test-Module: QCTestModule
Build-depends: base, QuickCheck, HUnit, test-framework
 
 
-- Then your Executable or Library Cabal sections/blocks under here.

Thanks for this, but the 'exitcode' variation won't work: cabal will report pass even though quickCheck reports a failure.

Also is detailed-1.0 even supported yet? Confused.

Owner

@jberryman Which version of QuickCheck are you using? With type: exitcode-stdio-1.0 our test suites run correctly (the exit code when a failure exists is 1 and when all properties pass the process exit code is 0). I admit I have never used type: detailed-1.0. It was added to the README.md since the Cabal docs had this as an option for the type in the test-suite block.

The versions (where the exitcode-stdio-1.0 method above works) are:

  • GHC 7.0.4
  • Cabal library 1.10.2.0
  • cabal-install 0.10.2
  • QuickCheck 2.4.1.1

The only difference I have is I am using cabal v1.10.1.0, but it doesn't seem to be related to cabal.

I have a test module that looks like:

prop_simple_creation :: [Char] -> Property
prop_simple_creation a = length a < 5

main = do
     quickCheck prop_simple_creation

When I run this directly (from the cabal-built binary in the dist directory) it prints this:

*** Failed! Falsifiable (after 8 tests and 7 shrinks):    
"aaaaa"

but returns status 0. Very weird.

quickCheck and those similar functions are run in the IO monad only for printing the result to stdout,
however they will not abort the computation with some exception (e.g. exitFailure).
This is why Cabal reports Pass no matter if the properties passed or not.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Something went wrong with that request. Please try again.