If you like
test-framework you may also want to check out Oscar Finnsson's
test-framework-th package for generating
your test groups programatically.
The latest version of the example is always available online:
-- Adapted from an example by Don Stewart. For licensing information -- please see the file "example/Test/Framework/Example.lhs" in the source tree. import Test.Framework (defaultMain, testGroup) import Test.Framework.Providers.HUnit import Test.Framework.Providers.QuickCheck (testProperty) import Test.QuickCheck import Test.HUnit import Data.List main = defaultMain tests tests = [ testGroup "Sorting Group 1" [ testProperty "sort1" prop_sort1, testProperty "sort2" prop_sort2, testProperty "sort3" prop_sort3 ], testGroup "Sorting Group 2" [ testProperty "sort4" prop_sort4, testProperty "sort5" prop_sort5, testProperty "sort6" prop_sort6, testCase "sort7" test_sort7, testCase "sort8" test_sort8 ] ] prop_sort1 xs = sort xs == sortBy compare xs where types = (xs :: [Int]) prop_sort2 xs = (not (null xs)) ==> (head (sort xs) == minimum xs) where types = (xs :: [Int]) prop_sort3 xs = (not (null xs)) ==> last (sort xs) == maximum xs where types = (xs :: [Int]) prop_sort4 xs ys = (not (null xs)) ==> (not (null ys)) ==> (head (sort (xs ++ ys)) == min (minimum xs) (minimum ys)) where types = (xs :: [Int], ys :: [Int]) prop_sort5 xs ys = (not (null xs)) ==> (not (null ys)) ==> (head (sort (xs ++ ys)) == max (maximum xs) (maximum ys)) where types = (xs :: [Int], ys :: [Int]) prop_sort6 xs ys = (not (null xs)) ==> (not (null ys)) ==> (last (sort (xs ++ ys)) == max (maximum xs) (maximum ys)) where types = (xs :: [Int], ys :: [Int]) test_sort7 = sort [8, 7, 2, 5, 4, 9, 6, 1, 0, 3] @?= [0..9] test_sort8 = error "This test deliberately contains a user error"
We can run these tests from the command line (in parallel!) like so:
$ ghc -package test-framework -package test-framework-quickcheck \ -package test-framework-hunit -threaded Example.hs -o Example $ ./Example --maximum-generated-tests=5000 +RTS -N2
You may get an error like the following:
No instance for (QuickCheck-184.108.40.206:Test.QuickCheck.Testable (Gen Prop)) arising from a use of `testProperty' at Example.lhs:68:16-46
The reason is that GHC is building the example (which uses QuickCheck 1) with the QuickCheck 2 library. You can fix this by running:
$ ghc-pkg hide QuickCheck-220.127.116.11
You may need adjust the version number to match the one on your machine, as reported by
Alternatively, you can supply the
-package QuickCheck-18.104.22.168 flag when invoking GHC (again you may
need to change the version number to match your installed QuickCheck version).
errorbut the tests pass
This happens because GHC finds it convenient to eta-expand the calls to error, so we can no longer observe the failure.
This is dodgy behaviour by GHC, but it's unlikely to change any time soon, so the workaround is to compile your tests
-O0 (the default).
If you really do want to compile your tests with optimisations enabled, you should also be able to supply the
-fno-full-laziness flag along with your
-O2 flag. However, if new optimisations
are added to GHC they may break this workaround - it's probably safest to turn optimisation off altogether.
The framework is built with an extensible architecture that allows anyone to add their own test "provider".
All of the supplied providers are built using this mechanism, and so you probably want to install not only the
infrastructure package (
test-framework), but also the commonly-used providers
A description of the options available can be obtained by using the option
--help on the command line.
The test-selection syntax for use with the
-s command line option is based on that of shell globs or
Git .gitignore files. Test patterns consist of the following:
!which negates the pattern.
foo/will match a group called
fooand any tests underneath it, but will not match a regular test
/, the framework checks for a match against any single component of the path.
*matches anything within a single path component (i.e.
**matches anything (i.e.
foowould only match a component of the test path called
foo(or a substring of that form).
group/test1 but not
both examples would be matched by
group/**1. A leading slash matches the beginning of the test path; for example,
test1 but not
A test will be run if it matches any of the patterns supplied with
Haddock documentation is available at Hackage: