Project 10: Property-Based Testing Framework
Project 10: Property-Based Testing Framework
Building QuickCheck from Scratch: Where Randomness Meets Rigor
- Main Programming Language: Haskell
- Alternative Languages: Scala (ScalaCheck), F# (FsCheck), Python (Hypothesis)
- Coolness Level: Level 4: Hardcore Tech Flex
- Difficulty: Level 3: Advanced
- Knowledge Area: Testing / Generators / Type Classes
- Estimated Time: 2-3 weeks
- Prerequisites: Projects 5 (Algebraic Data Types), Project 6 (Functor/Applicative/Monad), understanding of randomness
Learning Objectives
After completing this project, you will be able to:
- Implement the Gen monad from scratch - Build a monadic random value generator that composes elegantly using do-notation
- Design and implement the Arbitrary type class - Create a polymorphic interface for generating random values of any type
- Build shrinking algorithms - Implement strategies to find minimal counterexamples when properties fail
- Compose properties using combinators - Design higher-order functions that combine and modify properties
- Implement the core QuickCheck loop - Build the machinery that runs tests, catches failures, and reports results
- Apply property-based testing to real code - Identify invariants in your code and express them as testable properties
- Understand the relationship between types and generators - See how types guide the generation of test data
Conceptual Foundation
The Revolution in Testing
In 1999, Koen Claessen and John Hughes published âQuickCheck: A Lightweight Tool for Random Testing of Haskell Programs,â and it changed how we think about testing. Before QuickCheck, testing meant writing example-based tests:
-- Traditional unit tests
test1 = reverse [1,2,3] == [3,2,1]
test2 = reverse [] == []
test3 = reverse [1] == [1]
These tests check specific examples, but they can never cover all cases. QuickCheck introduced a fundamentally different idea: instead of testing specific examples, test properties that should hold for ALL inputs.
-- Property-based test
prop_reverse_reverse :: [Int] -> Bool
prop_reverse_reverse xs = reverse (reverse xs) == xs
-- This property is tested with HUNDREDS of random inputs!
This shift from âdoes my code produce the right output for these examples?â to âdoes my code satisfy this invariant for all inputs?â is profound. Itâs the difference between testing instances and testing laws.
What Is a Property?
A property is a boolean predicate that should hold for all inputs of a given type. Properties capture invariantsâthings that must always be true about your code.
Consider the reverse function. What invariants should it satisfy?
- Involution:
reverse (reverse xs) == xs - Length preservation:
length (reverse xs) == length xs - First becomes last:
head (reverse xs) == last xs(for non-empty lists) - Concatenation reversal:
reverse (xs ++ ys) == reverse ys ++ reverse xs
Each of these is a property. If reverse is implemented correctly, all these properties should hold for ANY list. QuickCheck tests this by generating hundreds of random lists and checking each property.
The Power of Random Testing
Why is random testing so effective? Consider testing a sorting function:
sort :: Ord a => [a] -> [a]
Properties of a correct sort:
- Output is sorted:
isSorted (sort xs) - Output is a permutation:
sort xs \isPermutationOf` xs` - Idempotent:
sort (sort xs) == sort xs - Length preserved:
length (sort xs) == length xs
If you write example-based tests, you might test sort [3,1,2], sort [], sort [1], etc. But what about:
- Lists with duplicates?
- Lists that are already sorted?
- Lists sorted in reverse?
- Very long lists?
- Lists with negative numbers?
Random testing automatically explores this space. The generator produces all sorts of weird edge cases you might not think of. This is why property-based tests often find bugs that example-based tests miss.
The Gen Monad: Generating Random Values
At the heart of QuickCheck is the Gen monadâa way to describe random value generation compositionally. Letâs understand why it needs to be a monad.
The Problem with Naive Random Generation
Imagine you want to generate a random person:
data Person = Person { name :: String, age :: Int }
With imperative randomness:
# Imperative approach
import random
def random_person():
name = random_string() # Uses global random state
age = random.randint(0, 100) # Uses global random state
return Person(name, age)
This works but has problems:
- Hidden state: The random state is global and mutable
- Not reproducible: Canât replay a specific test run
- Not composable: Hard to combine generators in complex ways
The Monadic Solution
Haskellâs Gen monad makes randomness explicit and composable:
newtype Gen a = Gen { runGen :: StdGen -> (a, StdGen) }
A Gen a is a stateful computation that takes a random generator and produces a value a along with a new generator state. This is exactly the State monad pattern!
Why is this beautiful?
- Explicit threading: The random state is threaded through explicitly
- Pure: Given the same seed, you get the same sequence of values
- Composable: Monadic bind
(>>=)automatically threads the state
instance Monad Gen where
return x = Gen $ \g -> (x, g)
(Gen f) >>= k = Gen $ \g ->
let (a, g') = f g -- Run first generator
Gen h = k a -- Get second generator from result
in h g' -- Run second generator with new state
Now generating a Person becomes:
genPerson :: Gen Person
genPerson = do
name <- genString
age <- choose (0, 100)
return $ Person name age
The do notation hides the state threading. Each generator consumes some randomness and passes the rest along.
Combinators: Building Complex Generators from Simple Ones
The real power of Gen comes from combinatorsâfunctions that build complex generators from simple ones.
Basic Combinators
-- Choose from a range
choose :: Random a => (a, a) -> Gen a
choose (lo, hi) = Gen $ \g -> randomR (lo, hi) g
-- Pick from a list
elements :: [a] -> Gen a
elements xs = do
i <- choose (0, length xs - 1)
return (xs !! i)
-- One of several generators
oneof :: [Gen a] -> Gen a
oneof gens = do
i <- choose (0, length gens - 1)
gens !! i
-- Weighted choice
frequency :: [(Int, Gen a)] -> Gen a
frequency xs = do
let total = sum (map fst xs)
n <- choose (1, total)
pick n xs
where
pick n ((k,g):rest)
| n <= k = g
| otherwise = pick (n-k) rest
Structural Recursion for Recursive Types
Generating recursive data types (like lists or trees) requires careful handling to ensure termination:
-- Generate a list (with bounded recursion)
genList :: Gen a -> Gen [a]
genList genA = do
len <- choose (0, 30) -- Bound the length!
replicateM len genA
-- Generate a tree (using sized to control depth)
genTree :: Gen a -> Gen (Tree a)
genTree genA = sized $ \n ->
if n <= 0
then Leaf <$> genA
else oneof
[ Leaf <$> genA
, Branch <$> resize (n `div` 2) (genTree genA)
<*> resize (n `div` 2) (genTree genA)
]
The sized combinator passes a âsizeâ parameter that decreases with recursion depth, ensuring termination.
The Arbitrary Type Class: Polymorphic Generation
Different types need different generators. The Arbitrary type class provides a uniform interface:
class Arbitrary a where
arbitrary :: Gen a
shrink :: a -> [a]
shrink _ = [] -- Default: no shrinking
Any type with an Arbitrary instance can be randomly generated. This is type class polymorphism in actionâthe same interface works for any type:
instance Arbitrary Int where
arbitrary = choose (-1000, 1000)
shrink 0 = []
shrink n = [0, n `div` 2] -- Shrink toward 0
instance Arbitrary Bool where
arbitrary = elements [True, False]
shrink True = [False]
shrink False = []
instance Arbitrary a => Arbitrary [a] where
arbitrary = do
len <- choose (0, 30)
replicateM len arbitrary
shrink [] = []
shrink (x:xs) =
[] : -- Remove all elements
xs : -- Remove first element
[x':xs | x' <- shrink x] ++ -- Shrink first element
[x:xs' | xs' <- shrink xs] -- Shrink rest
Notice how Arbitrary [a] requires Arbitrary aâthis is the power of type class constraints propagating through instances.
Shrinking: Finding Minimal Counterexamples
When a property fails, the failing input might be huge and hard to understand. Shrinking finds a minimal counterexample.
Example: Suppose testing prop_bad_sort fails on [7, -3, 12, 4, -8, 1, 9]. Thatâs a lot to debug! But after shrinking, we might find the minimal failure is [1, 0]âmuch easier to understand.
How Shrinking Works
The shrink function returns a list of âsmallerâ values:
shrink :: a -> [a]
For example:
shrink 10might return[0, 5, 8, 9](smaller integers)shrink [1,2,3]might return[[], [2,3], [1,3], [1,2], ...](shorter lists or smaller elements)
The shrinking algorithm:
shrinkLoop :: (a -> Bool) -> a -> a
shrinkLoop prop value =
case filter (not . prop) (shrink value) of
[] -> value -- Can't shrink further; this is minimal
(smaller:_) -> shrinkLoop prop smaller -- Found a smaller failure!
This is a greedy search: keep finding smaller failing inputs until none can be found. The result is a local minimumâthe smallest failure in the neighborhood defined by shrink.
Designing Good Shrink Functions
A good shrink function should:
- Move toward âsimplerâ values: Smaller numbers, shorter lists, simpler structures
- Preserve type invariants: Donât generate invalid values
- Make progress: Eventually terminate (shrink values should be strictly âsmallerâ)
- Be thorough: Explore multiple shrinking directions
For lists, shrinking can:
- Remove elements (makes it shorter)
- Shrink individual elements (makes elements simpler)
shrink (x:xs) =
-- Remove the first element
xs :
-- Remove the first element completely
[] :
-- Keep first, shrink rest
[x:xs' | xs' <- shrink xs] ++
-- Shrink first, keep rest
[x':xs | x' <- shrink x]
The Property-Checking Loop
The core loop that ties everything together:
quickCheck :: Testable prop => prop -> IO ()
quickCheck prop = do
seed <- newStdGen
let result = runTests 100 seed
case result of
Success n -> putStrLn $ "OK, passed " ++ show n ++ " tests."
Failure input -> do
putStrLn $ "FAILED: " ++ show input
putStrLn $ "Shrinking..."
let minimal = shrinkLoop prop input
putStrLn $ "Minimal counterexample: " ++ show minimal
The Testable class abstracts over things that can be tested:
class Testable prop where
property :: prop -> Property
instance Testable Bool where
property b = Result b
instance (Arbitrary a, Show a, Testable prop) => Testable (a -> prop) where
property f = forAll arbitrary $ \x -> property (f x)
This instance for functions is the key insight: a function a -> Bool is testable by generating random a values!
Why Property-Based Testing Matters
Property-based testing changes how you think about correctness:
- Forces you to articulate invariants: What should always be true?
- Tests edge cases automatically: Random generation explores the space
- Finds subtle bugs: Counterexamples youâd never think to test
- Documents behavior: Properties serve as executable specifications
- Compositional: Combine generators and properties elegantly
Real bugs found by QuickCheck:
- Concurrency bugs in Erlang/OTP (John Hughesâ famous work)
- Protocol conformance issues
- Off-by-one errors in array algorithms
- Unicode handling bugs
The Mathematical Foundation
Property-based testing connects to several deep ideas:
Free Theorems: In polymorphic code, types impose constraints that lead to properties. For example, any function f :: [a] -> [a] must satisfy:
map g . f = f . map g
This is a âfree theoremâ derivable purely from the type!
Algebraic Laws: Properties often express algebraic laws:
- Monoid:
x <> mempty = x,mempty <> x = x,(x <> y) <> z = x <> (y <> z) - Functor:
fmap id = id,fmap (f . g) = fmap f . fmap g
QuickCheck tests these laws for specific instances.
Specification as Code: Properties are executable specifications. Instead of writing informal documentation (âreverse should undo itselfâ), you write code that is the specification.
Historical Context
QuickCheck emerged from the functional programming communityâs focus on equational reasoning. In Haskell, we reason about code by substituting equals for equals:
-- If we know:
reverse (reverse xs) = xs
-- Then we can transform:
reverse (reverse (reverse ys))
= reverse ys -- by the law
QuickCheck lets us test these laws empirically. It bridges the gap between formal proofs and practical testing.
The idea spread rapidly:
- ScalaCheck (2007): QuickCheck for Scala
- FsCheck (2008): For F#/.NET
- Hypothesis (2015): For Python, adding advanced shrinking
- PropEr (2011): For Erlang
- fast-check (2018): For JavaScript/TypeScript
Every major language now has a property-based testing library, all descended from the original Haskell QuickCheck.
Project Specification
You will build a property-based testing framework from scratch. Your framework should support:
Core Requirements
- Gen Monad
- Random value generation with composable generators
- Combinators:
choose,elements,oneof,frequency,listOf,vectorOf - Size control with
sizedandresize - Monad instance for do-notation support
- Arbitrary Type Class
- Instances for:
Bool,Int,Integer,Char,[a],Maybe a,Either a b,(a, b) - Shrink function for each type
- Derived generators using type class constraints
- Instances for:
- Property Testing
quickCheckfunction that runs tests- Configurable number of tests
- Seed control for reproducibility
- Failure reporting with the failing input
- Shrinking
- Automatic shrinking of failing inputs
- Report the minimal counterexample
- Track shrinking steps
- Property Combinators
(==>)for conditional properties (implications)forAllfor explicit generator specificationclassifyfor input classification/statisticslabelfor labeling test cases
Stretch Goals
- Coverage checking: Ensure properties are tested with diverse inputs
- Parallel testing: Run tests in parallel for speed
- Stateful testing: Test stateful systems with sequences of commands
- Custom shrinking strategies: Allow users to define domain-specific shrinking
Solution Architecture
Module Structure
src/
QuickCheck/
Gen.hs -- Gen monad and combinators
Arbitrary.hs -- Arbitrary class and instances
Property.hs -- Property type and combinators
Test.hs -- Testing loop and runners
Shrink.hs -- Shrinking algorithms
Random.hs -- Random number utilities
QuickCheck.hs -- Re-exports public API
Core Data Types
-- The generator monad
newtype Gen a = Gen { unGen :: StdGen -> Int -> (a, StdGen) }
-- The Int is the "size" parameter for recursive generators
-- The arbitrary class
class Arbitrary a where
arbitrary :: Gen a
shrink :: a -> [a]
shrink _ = []
-- Property result
data Result = Success | Failure String | Gave Up String
-- Property wrapper
newtype Property = MkProperty { unProperty :: Gen Result }
-- Test configuration
data Config = Config
{ numTests :: Int
, maxDiscards :: Int
, seed :: Maybe Int
}
Key Design Decisions
- Gen includes size: The size parameter allows recursive generators to terminate
- Result includes messages: For better error reporting
- Property is Gen Result: Properties are themselves generators of results
- Shrink returns list: Multiple shrink candidates allow thorough searching
Implementation Guide
Phase 1: The Gen Monad (Days 1-2)
Goal: Implement the generator monad with basic combinators.
Milestone 1.1: Basic Gen type and Monad instance
newtype Gen a = Gen { runGen :: StdGen -> (a, StdGen) }
instance Functor Gen where
fmap f (Gen g) = Gen $ \s -> let (a, s') = g s in (f a, s')
instance Applicative Gen where
pure a = Gen $ \s -> (a, s)
Gen f <*> Gen a = Gen $ \s ->
let (fn, s') = f s
(x, s'') = a s'
in (fn x, s'')
instance Monad Gen where
Gen m >>= k = Gen $ \s ->
let (a, s') = m s
in runGen (k a) s'
Milestone 1.2: Basic combinators
choose :: Random a => (a, a) -> Gen a
elements :: [a] -> Gen a
oneof :: [Gen a] -> Gen a
frequency :: [(Int, Gen a)] -> Gen a
Milestone 1.3: Size-aware generation
sized :: (Int -> Gen a) -> Gen a
resize :: Int -> Gen a -> Gen a
Test: Generate random integers and lists, verify distribution.
Phase 2: The Arbitrary Class (Days 3-4)
Goal: Implement the type class and instances for common types.
Milestone 2.1: Class definition and primitive instances
class Arbitrary a where
arbitrary :: Gen a
shrink :: a -> [a]
shrink _ = []
instance Arbitrary Bool where ...
instance Arbitrary Int where ...
instance Arbitrary Char where ...
Milestone 2.2: Compound type instances
instance Arbitrary a => Arbitrary [a] where ...
instance Arbitrary a => Arbitrary (Maybe a) where ...
instance (Arbitrary a, Arbitrary b) => Arbitrary (Either a b) where ...
instance (Arbitrary a, Arbitrary b) => Arbitrary (a, b) where ...
Milestone 2.3: Shrinking for all types
shrink :: Int -> [Int] -- Shrink toward 0
shrink :: [a] -> [[a]] -- Remove elements, shrink elements
Test: Generate values, verify theyâre in expected ranges, test shrinking produces smaller values.
Phase 3: Properties and Testing (Days 5-7)
Goal: Build the property checking machinery.
Milestone 3.1: Property type
data Result = Success | Failure String | GaveUp
newtype Property = Property { unProperty :: Gen Result }
Milestone 3.2: The Testable class
class Testable a where
property :: a -> Property
instance Testable Bool where ...
instance Testable Result where ...
instance (Arbitrary a, Show a, Testable b) => Testable (a -> b) where ...
Milestone 3.3: The quickCheck function
quickCheck :: Testable prop => prop -> IO ()
Test: Run simple properties, verify they pass/fail correctly.
Phase 4: Shrinking Loop (Days 8-10)
Goal: Implement automatic shrinking of counterexamples.
Milestone 4.1: Shrinking algorithm
shrinkLoop :: (a -> Bool) -> a -> a
-- Find minimal failing input
Milestone 4.2: Integration with quickCheck
- When a test fails, automatically shrink
- Report shrinking progress
- Show minimal counterexample
Milestone 4.3: Shrink tracing
- Count number of shrink steps
- Show intermediate values (optional verbose mode)
Test: Verify that shrinking finds known minimal counterexamples.
Phase 5: Advanced Combinators (Days 11-14)
Goal: Add property combinators for expressiveness.
Milestone 5.1: Implication
(==>) :: Testable prop => Bool -> prop -> Property
-- Only test when precondition holds
Milestone 5.2: forAll
forAll :: (Show a, Testable prop) => Gen a -> (a -> prop) -> Property
-- Use custom generator
Milestone 5.3: Classification
classify :: Testable prop => Bool -> String -> prop -> Property
label :: Testable prop => String -> prop -> Property
collect :: (Show a, Testable prop) => a -> prop -> Property
-- For gathering statistics on test inputs
Test: Use combinators in real properties, verify statistics collection.
Testing Strategy
Unit Tests for Gen
-- Distribution tests
test_choose_bounds = do
let values = generate 1000 $ choose (0, 10)
assert $ all (\x -> x >= 0 && x <= 10) values
test_elements_coverage = do
let values = generate 1000 $ elements ['a'..'z']
assert $ all (`elem` ['a'..'z']) values
Property Tests for Your Framework
Use your framework to test itself! (Meta-property testing)
-- Shrinking properties
prop_shrink_smaller :: Int -> Bool
prop_shrink_smaller n = all (< abs n) (map abs $ shrink n) || n == 0
-- Generator distribution
prop_choose_uniform :: Int -> Int -> Property
prop_choose_uniform lo hi = lo < hi ==>
let gen = choose (lo, hi)
in forAll gen $ \x -> x >= lo && x <= hi
Known Failure Cases
Create functions with known bugs and verify your framework catches them:
-- Bug: fails to handle negative numbers
badSort :: [Int] -> [Int]
badSort = sort . map abs -- Oops! Loses negative info
prop_sort_preserves :: [Int] -> Bool
prop_sort_preserves xs = sort xs `isPermutationOf` xs
-- Should fail with minimal case like [-1]
Common Pitfalls
1. Non-Terminating Generators
Problem: Recursive generators that donât decrease size:
-- WRONG: Infinite recursion
genTree :: Gen Tree
genTree = oneof [Leaf <$> arbitrary, Branch <$> genTree <*> genTree]
Solution: Use sized to bound recursion:
genTree :: Gen Tree
genTree = sized go
where
go 0 = Leaf <$> arbitrary
go n = oneof
[ Leaf <$> arbitrary
, Branch <$> go (n `div` 2) <*> go (n `div` 2)
]
2. Biased Generators
Problem: Generators that favor certain values:
-- WRONG: Empty lists almost never generated
genList :: Arbitrary a => Gen [a]
genList = do
x <- arbitrary
xs <- genList
return (x:xs) -- Always adds at least one element!
Solution: Include base cases with appropriate frequency:
genList = frequency [(1, return []), (4, (:) <$> arbitrary <*> genList)]
3. Shrinking Creates Invalid Values
Problem: Shrink produces values that violate invariants:
-- Positive integers represented as regular Int
newtype Positive = Positive Int
-- WRONG: Can shrink to 0 or negative!
shrink (Positive n) = map Positive (shrink n)
Solution: Filter invalid shrinks:
shrink (Positive n) = [Positive m | m <- shrink n, m > 0]
4. Weak Properties
Problem: Properties that donât actually test anything:
-- WRONG: Always true, tests nothing useful
prop_sort_is_list :: [Int] -> Bool
prop_sort_is_list xs = length (sort xs) >= 0
Solution: Write properties that would fail on buggy implementations:
-- BETTER: Actually tests sorting behavior
prop_sort_ordered xs = isSorted (sort xs)
prop_sort_permutation xs = sort xs `isPermutationOf` xs
5. State Threading Bugs
Problem: Forgetting to thread the random generator state:
-- WRONG: Reuses same generator state
genPair :: Gen (Int, Int)
genPair = Gen $ \g ->
let (a, _) = runGen arbitrary g -- Ignores new state!
(b, g') = runGen arbitrary g -- Same g, same value!
in ((a, b), g')
Solution: Thread state properly (or use monadic bind):
genPair = do
a <- arbitrary
b <- arbitrary
return (a, b)
Extensions and Challenges
1. Stateful Testing
Test stateful systems by generating sequences of commands:
data Command = Push Int | Pop | Peek
type Model = [Int]
prop_stack_model :: [Command] -> Bool
prop_stack_model cmds = runModel [] cmds == runReal newStack cmds
2. Coverage-Guided Generation
Track which branches are tested and guide generation accordingly.
3. Parallel Test Execution
Run independent tests in parallel for speed.
4. Golden Master Testing
Combine with snapshot testing: generate inputs, save expected outputs.
5. Mutation Testing Integration
Generate properties that detect code mutations.
Real-World Connections
Where Property-Based Testing Appears
- Cryptocurrency: Formal verification of smart contracts
- Distributed Systems: Testing consensus protocols (Jepsen-style)
- Compilers: Testing optimizations preserve semantics
- Databases: Testing query planners and executors
- Serialization: Testing encode/decode roundtrips
Industry Adoption
- Quviq AB: John Hughesâ company uses QuickCheck for industrial testing
- Dropbox: Tests file sync with property-based testing
- Amazon: Uses property-based testing for S3 and other services
- Basho: Used QuickCheck extensively for Riak database
How This Connects to Other Concepts
- Formal Methods: Properties are like lightweight specifications
- Fuzzing: Random generation is related to fuzz testing
- Model Checking: Stateful testing explores state spaces
- Dependent Types: Types that guarantee properties at compile time
Interview Questions
Prepare to answer these after completing the project:
- âWhat is property-based testing and how does it differ from example-based testing?â
- Answer: Property-based testing tests invariants that should hold for all inputs, while example-based testing checks specific input/output pairs. Property-based testing uses random generation to explore the input space automatically.
- âExplain how the Gen monad works and why it needs to be a monad.â
- Answer: Gen wraps a function from random seed to (value, new seed). Itâs a monad because we need to thread random state through composed generators. Monadic bind automatically handles state threading.
- âWhat is shrinking and why is it important?â
- Answer: Shrinking finds minimal counterexamples when tests fail. It repeatedly tries âsmallerâ inputs until it finds one that still fails but has no smaller failing version. This makes debugging much easier.
- âHow would you test a sorting function with properties?â
- Answer: Key properties include: output is sorted, output is a permutation of input, length is preserved, idempotence (sort . sort = sort), and stability (for stable sorts).
- âWhat makes a good property? What makes a weak property?â
- Answer: Good properties would fail on buggy implementations and capture essential invariants. Weak properties are always true or rarely fail. Example: âlength xs >= 0â is weak; âreverse (reverse xs) == xsâ is strong.
- âHow do you handle recursive data types in generation?â
- Answer: Use the
sizedcombinator to pass a decreasing size parameter. At size 0, generate base cases only. This ensures termination while still generating deep structures for larger sizes.
- Answer: Use the
- âHow does the Arbitrary type class enable polymorphic testing?â
- Answer: Arbitrary provides a uniform interface for generating any type. A property like
prop :: [a] -> Boolis automatically testable for anyawith an Arbitrary instance, thanks to the instance for functions.
- Answer: Arbitrary provides a uniform interface for generating any type. A property like
Self-Assessment Checklist
Youâve mastered this project when you can:
- Implement the Gen monad from scratch with correct state threading
- Explain why Gen is a monad and what each typeclass law means
- Write Arbitrary instances for compound types using constraints
- Design shrink functions that make progress and preserve invariants
- Implement the main testing loop with failure detection
- Add shrinking that finds minimal counterexamples
- Write properties that would catch real bugs
- Use combinators like
==>,forAll, andclassify - Generate recursive data without infinite loops
- Explain the tradeoffs of random testing vs. exhaustive testing
- Debug your framework when tests donât behave as expected
- Extend the framework with new features (coverage, parallelism)
Resources
Primary References
| Topic | Resource | Specific Section |
|---|---|---|
| Original QuickCheck paper | âQuickCheck: A Lightweight Tool for Random Testingâ by Claessen & Hughes | Full paper |
| QuickCheck implementation | âReal World Haskellâ by OâSullivan, Stewart, Goerzen | Chapter 11 |
| Monads and state | âHaskell Programming from First Principlesâ | Chapter 18-23 |
| Type classes | âHaskell Programming from First Principlesâ | Chapter 6 |
| Random generation | âLearn You a Haskell for Great Good!â | Chapter 9 |
Advanced Reading
| Topic | Resource |
|---|---|
| State machine testing | âTesting the Hard Stuff and Staying Saneâ by John Hughes (talk) |
| Shrinking strategies | âIntegrated vs Type-Based Shrinkingâ by Hughes |
| Hypothesis design | Hypothesis documentation on shrinking |
| Industrial applications | âQuickCheck: A Lightweight Toolâ case studies |
Online Resources
- QuickCheck source code on Hackage
- Hypothesis (Python) documentation for modern shrinking ideas
- PropEr (Erlang) for stateful testing patterns
- John Hughesâ talks on YouTube
âDonât write tests. Write specifications and let the computer generate tests.â - The philosophy behind property-based testing.
After building this framework, youâll never look at testing the same way again.