from peckcheck import TestCase, an_int, main
class TestArithmetic(TestCase):
def testAddCommutes(self, x=an_int, y=an_int):
assert x + y == y + x
def testAddAssociates(self, x=an_int, y=an_int, z=an_int):
assert x + (y + z) == (x + y) + z
if __name__ == '__main__':
main()
You can create a test-data generator of your own by defining a function with one parameter, the size bound. For example:
def a_weekday(size):
import random
return random.choice(['Sun','Mon','Tue','Wed','Thu','Fri','Sat'])
CL-USER> (load "clickcheck.lisp")
CL-USER> (use-package :click-check)
CL-USER> (click-check (is= (+ 2 3) 5))
Starting tests with seed
#S(RANDOM-STATE
#*0110000000100101111101000101000111001100101100111001010111011101)
.
1 test submitted; all passed.
T
Why display the seed? For repeatability when we run random test cases:
CL-USER> (click-check (for-all ((n an-integer))
(is= (* 2 n) (+ n n))))
Starting tests with seed
#S(RANDOM-STATE
#*0110000000100101111101000101000111001100101100111001010111011101)
...............................................................................
.....................
1 test submitted; all passed.
T
The line of dots shows a bunch of successful test cases, one dot for each. The summary line "1 test submitted" means the same single test was done on all those random values of N. We could have done more tests:
CL-USER> (click-check (for-all ((n an-integer))
(is= (* 2 n) (+ n n))
(is= (* 3 n) (+ n n n)))
(for-all ((m an-integer) (n an-integer))
(is= (+ m n) (+ n m))))
Starting tests with seed
#S(RANDOM-STATE
#*1101011110111110101110111111111010110000111011101101010000100110)
...............................................................................
...............................................................................
...............................................................................
...............................................................
3 tests submitted; all passed.
T
In general, (CLICK-CHECK expression...) evaluates the argument
expressions and reports on any tests performed in that dynamic extent.
This means you can put the tests off in functions or files or
wherever; typical usage is then like (click-check (my-test-suite)) or
(click-check (load "tests1.lisp") (load "tests2.lisp")).
Let's see what failure looks like:
CL-USER> (click-check (for-all ((n an-integer))
(test (evenp n))))
Starting tests with seed
#S(RANDOM-STATE
#*1011111110011110110110101000010101100100101110101010000011111001)
.X
FAIL (TEST (EVENP N))
for ((N -3))
1/2 counterexamples.
1 test submitted; 1 FAILED.
NIL
Now along with the dot for a passing test case, there's an X for a
failing one. Testing continues after a failure unless
*BREAK-ON-FAILURE* is true; the reason this test run stopped early was
because FOR-ALL exits early once all the tests in its body have
failed, on the grounds that finding more counterexamples is probably a
waste of effort.
You can define test-data generators for your own types of data:
CL-USER> (defparameter a-color
(lambda ()
(pick-weighted
(1 'red)
(2 'green)
(1 'blue))))
A-COLOR
CL-USER> (click-check (for-all ((c a-color))
(test (symbolp c))))
Starting tests with seed
#S(RANDOM-STATE
#*1111010101010001100010110010000111111000110000001110000100100111)
...............................................................................
.....................
1 test submitted; all passed.
T
So far we've been running the tests all together in one sequence of
expressions, with any side effects in one test potentially interfering
with the next. Much of the time this is fine, since Lisp encourages a
mostly-functional style; but for the sake of other styles we can
isolate tests using WRAP-EACH:
CL-USER> (defun x-tests ()
(wrap-each (let ((x (set-up-an-x)))
(unwind-protect WRAPPEE
(tear-down-an-x x)))
(test (foo x))
(test (bar x))))
which works as if we'd written
CL-USER> (defun x-tests ()
(let ((x (set-up-an-x)))
(unwind-protect (test (foo x))
(tear-down-an-x x)))
(let ((x (set-up-an-x)))
(unwind-protect (test (bar x))
(tear-down-an-x x))))
That's the gist of this package; the manual and examples cover more features, conveniences, and details.
(for-all ((x <set>) (y <set>)) (set-equal (union x y) (union y x)))
That was as far as I got; QuickCheck had the same basic idea and added conditional laws, histograms of test data distribution, and randomly generated functions. Plus, they published! Meanwhile I just drifted away from this out of too much sloth to maintain fine-grained test suites in my free time. Now that Extreme Programming has gotten so popular I figure it's past time to dust this stuff off and give it another go, trying to build on the work others have done.