lisp-unit is a Common Lisp library that supports unit testing. It is an extension of the library written by Chris Riesbeck . There is a long history of testing packages in Lisp, usually called "regression" testers. More recent packages in Lisp and other languages have been inspired by JUnit for Java. For more information on both unit testing and JUnit, visit www.junit.org.
This page has three parts:
The main goal for lisp-unit was to make it simple to use. The advantages of lisp-unit are:
(asdf:operate 'asdf:load-op :lisp-unit).(in-package :lisp-unit).run-tests.Any test failures will be printed, along with a summary of how many tests were run, how many passed, and how many failed.
You define a test with define-test:
(define-test name exp1 exp2 ...)
This defines a test called name. The expressions can be anything, but typically most will be assertion forms.
Tests can be defined before the code they test, even if they're testing macros. This is to support test-first programming.
After defining your tests and the code they test, run the tests with
(run-tests)
This runs every test defined in the current package. To run just certain specific tests, use
(run-tests name1 name2 ...)
e.g., (run-tests greater summit).
The following example
my-max returns the
larger of two argumentsmy-maxFirst, we define some tests.
> (in-package :example) #<PACKAGE EXAMPLE> > (define-test test-my-max (assert-equal 5 (my-max 2 5)) (assert-equal 5 (my-max 5 2)) (assert-equal 10 (my-max 10 10)) (assert-equal 0 (my-max -5 0))) TEST-MY-MAX
Following good test-first programming practice, we run these tests before writing any code.
> (run-tests test-my-max) TEST-MY-MAX: Undefined function MY-MAX called with arguments (2 5).
This shows that we need to do some work. So we define our broken
version of my-max.
> (defun my-max (x y) x) ;; deliberately wrong MY-MAX
Now we run the tests again:
> (run-tests my-max) MY-MAX: (MY-MAX 2 5) failed: Expected 5 but saw 2 MY-MAX: (MY-MAX -5 0) failed: Expected 0 but saw -5 MY-MAX: 2 assertions passed, 2 failed.
This shows two failures. In both cases, the equality test returned
NIL. In the first case it was because (my-max 2 5)
returned 2 when 5 was expected, and in the second case, it was
because (my-max -5 0) returned -5 when 0 was expected.
The most commonly used assertion form is
(assert-equal value form)
This tallies a failure if form returns a value not
equal to value. Both value and
test are evaluated in the local lexical environment. This
means that you can use local variables in tests. In particular, you
can write loops that run many tests at once:
> (define-test my-sqrt
(dotimes (i 5)
(assert-equal i (my-sqrt (* i i)))))
MY-SQRT
> (defun my-sqrt (n) (/ n 2)) ;; wrong!!
> (run-tests my-sqrt)
MY-SQRT: (MY-SQRT (* I I)) failed: Expected 1 but saw 1/2
MY-SQRT: (MY-SQRT (* I I)) failed: Expected 3 but saw 9/2
MY-SQRT: (MY-SQRT (* I I)) failed: Expected 4 but saw 8
MY-SQRT: 2 assertions passed, 3 failed.
However, the above output doesn't tell us for which values of
i the code failed. Fortunately, you can fix this by
adding expressions at the end of the assert-equal.
These expression and their values will be printed on failure.
> (define-test my-sqrt
(dotimes (i 5)
(assert-equal i (my-sqrt (* i i)) i))) ;; added i at the end
MY-SQRT
> (run-tests my-sqrt)
MY-SQRT: (MY-SQRT (* I I)) failed: Expected 1 but saw 1/2
I => 1
MY-SQRT: (MY-SQRT (* I I)) failed: Expected 3 but saw 9/2
I => 3
MY-SQRT: (MY-SQRT (* I I)) failed: Expected 4 but saw 8
I => 4
MY-SQRT: 2 assertions passed, 3 failed.
The next most useful assertion form is
(assert-true test)
This tallies a failure if test returns false. Again, if you need to print out extra information, just add expressions after test.
There are also assertion forms to test what code prints, what errors code returns, or what a macro expands into. A complete list of assertion forms is in the reference and extensions sections.
Do not confuse assert-true with Common Lisp's
assert macro. assert is used in code to
guarantee that some condition is true. If it isn't, the code
halts. assert has options you can use to let a user fix
what's wrong and resume execution. A similar collision of names
exists in JUnit and Java.
Tests are grouped internally by the current package, so that a set of tests can be defined for one package of code without interfering with tests for other packages.
If your code is being defined in cl-user, which is
common when learning Common Lisp, but not for production-level code,
then you should define your tests in cl-user as well.
If your code is being defined in its own package, you should define your tests either in that same package, or in another package for test code. The latter approach has the advantage of making sure that your tests have access to only the exported symbols of your code package.
For example, if you were defining a date package, your
date.lisp file would look like this:
(defpackage :date (:use :common-lisp) (:export :date->string :string->date)) (in-package :date) (defun date->string (date) ...) (defun string->date (string) ...)
Your date-tests.lisp file would look like this:
(defpackage :date-tests (:use :common-lisp :lisp-unit :date)) (in-package :date-tests) (define-test date->string (assert-true (string= ... (date->string ...))) ...) ...
You could then run all your date tests in the test package:
(in-package :date-tests) (run-tests)
Alternately, you could run all your date tests from any package with:
(lisp-unit:run-all-tests :date-tests)
The list of the basic functions and macros exported by lisp-unit are presented in this section.
*package* in effect when define-test is
executed. The expresssions are assembled into runnable
code whenever needed by run-tests. Hence you can
define or redefine macros without reloading tests using those
macros.
*package* is used.
*package* is used.
*package* is used.
nil is given, it removes all tests for all
packages.
*package* in effect when
the macro is expanded. If no names are given, all tests for
that package are run.
All of the assertion forms are macros. They tally a failure if the associated predication returns false. Assertions can be made about return values, printed output, macro expansions, and even expected errors. Assertion form arguments are evaluated in the local lexical environment.
All assertion forms allow you to include additional expressions at the end of the form. These expressions and their values will be printed only when the test fails.
Return values are unspecified for all assertion forms.
assert-equal is used for most
tests. Example use of assert-equality:
(assert-equality #'set-equal '(a b c) (unique-atoms '((b c) a ((b a) c))))
assert-true tallies a failure if test
returns false. assert-false tallies a failure if
test returns true.
(macroexpand-1
form) does not produce a value equal to
expansion.
error to refer to any
kind of error. See condition
types in the Common Lisp Hyperspec for other possible
names. For example,
(assert-error 'arithmetic-error (foo 0))would assert that
foo is supposed to signal an
arithmetic error when passed zero.
Several predicate functions are exported that are often useful in
writing tests with assert-equality.
:test can be used to specify
an equality predicate. The default is eql.
The extensions to lisp-unit are presented in this
section. The original lisp-unit has been extended
with predicate functions and assertions to support numerical
testing. The predicates can be used with
assert-equality or with the corresponding
assertions. All extensions are implemented using generic functions
and consequently can be specialized for user classes.
The internal default value of epsilon is is twice the
appropriate float epsilon (i.e. 2*single-float-epsilon
or 2*double-float-epsilon). Epsilon can be controlled
at a lexical level using the package variable
*epsilon*. If *epsilon* is set to
nil, the internal epsilon values are used. This is the
default value of epsilon.
data1 and data2
is less than epsilon. The assertion tallies the
failure if value is not equal to the result returned
from form, using float-equal.
float1 and
float2 are equal to the specified
significant-figures. The default value of significant
figures is 4, set by the global variable
*significant-figures. The test tallies the failure
if value is not equal to the result returned from
form, using sigfig-equal.
data1 and data2 is
less than epsilon.
number1
and number2 is less than
epsilon. The numbers must be the same type unless
type-eq-p is t. For the comparison, both
numbers are coerced to (complex double-float) and
passed to float-equal. The test tallies the failure
if value is not equal to the result returned from
form, using number-equal.
result1 is
equal to result2 as defined by
:test. The results can be numbers, sequences, nested
sequences and arrays. The test tallies the failure if
value is not equal to the result returned from
form, using numerical-equal. In general,
test must be a function that accepts 2 arguments and
returns true or false.
The floating point functions can be specialized for user data types and aid in writing user specific predicates.
value.