Mercurial > lasercutter
view src/clojure/test.clj @ 10:ef7dbbd6452c
added clojure source goodness
author | Robert McIntyre <rlm@mit.edu> |
---|---|
date | Sat, 21 Aug 2010 06:25:44 -0400 |
parents | |
children |
line wrap: on
line source
1 ; Copyright (c) Rich Hickey. All rights reserved.2 ; The use and distribution terms for this software are covered by the3 ; Eclipse Public License 1.0 (http://opensource.org/licenses/eclipse-1.0.php)4 ; which can be found in the file epl-v10.html at the root of this distribution.5 ; By using this software in any fashion, you are agreeing to be bound by6 ; the terms of this license.7 ; You must not remove this notice, or any other, from this software.9 ;;; test.clj: test framework for Clojure11 ;; by Stuart Sierra12 ;; March 28, 200914 ;; Thanks to Chas Emerick, Allen Rohner, and Stuart Halloway for15 ;; contributions and suggestions.17 (ns18 ^{:author "Stuart Sierra, with contributions and suggestions by19 Chas Emerick, Allen Rohner, and Stuart Halloway",20 :doc "A unit testing framework.22 ASSERTIONS24 The core of the library is the \"is\" macro, which lets you make25 assertions of any arbitrary expression:27 (is (= 4 (+ 2 2)))28 (is (instance? Integer 256))29 (is (.startsWith \"abcde\" \"ab\"))31 You can type an \"is\" expression directly at the REPL, which will32 print a message if it fails.34 user> (is (= 5 (+ 2 2)))36 FAIL in (:1)37 expected: (= 5 (+ 2 2))38 actual: (not (= 5 4))39 false41 The \"expected:\" line shows you the original expression, and the42 \"actual:\" shows you what actually happened. In this case, it43 shows that (+ 2 2) returned 4, which is not = to 5. Finally, the44 \"false\" on the last line is the value returned from the45 expression. The \"is\" macro always returns the result of the46 inner expression.48 There are two special assertions for testing exceptions. The49 \"(is (thrown? c ...))\" form tests if an exception of class c is50 thrown:52 (is (thrown? ArithmeticException (/ 1 0)))54 \"(is (thrown-with-msg? c re ...))\" does the same thing and also55 tests that the message on the exception matches the regular56 expression re:58 (is (thrown-with-msg? ArithmeticException #\"Divide by zero\"59 (/ 1 0)))61 DOCUMENTING TESTS63 \"is\" takes an optional second argument, a string describing the64 assertion. This message will be included in the error report.66 (is (= 5 (+ 2 2)) \"Crazy arithmetic\")68 In addition, you can document groups of assertions with the69 \"testing\" macro, which takes a string followed by any number of70 assertions. The string will be included in failure reports.71 Calls to \"testing\" may be nested, and all of the strings will be72 joined together with spaces in the final report, in a style73 similar to RSpec <http://rspec.info/>75 (testing \"Arithmetic\"76 (testing \"with positive integers\"77 (is (= 4 (+ 2 2)))78 (is (= 7 (+ 3 4))))79 (testing \"with negative integers\"80 (is (= -4 (+ -2 -2)))81 (is (= -1 (+ 3 -4)))))83 Note that, unlike RSpec, the \"testing\" macro may only be used84 INSIDE a \"deftest\" or \"with-test\" form (see below).87 DEFINING TESTS89 There are two ways to define tests. The \"with-test\" macro takes90 a defn or def form as its first argument, followed by any number91 of assertions. The tests will be stored as metadata on the92 definition.94 (with-test95 (defn my-function [x y]96 (+ x y))97 (is (= 4 (my-function 2 2)))98 (is (= 7 (my-function 3 4))))100 As of Clojure SVN rev. 1221, this does not work with defmacro.101 See http://code.google.com/p/clojure/issues/detail?id=51103 The other way lets you define tests separately from the rest of104 your code, even in a different namespace:106 (deftest addition107 (is (= 4 (+ 2 2)))108 (is (= 7 (+ 3 4))))110 (deftest subtraction111 (is (= 1 (- 4 3)))112 (is (= 3 (- 7 4))))114 This creates functions named \"addition\" and \"subtraction\", which115 can be called like any other function. Therefore, tests can be116 grouped and composed, in a style similar to the test framework in117 Peter Seibel's \"Practical Common Lisp\"118 <http://www.gigamonkeys.com/book/practical-building-a-unit-test-framework.html>120 (deftest arithmetic121 (addition)122 (subtraction))124 The names of the nested tests will be joined in a list, like125 \"(arithmetic addition)\", in failure reports. You can use nested126 tests to set up a context shared by several tests.129 RUNNING TESTS131 Run tests with the function \"(run-tests namespaces...)\":133 (run-tests 'your.namespace 'some.other.namespace)135 If you don't specify any namespaces, the current namespace is136 used. To run all tests in all namespaces, use \"(run-all-tests)\".138 By default, these functions will search for all tests defined in139 a namespace and run them in an undefined order. However, if you140 are composing tests, as in the \"arithmetic\" example above, you141 probably do not want the \"addition\" and \"subtraction\" tests run142 separately. In that case, you must define a special function143 named \"test-ns-hook\" that runs your tests in the correct order:145 (defn test-ns-hook []146 (arithmetic))149 OMITTING TESTS FROM PRODUCTION CODE151 You can bind the variable \"*load-tests*\" to false when loading or152 compiling code in production. This will prevent any tests from153 being created by \"with-test\" or \"deftest\".156 FIXTURES (new)158 Fixtures allow you to run code before and after tests, to set up159 the context in which tests should be run.161 A fixture is just a function that calls another function passed as162 an argument. It looks like this:164 (defn my-fixture [f]165 Perform setup, establish bindings, whatever.166 (f) Then call the function we were passed.167 Tear-down / clean-up code here.168 )170 Fixtures are attached to namespaces in one of two ways. \"each\"171 fixtures are run repeatedly, once for each test function created172 with \"deftest\" or \"with-test\". \"each\" fixtures are useful for173 establishing a consistent before/after state for each test, like174 clearing out database tables.176 \"each\" fixtures can be attached to the current namespace like this:177 (use-fixtures :each fixture1 fixture2 ...)178 The fixture1, fixture2 are just functions like the example above.179 They can also be anonymous functions, like this:180 (use-fixtures :each (fn [f] setup... (f) cleanup...))182 The other kind of fixture, a \"once\" fixture, is only run once,183 around ALL the tests in the namespace. \"once\" fixtures are useful184 for tasks that only need to be performed once, like establishing185 database connections, or for time-consuming tasks.187 Attach \"once\" fixtures to the current namespace like this:188 (use-fixtures :once fixture1 fixture2 ...)191 SAVING TEST OUTPUT TO A FILE193 All the test reporting functions write to the var *test-out*. By194 default, this is the same as *out*, but you can rebind it to any195 PrintWriter. For example, it could be a file opened with196 clojure.java.io/writer.199 EXTENDING TEST-IS (ADVANCED)201 You can extend the behavior of the \"is\" macro by defining new202 methods for the \"assert-expr\" multimethod. These methods are203 called during expansion of the \"is\" macro, so they should return204 quoted forms to be evaluated.206 You can plug in your own test-reporting framework by rebinding207 the \"report\" function: (report event)209 The 'event' argument is a map. It will always have a :type key,210 whose value will be a keyword signaling the type of event being211 reported. Standard events with :type value of :pass, :fail, and212 :error are called when an assertion passes, fails, and throws an213 exception, respectively. In that case, the event will also have214 the following keys:216 :expected The form that was expected to be true217 :actual A form representing what actually occurred218 :message The string message given as an argument to 'is'220 The \"testing\" strings will be a list in \"*testing-contexts*\", and221 the vars being tested will be a list in \"*testing-vars*\".223 Your \"report\" function should wrap any printing calls in the224 \"with-test-out\" macro, which rebinds *out* to the current value225 of *test-out*.227 For additional event types, see the examples in the code.228 "}229 clojure.test230 (:require [clojure.template :as temp]231 [clojure.stacktrace :as stack]))233 ;; Nothing is marked "private" here, so you can rebind things to plug234 ;; in your own testing or reporting frameworks.237 ;;; USER-MODIFIABLE GLOBALS239 (defonce240 ^{:doc "True by default. If set to false, no test functions will241 be created by deftest, set-test, or with-test. Use this to omit242 tests when compiling or loading production code."243 :added "1.1"}244 *load-tests* true)246 (def247 ^{:doc "The maximum depth of stack traces to print when an Exception248 is thrown during a test. Defaults to nil, which means print the249 complete stack trace."250 :added "1.1"}251 *stack-trace-depth* nil)254 ;;; GLOBALS USED BY THE REPORTING FUNCTIONS256 (def *report-counters* nil) ; bound to a ref of a map in test-ns258 (def *initial-report-counters* ; used to initialize *report-counters*259 {:test 0, :pass 0, :fail 0, :error 0})261 (def *testing-vars* (list)) ; bound to hierarchy of vars being tested263 (def *testing-contexts* (list)) ; bound to hierarchy of "testing" strings265 (def *test-out* *out*) ; PrintWriter for test reporting output267 (defmacro with-test-out268 "Runs body with *out* bound to the value of *test-out*."269 {:added "1.1"}270 [& body]271 `(binding [*out* *test-out*]272 ~@body))274 ;;; UTILITIES FOR REPORTING FUNCTIONS276 (defn file-position277 "Returns a vector [filename line-number] for the nth call up the278 stack.280 Deprecated in 1.2: The information needed for test reporting is281 now on :file and :line keys in the result map."282 {:added "1.1"283 :deprecated "1.2"}284 [n]285 (let [^StackTraceElement s (nth (.getStackTrace (new java.lang.Throwable)) n)]286 [(.getFileName s) (.getLineNumber s)]))288 (defn testing-vars-str289 "Returns a string representation of the current test. Renders names290 in *testing-vars* as a list, then the source file and line of291 current assertion."292 {:added "1.1"}293 [m]294 (let [{:keys [file line]} m]295 (str296 ;; Uncomment to include namespace in failure report:297 ;;(ns-name (:ns (meta (first *testing-vars*)))) "/ "298 (reverse (map #(:name (meta %)) *testing-vars*))299 " (" file ":" line ")")))301 (defn testing-contexts-str302 "Returns a string representation of the current test context. Joins303 strings in *testing-contexts* with spaces."304 {:added "1.1"}305 []306 (apply str (interpose " " (reverse *testing-contexts*))))308 (defn inc-report-counter309 "Increments the named counter in *report-counters*, a ref to a map.310 Does nothing if *report-counters* is nil."311 {:added "1.1"}312 [name]313 (when *report-counters*314 (dosync (commute *report-counters* assoc name315 (inc (or (*report-counters* name) 0))))))317 ;;; TEST RESULT REPORTING319 (defmulti320 ^{:doc "Generic reporting function, may be overridden to plug in321 different report formats (e.g., TAP, JUnit). Assertions such as322 'is' call 'report' to indicate results. The argument given to323 'report' will be a map with a :type key. See the documentation at324 the top of test_is.clj for more information on the types of325 arguments for 'report'."326 :dynamic true327 :added "1.1"}328 report :type)330 (defn- file-and-line331 [exception depth]332 (let [^StackTraceElement s (nth (.getStackTrace exception) depth)]333 {:file (.getFileName s) :line (.getLineNumber s)}))335 (defn do-report336 "Add file and line information to a test result and call report.337 If you are writing a custom assert-expr method, call this function338 to pass test results to report."339 {:added "1.2"}340 [m]341 (report342 (case343 (:type m)344 :fail (merge (file-and-line (new java.lang.Throwable) 1) m)345 :error (merge (file-and-line (:actual m) 0) m)346 m)))348 (defmethod report :default [m]349 (with-test-out (prn m)))351 (defmethod report :pass [m]352 (with-test-out (inc-report-counter :pass)))354 (defmethod report :fail [m]355 (with-test-out356 (inc-report-counter :fail)357 (println "\nFAIL in" (testing-vars-str m))358 (when (seq *testing-contexts*) (println (testing-contexts-str)))359 (when-let [message (:message m)] (println message))360 (println "expected:" (pr-str (:expected m)))361 (println " actual:" (pr-str (:actual m)))))363 (defmethod report :error [m]364 (with-test-out365 (inc-report-counter :error)366 (println "\nERROR in" (testing-vars-str m))367 (when (seq *testing-contexts*) (println (testing-contexts-str)))368 (when-let [message (:message m)] (println message))369 (println "expected:" (pr-str (:expected m)))370 (print " actual: ")371 (let [actual (:actual m)]372 (if (instance? Throwable actual)373 (stack/print-cause-trace actual *stack-trace-depth*)374 (prn actual)))))376 (defmethod report :summary [m]377 (with-test-out378 (println "\nRan" (:test m) "tests containing"379 (+ (:pass m) (:fail m) (:error m)) "assertions.")380 (println (:fail m) "failures," (:error m) "errors.")))382 (defmethod report :begin-test-ns [m]383 (with-test-out384 (println "\nTesting" (ns-name (:ns m)))))386 ;; Ignore these message types:387 (defmethod report :end-test-ns [m])388 (defmethod report :begin-test-var [m])389 (defmethod report :end-test-var [m])393 ;;; UTILITIES FOR ASSERTIONS395 (defn get-possibly-unbound-var396 "Like var-get but returns nil if the var is unbound."397 {:added "1.1"}398 [v]399 (try (var-get v)400 (catch IllegalStateException e401 nil)))403 (defn function?404 "Returns true if argument is a function or a symbol that resolves to405 a function (not a macro)."406 {:added "1.1"}407 [x]408 (if (symbol? x)409 (when-let [v (resolve x)]410 (when-let [value (get-possibly-unbound-var v)]411 (and (fn? value)412 (not (:macro (meta v))))))413 (fn? x)))415 (defn assert-predicate416 "Returns generic assertion code for any functional predicate. The417 'expected' argument to 'report' will contains the original form, the418 'actual' argument will contain the form with all its sub-forms419 evaluated. If the predicate returns false, the 'actual' form will420 be wrapped in (not...)."421 {:added "1.1"}422 [msg form]423 (let [args (rest form)424 pred (first form)]425 `(let [values# (list ~@args)426 result# (apply ~pred values#)]427 (if result#428 (do-report {:type :pass, :message ~msg,429 :expected '~form, :actual (cons ~pred values#)})430 (do-report {:type :fail, :message ~msg,431 :expected '~form, :actual (list '~'not (cons '~pred values#))}))432 result#)))434 (defn assert-any435 "Returns generic assertion code for any test, including macros, Java436 method calls, or isolated symbols."437 {:added "1.1"}438 [msg form]439 `(let [value# ~form]440 (if value#441 (do-report {:type :pass, :message ~msg,442 :expected '~form, :actual value#})443 (do-report {:type :fail, :message ~msg,444 :expected '~form, :actual value#}))445 value#))449 ;;; ASSERTION METHODS451 ;; You don't call these, but you can add methods to extend the 'is'452 ;; macro. These define different kinds of tests, based on the first453 ;; symbol in the test expression.455 (defmulti assert-expr456 (fn [msg form]457 (cond458 (nil? form) :always-fail459 (seq? form) (first form)460 :else :default)))462 (defmethod assert-expr :always-fail [msg form]463 ;; nil test: always fail464 `(do-report {:type :fail, :message ~msg}))466 (defmethod assert-expr :default [msg form]467 (if (and (sequential? form) (function? (first form)))468 (assert-predicate msg form)469 (assert-any msg form)))471 (defmethod assert-expr 'instance? [msg form]472 ;; Test if x is an instance of y.473 `(let [klass# ~(nth form 1)474 object# ~(nth form 2)]475 (let [result# (instance? klass# object#)]476 (if result#477 (do-report {:type :pass, :message ~msg,478 :expected '~form, :actual (class object#)})479 (do-report {:type :fail, :message ~msg,480 :expected '~form, :actual (class object#)}))481 result#)))483 (defmethod assert-expr 'thrown? [msg form]484 ;; (is (thrown? c expr))485 ;; Asserts that evaluating expr throws an exception of class c.486 ;; Returns the exception thrown.487 (let [klass (second form)488 body (nthnext form 2)]489 `(try ~@body490 (do-report {:type :fail, :message ~msg,491 :expected '~form, :actual nil})492 (catch ~klass e#493 (do-report {:type :pass, :message ~msg,494 :expected '~form, :actual e#})495 e#))))497 (defmethod assert-expr 'thrown-with-msg? [msg form]498 ;; (is (thrown-with-msg? c re expr))499 ;; Asserts that evaluating expr throws an exception of class c.500 ;; Also asserts that the message string of the exception matches501 ;; (with re-find) the regular expression re.502 (let [klass (nth form 1)503 re (nth form 2)504 body (nthnext form 3)]505 `(try ~@body506 (do-report {:type :fail, :message ~msg, :expected '~form, :actual nil})507 (catch ~klass e#508 (let [m# (.getMessage e#)]509 (if (re-find ~re m#)510 (do-report {:type :pass, :message ~msg,511 :expected '~form, :actual e#})512 (do-report {:type :fail, :message ~msg,513 :expected '~form, :actual e#})))514 e#))))517 (defmacro try-expr518 "Used by the 'is' macro to catch unexpected exceptions.519 You don't call this."520 {:added "1.1"}521 [msg form]522 `(try ~(assert-expr msg form)523 (catch Throwable t#524 (do-report {:type :error, :message ~msg,525 :expected '~form, :actual t#}))))529 ;;; ASSERTION MACROS531 ;; You use these in your tests.533 (defmacro is534 "Generic assertion macro. 'form' is any predicate test.535 'msg' is an optional message to attach to the assertion.537 Example: (is (= 4 (+ 2 2)) \"Two plus two should be 4\")539 Special forms:541 (is (thrown? c body)) checks that an instance of c is thrown from542 body, fails if not; then returns the thing thrown.544 (is (thrown-with-msg? c re body)) checks that an instance of c is545 thrown AND that the message on the exception matches (with546 re-find) the regular expression re."547 {:added "1.1"}548 ([form] `(is ~form nil))549 ([form msg] `(try-expr ~msg ~form)))551 (defmacro are552 "Checks multiple assertions with a template expression.553 See clojure.template/do-template for an explanation of554 templates.556 Example: (are [x y] (= x y)557 2 (+ 1 1)558 4 (* 2 2))559 Expands to:560 (do (is (= 2 (+ 1 1)))561 (is (= 4 (* 2 2))))563 Note: This breaks some reporting features, such as line numbers."564 {:added "1.1"}565 [argv expr & args]566 `(temp/do-template ~argv (is ~expr) ~@args))568 (defmacro testing569 "Adds a new string to the list of testing contexts. May be nested,570 but must occur inside a test function (deftest)."571 {:added "1.1"}572 [string & body]573 `(binding [*testing-contexts* (conj *testing-contexts* ~string)]574 ~@body))578 ;;; DEFINING TESTS580 (defmacro with-test581 "Takes any definition form (that returns a Var) as the first argument.582 Remaining body goes in the :test metadata function for that Var.584 When *load-tests* is false, only evaluates the definition, ignoring585 the tests."586 {:added "1.1"}587 [definition & body]588 (if *load-tests*589 `(doto ~definition (alter-meta! assoc :test (fn [] ~@body)))590 definition))593 (defmacro deftest594 "Defines a test function with no arguments. Test functions may call595 other tests, so tests may be composed. If you compose tests, you596 should also define a function named test-ns-hook; run-tests will597 call test-ns-hook instead of testing all vars.599 Note: Actually, the test body goes in the :test metadata on the var,600 and the real function (the value of the var) calls test-var on601 itself.603 When *load-tests* is false, deftest is ignored."604 {:added "1.1"}605 [name & body]606 (when *load-tests*607 `(def ~(vary-meta name assoc :test `(fn [] ~@body))608 (fn [] (test-var (var ~name))))))610 (defmacro deftest-611 "Like deftest but creates a private var."612 {:added "1.1"}613 [name & body]614 (when *load-tests*615 `(def ~(vary-meta name assoc :test `(fn [] ~@body) :private true)616 (fn [] (test-var (var ~name))))))619 (defmacro set-test620 "Experimental.621 Sets :test metadata of the named var to a fn with the given body.622 The var must already exist. Does not modify the value of the var.624 When *load-tests* is false, set-test is ignored."625 {:added "1.1"}626 [name & body]627 (when *load-tests*628 `(alter-meta! (var ~name) assoc :test (fn [] ~@body))))632 ;;; DEFINING FIXTURES634 (defn- add-ns-meta635 "Adds elements in coll to the current namespace metadata as the636 value of key."637 {:added "1.1"}638 [key coll]639 (alter-meta! *ns* assoc key coll))641 (defmulti use-fixtures642 "Wrap test runs in a fixture function to perform setup and643 teardown. Using a fixture-type of :each wraps every test644 individually, while:once wraps the whole run in a single function."645 {:added "1.1"}646 (fn [fixture-type & args] fixture-type))648 (defmethod use-fixtures :each [fixture-type & args]649 (add-ns-meta ::each-fixtures args))651 (defmethod use-fixtures :once [fixture-type & args]652 (add-ns-meta ::once-fixtures args))654 (defn- default-fixture655 "The default, empty, fixture function. Just calls its argument."656 {:added "1.1"}657 [f]658 (f))660 (defn compose-fixtures661 "Composes two fixture functions, creating a new fixture function662 that combines their behavior."663 {:added "1.1"}664 [f1 f2]665 (fn [g] (f1 (fn [] (f2 g)))))667 (defn join-fixtures668 "Composes a collection of fixtures, in order. Always returns a valid669 fixture function, even if the collection is empty."670 {:added "1.1"}671 [fixtures]672 (reduce compose-fixtures default-fixture fixtures))677 ;;; RUNNING TESTS: LOW-LEVEL FUNCTIONS679 (defn test-var680 "If v has a function in its :test metadata, calls that function,681 with *testing-vars* bound to (conj *testing-vars* v)."682 {:dynamic true, :added "1.1"}683 [v]684 (when-let [t (:test (meta v))]685 (binding [*testing-vars* (conj *testing-vars* v)]686 (do-report {:type :begin-test-var, :var v})687 (inc-report-counter :test)688 (try (t)689 (catch Throwable e690 (do-report {:type :error, :message "Uncaught exception, not in assertion."691 :expected nil, :actual e})))692 (do-report {:type :end-test-var, :var v}))))694 (defn test-all-vars695 "Calls test-var on every var interned in the namespace, with fixtures."696 {:added "1.1"}697 [ns]698 (let [once-fixture-fn (join-fixtures (::once-fixtures (meta ns)))699 each-fixture-fn (join-fixtures (::each-fixtures (meta ns)))]700 (once-fixture-fn701 (fn []702 (doseq [v (vals (ns-interns ns))]703 (when (:test (meta v))704 (each-fixture-fn (fn [] (test-var v)))))))))706 (defn test-ns707 "If the namespace defines a function named test-ns-hook, calls that.708 Otherwise, calls test-all-vars on the namespace. 'ns' is a709 namespace object or a symbol.711 Internally binds *report-counters* to a ref initialized to712 *inital-report-counters*. Returns the final, dereferenced state of713 *report-counters*."714 {:added "1.1"}715 [ns]716 (binding [*report-counters* (ref *initial-report-counters*)]717 (let [ns-obj (the-ns ns)]718 (do-report {:type :begin-test-ns, :ns ns-obj})719 ;; If the namespace has a test-ns-hook function, call that:720 (if-let [v (find-var (symbol (str (ns-name ns-obj)) "test-ns-hook"))]721 ((var-get v))722 ;; Otherwise, just test every var in the namespace.723 (test-all-vars ns-obj))724 (do-report {:type :end-test-ns, :ns ns-obj}))725 @*report-counters*))729 ;;; RUNNING TESTS: HIGH-LEVEL FUNCTIONS731 (defn run-tests732 "Runs all tests in the given namespaces; prints results.733 Defaults to current namespace if none given. Returns a map734 summarizing test results."735 {:added "1.1"}736 ([] (run-tests *ns*))737 ([& namespaces]738 (let [summary (assoc (apply merge-with + (map test-ns namespaces))739 :type :summary)]740 (do-report summary)741 summary)))743 (defn run-all-tests744 "Runs all tests in all namespaces; prints results.745 Optional argument is a regular expression; only namespaces with746 names matching the regular expression (with re-matches) will be747 tested."748 {:added "1.1"}749 ([] (apply run-tests (all-ns)))750 ([re] (apply run-tests (filter #(re-matches re (name (ns-name %))) (all-ns)))))752 (defn successful?753 "Returns true if the given test summary indicates all tests754 were successful, false otherwise."755 {:added "1.1"}756 [summary]757 (and (zero? (:fail summary 0))758 (zero? (:error summary 0))))