1 | # node-tap
|
2 |
|
3 | A <abbr title="Test Anything Protocol">TAP</abbr> test framework
|
4 | for Node.js.
|
5 |
|
6 | _Just wanna see some code? [Get
|
7 | started!](http://www.node-tap.org/basics/)_
|
8 |
|
9 | It includes a command line test runner for consuming
|
10 | TAP-generating test scripts, and a JavaScript framework for
|
11 | writing such scripts.
|
12 |
|
13 |
|
14 |
|
15 | - [Getting started guide](http://node-tap.org/basics/)
|
16 | - Built-in [test coverage](http://node-tap.org/coverage/)
|
17 | - Many [reporter formats](http://node-tap.org/reporter/)
|
18 | - Extensive [API](http://node-tap.org/api/)
|
19 | - [Command-line interface](http://node-tap.org/cli/) for
|
20 | running tests (whether they use node-tap or not)
|
21 | - [Machine-generated API docs](https://tapjs.github.io/tapjs)
|
22 |
|
23 | See [the changelog](http://node-tap.org/changelog/) for recent
|
24 | updates, or just get started with [the
|
25 | basics](http://www.node-tap.org/basics/).
|
26 |
|
27 | All this is too much to manage in a single README file, so head
|
28 | over to [the website](http://node-tap.org/) to learn more.
|
29 |
|
30 | ## Why TAP?
|
31 |
|
32 | Why should you use this thing!? **LET ME TELL YOU!**
|
33 |
|
34 | Just kidding.
|
35 |
|
36 | Most frameworks spend a lot of their documentation telling you
|
37 | why they're the greatest. This isn't that.
|
38 |
|
39 | ### <i lang="it" title="all tastes are tastes">tutti i gusti sono gusti</i>
|
40 |
|
41 | Software testing is a software and user experience design
|
42 | challenge that balances on the intersection of many conflicting
|
43 | demands.
|
44 |
|
45 | Node-tap is based on [my](http://izs.me) opinions about how a
|
46 | test framework should work, and what it should let you do. I do
|
47 | _not_ have any opinion about whether or not you share those
|
48 | opinions. If you do share them, you will probably enjoy this test
|
49 | library.
|
50 |
|
51 | Here are the design principles that shape this test framework.
|
52 |
|
53 | ### Test files are "normal" programs
|
54 |
|
55 | Any TAP test can be run directly as a plain old JavaScript
|
56 | program. Of course, if it's written in TypeScript, you'll
|
57 | have to run it with a TypeScript loader, but otherwise, they
|
58 | should be just like normal programs that run in a normal
|
59 | environment.
|
60 |
|
61 | But there's no runner required to run tests, they don't
|
62 | execute in a special simulated memory space with injected
|
63 | globals, and so on. Because each test runs in its own process,
|
64 | there's no chance of tests becoming dependent on one another's
|
65 | leaked globals or causing other confusing situations.
|
66 |
|
67 | ### Tests should help, not get in the way
|
68 |
|
69 | The goal of tests is to help you write code. They add reliability
|
70 | to your program by adding a layer of "yes, this does what I think
|
71 | it does". Whether you're doing strict Red-Green-Refactor style
|
72 | TDD, or just finger-painting until it feels right and then
|
73 | writing tests to verify what it actually does, writing the tests
|
74 | should feel empowering and straightforward, _reducing_ cognitive
|
75 | load rather than increasing it.
|
76 |
|
77 | ### All types must be accurate and complete
|
78 |
|
79 | This is simply not reasonable to do with a hand-edited type
|
80 | definition in `.d.ts` file.
|
81 |
|
82 | TAP's exported types are built up from its set of plugins and
|
83 | internal classes, assembled into the `Test` class that your test
|
84 | programs interact with. When a plugin is added or removed, the
|
85 | `t` in your editor can accurately tell you its new shape.
|
86 |
|
87 | If you have to look at the docs too often, that's a bug in my
|
88 | opinion. Lean into the beautiful power of code completion.
|
89 |
|
90 | ### TypeScript, ESM, and CommonJS supported out of the box
|
91 |
|
92 | With the changes to the module system in Node.js over the
|
93 | last several years, TAP fell down on this requirement in
|
94 | versions prior to v18. As of version 18, the entire system has
|
95 | been rewritten in TypeScript, and built as hybrid ESM/CommonJS
|
96 | packages.
|
97 |
|
98 | Your tests should be written just like your program, with as few
|
99 | barriers as possible. If you can do it in CommonJS, you can do it
|
100 | in ESM, and vice versa (at least as far as TAP is concerned).
|
101 | Whatever is in your `tsconfig.json` or `package.json`, it should
|
102 | Just Work.
|
103 |
|
104 | ### Anything that _can_ be a plugin _is_ a plugin
|
105 |
|
106 | The plugin system is leveraged for anything that does not
|
107 | absolutely need to be included in the core.
|
108 |
|
109 | Basic [TAP](https://testanything.org) generation and flow
|
110 | control, error handling, config loading, process management and
|
111 | so on, are all included in the core. But TypeScript support,
|
112 | mocking, almost all assertion methods, method and property
|
113 | interception and spying, spawning/forking subtests, creating
|
114 | fixtures, snapshots, and attaching lifecycle methods (among
|
115 | others) are all relegated to plugins.
|
116 |
|
117 | This means that features can be switched on or off or extended
|
118 | very easily.
|
119 |
|
120 | ### Plugins must be powerful and trivial to write correctly
|
121 |
|
122 | The plugin interface is extremely simple. Export a `plugin`
|
123 | function that returns an object. That's it, that's a plugin.
|
124 |
|
125 | Plugins can also export configuration definitions, which are
|
126 | folded into the set of fields that TAP knows how to parse from
|
127 | the command line or from your `.taprc` file, or export a `loader`
|
128 | string, which will be invoked when spawning test processes.
|
129 |
|
130 | ### High Signal, Low Noise
|
131 |
|
132 | It is important to give a lot of information about test failures,
|
133 | throws, and so on, so that you can easily jump straight to the
|
134 | appropriate place in the code to fix the problem. And, it's
|
135 | usually helpful to see which tests are actually running.
|
136 |
|
137 | However, a screen full of green checkmarks and `100% Covered!`
|
138 | isn't very useful. It should be just enough to know what happened
|
139 | and easily diagnose any problems, and otherwise fairly quiet.
|
140 |
|
141 | Low information output has been trimmed down as much as possible
|
142 | from the default reporters. Coverage information is only shown
|
143 | when it has something relevant to say. TAP tries to show you
|
144 | exactly what you need to see, and nothing else. Stack traces have
|
145 | noisy internals trimmed out, so it's easier to see exactly where
|
146 | in _your_ code the problem happened. Source maps are always
|
147 | enabled, because you need to know where the actual code is, not
|
148 | just which built artifact failed.
|
149 |
|
150 | If the default reporter isn't terse enough for your liking, try
|
151 | `tap -Rterse`.
|
152 |
|
153 | ### Assertions don't throw (but throws are handled nicely)
|
154 |
|
155 | I frequently write programs that have many hundreds of assertions
|
156 | based on some list of test cases. If the first failure throws,
|
157 | then I don't know if I've failed 100 tests or 1, without wrapping
|
158 | everything in a try-catch. Furthermore, I usually want to see
|
159 | some kind of output or reporting to verify that each one actually
|
160 | ran.
|
161 |
|
162 | Basically, it should be your decision whether you want to throw
|
163 | or not. The test framework shouldn't force that on you, and
|
164 | should make either case easy.
|
165 |
|
166 | ### Test reporting should be useful, extensible, and accessible
|
167 |
|
168 | The [raw test output](https://www.node-tap.org/tap-format/)
|
169 | is machine-parseable and human-intelligible, a separate component
|
170 | consumes test output and turns it into a [pretty summarized
|
171 | report](https://www.node-tap.org/reporting/). This means that
|
172 | test data can be stored and parsed later, dug into for additional
|
173 | details, and so on.
|
174 |
|
175 | Red and green are the conventional colors meaning "removed" and
|
176 | "added", but they're also exactly the same color for many people.
|
177 | All of the color choices in the reporter are tested rigorously
|
178 | against simulators for protanopia, deuteranopia, tritanopia, and
|
179 | monochromicity.
|
180 |
|
181 | ### Test coverage is always on
|
182 |
|
183 | Running tests with coverage changes the way that you think
|
184 | about your programs, and provides much deeper insight.
|
185 | TAP uses V8's internal coverage mechanisms directly, and verifies
|
186 | that tests provide 100% coverage of all lines, branches,
|
187 | functions, and statements in the system under test. It uses
|
188 | [C8](https://npmjs.com/c8) to analyze the V8 coverage data and
|
189 | generate coverage reports.
|
190 |
|
191 | Missing coverage means that you are relying on untested code, so
|
192 | this is treated as a test failure. If you have some bit of code
|
193 | which is actually _impossible_ to test for some reason, wrap it
|
194 | in the appropriate `/* c8 ignore start */` / `/* c8 ignore end
|
195 | */` comments to exclude those lines from the analysis. But think
|
196 | carefully about whether that's really the case. Usually, if you
|
197 | have to coverage-ignore something, it's a sign that you need to
|
198 | either delete that code or refactor it into a more easily tested
|
199 | module.
|
200 |
|
201 | -----
|
202 |
|
203 | Software testing should help you build software. It should be a
|
204 | security blanket and a quality ratchet, giving you the support to
|
205 | undertake massive refactoring and fix bugs without worrying. It
|
206 | shouldn't be a purification rite or a hazing ritual. It should be
|
207 | fun, because making stuff is fun, and it helps you make better
|
208 | stuff.
|
209 |
|
210 | There are many opinions left off of this list! Reasonable people
|
211 | can disagree. But if you find yourself nodding along, [maybe tap
|
212 | is for you](https://www.node-tap.org/basics/).
|