1 | ![](http://res.cloudinary.com/adonisjs/image/upload/v1484834197/monk_di16hz.png)
|
2 |
|
3 | # Japa
|
4 | > A test runner to create test runners
|
5 |
|
6 | Japa is a tiny **Node.js** test runner that you can use to [**test your apps**]() or even [**create your test runner**](https://github.com/thetutlage/japa/wiki/Create-your-test-runner-using-Japa).
|
7 |
|
8 | Japa is **simple**, **fast** and has **minimal core**. Japa doesn't ship with any CLI. You run your test files as standard Node.js scripts.
|
9 |
|
10 | ```js
|
11 | node test/list-users.spec.js
|
12 | ```
|
13 |
|
14 |
|
15 |
|
16 | ## Table of contents
|
17 |
|
18 | - [Features](#features)
|
19 | - [Why Japa?](#why-japa)
|
20 | - [Faster boot time ⏰](#faster-boot-time-)
|
21 | - [Simpler Syntax 💅](#simpler-syntax-)
|
22 | - [Test your apps](#test-your-apps)
|
23 | - [Installation](#installation)
|
24 | - [Writing your first test](#writing-your-first-test)
|
25 | - [async/await](#asyncawait)
|
26 | - [Test timeouts](#test-timeouts)
|
27 | - [Test groups](#test-groups)
|
28 | - [Skipping tests](#skipping-tests)
|
29 | - [Skipping/Running tests in CI](#skippingrunning-tests-in-ci)
|
30 | - [Run selected tests](#run-selected-tests)
|
31 | - [Retry flaky tests](#retry-flaky-tests)
|
32 | - [Regression tests](#regression-tests)
|
33 | - [Assertion Planning](#assertion-planning)
|
34 | - [Cleaner Error Stack](#cleaner-error-stack)
|
35 | - [Runner hooks](#runner-hooks)
|
36 | - [Japa flow](#japa-flow)
|
37 | - [Running multiple test files](#running-multiple-test-files)
|
38 | - [Filtering files](#filtering-files)
|
39 | - [Configure options](#configure-options)
|
40 | - [Running typescript tests](#running-typescript-tests)
|
41 | - [ES modules support](#es-modules-support)
|
42 | - [Coverage](#coverage)
|
43 |
|
44 |
|
45 |
|
46 | <br>
|
47 |
|
48 | ---
|
49 |
|
50 | ## Features
|
51 |
|
52 | - Supports [ES6 async/await](#asyncawait) style tests.
|
53 | - Support for ES modules.
|
54 | - Doesn't pollute the global namespace.
|
55 | - First class support for [regression](#regression-tests) tests.
|
56 | - Conditionally [skip](#skipping-tests) when running tests in CI like **travis**.
|
57 | - [Retry flaky tests](#retry-flaky-tests).
|
58 | - Define [test timeout](#test-timeouts).
|
59 | - Cleaner [error stack](#cleaner-error-stack).
|
60 | - [Test groups](#test-groups) with lifecycle hooks.
|
61 | - Inbuilt assertion library with [assertion planning](#assertion-planning).
|
62 |
|
63 | <br>
|
64 |
|
65 | ---
|
66 |
|
67 | ## Why Japa?
|
68 |
|
69 | The primary reason to use Japa is that you can [**create your test runner**](https://github.com/thetutlage/japa/wiki/Create-your-test-runner-using-Japa) using it. Which is impossible or cumbersome with other test runners like Ava or Mocha.
|
70 |
|
71 | However, Japa also shines as a standalone test runner.
|
72 |
|
73 | <br>
|
74 |
|
75 | ---
|
76 |
|
77 |
|
78 | ### Faster boot time ⏰
|
79 | Since Japa core is minimal doesn't have any CLI to run tests, it boots faster than Mocha and Ava.
|
80 |
|
81 | > The following benchmark is not a comparison to look down at Mocha or Ava, they are great test runners and offers a lot more than Japa.
|
82 |
|
83 |
|
84 | 1. First is mocha **( 0.20 seconds )**
|
85 | 2. Seconds is Ava **( 0.73 seconds )**
|
86 | 3. Third is Japa **( 0.12 seconds )**
|
87 |
|
88 | ![](https://res.cloudinary.com/adonisjs/image/upload/q_100/v1537446966/japa-boot-time_cy5crk.gif)
|
89 |
|
90 | <br>
|
91 |
|
92 | ---
|
93 |
|
94 |
|
95 | ### Simpler Syntax 💅
|
96 |
|
97 | The Japa syntax is very similar to Ava. Additionally, it allows grouping tests using the `group` method.
|
98 |
|
99 | ```js
|
100 | const test = require('japa')
|
101 |
|
102 | test('list users', () => {
|
103 | })
|
104 | ```
|
105 |
|
106 | Group tests
|
107 |
|
108 | ```js
|
109 | const test = require('japa')
|
110 |
|
111 | test.group('User', (group) => {
|
112 | group.beforeEach(() => {
|
113 | })
|
114 |
|
115 | test('list users', () => {
|
116 |
|
117 | })
|
118 | })
|
119 | ```
|
120 |
|
121 | ## Test your apps
|
122 |
|
123 | This section covers the topics to write tests for your apps using Japa. If you are looking to create a test runner, then read the [**custom test runner guides**]().
|
124 |
|
125 | <br>
|
126 |
|
127 | ---
|
128 |
|
129 | ### Installation
|
130 |
|
131 | The installation is like any other Node.js package.
|
132 |
|
133 | ```shell
|
134 | npm i --save-dev japa
|
135 |
|
136 | # yarn
|
137 | yarn add --dev japa
|
138 | ```
|
139 |
|
140 | <br>
|
141 |
|
142 | ---
|
143 |
|
144 |
|
145 | ### Writing your first test
|
146 |
|
147 | Let's start by writing the first test for a method that returns the user object.
|
148 |
|
149 | **src/getUser.js**
|
150 |
|
151 | ```js
|
152 | module.exports = function getUser () {
|
153 | return {
|
154 | username: 'virk',
|
155 | age: 28
|
156 | }
|
157 | }
|
158 | ```
|
159 |
|
160 | **test/get-user.spec.js**
|
161 |
|
162 | ```js
|
163 | const test = require('japa')
|
164 | const getUser = require('../src/getUser')
|
165 |
|
166 | test('get user with username and age', (assert) => {
|
167 | assert.deepEqual(getUser(), {
|
168 | username: 'virk',
|
169 | age: 28
|
170 | })
|
171 | })
|
172 | ```
|
173 |
|
174 | Now run the test file as follows.
|
175 |
|
176 | ```shell
|
177 | node test/get-user.spec.js
|
178 | ```
|
179 |
|
180 | <img src="assets/run-test.png" width="400px" />
|
181 |
|
182 | That's all there to learn! See you tomorrow 😝. Okay wait, let's explore all the features of Japa.
|
183 |
|
184 | <br>
|
185 |
|
186 | ---
|
187 |
|
188 | ### async/await
|
189 | Async/await one of the best ways to write async code, without spending your entire day in opening and closing curly braces.
|
190 |
|
191 | Japa has out of the box support for them.
|
192 |
|
193 | ```js
|
194 | test('get user', async () => {
|
195 | const user = await Db.getUser()
|
196 | })
|
197 | ```
|
198 |
|
199 | Also, you can return Promises from your tests and Japa will resolve them for you. If the promise rejects, the test will be marked as failed.
|
200 |
|
201 | ```js
|
202 | test('get user', () => {
|
203 | return Db.getUser()
|
204 | })
|
205 | ```
|
206 |
|
207 | <br>
|
208 |
|
209 | ---
|
210 |
|
211 | ### Test timeouts
|
212 |
|
213 | Your tests must always timeout, otherwise you may end up with a never-ending process if one of your tests gets stuck.
|
214 |
|
215 | Japa adds a timeout of `2000 milliseconds` by default to all of your tests, which is a reasonable time. However, tests which interact with a **Browser** or **3rd party API** may need a larger timeout.
|
216 |
|
217 | **Passing 0 as the timeout will disable the timeout**
|
218 |
|
219 | ```js
|
220 | test('query contributors from github', async () => {
|
221 | await gh.getContributors()
|
222 | }).timeout(6000)
|
223 | ```
|
224 |
|
225 | You can also configure the timeout for group using the `group` object.
|
226 |
|
227 | ```ts
|
228 | test.group('Test group', (group) => {
|
229 | group.timeout(6000)
|
230 |
|
231 | test('slow test', () => {
|
232 | // timeout is set 6000
|
233 | })
|
234 |
|
235 | test('fast test', () => {
|
236 | // timeout is set 2000
|
237 | }).timeout(2000)
|
238 | })
|
239 | ```
|
240 |
|
241 | <br>
|
242 |
|
243 | ---
|
244 |
|
245 | ### Test groups
|
246 |
|
247 | Grouping tests are helpful when you want to perform actions `before,` `after` the group or `beforeEach` or `afterEach` test.
|
248 |
|
249 | ```js
|
250 | const test = require('japa')
|
251 |
|
252 | test.group('Group name', (group) => {
|
253 |
|
254 | group.before(async () => {
|
255 | })
|
256 |
|
257 | group.beforeEach(async () => {
|
258 | })
|
259 |
|
260 | group.after(async () => {
|
261 | })
|
262 |
|
263 | group.afterEach(async () => {
|
264 | })
|
265 |
|
266 | })
|
267 | ```
|
268 |
|
269 | - The `before` hook is executed only once before all the tests.
|
270 | - The `after` hook is executed only once after all the tests.
|
271 | - The `beforeEach` hook is executed before every test.
|
272 | - The `afterEach` hook is executed after every test.
|
273 |
|
274 | <br>
|
275 |
|
276 | ---
|
277 |
|
278 | ### Skipping tests
|
279 |
|
280 | When refactoring code bases, you may break a bunch of existing functionality causing existing tests to fail.
|
281 |
|
282 | Instead of removing those tests, it's better to skip them and once your codebase gets stable, re-run them to make sure everything is working fine.
|
283 |
|
284 | ```js
|
285 | test.skip('I exists, but will be skipped', () => {
|
286 |
|
287 | })
|
288 | ```
|
289 |
|
290 | The default `list` reporter will show skipped tests in yellow.
|
291 |
|
292 | <img src="assets/skipping-tests.png" width="400px" />
|
293 |
|
294 | <br>
|
295 |
|
296 | ---
|
297 |
|
298 | ### Skipping/Running tests in CI
|
299 |
|
300 | Some projects are highly dependent on the execution environment. For example, A test of yours depends on an `API_KEY` which cannot be shared with everyone in the company. In this case, you can save the key with a CI like Travis and only run dependent tests in CI and not on local machine.
|
301 |
|
302 | Japa supports this behavior as a first class citizen using `runInCI` method.
|
303 |
|
304 | ```js
|
305 | test.runInCI('I will execute in CI only', () => {
|
306 | })
|
307 | ```
|
308 |
|
309 | The opposite of same is also available.
|
310 |
|
311 | ```js
|
312 | test.skipInCI('I will be skipped in CI', () => {
|
313 | })
|
314 | ```
|
315 |
|
316 | <br>
|
317 |
|
318 | ---
|
319 |
|
320 | ### Run selected tests
|
321 | Just like skipping tests, you can also run a specific test using `test.only` method.
|
322 |
|
323 | - If multiple tests uses `test.only`, then only the last one will be entertained.
|
324 |
|
325 | ```js
|
326 | test.only('all other tests will be skipped, except me', () => {
|
327 | })
|
328 | ```
|
329 |
|
330 | <br>
|
331 |
|
332 | ---
|
333 |
|
334 | ### Retry flaky tests
|
335 |
|
336 | Flaky tests are those, which needs a couple of retries before they can pass. Japa exposes a helpful method to retry the same test (until it passes) before marking it as failed.
|
337 |
|
338 | **Following will be executed four times in total. `3 retries + 1 actual`.**
|
339 |
|
340 | ```js
|
341 | test('I am flaky attimes', () => {
|
342 |
|
343 | }).retry(3)
|
344 | ```
|
345 |
|
346 | <br>
|
347 |
|
348 | ---
|
349 |
|
350 |
|
351 | ### Regression tests
|
352 |
|
353 | The best way to accept code contributions is to ask people to write failing tests for the bugs. Later, you fix that bug and keep the test in its place.
|
354 |
|
355 | When creating a regression that, you are telling `japa`, **I want this test to fail** and when it fails, Japa marks it passed to keep the whole tests suite green.
|
356 |
|
357 | ```js
|
358 | test.failing('user must be verified when logging', async (assert) => {
|
359 | const user = await Db.getUser()
|
360 | user.verified = false
|
361 |
|
362 | const loggedIn = await auth.login(user)
|
363 | assert.isFalse(loggedIn, 'Expected logged in to be false for unverified user')
|
364 | })
|
365 | ```
|
366 |
|
367 | Now if `login` returns true, Japa will mark this test as passed and will print the assertion note.
|
368 |
|
369 | <img src="assets/regression-test.png" width="600px" />
|
370 |
|
371 | When you fix this behavior, remove `.failing` and everything should be green.
|
372 |
|
373 | <br>
|
374 |
|
375 | ---
|
376 |
|
377 |
|
378 | ### Assertion Planning
|
379 |
|
380 | Have you ever wrote one of those `try/catch` tests, in which you want the test to throw an exception?
|
381 |
|
382 | ```js
|
383 | test('raise error when user is null', async (assert) => {
|
384 | try {
|
385 | await auth.login(null)
|
386 | } catch ({ message }) {
|
387 | assert.equal(message, 'Please provide a user')
|
388 | }
|
389 | })
|
390 | ```
|
391 |
|
392 | Now, what if `auth.login` never throws an exception? The test will still be green since the `catch` block was never executed.
|
393 |
|
394 | This is one of those things, which even bites seasoned programmers. To prevent this behavior, Japa asks you to plan assertions.
|
395 |
|
396 | ```js
|
397 | test('raise error when user is null', async (assert) => {
|
398 | assert.plan(1)
|
399 |
|
400 | try {
|
401 | await auth.login(null)
|
402 | } catch ({ message }) {
|
403 | assert.equal(message, 'Please provide a user')
|
404 | }
|
405 | })
|
406 | ```
|
407 |
|
408 | Now, if `catch` block is never executed, Japa will complain that you planned for `1` assertion, but never ran any.
|
409 |
|
410 | <img src="assets/assertion-planning.png" width="400px" />
|
411 |
|
412 | <br>
|
413 |
|
414 | ---
|
415 |
|
416 |
|
417 | ### Cleaner Error Stack
|
418 |
|
419 | The default `list` reporter, will clean the stack traces by removing Japa core from it. It helps you in tracing the errors quickly.
|
420 |
|
421 | <img src="assets/stack-traces.png" width="600px" />
|
422 |
|
423 | <br>
|
424 |
|
425 | ---
|
426 |
|
427 | ## Runner hooks
|
428 | Runner hooks are executed before and after running the entire test suite. These hooks can be used to perform global actions, which are required for all test groups.
|
429 |
|
430 | The `before` and `after` hooks can be defined inside the configure object.
|
431 |
|
432 | ```js
|
433 | const { configure } = require('japa')
|
434 |
|
435 | configure({
|
436 | before: [
|
437 | async (runner) => {
|
438 | // setup db
|
439 | }
|
440 | ],
|
441 | after: [
|
442 | async (runner) => {
|
443 | // cleanup db
|
444 | }
|
445 | ]
|
446 | })
|
447 | ```
|
448 |
|
449 | <br>
|
450 |
|
451 | ---
|
452 |
|
453 | ## Japa flow
|
454 |
|
455 | Japa will attempt to run as many tests as possible when tests or group hooks start failing.
|
456 |
|
457 | - If one of the group **fails**, it will not impact the other groups.
|
458 | - If one of the test **fails**, it will not affect other tests.
|
459 |
|
460 | However, if `lifecycle hooks` will fail, they will exit early and will mark the entire group as failed. This behavior is done intentionally since if `beforeEach` hook is failing, then tests will get unstable, and there is no point in running them.
|
461 |
|
462 | Check out the following flowchart to understand it better.
|
463 |
|
464 | <img src="assets/japa-flow-chart.png" width="400px">
|
465 |
|
466 | <br>
|
467 |
|
468 | ---
|
469 |
|
470 | ## Running multiple test files
|
471 | Sooner or later, you will have multiple tests files, that you would like to run together, instead of running one file at a time. Doing same is very simple and is achieved using a **master test file**.
|
472 |
|
473 | Let's create a master test file called `japaFile.js` in the project root and write following code inside it.
|
474 |
|
475 | ```js
|
476 | const { configure } = require('japa')
|
477 | configure({
|
478 | files: ['test/*.spec.js']
|
479 | })
|
480 | ```
|
481 |
|
482 | Now execute the file using the node command `node japaFile.js`.
|
483 |
|
484 | ### Filtering files
|
485 | Not only you can define the glob, you can also filter the test files.
|
486 |
|
487 | ```js
|
488 | const { configure } = require('japa')
|
489 |
|
490 | configure({
|
491 | filter (filePath) {
|
492 | // return true for file(s) that you want to load
|
493 | },
|
494 | files: ['test/*.spec.js']
|
495 | })
|
496 | ```
|
497 |
|
498 | Also, you can define files to ignore with the `files` property. The files property is 100% compatible with [fast-glob](https://www.npmjs.com/package/fast-glob).
|
499 |
|
500 | ```js
|
501 | const { configure } = require('japa')
|
502 | configure({
|
503 | files: ['test/*.spec.js', '!test/users.spec.js']
|
504 | })
|
505 | ```
|
506 |
|
507 | <br>
|
508 |
|
509 | ---
|
510 |
|
511 | ## Configure options
|
512 | Here's the list of the options the `configure` method accepts.
|
513 |
|
514 | <table>
|
515 | <tr>
|
516 | <td colspan="2"><code>{</code></td>
|
517 | </tr>
|
518 | <tr>
|
519 | <td valign="top"><code>timeout:</code></td>
|
520 | <td>
|
521 | <p>The global timeout for all the tests. However, you can override this value by defining explicit timeout of the group of the test.</p>
|
522 | </td>
|
523 | </tr>
|
524 | <tr>
|
525 | <td valign="top"><code>bail:</code></td>
|
526 | <td>
|
527 | <p>If value of <code>bail = true</code>, then the first failing test will exit the process.</p>
|
528 | </td>
|
529 | </tr>
|
530 | <tr>
|
531 | <td valign="top"><code>grep:</code></td>
|
532 | <td>
|
533 | <p>Define a string or regex to match the test titles and only run matching tests.</p>
|
534 | </td>
|
535 | </tr>
|
536 | <tr>
|
537 | <td valign="top"><code>files:</code></td>
|
538 | <td>
|
539 | <p>An array glob patterns to load test files from.</p>
|
540 | </td>
|
541 | </tr>
|
542 | <tr>
|
543 | <td valign="top"><code>filter:</code></td>
|
544 | <td>
|
545 | <p>A custom callback to dynamically filter tests.</p>
|
546 | </td>
|
547 | </tr>
|
548 | <tr>
|
549 | <td valign="top"><code>before:</code></td>
|
550 | <td>
|
551 | <p>An array of hooks to be executed before running all the tests. <strong>Hooks are executed in sequence</strong></p>
|
552 | </td>
|
553 | </tr>
|
554 | <tr>
|
555 | <td valign="top"><code>after
|
556 | :</code></td>
|
557 | <td>
|
558 | <p>An array of hooks to be executed after running all the tests. <strong>Hooks are executed in sequence</strong></p>
|
559 | </td>
|
560 | </tr>
|
561 | <tr>
|
562 | <td colspan="2"><code>}</code></td>
|
563 | </tr>
|
564 | </table>
|
565 |
|
566 | <br>
|
567 |
|
568 | ---
|
569 |
|
570 | ## Running typescript tests
|
571 | Running test files written in `Typescript` is a piece of cake for Japa. Since everything is done inside the Javascript files, we can ask `japaFile.js` to load `ts-node`.
|
572 |
|
573 | ```js
|
574 | // Load ts-node
|
575 | require('ts-node').register()
|
576 |
|
577 | const { configure } = require('japa')
|
578 | configure({
|
579 | files: ['test/*.ts']
|
580 | })
|
581 | ```
|
582 |
|
583 | ## ES modules support
|
584 | Japa automatically imports the ES modules with the `.mjs` extension. However, for files with `.js` extension, you will have to explicitly tell japa to import test files as ES modules and not CJS.
|
585 |
|
586 | ```ts
|
587 | const { configure } = require('japa')
|
588 | configure({
|
589 | files: ['test/*.js'],
|
590 | experimentalEsmSupport: true,
|
591 | })
|
592 | ```
|
593 |
|
594 | ## Coverage
|
595 | You can use [nyc](https://www.npmjs.com/package/nyc) to check the coverage of your code. It works for Typescript files too.
|
596 |
|
597 | ```
|
598 | npm install -D nyc
|
599 | ```
|
600 |
|
601 | Following is a screenshot from one of my projects.
|
602 |
|
603 | <img width="789" alt="Coverage image with NYC" src="/assets/coverage.png">
|
604 |
|
605 | Then, in your package.json file, just add:
|
606 |
|
607 | ```json
|
608 | {
|
609 | "scripts": {
|
610 | "test": "node japaFile.js",
|
611 | "coverage": "nyc npm run test"
|
612 | }
|
613 | }
|
614 | ```
|