1 | # Puppeteer
|
2 |
|
3 |
|
4 | [![Linux Build Status](https://img.shields.io/travis/com/GoogleChrome/puppeteer/master.svg)](https://travis-ci.com/GoogleChrome/puppeteer) [![Windows Build Status](https://img.shields.io/appveyor/ci/aslushnikov/puppeteer/master.svg?logo=appveyor)](https://ci.appveyor.com/project/aslushnikov/puppeteer/branch/master) [![Build Status](https://api.cirrus-ci.com/github/GoogleChrome/puppeteer.svg)](https://cirrus-ci.com/github/GoogleChrome/puppeteer) [![NPM puppeteer package](https://img.shields.io/npm/v/puppeteer.svg)](https://npmjs.org/package/puppeteer) [![Issue resolution status](https://isitmaintained.com/badge/resolution/GoogleChrome/puppeteer.svg)](https://github.com/GoogleChrome/puppeteer/issues)
|
5 |
|
6 |
|
7 | <img src="https://user-images.githubusercontent.com/10379601/29446482-04f7036a-841f-11e7-9872-91d1fc2ea683.png" height="200" align="right">
|
8 |
|
9 | ###### [API](https://github.com/GoogleChrome/puppeteer/blob/v2.0.0/docs/api.md) | [FAQ](#faq) | [Contributing](https://github.com/GoogleChrome/puppeteer/blob/master/CONTRIBUTING.md) | [Troubleshooting](https://github.com/GoogleChrome/puppeteer/blob/master/docs/troubleshooting.md)
|
10 |
|
11 | > Puppeteer is a Node library which provides a high-level API to control Chrome or Chromium over the [DevTools Protocol](https://chromedevtools.github.io/devtools-protocol/). Puppeteer runs [headless](https://developers.google.com/web/updates/2017/04/headless-chrome) by default, but can be configured to run full (non-headless) Chrome or Chromium.
|
12 |
|
13 |
|
14 | ###### What can I do?
|
15 |
|
16 | Most things that you can do manually in the browser can be done using Puppeteer! Here are a few examples to get you started:
|
17 |
|
18 | * Generate screenshots and PDFs of pages.
|
19 | * Crawl a SPA (Single-Page Application) and generate pre-rendered content (i.e. "SSR" (Server-Side Rendering)).
|
20 | * Automate form submission, UI testing, keyboard input, etc.
|
21 | * Create an up-to-date, automated testing environment. Run your tests directly in the latest version of Chrome using the latest JavaScript and browser features.
|
22 | * Capture a [timeline trace](https://developers.google.com/web/tools/chrome-devtools/evaluate-performance/reference) of your site to help diagnose performance issues.
|
23 | * Test Chrome Extensions.
|
24 |
|
25 |
|
26 | Give it a spin: https://try-puppeteer.appspot.com/
|
27 |
|
28 |
|
29 | ## Getting Started
|
30 |
|
31 | ### Installation
|
32 |
|
33 | To use Puppeteer in your project, run:
|
34 |
|
35 | ```bash
|
36 | npm i puppeteer
|
37 | # or "yarn add puppeteer"
|
38 | ```
|
39 |
|
40 | Note: When you install Puppeteer, it downloads a recent version of Chromium (~170MB Mac, ~282MB Linux, ~280MB Win) that is guaranteed to work with the API. To skip the download, see [Environment variables](https://github.com/GoogleChrome/puppeteer/blob/v2.0.0/docs/api.md#environment-variables).
|
41 |
|
42 |
|
43 | ### puppeteer-core
|
44 |
|
45 | Since version 1.7.0 we publish the [`puppeteer-core`](https://www.npmjs.com/package/puppeteer-core) package,
|
46 | a version of Puppeteer that doesn't download Chromium by default.
|
47 |
|
48 | ```bash
|
49 | npm i puppeteer-core
|
50 | # or "yarn add puppeteer-core"
|
51 | ```
|
52 |
|
53 | `puppeteer-core` is intended to be a lightweight version of Puppeteer for launching an existing browser installation or for connecting to a remote one. Be sure that the version of puppeteer-core you install is compatible with the
|
54 | browser you intend to connect to.
|
55 |
|
56 | See [puppeteer vs puppeteer-core](https://github.com/GoogleChrome/puppeteer/blob/master/docs/api.md#puppeteer-vs-puppeteer-core).
|
57 |
|
58 | ### Usage
|
59 |
|
60 | Puppeteer follows the latest [maintenance LTS](https://github.com/nodejs/Release#release-schedule) version of Node.
|
61 |
|
62 | Note: Prior to v1.18.1, Puppeteer required at least Node v6.4.0. All subsequent versions rely on
|
63 | Node 8.9.0+. All examples below use async/await which is only supported in Node v7.6.0 or greater.
|
64 |
|
65 | Puppeteer will be familiar to people using other browser testing frameworks. You create an instance
|
66 | of `Browser`, open pages, and then manipulate them with [Puppeteer's API](https://github.com/GoogleChrome/puppeteer/blob/v2.0.0/docs/api.md#).
|
67 |
|
68 | **Example** - navigating to https://example.com and saving a screenshot as *example.png*:
|
69 |
|
70 | Save file as **example.js**
|
71 |
|
72 | ```js
|
73 | const puppeteer = require('puppeteer');
|
74 |
|
75 | (async () => {
|
76 | const browser = await puppeteer.launch();
|
77 | const page = await browser.newPage();
|
78 | await page.goto('https://example.com');
|
79 | await page.screenshot({path: 'example.png'});
|
80 |
|
81 | await browser.close();
|
82 | })();
|
83 | ```
|
84 |
|
85 | Execute script on the command line
|
86 |
|
87 | ```bash
|
88 | node example.js
|
89 | ```
|
90 |
|
91 | Puppeteer sets an initial page size to 800×600px, which defines the screenshot size. The page size can be customized with [`Page.setViewport()`](https://github.com/GoogleChrome/puppeteer/blob/v2.0.0/docs/api.md#pagesetviewportviewport).
|
92 |
|
93 | **Example** - create a PDF.
|
94 |
|
95 | Save file as **hn.js**
|
96 |
|
97 | ```js
|
98 | const puppeteer = require('puppeteer');
|
99 |
|
100 | (async () => {
|
101 | const browser = await puppeteer.launch();
|
102 | const page = await browser.newPage();
|
103 | await page.goto('https://news.ycombinator.com', {waitUntil: 'networkidle2'});
|
104 | await page.pdf({path: 'hn.pdf', format: 'A4'});
|
105 |
|
106 | await browser.close();
|
107 | })();
|
108 | ```
|
109 |
|
110 | Execute script on the command line
|
111 |
|
112 | ```bash
|
113 | node hn.js
|
114 | ```
|
115 |
|
116 | See [`Page.pdf()`](https://github.com/GoogleChrome/puppeteer/blob/v2.0.0/docs/api.md#pagepdfoptions) for more information about creating pdfs.
|
117 |
|
118 | **Example** - evaluate script in the context of the page
|
119 |
|
120 | Save file as **get-dimensions.js**
|
121 |
|
122 | ```js
|
123 | const puppeteer = require('puppeteer');
|
124 |
|
125 | (async () => {
|
126 | const browser = await puppeteer.launch();
|
127 | const page = await browser.newPage();
|
128 | await page.goto('https://example.com');
|
129 |
|
130 | // Get the "viewport" of the page, as reported by the page.
|
131 | const dimensions = await page.evaluate(() => {
|
132 | return {
|
133 | width: document.documentElement.clientWidth,
|
134 | height: document.documentElement.clientHeight,
|
135 | deviceScaleFactor: window.devicePixelRatio
|
136 | };
|
137 | });
|
138 |
|
139 | console.log('Dimensions:', dimensions);
|
140 |
|
141 | await browser.close();
|
142 | })();
|
143 | ```
|
144 |
|
145 | Execute script on the command line
|
146 |
|
147 | ```bash
|
148 | node get-dimensions.js
|
149 | ```
|
150 |
|
151 | See [`Page.evaluate()`](https://github.com/GoogleChrome/puppeteer/blob/v2.0.0/docs/api.md#pageevaluatepagefunction-args) for more information on `evaluate` and related methods like `evaluateOnNewDocument` and `exposeFunction`.
|
152 |
|
153 |
|
154 |
|
155 |
|
156 | ## Default runtime settings
|
157 |
|
158 | **1. Uses Headless mode**
|
159 |
|
160 | Puppeteer launches Chromium in [headless mode](https://developers.google.com/web/updates/2017/04/headless-chrome). To launch a full version of Chromium, set the [`headless` option](https://github.com/GoogleChrome/puppeteer/blob/v2.0.0/docs/api.md#puppeteerlaunchoptions) when launching a browser:
|
161 |
|
162 | ```js
|
163 | const browser = await puppeteer.launch({headless: false}); // default is true
|
164 | ```
|
165 |
|
166 | **2. Runs a bundled version of Chromium**
|
167 |
|
168 | By default, Puppeteer downloads and uses a specific version of Chromium so its API
|
169 | is guaranteed to work out of the box. To use Puppeteer with a different version of Chrome or Chromium,
|
170 | pass in the executable's path when creating a `Browser` instance:
|
171 |
|
172 | ```js
|
173 | const browser = await puppeteer.launch({executablePath: '/path/to/Chrome'});
|
174 | ```
|
175 |
|
176 | See [`Puppeteer.launch()`](https://github.com/GoogleChrome/puppeteer/blob/v2.0.0/docs/api.md#puppeteerlaunchoptions) for more information.
|
177 |
|
178 | See [`this article`](https://www.howtogeek.com/202825/what%E2%80%99s-the-difference-between-chromium-and-chrome/) for a description of the differences between Chromium and Chrome. [`This article`](https://chromium.googlesource.com/chromium/src/+/master/docs/chromium_browser_vs_google_chrome.md) describes some differences for Linux users.
|
179 |
|
180 | **3. Creates a fresh user profile**
|
181 |
|
182 | Puppeteer creates its own Chromium user profile which it **cleans up on every run**.
|
183 |
|
184 |
|
185 |
|
186 | ## Resources
|
187 |
|
188 | - [API Documentation](https://github.com/GoogleChrome/puppeteer/blob/v2.0.0/docs/api.md)
|
189 | - [Examples](https://github.com/GoogleChrome/puppeteer/tree/master/examples/)
|
190 | - [Community list of Puppeteer resources](https://github.com/transitive-bullshit/awesome-puppeteer)
|
191 |
|
192 |
|
193 |
|
194 |
|
195 | ## Debugging tips
|
196 |
|
197 | 1. Turn off headless mode - sometimes it's useful to see what the browser is
|
198 | displaying. Instead of launching in headless mode, launch a full version of
|
199 | the browser using `headless: false`:
|
200 |
|
201 | const browser = await puppeteer.launch({headless: false});
|
202 |
|
203 | 2. Slow it down - the `slowMo` option slows down Puppeteer operations by the
|
204 | specified amount of milliseconds. It's another way to help see what's going on.
|
205 |
|
206 | const browser = await puppeteer.launch({
|
207 | headless: false,
|
208 | slowMo: 250 // slow down by 250ms
|
209 | });
|
210 |
|
211 | 3. Capture console output - You can listen for the `console` event.
|
212 | This is also handy when debugging code in `page.evaluate()`:
|
213 |
|
214 | page.on('console', msg => console.log('PAGE LOG:', msg.text()));
|
215 |
|
216 | await page.evaluate(() => console.log(`url is ${location.href}`));
|
217 |
|
218 | 4. Use debugger in application code browser
|
219 |
|
220 | There are two execution context: node.js that is running test code, and the browser
|
221 | running application code being tested. This lets you debug code in the
|
222 | application code browser; ie code inside `evaluate()`.
|
223 |
|
224 | - Use `{devtools: true}` when launching Puppeteer:
|
225 |
|
226 | `const browser = await puppeteer.launch({devtools: true});`
|
227 |
|
228 | - Change default test timeout:
|
229 |
|
230 | jest: `jest.setTimeout(100000);`
|
231 |
|
232 | jasmine: `jasmine.DEFAULT_TIMEOUT_INTERVAL = 100000;`
|
233 |
|
234 | mocha: `this.timeout(100000);` (don't forget to change test to use [function and not '=>'](https://stackoverflow.com/a/23492442))
|
235 |
|
236 | - Add an evaluate statement with `debugger` inside / add `debugger` to an existing evaluate statement:
|
237 |
|
238 | `await page.evaluate(() => {debugger;});`
|
239 |
|
240 | The test will now stop executing in the above evaluate statement, and chromium will stop in debug mode.
|
241 |
|
242 | 5. Use debugger in node.js
|
243 |
|
244 | This will let you debug test code. For example, you can step over `await page.click()` in the node.js script and see the click happen in the application code browser.
|
245 |
|
246 | Note that you won't be able to run `await page.click()` in
|
247 | DevTools console due to this [Chromium bug](https://bugs.chromium.org/p/chromium/issues/detail?id=833928). So if
|
248 | you want to try something out, you have to add it to your test file.
|
249 |
|
250 | - Add `debugger;` to your test, eg:
|
251 | ```
|
252 | debugger;
|
253 | await page.click('a[target=_blank]');
|
254 | ```
|
255 | - Set `headless` to `false`
|
256 | - Run `node --inspect-brk`, eg `node --inspect-brk node_modules/.bin/jest tests`
|
257 | - In Chrome open `chrome://inspect/#devices` and click `inspect`
|
258 | - In the newly opened test browser, type `F8` to resume test execution
|
259 | - Now your `debugger` will be hit and you can debug in the test browser
|
260 |
|
261 |
|
262 | 6. Enable verbose logging - internal DevTools protocol traffic
|
263 | will be logged via the [`debug`](https://github.com/visionmedia/debug) module under the `puppeteer` namespace.
|
264 |
|
265 | # Basic verbose logging
|
266 | env DEBUG="puppeteer:*" node script.js
|
267 |
|
268 | # Protocol traffic can be rather noisy. This example filters out all Network domain messages
|
269 | env DEBUG="puppeteer:*" env DEBUG_COLORS=true node script.js 2>&1 | grep -v '"Network'
|
270 |
|
271 | 7. Debug your Puppeteer (node) code easily, using [ndb](https://github.com/GoogleChromeLabs/ndb)
|
272 |
|
273 | - `npm install -g ndb` (or even better, use [npx](https://github.com/zkat/npx)!)
|
274 |
|
275 | - add a `debugger` to your Puppeteer (node) code
|
276 |
|
277 | - add `ndb` (or `npx ndb`) before your test command. For example:
|
278 |
|
279 | `ndb jest` or `ndb mocha` (or `npx ndb jest` / `npx ndb mocha`)
|
280 |
|
281 | - debug your test inside chromium like a boss!
|
282 |
|
283 |
|
284 |
|
285 |
|
286 | ## Contributing to Puppeteer
|
287 |
|
288 | Check out [contributing guide](https://github.com/GoogleChrome/puppeteer/blob/master/CONTRIBUTING.md) to get an overview of Puppeteer development.
|
289 |
|
290 |
|
291 |
|
292 | # FAQ
|
293 |
|
294 | #### Q: Who maintains Puppeteer?
|
295 |
|
296 | The Chrome DevTools team maintains the library, but we'd love your help and expertise on the project!
|
297 | See [Contributing](https://github.com/GoogleChrome/puppeteer/blob/master/CONTRIBUTING.md).
|
298 |
|
299 | #### Q: What are Puppeteer’s goals and principles?
|
300 |
|
301 | The goals of the project are:
|
302 |
|
303 | - Provide a slim, canonical library that highlights the capabilities of the [DevTools Protocol](https://chromedevtools.github.io/devtools-protocol/).
|
304 | - Provide a reference implementation for similar testing libraries. Eventually, these other frameworks could adopt Puppeteer as their foundational layer.
|
305 | - Grow the adoption of headless/automated browser testing.
|
306 | - Help dogfood new DevTools Protocol features...and catch bugs!
|
307 | - Learn more about the pain points of automated browser testing and help fill those gaps.
|
308 |
|
309 | We adapt [Chromium principles](https://www.chromium.org/developers/core-principles) to help us drive product decisions:
|
310 | - **Speed**: Puppeteer has almost zero performance overhead over an automated page.
|
311 | - **Security**: Puppeteer operates off-process with respect to Chromium, making it safe to automate potentially malicious pages.
|
312 | - **Stability**: Puppeteer should not be flaky and should not leak memory.
|
313 | - **Simplicity**: Puppeteer provides a high-level API that’s easy to use, understand, and debug.
|
314 |
|
315 | #### Q: Is Puppeteer replacing Selenium/WebDriver?
|
316 |
|
317 | **No**. Both projects are valuable for very different reasons:
|
318 | - Selenium/WebDriver focuses on cross-browser automation; its value proposition is a single standard API that works across all major browsers.
|
319 | - Puppeteer focuses on Chromium; its value proposition is richer functionality and higher reliability.
|
320 |
|
321 | That said, you **can** use Puppeteer to run tests against Chromium, e.g. using the community-driven [jest-puppeteer](https://github.com/smooth-code/jest-puppeteer). While this probably shouldn’t be your only testing solution, it does have a few good points compared to WebDriver:
|
322 |
|
323 | - Puppeteer requires zero setup and comes bundled with the Chromium version it works best with, making it [very easy to start with](https://github.com/GoogleChrome/puppeteer/#getting-started). At the end of the day, it’s better to have a few tests running chromium-only, than no tests at all.
|
324 | - Puppeteer has event-driven architecture, which removes a lot of potential flakiness. There’s no need for evil “sleep(1000)” calls in puppeteer scripts.
|
325 | - Puppeteer runs headless by default, which makes it fast to run. Puppeteer v1.5.0 also exposes browser contexts, making it possible to efficiently parallelize test execution.
|
326 | - Puppeteer shines when it comes to debugging: flip the “headless” bit to false, add “slowMo”, and you’ll see what the browser is doing. You can even open Chrome DevTools to inspect the test environment.
|
327 |
|
328 | #### Q: Why doesn’t Puppeteer v.XXX work with Chromium v.YYY?
|
329 |
|
330 | We see Puppeteer as an **indivisible entity** with Chromium. Each version of Puppeteer bundles a specific version of Chromium – **the only** version it is guaranteed to work with.
|
331 |
|
332 | This is not an artificial constraint: A lot of work on Puppeteer is actually taking place in the Chromium repository. Here’s a typical story:
|
333 | - A Puppeteer bug is reported: https://github.com/GoogleChrome/puppeteer/issues/2709
|
334 | - It turned out this is an issue with the DevTools protocol, so we’re fixing it in Chromium: https://chromium-review.googlesource.com/c/chromium/src/+/1102154
|
335 | - Once the upstream fix is landed, we roll updated Chromium into Puppeteer: https://github.com/GoogleChrome/puppeteer/pull/2769
|
336 |
|
337 | However, oftentimes it is desirable to use Puppeteer with the official Google Chrome rather than Chromium. For this to work, you should install a `puppeteer-core` version that corresponds to the Chrome version.
|
338 |
|
339 | For example, in order to drive Chrome 71 with puppeteer-core, use `chrome-71` npm tag:
|
340 | ```bash
|
341 | npm install puppeteer-core@chrome-71
|
342 | ```
|
343 |
|
344 | #### Q: Which Chromium version does Puppeteer use?
|
345 |
|
346 | Look for `chromium_revision` in [package.json](https://github.com/GoogleChrome/puppeteer/blob/master/package.json). To find the corresponding Chromium commit and version number, search for the revision prefixed by an `r` in [OmahaProxy](https://omahaproxy.appspot.com/)'s "Find Releases" section.
|
347 |
|
348 | #### Q: What’s considered a “Navigation”?
|
349 |
|
350 | From Puppeteer’s standpoint, **“navigation” is anything that changes a page’s URL**.
|
351 | Aside from regular navigation where the browser hits the network to fetch a new document from the web server, this includes [anchor navigations](https://www.w3.org/TR/html5/single-page.html#scroll-to-fragid) and [History API](https://developer.mozilla.org/en-US/docs/Web/API/History_API) usage.
|
352 |
|
353 | With this definition of “navigation,” **Puppeteer works seamlessly with single-page applications.**
|
354 |
|
355 | #### Q: What’s the difference between a “trusted" and "untrusted" input event?
|
356 |
|
357 | In browsers, input events could be divided into two big groups: trusted vs. untrusted.
|
358 |
|
359 | - **Trusted events**: events generated by users interacting with the page, e.g. using a mouse or keyboard.
|
360 | - **Untrusted event**: events generated by Web APIs, e.g. `document.createEvent` or `element.click()` methods.
|
361 |
|
362 | Websites can distinguish between these two groups:
|
363 | - using an [`Event.isTrusted`](https://developer.mozilla.org/en-US/docs/Web/API/Event/isTrusted) event flag
|
364 | - sniffing for accompanying events. For example, every trusted `'click'` event is preceded by `'mousedown'` and `'mouseup'` events.
|
365 |
|
366 | For automation purposes it’s important to generate trusted events. **All input events generated with Puppeteer are trusted and fire proper accompanying events.** If, for some reason, one needs an untrusted event, it’s always possible to hop into a page context with `page.evaluate` and generate a fake event:
|
367 |
|
368 | ```js
|
369 | await page.evaluate(() => {
|
370 | document.querySelector('button[type=submit]').click();
|
371 | });
|
372 | ```
|
373 |
|
374 | #### Q: What features does Puppeteer not support?
|
375 |
|
376 | You may find that Puppeteer does not behave as expected when controlling pages that incorporate audio and video. (For example, [video playback/screenshots is likely to fail](https://github.com/GoogleChrome/puppeteer/issues/291).) There are two reasons for this:
|
377 |
|
378 | * Puppeteer is bundled with Chromium — not Chrome — and so by default, it inherits all of [Chromium's media-related limitations](https://www.chromium.org/audio-video). This means that Puppeteer does not support licensed formats such as AAC or H.264. (However, it is possible to force Puppeteer to use a separately-installed version Chrome instead of Chromium via the [`executablePath` option to `puppeteer.launch`](https://github.com/GoogleChrome/puppeteer/blob/v2.0.0/docs/api.md#puppeteerlaunchoptions). You should only use this configuration if you need an official release of Chrome that supports these media formats.)
|
379 | * Since Puppeteer (in all configurations) controls a desktop version of Chromium/Chrome, features that are only supported by the mobile version of Chrome are not supported. This means that Puppeteer [does not support HTTP Live Streaming (HLS)](https://caniuse.com/#feat=http-live-streaming).
|
380 |
|
381 | #### Q: I am having trouble installing / running Puppeteer in my test environment. Where should I look for help?
|
382 | We have a [troubleshooting](https://github.com/GoogleChrome/puppeteer/blob/master/docs/troubleshooting.md) guide for various operating systems that lists the required dependencies.
|
383 |
|
384 | #### Q: How do I try/test a prerelease version of Puppeteer?
|
385 |
|
386 | You can check out this repo or install the latest prerelease from npm:
|
387 |
|
388 | ```bash
|
389 | npm i --save puppeteer@next
|
390 | ```
|
391 |
|
392 | Please note that prerelease may be unstable and contain bugs.
|
393 |
|
394 | #### Q: I have more questions! Where do I ask?
|
395 |
|
396 | There are many ways to get help on Puppeteer:
|
397 | - [bugtracker](https://github.com/GoogleChrome/puppeteer/issues)
|
398 | - [Stack Overflow](https://stackoverflow.com/questions/tagged/puppeteer)
|
399 | - [slack channel](https://join.slack.com/t/puppeteer/shared_invite/enQtMzU4MjIyMDA5NTM4LWI0YTE0MjM0NWQzYmE2MTRmNjM1ZTBkN2MxNmJmNTIwNTJjMmFhOWFjMGExMDViYjk2YjU2ZmYzMmE1NmExYzc)
|
400 |
|
401 | Make sure to search these channels before posting your question.
|
402 |
|
403 |
|
404 |
|