1 | # puppeteer-extra-plugin-stealth
|
2 |
|
3 | > A plugin for [puppeteer-extra](https://github.com/berstend/puppeteer-extra) to prevent detection.
|
4 |
|
5 | ### Install
|
6 |
|
7 | ```bash
|
8 | yarn add puppeteer-extra-plugin-stealth
|
9 | # - or -
|
10 | npm install puppeteer-extra-plugin-stealth
|
11 | ```
|
12 |
|
13 | ### Usage
|
14 |
|
15 | ```js
|
16 | const puppeteer = require("puppeteer-extra")
|
17 | const pluginStealth = require("puppeteer-extra-plugin-stealth")
|
18 | puppeteer.use(pluginStealth())
|
19 | ```
|
20 |
|
21 | ## Changelog
|
22 |
|
23 | ### `v2.1.2`
|
24 |
|
25 | - Improved: `navigator.plugins` - we fully emulate plugins/mimetypes in headless now 🎉
|
26 | - New: `webgl.vendor` - is otherwise set to "Google" in headless
|
27 | - New: `window.outerdimensions` - fix missing window.outerWidth/outerHeight and viewport
|
28 | - Fixed: `navigator.webdriver` now returns undefined instead of false
|
29 |
|
30 | ## Test results (red is bad)
|
31 |
|
32 | #### Vanilla puppeteer <strong>without stealth 😢</strong>
|
33 |
|
34 | <table class="image">
|
35 | <tr>
|
36 |
|
37 | <td><figure class="image"><a href="./stealthtests/_results/headless-chromium-vanilla.js.png"><img src="./stealthtests/_results/_thumbs/headless-chromium-vanilla.js.png"></a><figcaption>Chromium + headless</figcaption></figure></td>
|
38 | <td><figure class="image"><a href="./stealthtests/_results/headful-chromium-vanilla.js.png"><img src="./stealthtests/_results/_thumbs/headful-chromium-vanilla.js.png"></a><figcaption>Chromium + headful</figcaption></figure></td>
|
39 | <td><figure class="image"><a href="./stealthtests/_results/headless-chrome-vanilla.js.png"><img src="./stealthtests/_results/_thumbs/headless-chrome-vanilla.js.png"></a><figcaption>Chrome + headless</figcaption></figure></td>
|
40 | <td><figure class="image"><a href="./stealthtests/_results/headful-chrome-vanilla.js.png"><img src="./stealthtests/_results/_thumbs/headful-chrome-vanilla.js.png"></a><figcaption>Chrome + headful</figcaption></figure></td>
|
41 |
|
42 | </tr>
|
43 | </table>
|
44 |
|
45 | #### Puppeteer <strong>with stealth plugin 💯</strong>
|
46 |
|
47 | <table class="image">
|
48 | <tr>
|
49 |
|
50 | <td><figure class="image"><a href="./stealthtests/_results/headless-chromium-stealth.js.png"><img src="./stealthtests/_results/_thumbs/headless-chromium-stealth.js.png"></a><figcaption>Chromium + headless</figcaption></figure></td>
|
51 | <td><figure class="image"><a href="./stealthtests/_results/headful-chromium-stealth.js.png"><img src="./stealthtests/_results/_thumbs/headful-chromium-stealth.js.png"></a><figcaption>Chromium + headful</figcaption></figure></td>
|
52 | <td><figure class="image"><a href="./stealthtests/_results/headless-chrome-stealth.js.png"><img src="./stealthtests/_results/_thumbs/headless-chrome-stealth.js.png"></a><figcaption>Chrome + headless</figcaption></figure></td>
|
53 | <td><figure class="image"><a href="./stealthtests/_results/headful-chrome-stealth.js.png"><img src="./stealthtests/_results/_thumbs/headful-chrome-stealth.js.png"></a><figcaption>Chrome + headful</figcaption></figure></td>
|
54 |
|
55 | </tr>
|
56 | </table>
|
57 |
|
58 | Tests have been done using [this test site](https://bot.sannysoft.com/) and [these scripts](./stealthtests/).
|
59 |
|
60 | ## API
|
61 |
|
62 |
|
63 |
|
64 | #### Table of Contents
|
65 |
|
66 | - [Plugin](#plugin)
|
67 | - [Purpose](#purpose)
|
68 | - [Modularity](#modularity)
|
69 | - [Contributing](#contributing)
|
70 | - [Kudos](#kudos)
|
71 | - [availableEvasions](#availableevasions)
|
72 | - [enabledEvasions](#enabledevasions)
|
73 |
|
74 | ### [Plugin](https://git@github.com/:berstend/puppeteer-extra/blob/ff112879545e8e68d6500d731ceeafc22d187dd3/packages/puppeteer-extra-plugin-stealth/index.js#L72-L151)
|
75 |
|
76 | **Extends: PuppeteerExtraPlugin**
|
77 |
|
78 | Stealth mode: Applies various techniques to make detection of headless puppeteer harder. 💯
|
79 |
|
80 | #### Purpose
|
81 |
|
82 | There are a couple of ways the use of puppeteer can easily be detected by a target website.
|
83 | The addition of `HeadlessChrome` to the user-agent being only the most obvious one.
|
84 |
|
85 | The goal of this plugin is to be the definite companion to puppeteer to avoid
|
86 | detection, applying new techniques as they surface.
|
87 |
|
88 | As this cat & mouse game is in it's infancy and fast-paced the plugin
|
89 | is kept as flexibile as possible, to support quick testing and iterations.
|
90 |
|
91 | #### Modularity
|
92 |
|
93 | This plugin uses `puppeteer-extra`'s dependency system to only require
|
94 | code mods for evasions that have been enabled, to keep things modular and efficient.
|
95 |
|
96 | The `stealth` plugin is a convenience wrapper that requires multiple [evasion techniques](./evasions/)
|
97 | automatically and comes with defaults. You could also bypass the main module and require
|
98 | specific evasion plugins yourself, if you whish to do so (as they're standalone `puppeteer-extra` plugins):
|
99 |
|
100 | ```es6
|
101 | // bypass main module and require a specific stealth plugin directly:
|
102 | puppeteer.use(require('puppeteer-extra-plugin-stealth/evasions/console.debug')())
|
103 | ```
|
104 |
|
105 | #### Contributing
|
106 |
|
107 | PRs are welcome, if you want to add a new evasion technique I suggest you
|
108 | look at the [template](./evasions/_template) to kickstart things.
|
109 |
|
110 | #### Kudos
|
111 |
|
112 | Thanks to [Evan Sangaline](https://intoli.com/blog/not-possible-to-block-chrome-headless/) and [Paul Irish](https://github.com/paulirish/headless-cat-n-mouse) for kickstarting the discussion!
|
113 |
|
114 | * * *
|
115 |
|
116 | Type: `function (opts)`
|
117 |
|
118 | - `opts` **[Object](https://developer.mozilla.org/docs/Web/JavaScript/Reference/Global_Objects/Object)** Options (optional, default `{}`)
|
119 | - `opts.enabledEvasions` **[Set](https://developer.mozilla.org/docs/Web/JavaScript/Reference/Global_Objects/Set)<[string](https://developer.mozilla.org/docs/Web/JavaScript/Reference/Global_Objects/String)>?** Specify which evasions to use (by default all)
|
120 |
|
121 | Example:
|
122 |
|
123 | ```javascript
|
124 | const puppeteer = require('puppeteer-extra')
|
125 | // Enable stealth plugin with all evasions
|
126 | puppeteer.use(require('puppeteer-extra-plugin-stealth')())
|
127 |
|
128 |
|
129 | ;(async () => {
|
130 | // Launch the browser in headless mode and set up a page.
|
131 | const browser = await puppeteer.launch({ args: ['--no-sandbox'], headless: true })
|
132 | const page = await browser.newPage()
|
133 |
|
134 | // Navigate to the page that will perform the tests.
|
135 | const testUrl = 'https://intoli.com/blog/' +
|
136 | 'not-possible-to-block-chrome-headless/chrome-headless-test.html'
|
137 | await page.goto(testUrl)
|
138 |
|
139 | // Save a screenshot of the results.
|
140 | const screenshotPath = '/tmp/headless-test-result.png'
|
141 | await page.screenshot({path: screenshotPath})
|
142 | console.log('have a look at the screenshot:', screenshotPath)
|
143 |
|
144 | await browser.close()
|
145 | })()
|
146 | ```
|
147 |
|
148 | * * *
|
149 |
|
150 | #### [availableEvasions](https://git@github.com/:berstend/puppeteer-extra/blob/ff112879545e8e68d6500d731ceeafc22d187dd3/packages/puppeteer-extra-plugin-stealth/index.js#L124-L126)
|
151 |
|
152 | Get all available evasions.
|
153 |
|
154 | Please look into the [evasions directory](./evasions/) for an up to date list.
|
155 |
|
156 | Type: [Set](https://developer.mozilla.org/docs/Web/JavaScript/Reference/Global_Objects/Set)<[string](https://developer.mozilla.org/docs/Web/JavaScript/Reference/Global_Objects/String)>
|
157 |
|
158 | Example:
|
159 |
|
160 | ```javascript
|
161 | const pluginStealth = require('puppeteer-extra-plugin-stealth')()
|
162 | console.log(pluginStealth.availableEvasions) // => Set { 'user-agent', 'console.debug' }
|
163 | puppeteer.use(pluginStealth)
|
164 | ```
|
165 |
|
166 | * * *
|
167 |
|
168 | #### [enabledEvasions](https://git@github.com/:berstend/puppeteer-extra/blob/ff112879545e8e68d6500d731ceeafc22d187dd3/packages/puppeteer-extra-plugin-stealth/index.js#L141-L143)
|
169 |
|
170 | Get all enabled evasions.
|
171 |
|
172 | Enabled evasions can be configured either through `opts` or by modifying this property.
|
173 |
|
174 | Type: [Set](https://developer.mozilla.org/docs/Web/JavaScript/Reference/Global_Objects/Set)<[string](https://developer.mozilla.org/docs/Web/JavaScript/Reference/Global_Objects/String)>
|
175 |
|
176 | Example:
|
177 |
|
178 | ```javascript
|
179 | // Remove specific evasion from enabled ones dynamically
|
180 | const pluginStealth = require('puppeteer-extra-plugin-stealth')()
|
181 | pluginStealth.enabledEvasions.delete('console.debug')
|
182 | puppeteer.use(pluginStealth)
|
183 | ```
|
184 |
|
185 | * * *
|