Skip to content

Instantly share code, notes, and snippets.

@azl397985856
Created August 9, 2019 09:54
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save azl397985856/b9c26c4dfc6bc3c402bf3d46e53a11f5 to your computer and use it in GitHub Desktop.
Save azl397985856/b9c26c4dfc6bc3c402bf3d46e53a11f5 to your computer and use it in GitHub Desktop.
检查是否是无头浏览器

It seems he's doing something with header detection. I used Puppeteer to play around with the site and various configurations I use when scraping. In headless Chrome, the "Accept-Language" header is not sent. In Puppeteer, one can force the header to be sent by doing:

page.setExtraHTTPHeaders({ 'Accept-Language': 'en-US,en;q=0.9' })

However, Puppeteer sends that header as lowercase: accept-language: en-US,en;q=0.9

So it seems the detection is as simply: if there is no 'Accept-Language' header (case-sensitive), then "Headless Chrome"; else, "Not Headless Chrome". This is a completely server-side check, which is why he can say the fpcollect client-side javascript library isn't involved.

Here are some curl commands that demonstrate:

Detected: not headless

curl 'https://arh.antoinevastel.com/bots/areyouheadless'
-H 'User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/76.0.3803.0 Safari/537.36'
-H 'Accept-Language: en-US,en;q=0.9' Detected: headless curl 'https://arh.antoinevastel.com/bots/areyouheadless'
-H 'User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/76.0.3803.0 Safari/537.36' Detected: headless curl 'https://arh.antoinevastel.com/bots/areyouheadless'
-H 'User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/76.0.3803.0 Safari/537.36'
-H 'accept-language: en-US,en;q=0.9'

see: https://mishravikas.com/articles/2019-07/bypassing-anti-incognito-detection-google-chrome.html

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment