Skip to content

Instantly share code, notes, and snippets.

@addyosmani
Last active July 18, 2023 18:47
Show Gist options
  • Save addyosmani/5336747 to your computer and use it in GitHub Desktop.
Save addyosmani/5336747 to your computer and use it in GitHub Desktop.
So, you want to run Chrome headless.

Update May 2017

Eric Bidelman has documented some of the common workflows possible with headless Chrome over in https://developers.google.com/web/updates/2017/04/headless-chrome.

Update

If you're looking at this in 2016 and beyond, I strongly recommend investigating real headless Chrome: https://chromium.googlesource.com/chromium/src/+/lkgr/headless/README.md

Windows and Mac users might find using Justin Ribeiro's Docker setup useful here while full support for these platforms is being worked out.

You can use chrome --headless on Linux as of M57 but note you'll need to build the binaries yourself for now.

The metabug for adding headless mode to Chromium is over here.

Otherwise, if you're on Ubuntu or working with a Linux VM that isn't working with Linux:

Step 1: Install the Ubuntu dependencies needed:

SSH into your server as a root or do sudo -i.

Then install necessary software:

apt-get update
apt-get install -y xvfb fluxbox x11vnc dbus libasound2 libqt4-dbus libqt4-network libqtcore4 libqtgui4 libxss1 libpython2.7 libqt4-xml libaudio2 libmng1 fontconfig liblcms1 lib32stdc++6 lib32asound2 ia32-libs libc6-i386 lib32gcc1 nano
apt-get install -y python-gobject-2
apt-get install -y curl git

Step 2. Options for starting:

Normal:

xvfb-run --server-args='-screen 0, 1024x768x16' google-chrome -start-maximized > /dev/null &

Start and open a page:

xvfb-run --server-args='-screen 0, 1024x768x16' google-chrome -start-maximized http://www.example.com > /dev/null &

If you're on Windows..

Use Desktops from the MS SysInternals package. You can spawn your Chrome process directly from the "hidden" desktop and keep it working there (I have not personally tested this).

#Other options...

You could try spawning Chrome as a child process using something like Node.js and piping the output back to the terminal. See runChrome.js for an example of how to do this (todo: update to latest Express).

var express = require('express');
var http = require('http');
var cp = require('child_process');
var port = 3000;
var app = express();
//var app = express();
//var server = http.createServer(app);
var chrome_count = 0;
app.get('/', function(req, res){
startChrome(chrome_count++,9222,function chromeStared(err,id,chrome){
res.send('Chrome '+id+' Started\r\n');
console.log('Simulating some data parsing');
setTimeout(function(){
console.log('Shutdown chrome '+id);
chrome.kill('doh!');
},1000);
});
});
app.listen(port);
console.log('Server is listening on port ' + port);
function startChrome(id,port,callback){
var terminal = cp.spawn('bash');
var chrome = {};
terminal.on('exit', function (code) {
console.log('Starting chrome');
chrome = cp.spawn('/Applications/Google\ Chrome.app/Contents/MacOS/Google\ Chrome',[
'--remote-debugging-port='+port,
'--user-data-dir=/Volumes/DATA/repos/scrapper/userdata',
'http://www.chrome.com'
]);
callback(null,id,chrome);
});
setTimeout(function() {
console.log('Sending stdin to terminal');
//terminal.stdin.write('echo "Hello $USER"');
terminal.stdin.write('rm -rf /Volumes/DATA/repos/scrapper/userdata'+'\n');
terminal.stdin.write('mkdir /Volumes/DATA/repos/scrapper/userdata'+'\n');
terminal.stdin.write('touch "/Volumes/DATA/repos/scrapper/userdata/First Run"'+'\n');
terminal.stdin.write('chmod 777 "/Volumes/DATA/repos/scrapper/userdata/First Run"'+'\n');
terminal.stdin.end();
}, 5000);
}
@dankonino
Copy link

@positlabs yeah, google states that in official docs. For me the biggest problem is the timeout and for me to pass tests timeout needed to be 60k * 5 ms. the same issue I see others having. It actually discourage me to use it now.

@joelgriffith
Copy link

Just want to throw it out there that if anyone is looking for a turnkey solution, I've been building https://browserless.io. Aim here is help folks get their stuff off the ground and in production without friction. I'd be more than happy to answer questions or help others' out as it's been something I've worked through about 10 times now.

@talamaska
Copy link

there is this puppeteer library which can control and spawn headless chrome.

@nix86
Copy link

nix86 commented Jan 11, 2018

Is it possible to perform integrated authentication with Chrome headless?

@bearkat2173
Copy link

Yes, please fix integrated authentication for headless. We are using Selenium with PhantomJS right now, which does support integrated auth, but headless Chrome works so much better otherwise.

@AshCoolman
Copy link

AshCoolman commented Feb 1, 2018

When I run:

apt-get install -y xvfb fluxbox x11vnc dbus libasound2 libqt4-dbus libqt4-network libqtcore4 libqtgui4 libxss1 libpython2.7 libqt4-xml libaudio2 libmng1 fontconfig liblcms1 lib32stdc++6 lib32asound2 ia32-libs libc6-i386 lib32gcc1 nano

I get:

Reading package lists... Done
Building dependency tree
Reading state information... Done
Package ia32-libs is not available, but is referred to by another package.
This may mean that the package is missing, has been obsoleted, or
is only available from another source
However the following packages replace it:
  lib32ncurses5 lib32z1

E: Unable to locate package libmng1
E: Unable to locate package liblcms1
E: Unable to locate package lib32asound2
E: Package 'ia32-libs' has no installation candidate

Anyone know what this is about?

Using ubuntu:xenial via docker

Update:

Found these packages here, works for me:

apt-get update && \
    apt-get install -yq gconf-service libasound2 libatk1.0-0 libc6 libcairo2 libcups2 libdbus-1-3 \
    libexpat1 libfontconfig1 libgcc1 libgconf-2-4 libgdk-pixbuf2.0-0 libglib2.0-0 libgtk-3-0 libnspr4 \
    libpango-1.0-0 libpangocairo-1.0-0 libstdc++6 libx11-6 libx11-xcb1 libxcb1 libxcomposite1 \
    libxcursor1 libxdamage1 libxext6 libxfixes3 libxi6 libxrandr2 libxrender1 libxss1 libxtst6 \
    ca-certificates fonts-liberation libappindicator1 libnss3 lsb-release xdg-utils wget

@rrjanbiah
Copy link

@addyosmani or @paulirish Are we having any better solution to record (video recording of screen) in 2020? TIA

@xhimalaya
Copy link

display = Display(visible=0, size=(800, 600))
display.start()

thank you @jond3k , you just saved me..

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment