Skip to content

Instantly share code, notes, and snippets.

@addyosmani
Last active May 17, 2024 03:38
Show Gist options
  • Save addyosmani/5336747 to your computer and use it in GitHub Desktop.
Save addyosmani/5336747 to your computer and use it in GitHub Desktop.
So, you want to run Chrome headless.

Update May 2017

Eric Bidelman has documented some of the common workflows possible with headless Chrome over in https://developers.google.com/web/updates/2017/04/headless-chrome.

Update

If you're looking at this in 2016 and beyond, I strongly recommend investigating real headless Chrome: https://chromium.googlesource.com/chromium/src/+/lkgr/headless/README.md

Windows and Mac users might find using Justin Ribeiro's Docker setup useful here while full support for these platforms is being worked out.

You can use chrome --headless on Linux as of M57 but note you'll need to build the binaries yourself for now.

The metabug for adding headless mode to Chromium is over here.

Otherwise, if you're on Ubuntu or working with a Linux VM that isn't working with Linux:

Step 1: Install the Ubuntu dependencies needed:

SSH into your server as a root or do sudo -i.

Then install necessary software:

apt-get update
apt-get install -y xvfb fluxbox x11vnc dbus libasound2 libqt4-dbus libqt4-network libqtcore4 libqtgui4 libxss1 libpython2.7 libqt4-xml libaudio2 libmng1 fontconfig liblcms1 lib32stdc++6 lib32asound2 ia32-libs libc6-i386 lib32gcc1 nano
apt-get install -y python-gobject-2
apt-get install -y curl git

Step 2. Options for starting:

Normal:

xvfb-run --server-args='-screen 0, 1024x768x16' google-chrome -start-maximized > /dev/null &

Start and open a page:

xvfb-run --server-args='-screen 0, 1024x768x16' google-chrome -start-maximized http://www.example.com > /dev/null &

If you're on Windows..

Use Desktops from the MS SysInternals package. You can spawn your Chrome process directly from the "hidden" desktop and keep it working there (I have not personally tested this).

#Other options...

You could try spawning Chrome as a child process using something like Node.js and piping the output back to the terminal. See runChrome.js for an example of how to do this (todo: update to latest Express).

var express = require('express');
var http = require('http');
var cp = require('child_process');
var port = 3000;
var app = express();
//var app = express();
//var server = http.createServer(app);
var chrome_count = 0;
app.get('/', function(req, res){
startChrome(chrome_count++,9222,function chromeStared(err,id,chrome){
res.send('Chrome '+id+' Started\r\n');
console.log('Simulating some data parsing');
setTimeout(function(){
console.log('Shutdown chrome '+id);
chrome.kill('doh!');
},1000);
});
});
app.listen(port);
console.log('Server is listening on port ' + port);
function startChrome(id,port,callback){
var terminal = cp.spawn('bash');
var chrome = {};
terminal.on('exit', function (code) {
console.log('Starting chrome');
chrome = cp.spawn('/Applications/Google\ Chrome.app/Contents/MacOS/Google\ Chrome',[
'--remote-debugging-port='+port,
'--user-data-dir=/Volumes/DATA/repos/scrapper/userdata',
'http://www.chrome.com'
]);
callback(null,id,chrome);
});
setTimeout(function() {
console.log('Sending stdin to terminal');
//terminal.stdin.write('echo "Hello $USER"');
terminal.stdin.write('rm -rf /Volumes/DATA/repos/scrapper/userdata'+'\n');
terminal.stdin.write('mkdir /Volumes/DATA/repos/scrapper/userdata'+'\n');
terminal.stdin.write('touch "/Volumes/DATA/repos/scrapper/userdata/First Run"'+'\n');
terminal.stdin.write('chmod 777 "/Volumes/DATA/repos/scrapper/userdata/First Run"'+'\n');
terminal.stdin.end();
}, 5000);
}
@amitaibu
Copy link

Here's a post about the docker option

@mohsen1
Copy link

mohsen1 commented May 3, 2016

@addyosmani is nano at the end of packages to install a joke?

@krtschmr
Copy link

@mohsen1 to make sure, people read what they are installing. haha!

@snewcomer
Copy link

snewcomer commented Aug 8, 2016

How about for centos? Just find the equivalent pkgs?

@lmayorga1980
Copy link

@snewcomer I figure out by using these packages but for some reason text is not displayed on the browser.

RUN yum update -y \
  && yum install  \
     ipa-gothic-fonts \
     xorg-x11-fonts-100dpi \
     xorg-x11-fonts-75dpi \
     xorg-x11-utils \
     xorg-x11-fonts-cyrillic \
     xorg-x11-fonts-Type1 \
     xorg-x11-fonts-misc -y

@Overdrivr
Copy link

@jimthedev
Copy link

@elgalu
Copy link

elgalu commented Apr 21, 2017

Does someone knows how to browser.manage().window().setSize() when using google-chrome-57 --headless ?

@elgalu
Copy link

elgalu commented Apr 21, 2017

@ripper2hl
Copy link

Very nice works perfectly with Selenium, no window size .
This run on ubuntu 17.04

    @BeforeTest
    public void setupTest() {
        ChromeDriverManager.getInstance().setup();
        ChromeOptions options = new ChromeOptions();
        options.addArguments("--headless", "--disable-gpu");
        driver =  new ChromeDriver( options );
    }

ChromeDriverManager its a project of Boni Garcia
https://github.com/bonigarcia/webdrivermanager

@ragingwind
Copy link

@gwuhaolin
Copy link

gwuhaolin commented Jun 19, 2017

here are some cool project about headless-chrome you gays may like:

  • chrome-render general server render base on chrome.
  • koa-chrome-render chrome-render middleware for koa.
  • koa-seo koa SEO middleware.
  • chrome-pool Headless chrome tabs manage pool, concept come from database connection pool for reuse chrome tab improve performance.

@angeloassis13
Copy link

Hi,

I have a java application using cdp to evaluate some javascript code on chrome headless. When I send a large string to be evaluated chrome returns null and the following line is shown on chrome's log: "Too large read data is pending: capacity=1048576, max_buffer_size=1048576, read=1048576"

Is there a way to increase buffer size?

@hl2055
Copy link

hl2055 commented Jul 11, 2017

Virtual display belongs to the past, with almighty chrome doing it all.

@vpassapera
Copy link

vpassapera commented Jul 20, 2017

On Centos 7, with chrome 59, using the --headless flag by itself still causes issues (it straight up doesn't work and you end up having to use real chrome to connect to karma). I still have to use export DISPLAY=99.0 and xvfb-run before running ng test to do headless.

@positlabs
Copy link

Does anyone know if there are plans to make it work without --disable-gpu? I'd like to render webgl content.

@dankonino
Copy link

@positlabs yeah, google states that in official docs. For me the biggest problem is the timeout and for me to pass tests timeout needed to be 60k * 5 ms. the same issue I see others having. It actually discourage me to use it now.

@joelgriffith
Copy link

Just want to throw it out there that if anyone is looking for a turnkey solution, I've been building https://browserless.io. Aim here is help folks get their stuff off the ground and in production without friction. I'd be more than happy to answer questions or help others' out as it's been something I've worked through about 10 times now.

@talamaska
Copy link

there is this puppeteer library which can control and spawn headless chrome.

@nix86
Copy link

nix86 commented Jan 11, 2018

Is it possible to perform integrated authentication with Chrome headless?

@bearkat2173
Copy link

Yes, please fix integrated authentication for headless. We are using Selenium with PhantomJS right now, which does support integrated auth, but headless Chrome works so much better otherwise.

@AshCoolman
Copy link

AshCoolman commented Feb 1, 2018

When I run:

apt-get install -y xvfb fluxbox x11vnc dbus libasound2 libqt4-dbus libqt4-network libqtcore4 libqtgui4 libxss1 libpython2.7 libqt4-xml libaudio2 libmng1 fontconfig liblcms1 lib32stdc++6 lib32asound2 ia32-libs libc6-i386 lib32gcc1 nano

I get:

Reading package lists... Done
Building dependency tree
Reading state information... Done
Package ia32-libs is not available, but is referred to by another package.
This may mean that the package is missing, has been obsoleted, or
is only available from another source
However the following packages replace it:
  lib32ncurses5 lib32z1

E: Unable to locate package libmng1
E: Unable to locate package liblcms1
E: Unable to locate package lib32asound2
E: Package 'ia32-libs' has no installation candidate

Anyone know what this is about?

Using ubuntu:xenial via docker

Update:

Found these packages here, works for me:

apt-get update && \
    apt-get install -yq gconf-service libasound2 libatk1.0-0 libc6 libcairo2 libcups2 libdbus-1-3 \
    libexpat1 libfontconfig1 libgcc1 libgconf-2-4 libgdk-pixbuf2.0-0 libglib2.0-0 libgtk-3-0 libnspr4 \
    libpango-1.0-0 libpangocairo-1.0-0 libstdc++6 libx11-6 libx11-xcb1 libxcb1 libxcomposite1 \
    libxcursor1 libxdamage1 libxext6 libxfixes3 libxi6 libxrandr2 libxrender1 libxss1 libxtst6 \
    ca-certificates fonts-liberation libappindicator1 libnss3 lsb-release xdg-utils wget

@rrjanbiah
Copy link

@addyosmani or @paulirish Are we having any better solution to record (video recording of screen) in 2020? TIA

@xhimalaya
Copy link

display = Display(visible=0, size=(800, 600))
display.start()

thank you @jond3k , you just saved me..

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment