Create a gist now

Instantly share code, notes, and snippets.

What would you like to do?
Headless web browsers

Here are a list of headless browsers that I know about:

  • HtmlUnit - Java. Custom browser engine. JavaScript support/DOM emulated. Open source.
  • Ghost - Python only. WebKit-based. Full JavaScript support. Open source.
  • Twill - Python/command line. Custom browser engine. No JavaScript. Open source.
  • PhantomJS - Command line/all platforms. WebKit-based. Full JavaScript support. Open source.
  • Awesomium - C++/.Net/all platforms. Chromium-based. Full JavaScript support. Commercial/free.
  • SimpleBrowser - .Net 4/C#. Custom browser engine. No JavaScript support. Open source.
  • ZombieJS - Node.js. Custom browser engine. JavaScript support/emulated DOM. Open source.
  • EnvJS - JavaScript via Java/Rhino. Custom browser engine. JavaScript support/emulated DOM. Open source.
  • Watir-webdriver with headless gem - Ruby via WebDriver. Full JS Support via Browsers (Firefox/Chrome/Safari/IE).
  • Spynner - Python only. PyQT and WebKit.

Headless browsers that have JavaScript support via an emulated DOM generally have issues with some sites that use more advanced/obscure browser features, or have functionality that has visual dependencies (e.g. via CSS positions and so forth), so whilst the pure JavaScript support in these browsers is generally complete, the actual supported browser functionality should be considered as partial only.

(note: original version of this post only mentioned HtmlUnit, hence the comments. If you know of other headless browser implementations and have edit rights, feel free to edit this post and add them.)

@badsyntax

This comment has been minimized.

Show comment Hide comment
@badsyntax

badsyntax Nov 13, 2013

Here's another one for the list: "A headless Internet Explorer browser with a Javascript API. PhantomJS for the Trident engine." https://github.com/sdesalas/trifleJS

Here's another one for the list: "A headless Internet Explorer browser with a Javascript API. PhantomJS for the Trident engine." https://github.com/sdesalas/trifleJS

@voidmonk

This comment has been minimized.

Show comment Hide comment
@voidmonk

voidmonk Jan 15, 2014

Stumbled upon Crowbar (mozilla-based): http://simile.mit.edu/wiki/Crowbar

Stumbled upon Crowbar (mozilla-based): http://simile.mit.edu/wiki/Crowbar

@sonnyp

This comment has been minimized.

Show comment Hide comment
@sonnyp

sonnyp Mar 24, 2014

http://casperjs.org/ PhantomJS API-compatible, rely on Gecko (Firefox engine)

sonnyp commented Mar 24, 2014

http://casperjs.org/ PhantomJS API-compatible, rely on Gecko (Firefox engine)

@FunnyDewd

This comment has been minimized.

Show comment Hide comment
@FunnyDewd

FunnyDewd Apr 29, 2014

http://slimerjs.org/ Like phantom.js except uses the firefox engine.

http://slimerjs.org/ Like phantom.js except uses the firefox engine.

@mickaelandrieu

This comment has been minimized.

Show comment Hide comment
@mickaelandrieu

mickaelandrieu May 4, 2014

BrowserKit, the Symfony2 component. PHP, no javascript support

BrowserKit, the Symfony2 component. PHP, no javascript support

@christophebe

This comment has been minimized.

Show comment Hide comment
@christophebe

christophebe Jun 4, 2014

@nwohaibi

This comment has been minimized.

Show comment Hide comment
@bass14

This comment has been minimized.

Show comment Hide comment
@bass14

bass14 Feb 13, 2015

I tried htmlunit, jaunt, webview head less browser in java. but those browser are Rendering Css/js web page source code.

bass14 commented Feb 13, 2015

I tried htmlunit, jaunt, webview head less browser in java. but those browser are Rendering Css/js web page source code.

@chaserhkj

This comment has been minimized.

Show comment Hide comment
@chaserhkj

chaserhkj Feb 16, 2015

@ldong

This comment has been minimized.

Show comment Hide comment
@hollingsworthd

This comment has been minimized.

Show comment Hide comment
@hollingsworthd

hollingsworthd Feb 24, 2016

Also check out https://github.com/MachinePublishers/jBrowserDriver which is Selenium-compatible and WebKit-based. Pure Java.

Also check out https://github.com/MachinePublishers/jBrowserDriver which is Selenium-compatible and WebKit-based. Pure Java.

@zhuyingda

This comment has been minimized.

Show comment Hide comment
@zhuyingda

zhuyingda Oct 17, 2016

Check out https://github.com/segmentio/nightmare which is a chromium-based.Node.js/Javascript.

Check out https://github.com/segmentio/nightmare which is a chromium-based.Node.js/Javascript.

@JABirchall

This comment has been minimized.

Show comment Hide comment
@JABirchall

JABirchall Feb 21, 2017

Laravel just released their browser emulator written in PHP Laravel/Dusk

JABirchall commented Feb 21, 2017

Laravel just released their browser emulator written in PHP Laravel/Dusk

@samlucax

This comment has been minimized.

Show comment Hide comment
@samlucax

samlucax Aug 3, 2017

Can you add chrome headless: chrome headless

samlucax commented Aug 3, 2017

Can you add chrome headless: chrome headless

@Akagi201

This comment has been minimized.

Show comment Hide comment
@RichardBray

This comment has been minimized.

Show comment Hide comment
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment