zwarag

Using CasperJS to scrape an infinite scroll page

This is a short script to scrape an infinite scroll page and write the resulting HTML to a file

You'll need to install CasperJS; on a Mac that will involve brew install casperjs --devel (if you don't have Homebrew, you can read how to do so here)

To run the script, you can use casperjs scrape.js --ssl-protocol=any from the command line.

zwarag

var casper = require('casper').create({
	verbose : true,
	logLevel : 'info'
});
var images = [];
var fs=require("fs")

/**
 * Configuration here
 */

zwarag

Keybase proof

I hereby claim:

• I am zwarag on github.
• I am zwarag (https://keybase.io/zwarag) on keybase.
• I have a public key ASA9LI6axmYcKLDDU04xL_rdP19DbGHJ6rFOs5TVaaK9dAo

To claim this, I am signing this object: