Skip to content

Instantly share code, notes, and snippets.

@zhangyuan
Last active November 9, 2020 06:44
Show Gist options
  • Star 2 You must be signed in to star a gist
  • Fork 1 You must be signed in to fork a gist
  • Save zhangyuan/7037980 to your computer and use it in GitHub Desktop.
Save zhangyuan/7037980 to your computer and use it in GitHub Desktop.
Scrap pages generated by javascript with casperjs
// http://casperjs.org/
var casper = require('casper').create({
pageSettings: {
loadImages: false,
}
});
var fs = require('fs');
casper.userAgent('Mozilla/5.0 (Macintosh; Intel Mac OS X 10_7_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/30.0.1599.101 Safari/537.36');
casper.start();
casper.viewport(1024, 768);
casper.then(function() {
this.open('http://music.163.com/#/m/discover/artist/').then(function(){
this.page.switchToChildFrame(0);
fs.write("1.html", this.getPageContent(), 'w');
this.page.switchToParentFrame();
fs.write("2.html", this.getPageContent(), 'w');
});
});
casper.run();
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment