Skip to content

Instantly share code, notes, and snippets.

@berkant
Created July 24, 2018 11:06
Show Gist options
  • Save berkant/4e728dbc9840f1463aa004a9ac73359a to your computer and use it in GitHub Desktop.
Save berkant/4e728dbc9840f1463aa004a9ac73359a to your computer and use it in GitHub Desktop.
Spys.One Socks Proxy Scraper
var casper = require('casper').create();
// meaning 30 proxies in default
var ProxiesPerPage = 0;
switch(casper.cli.get('count')) {
case 30:
ProxiesPerPage = 0;
break;
case 50:
ProxiesPerPage = 1;
break;
case 100:
ProxiesPerPage = 2;
break;
case 200:
ProxiesPerPage = 3;
break;
case 300:
ProxiesPerPage = 4;
break;
case 500:
ProxiesPerPage = 5;
}
// casper.options.verbose = true;
casper.options.pageSettings = {
loadImages: false,
webSecurityEnabled: false,
allowMedia: false,
userAgent: 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_6) AppleWebKit/601.7.7 (KHTML, like Gecko) Version/9.1.2 Safari/601.7.7'
};
casper.start();
casper.open('http://spys.one/en/socks-proxy-list/', {
method: 'POST',
data: {
xpp: ProxiesPerPage,
xf1: 0,
xf2: 0,
xf4: 0,
xf5: 0
}
});
casper.then(function() {
var regexp = /[0-9]+.[0-9]+.[0-9]+.[0-9]+:[0-9]+/g;
var match;
while((match = regexp.exec(this.page.plainText)) != null) {
console.log(match);
}
});
casper.run();
@berkant
Copy link
Author

berkant commented Jul 24, 2018

  • You must have PhantomJS installed to use CasperJS since it doesn't have its own engine.
  • There are hard-coded counts of proxies that can be requested when running script:
    • --count=30 # which will print 30 proxies.
    • --count=50 # which will print 50 proxies.
    • --count=100 # which will print 100 proxies.
    • --count=200 # which will print 200 proxies.
    • --count=300 # which will print 300 proxies.
    • --count=500 # which will print 500 proxies.
    • If count is other than these values or it's not specified, it will fall back to 30.

$ casperjs --count=30 bot.js # prints 30 proxies
$ casperjs --count=546532 bot.js # falls back to and prints 30 proxies


Check out how it works: https://asciinema.org/a/k4CSTsEXWzfVtoT7sg7bvNHlY

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment