Skip to content

Instantly share code, notes, and snippets.

@tarvos21
Forked from chilts/alexa.js
Created September 30, 2017 15:33
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save tarvos21/9e3917a2a4cc28150d0471db977c2595 to your computer and use it in GitHub Desktop.
Save tarvos21/9e3917a2a4cc28150d0471db977c2595 to your computer and use it in GitHub Desktop.
Getting the Alexa top 1 million sites directly from the server, unzipping it, parsing the csv and getting each line as an array.
var request = require('request');
var unzip = require('unzip');
var csv2 = require('csv2');
request.get('http://s3.amazonaws.com/alexa-static/top-1m.csv.zip')
.pipe(unzip.Parse())
.on('entry', function (entry) {
entry.pipe(csv2()).on('data', console.log);
})
;
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment