Skip to content

Instantly share code, notes, and snippets.

View kyrcha's full-sized avatar

Kyriakos Chatzidimitriou kyrcha

View GitHub Profile
@kyrcha
kyrcha / crawler.js
Last active February 27, 2024 03:03
npm registry crawler
var request = require('request'),
cheerio = require('cheerio'),
async = require('async');
var base = 'https://www.npmjs.org',
concurrency = 2;
var q = async.queue(function(task, next) {
setTimeout(function() {
console.log('GET ' + task.url);