Skip to content

Instantly share code, notes, and snippets.

@geekingfrog
Created November 16, 2014 07:06
Show Gist options
  • Save geekingfrog/cf90cf52d2e645b4a53c to your computer and use it in GitHub Desktop.
Save geekingfrog/cf90cf52d2e645b4a53c to your computer and use it in GitHub Desktop.
Basic recursive call
function parsePage(url, depth) {
if(!depth) depth = 1;
if(depth > 3) return;
// here, fetch the page and save it to disk
// then process each links
extractLinksFromPage(body).forEach(function(link) {
parsePage(link, depth++);
})
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment