Skip to content

Instantly share code, notes, and snippets.

@dmdboi

dmdboi/scraper.js

Last active Apr 1, 2021
Embed
What would you like to do?
Downloads the JSON data for a given number of XKCD comics
const axios = require('axios')
const fs = require('fs');
const start = async (number) => {
for (let num = 0; num < 10; num++) {
let posts = []
for (let index = 0; index < number; index++) {
let comic = (num * number) + index + 1
console.log(comic)
try {
let response = await axios.get(`https://xkcd.com/${comic}/info.0.json`)
posts.push(response.data)
} catch(error) {
continue;
}
}
fs.writeFileSync(`xkcd_${num}.json`, JSON.stringify(posts), 'utf8');
}
}
//Number of comics to get per file
start(100);
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment