Skip to content

Instantly share code, notes, and snippets.

@littletsu
Created February 19, 2022 09:01
Show Gist options
  • Save littletsu/c79e53c9fe072be8f3263108cbf0aa72 to your computer and use it in GitHub Desktop.
Save littletsu/c79e53c9fe072be8f3263108cbf0aa72 to your computer and use it in GitHub Desktop.
Bulk Download Urls in Node.js
import fs from 'fs';
import fetch from 'node-fetch';
// urls.txt should be a text file containing the urls separated by a new line
const file = fs.readFileSync('./urls.txt', {'encoding': 'utf8'}).split('\n');
file.forEach(url => {
fetch(url).then(res => {
// modify to whichever format the downloads are
let write = fs.createWriteStream(`./urls/${Date.now()}.webp`);
res.body.pipe(write);
})
})
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment