Skip to content

Instantly share code, notes, and snippets.

@nsourov
Created June 27, 2020 06:47
Show Gist options
  • Save nsourov/6cd92f6ccacaf10bab6d35fe1835b1fe to your computer and use it in GitHub Desktop.
Save nsourov/6cd92f6ccacaf10bab6d35fe1835b1fe to your computer and use it in GitHub Desktop.
Puppeteer with p-queue
const puppeteer = require('puppeteer');
const PQueue = require('p-queue');
const queue = new PQueue({ concurrency: 2 });
const singleRunner = async (url) => {
const browser = await puppeteer.launch({ headless: false, executablePath: process.env.CHROME_PATH, });
const page = await browser.newPage();
await page.goto(url);
const title = await page.title();
console.log({title});
await browser.close();
};
const concurrentRunner = (url) =>
queue.add(() => singleRunner(url)).catch(console.error);
const urls = ['http://example.com/', 'https://google.com/', 'https://yahoo.com/', 'https://app.automatio.co/'];
for (const url of urls) {
concurrentRunner(url);
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment