Skip to content

Instantly share code, notes, and snippets.

@erasmo-marin
Last active December 2, 2017 18:08
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save erasmo-marin/970793b946b8136ae7021e87c946378f to your computer and use it in GitHub Desktop.
Save erasmo-marin/970793b946b8136ae7021e87c946378f to your computer and use it in GitHub Desktop.
async function fetchAllPages() {
const domain = "https://mycooleebsite.com";
const pages = [
"/",
"/about",
"/pricing",
"/contact"
]
const browser = await puppeteer.launch({ args: ["--no-sandbox"] });
for(let i = 0; i < pages.lenght; i++) {
const html = await crawler({
url: `${domain}${pages[i]}`,
browser
});
saveUrlToFile(html, pages[i], __dirname);
}
await browser.close();
return true;
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment