Skip to content

Instantly share code, notes, and snippets.

@diogojorgebasso
Created January 29, 2021 17:43
Show Gist options
  • Save diogojorgebasso/011fd4c3df2af764826b59205fbdda67 to your computer and use it in GitHub Desktop.
Save diogojorgebasso/011fd4c3df2af764826b59205fbdda67 to your computer and use it in GitHub Desktop.
Snippet for cloning a website with node.js
// espectacular modules:
const scrape = require("website-scraper");
const PuppeteerPlugin = require("website-scraper-puppeteer");
const path = require("path");
scrape({
// paste it down here the URL(s) of the site(s) that you want to clone:
urls: ["url#1", "url#2"],
// Specify the path inside this folder where the content should be saved
directory: path.resolve(__dirname, "fitness-prod"),
// Load the Puppeteer plugin
plugins: [
new PuppeteerPlugin({
launchOptions: {
headless: true,
},
}),
],
});
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment