Skip to content

Instantly share code, notes, and snippets.

@AnderRV

AnderRV/index.js Secret

Last active August 29, 2021 15:07
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save AnderRV/289563313c88fe6d597a67f43e52bdf7 to your computer and use it in GitHub Desktop.
Save AnderRV/289563313c88fe6d597a67f43e52bdf7 to your computer and use it in GitHub Desktop.
const axios = require('axios');
const cheerio = require('cheerio');
const extractLinks = $ => [
...new Set(
$('.page-numbers a') // Select pagination links
.map((_, a) => $(a).attr('href')) // Extract the href (url) from each link
.toArray() // Convert cheerio object to array
),
];
axios.get('https://scrapeme.live/shop/').then(({ data }) => {
const $ = cheerio.load(data); // Initialize cheerio
const links = extractLinks($);
console.log(links);
// ['https://scrapeme.live/shop/page/2/', 'https://scrapeme.live/shop/page/3/', ... ]
});
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment