Skip to content

Instantly share code, notes, and snippets.

@zivce
Created June 7, 2020 08:35
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save zivce/5d94a5d6ea17253d9d8ca50b7dc9301a to your computer and use it in GitHub Desktop.
Save zivce/5d94a5d6ea17253d9d8ca50b7dc9301a to your computer and use it in GitHub Desktop.
Scraping for police info on local FB page
const puppeteer = require('puppeteer');
const dotenv = require('dotenv');
dotenv.config();
const WAIT_FOR_PAGE = 5000;
const DELAY_USER_INPUT = 2000;
const DELAY_PW_INPUT = 1000;
(async () => {
const browser = await puppeteer.launch({headless: false});
const context = browser.defaultBrowserContext();
await context.overridePermissions(process.env.LEVAK_POLICIJE, ['notifications']);
const page = await browser.newPage();
await page.goto(process.env.LEVAK_POLICIJE);
await page.type('#email', process.env.FB_USER, {delay: DELAY_USER_INPUT});
await page.type('#pass', process.env.FB_PW, {delay: DELAY_PW_INPUT});
await page.click('#loginbutton');
await page.waitFor(".userContentWrapper");
await delay(WAIT_FOR_PAGE);
const elems = await page.evaluate(() => {
const infos = Array.from(document.querySelectorAll(".userContentWrapper"));
return infos.map(info => {
const array = info.innerText.split('\n');
return array.slice(3,4);
});
});
console.log(elems);
})();
function delay(time) {
return new Promise(function(resolve) {
setTimeout(resolve, time)
});
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment