Skip to content

Instantly share code, notes, and snippets.

@apal21
Created October 28, 2019 18:17
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save apal21/4746f9d3935f24485e40d5fd054f3202 to your computer and use it in GitHub Desktop.
Save apal21/4746f9d3935f24485e40d5fd054f3202 to your computer and use it in GitHub Desktop.
Puppeteer + Tor Proxy file entry point.
const puppeteer = require('puppeteer');
const proxy = require('./proxy'); // Our own proxy module
(async () => {
// Connect to the Tor network
const ip = await proxy();
console.log('Connected to IP:', ip);
let browser;
try {
browser = await puppeteer.launch({
args: ['--proxy-server=socks5://127.0.0.1:9050'],
});
const page = await browser.newPage();
await page.goto('http://example.com'); // Add the URL you want to fetch
await page.waitForSelector('.some-selector', {
timeout: 15000,
});
page.on('console', msg => {
console.log(msg);
});
const body = await page.evaluate(() => {
// I was using this to fetch the href
// Modify this section accordingly
return [...document.querySelectorAll('.some-selector a')].map(
element => element.href
);
});
console.log(body);
} catch (e) {
console.log(e);
} finally {
await browser.close();
// Close the Tor Connection
console.log(await proxy(true));
}
})();
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment