Skip to content

Instantly share code, notes, and snippets.

@zerebral
Last active September 1, 2023 22:35
Show Gist options
  • Save zerebral/c08810f23290d92fe2a749f7bea41e31 to your computer and use it in GitHub Desktop.
Save zerebral/c08810f23290d92fe2a749f7bea41e31 to your computer and use it in GitHub Desktop.
Use puppeteer-extra-plugin-stealth with Browserless
const puppeteer_extra = require('puppeteer-extra');
puppeteer_extra.use(require('puppeteer-extra-plugin-anonymize-ua')());
const StealthPlugin = require('puppeteer-extra-plugin-stealth');
puppeteer_extra.use(StealthPlugin());
module.exports = async ({ context, browser }) => {
const { url } = context;
const ws_endpoint_url = browser.wsEndpoint();
const newBrowser = await puppeteer_extra.connect({
browserWSEndpoint: ws_endpoint_url
});
const page_1 = await newBrowser.newPage();
await page_1.setRequestInterception(true);
await page_1.authenticate({
username: 'your-proxy-username',
password: 'your-proxy-password',
});
page_1.on('request', (req) => {
if(req.resourceType() == 'stylesheet' || req.resourceType() == 'font' || req.resourceType() == 'image'){
req.abort();
}else{
req.continue();
}
});
await page_1.goto(url, {'timeout': 30000, 'waitUntil':'networkidle2'});
const data = await page_1.content();
await newBrowser.close();
return {
data,
// Make sure to match the appropriate content here\n" +
// You'll likely want 'application/json'\n" +
type: 'application/html',
};
};
@tarek369
Copy link

I did every thing, but seems i cant send the code to /function
here the error I get from the logs:
browserless:job LYESRDNJ5MER2GVJRN303EF4EMBBU0BU: /function: Inbound HTTP request. Context: undefined

curl -X POST \ 'http://xxxxx/function' \ -H 'Content-Type: application/json' \ -d '{"code":"module.exports = async ({ page, context }) => { async function load_pe_recaptcha(page, two_captcha_key) { const puppeteer_extra = require(\"puppeteer-extra\"); const RecaptchaPlugin = require(\"puppeteer-extra-plugin-recaptcha\"); var browser = await page.browser(); var wsEndpoint = await browser._wsEndpoint; puppeteer_extra.use( RecaptchaPlugin({ provider: { id: \"2captcha\", token: two_captcha_key, }, visualFeedback: true, }) ); await puppeteer_extra.connect({ browserWSEndpoint: wsEndpoint, }); return wsEndpoint; }; const two_captcha_key = context[\"2captcha_key\"]; const wsEndpoint = load_pe_recaptcha(page, two_captcha_key); await page.goto(\"https://www.google.com/recaptcha/api2/demo\") // After calling load_pe_recaptcha() the additional // puppeteer extra functions become available const captcha_resp = await page.solveRecaptchas(); await Promise.all([ page.waitForNavigation(), page.click(#recaptcha-demo-submit) ]); const data = { endpoint: wsEndpoint, captcha_resp: captcha_resp, }; return { data, type: \"application/json\", } };"}'

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment