Skip to content

Instantly share code, notes, and snippets.

@jancurn
Last active January 11, 2023 17:31
Show Gist options
  • Star 2 You must be signed in to star a gist
  • Fork 1 You must be signed in to fork a gist
  • Save jancurn/ecf3f04aea3c245498e06a913ce54115 to your computer and use it in GitHub Desktop.
Save jancurn/ecf3f04aea3c245498e06a913ce54115 to your computer and use it in GitHub Desktop.
Puppeteer's page.authenticate() does not work for proxy authorization!
const puppeteer = require('puppeteer');
(async() => {
const proxyUrl = 'http://proxy.example.com:8000';
const username = 'bob';
const password = 'password123';
const browser = await puppeteer.launch({
args: [`--proxy-server=${proxyUrl}`],
headless: false,
});
const page = await browser.newPage();
await page.authenticate({ username, password });
await page.goto('https://www.example.com');
await browser.close();
})();
@dastanaron
Copy link

dastanaron commented Nov 10, 2020

@mnmkng yes I tried the proxies in firefox browser and all of them is working

@dastanaron
Copy link

dastanaron commented Nov 10, 2020

@mnmkng I took proxy in the https://fineproxy.org/ and they is working, but without authorization

@mnmkng
Copy link

mnmkng commented Nov 10, 2020

I tried the IP in your screenshot in my Firefox and could not make it work. http://example.com gave me a cloudflare DNS error. I also tried the code example once again with a different proxy provider and it worked. I'm afraid I can't help any further.

@dastanaron
Copy link

@mnmkng Sure, I have deleted it, because I show login and password for all))

@toxx91
Copy link

toxx91 commented Oct 21, 2021

It helped me to specify new paths for --user-data-dir =

'--user-data-dir=C:\\Program Files\\nodejs\\newprofile\\profile+rand(1,1000)

@aditodkar
Copy link

This did not work for me getting error: Protocol error (Target.createTarget): Target closed

@mnmkng
Copy link

mnmkng commented Jan 10, 2023

@aditodkar Weird, that usually happens when you're trying to work with a resource (browser, page) that crashed or is not available anymore. Sure you're not getting any errors or missing await?

@aditodkar
Copy link

@mnmkng I am running puppeteer + aws lambda with proxy servers. Check this code snippet. I also tried running crawlee in aws lambda but did not work for me please check this discord discussion on apify discord channel.

@mnmkng
Copy link

mnmkng commented Jan 10, 2023

Running Chromium or any headless browser on Lambda is always pain. I have not done it in a long time so can't help unfortunately. It's one of the reasons we created Apify. Because running browsers on existing serverless platforms is a constant struggle.

@aditodkar
Copy link

But Apify platform is also built on aws lambda right?

@mnmkng
Copy link

mnmkng commented Jan 10, 2023

Not at all :) Apify runs primarily on EC2 and Actors run in Docker.

@aditodkar
Copy link

Not at all :) Apify runs primarily on EC2 and Actors run in Docker.

Thanks a lot for the info :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment