Run Browser in a Virtual Display using Selenium and transfer session to Python Requests

browser.py

"""
If you use Chrome, get Chromedriver and put in your PATH: 
http://chromedriver.chromium.org/downloads

If you use Firefox, get Geckodriver and put in your PATH: 
https://github.com/mozilla/geckodriver/releases

Also install:

pip install requests
pip install selenium
pip install PyVirtualDisplay
"""

import requests

from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.common.desired_capabilities import DesiredCapabilities
from selenium.webdriver.support.ui import WebDriverWait, Select
from selenium.webdriver.support import expected_conditions as EC
from selenium.common.exceptions import NoSuchElementException, TimeoutException, UnexpectedAlertPresentException

from pyvirtualdisplay import Display


class Browser(object):
    # initialize with 'chrome' or 'firefox' as the driver 
    def __init__(self, flavor):
        self.display = Display(visible=0, size=(2880, 1800)).start()
        
        if flavor.lower() == "chrome":     
            self.driver = webdriver.Chrome()
        elif flavor.lower() == "firefox":
            self.driver = webdriver.Firefox()
        else:
            raise Exception("need to specify 'chrome' or firefox' when creating a Browser")
    
    
    def transfer_to_requests(self):
        headers = {
                "User-Agent":"Mozilla/5.0 (Windows NT 6.3; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/44.0.2403.157 Safari/537.36"
                }
        self.session = requests.session()
        self.session.headers.update(headers)
        for cookie in self.driver.get_cookies():
            c = {cookie['name']: cookie['value']}
            self.session.cookies.update(c)
        return self.session

# example usage
b = Browser("chrome")
b.driver.get("https://www.google.com")

'sesh' (line 48) supposed to be 'self' surely?

Yes!

It should be self.session. Fixed

thanks for confirming, cute kid in profile pic btw - reminds me of my son who just turned 5! Keep it real and PS - your coding looks strong/useful so always thankful for useful contributions in this regard. Best Regards, James

…

On Sun, 7 Feb 2021 at 18:00, Dhruv Kar ***@***.***> wrote: ***@***.**** commented on this gist. ------------------------------ Yes! It should be self.session. Fixed — You are receiving this because you commented. Reply to this email directly, view it on GitHub <https://gist.github.com/dd29fbf535d19db69e979c748bfd3236#gistcomment-3623416>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ADIRABHUWTESQWHU25UQVKDS53IKBANCNFSM4XHIO3WQ> .

Congrats man! They're growing up so fast! She's already 1.5 now.

Hope all is well with you and thanks for the kind words :)

-Dhruv

pyvirtualdisplay.abstractdisplay.XStartError: Xvfb program closed.

I am facing this error

• Aryan