This script creates a proxy server between the Tor Browser and Tor to capture requests/responses, using the seleniumwire library. You can access and modify the HTTP headers that are being sent/received, including the onion services. Note: You need to have Tor installed and running on the localhost while running this script.

tor-browser-selenium-wire.py

import os
from seleniumwire import webdriver
from selenium.webdriver.firefox.firefox_profile import FirefoxProfile
from selenium.webdriver.firefox.firefox_binary import FirefoxBinary

# Uncomment these if you need additional information for debugging
#import logging
#logging.basicConfig(level=logging.DEBUG)

# The location of the Tor Browser bundle
# Update this to match the location on your computer
tbb_dir = "/home/woswos/tor-browser_en-US/"

# Disable Tor Launcher to prevent it connecting the Tor Browser to Tor directly
os.environ['TOR_SKIP_LAUNCH'] = '1'
os.environ['TOR_TRANSPROXY'] = '1'

# Set the Tor Browser binary and profile
tb_binary = os.path.join(tbb_dir, 'Browser/firefox')
tb_profile = os.path.join(tbb_dir, 'Browser/TorBrowser/Data/Browser/profile.default')
binary = FirefoxBinary(os.path.join(tbb_dir, 'Browser/firefox'))
profile = FirefoxProfile(tb_profile)

# We need to disable HTTP Strict Transport Security (HSTS) in order to have
#   seleniumwire between the browser and Tor. Otherwise, we will not be able
#   to capture the requests and responses using seleniumwire.
profile.set_preference("security.cert_pinning.enforcement_level", 0)
profile.set_preference("network.stricttransportsecurity.preloadlist", False)

# Tell Tor Button it is OK to use seleniumwire
profile.set_preference("extensions.torbutton.local_tor_check", False)
profile.set_preference("extensions.torbutton.use_nontor_proxy", True)

# Required if you need JavaScript at all, otherwise JS stays disabled regardless
#   of the Tor Browser's security slider value
profile.set_preference("browser.startup.homepage_override.mstone", "68.8.0");

# Configure seleniumwire to upstream traffic to Tor running on port 9050
#   You might want to increase/decrease the timeout if you are trying
#   to a load page that requires a lot of requests. It is in seconds.
options = {
    'proxy': {
        'http': 'socks5h://127.0.0.1:9050',
        'https': 'socks5h://127.0.0.1:9050',
        'connection_timeout': 10
    }
}

driver = webdriver.Firefox(firefox_profile=profile,
                           firefox_binary=binary,
                           seleniumwire_options=options)

driver.get("https://check.torproject.org/")

for request in driver.requests:
    if request.response:
        print(
            request.path,
            request.response.status_code,
            request.response.headers['Content-Type']
        )

diver.quit()

Hi,

first of all, thanks for posting this snippet, it's really helpful.
For some reason, if I use this and request "https://check.torproject.org/", it works and confirms tor usage but NOT with any .onion url.
They fail with an error 502 - Bad gateway. My guesses so far...

• seleniumwire has a problem with the socks5h upstream proxy?
• some system time misconfiguration?
• incompatible tb update?

I had no luck messing aroung with those guesses.

Any idea what the culprit could be?

@woswos

// Edit: By the way, I made sure it is not a server side error (which 500s should be actually). Using my alternative setup (tor-browser-selenium) without seleniumwire works without any issues, same settings as far as I can tell.

You saved my life,thanks really much, at first it didnt work but i added this preferences and it worked like a charm:

profile.set_preference('network.proxy.type', 1)
profile.set_preference('network.proxy.socks_host', '127.0.0.1')
profile.set_preference('network.proxy.socks_port', 9050)
profile.update_preferences()

Is there a similar way to get this to work with tbselenium driver, especially with their STEM module? I can use the code I had before, without using Firefox objects, but I keep getting
selenium.common.exceptions.,WebDriverException: Message: Reached error page: about:neterror?e=dnsNotFound&u=...