Skip to content

Instantly share code, notes, and snippets.

@macloo
Last active April 9, 2019 23:46
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save macloo/57ae5591211205745986442f422fa109 to your computer and use it in GitHub Desktop.
Save macloo/57ae5591211205745986442f422fa109 to your computer and use it in GitHub Desktop.
Run Chrome headless with Selenium
import time
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
from bs4 import BeautifulSoup
chrome_options = Options()
chrome_options.add_argument("--headless")
# fill in your own path to installed chromedriver
driver = webdriver.Chrome(executable_path='/Users/dirname/dirname/dirname//chromedriver',
options=chrome_options)
# fill in URL for page you want to scrape
driver.get('https://somedomain.com');
time.sleep(2)
html = driver.page_source
soup = BeautifulSoup(html, "html.parser")
h1_list = soup.find_all('h1')
print(h1_list)
driver.quit()
@macloo
Copy link
Author

macloo commented Apr 9, 2019

Getting started with Selenium

http://bit.ly/selenium-intro

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment