Skip to content

Instantly share code, notes, and snippets.

@potatowave
Created November 26, 2019 04:04
Show Gist options
  • Save potatowave/d3dcaf2e99032672e0f75301a7dff884 to your computer and use it in GitHub Desktop.
Save potatowave/d3dcaf2e99032672e0f75301a7dff884 to your computer and use it in GitHub Desktop.
# Run `pip3 install beautifulsoup4` to install beautiful soup 4
# Run `pip3 install selenium` to install selenium
# Run `pip3 install webdriver-manager` to install webdriver-manager
# Run `brew cask install chromedriver` to install chromedriver
# to run this script, run `python3 scrape.py` from this folder
from bs4 import BeautifulSoup
from selenium import webdriver
driver = webdriver.Chrome()
driver.get('') # put the URL you want to scrape here. E.g driver.get('http://www.google.com')
page = driver.page_source
soup = BeautifulSoup(page, 'html.parser')
print(soup)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment