Skip to content

Instantly share code, notes, and snippets.

@EdwinTai

EdwinTai/Scraping-Dynamic-Site.py

Created April 22, 2021 10:19
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save EdwinTai/2898f15dddac66281abe71d9c38af4d8 to your computer and use it in GitHub Desktop.
Save EdwinTai/2898f15dddac66281abe71d9c38af4d8 to your computer and use it in GitHub Desktop.
Download ZIP
Raw
Scraping-Dynamic-Site.py
import time
from selenium import webdriver
from bs4 import BeautifulSoup
driver = webdriver.Chrome()
url = "https://tw.sports.yahoo.com/" #specific download url
driver.get(url)
time.sleep(5) #wait until page load finish
soup = BeautifulSoup(driver.page_source,'html.parser')
for content in soup.find_all('h3', {'class': 'M(0)'}):
print(content.text)
driver.close()
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment