Skip to content

Instantly share code, notes, and snippets.

@stephepush

stephepush/scraping Secret

Created April 25, 2017 18:14
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save stephepush/bfd2505d214e3572000e0d78b5a6dfa6 to your computer and use it in GitHub Desktop.
Save stephepush/bfd2505d214e3572000e0d78b5a6dfa6 to your computer and use it in GitHub Desktop.
Download ZIP
Raw
scraping
import bs4
from urllib.request import urlopen as uReq
from bs4 import BeautifulSoup as soup
my_url = 'https://site.com'
#opening up connection, grabbing the page
uClient = uReq(my_url)
page_html = uClient.read()
uClient.close()
#html parsing
page_soup = soup(page_html, "html.parser")
#grabs each group
items = page_soup.findAll("div", {"class":"group"})
len(items)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment