Skip to content

Instantly share code, notes, and snippets.

@kunalrustagi08
Last active May 6, 2020 14:21
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save kunalrustagi08/0a97ef5c46a88b0d41ba539a2ec4f697 to your computer and use it in GitHub Desktop.
Save kunalrustagi08/0a97ef5c46a88b0d41ba539a2ec4f697 to your computer and use it in GitHub Desktop.
def scrape_url(url):
book_title = []
star_rating = []
product_price = []
time.sleep(random.randint(1,10))
results = requests.get(url)
soup = BeautifulSoup(results.text, 'html.parser')
book_div = soup.find_all('li', class_='col-xs-6 col-sm-4 col-md-3 col-lg-3')
for container in book_div:
title = container.article.h3.a['title']
book_title.append(title)
price = container.article.find('div', class_='product_price').p.text
product_price.append(price)
rating = container.article.p['class'][-1]
star_rating.append(rating)
return (book_title, product_price, star_rating)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment