Skip to content

Instantly share code, notes, and snippets.

@EllieJaybee
Last active March 31, 2022 09:25
Show Gist options
  • Save EllieJaybee/5a6e4e918223a89442ee5918c0c44c27 to your computer and use it in GitHub Desktop.
Save EllieJaybee/5a6e4e918223a89442ee5918c0c44c27 to your computer and use it in GitHub Desktop.
Python - Scraping google results
import requests
from bs4 import BeautifulSoup as bs
import urllib
url = "https://www.google.com/search"
query = "your mom"
r = requests.get(url, params={'q': query}).text
soup = bs(r, 'html.parser')
soup = soup.find_all("div", class_="egMi0 kCrYT")
for i in soup:
if i.a is not None and i.a['href'].startswith("/url") and not 'scholar.google' in i.a['href']:
print(urllib.parse.unquote(i.a['href']).split('?q=')[1].split('&sa=')[0])
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment