Skip to content

Instantly share code, notes, and snippets.

@PaulSayantan
Created August 7, 2022 20:05
Show Gist options
  • Save PaulSayantan/68feb4c94f6371f7a0ad8682dac0774d to your computer and use it in GitHub Desktop.
Save PaulSayantan/68feb4c94f6371f7a0ad8682dac0774d to your computer and use it in GitHub Desktop.
Python Google Search Page Scraper
from bs4 import BeautifulSoup
from selenium import webdriver
from selenium.webdriver.chrome.service import Service
from webdriver_manager.chrome import ChromeDriverManager
chrome_options = webdriver.ChromeOptions()
chrome_options.add_argument("--headless")
driver = webdriver.Chrome(service=Service(ChromeDriverManager().install()), options=chrome_options)
query = 'pasilla peppers country of origin'
url = f"http://www.google.com/search?q={query}"
driver.get(url)
soup = BeautifulSoup(driver.page_source, 'html.parser')
answer_box = soup.find('div', class_="V3FYCf")
print('Answer:', answer_box.find_next('b').get_text())
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment