Skip to content

Instantly share code, notes, and snippets.

@chris-lovejoy
Created May 1, 2020 13:15
Show Gist options
  • Star 1 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save chris-lovejoy/66dd334a644a35ea4d1712955e7af463 to your computer and use it in GitHub Desktop.
Save chris-lovejoy/66dd334a644a35ea4d1712955e7af463 to your computer and use it in GitHub Desktop.
use driver to extract HTML soup
def make_job_search(job_title, location, driver):
driver.get('https://www.cwjobs.co.uk/')
# Select the job box
job_title_box = driver.find_element_by_name('Keywords')
# Send job information
job_title_box.send_keys(job_title)
# Selection location box
location_box = driver.find_element_by_id('location')
# Send location information
location_box.send_keys(location)
# Find Search button
search_button = driver.find_element_by_id('search-button')
search_button.click()
driver.implicitly_wait(5)
page_source = driver.page_source
job_soup = BeautifulSoup(page_source, "html.parser")
return job_soup
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment