Skip to content

Instantly share code, notes, and snippets.

@gitdagray

gitdagray/kanview_selenium2.py

Created February 18, 2018 17:43
Show Gist options
  • Save gitdagray/d75092b010b0fe20b7c57f4e4e54056b to your computer and use it in GitHub Desktop.
Save gitdagray/d75092b010b0fe20b7c57f4e4e54056b to your computer and use it in GitHub Desktop.
Download ZIP
Scraping Kanview with Selenium and Beautiful Soup
Raw
kanview_selenium2.py
#Selenium hands the page source to Beautiful Soup
soup_level1=BeautifulSoup(driver.page_source, 'lxml')
datalist = [] #empty list
x = 0 #counter
for link in soup_level1.find_all('a', id=re.compile("^MainContent_uxLevel2_JobTitles_uxJobTitleBtn_")):
##code to execute in for loop goes here
@211217613
Copy link

Instead of x = 0 #counter it's better to just call the var counter:

counter = 0

:)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment