How to use selenium

selenium-scrapy-guide.md

###How to selenium

Selenium is a browser automation tool used for testing elements in the browser and can also be used by our dear friend scrapy to scrape elements off the internet.

####Install selenium via pip

Run this command on your system. Make sure you have pip installed. If not then, download and run this python program and then run the command below.

pip install selenium

Here is some sample code for you to understand using selenium

from time import sleep
from selenium import webdriver

# You can select a specific browser webdriver
# Since FireFox is an open source web browser it is installed on all UNIX platforms
# So we select Firefox as our browser
firefox_driver = webdriver.Firefox()


#Navigate to website
firefox_driver.get("http://www.google.com")

# Find and click advertisement link in footer
advertisement_link = firefox_driver.find_element_by_css_selector("#fsl a._de")
advertisement_link.click()

# AND you navigate to the advertisement page. TADA !!!
# And finally close the browser after 10 seconds
sleep(10)
firefox_driver.close()