A simple tutorial of using selenium grid in python unittests

SeleniumGridTutorial.md

Prepare files & libraries

1. Download selenium standalone server from: http://goo.gl/rQhaxb, Create local folder at ~/selenium and copy or move jar to that path:

$ mkdir ~/selenium
$ cd ~/selenium
$ wget http://goo.gl/rQhaxb
$ mv selenium-server-standalone-2.49.1.jar ~/selenium/

2. Download and install selenium WebDriver bindings for Python: https://pypi.python.org/pypi/selenium

$ wget https://pypi.python.org/packages/source/s/selenium/selenium-2.49.2.tar.gz#md5=17cfe7c6adb2cad1f64a61cf753f0738
$ tar -zxvf selenium-2.49.2.tar.gz
$ cd selenium-2.49.2
$ python2.7 setup.py install --user
$ cd ..

3. Download chrome driver from http://chromedriver.storage.googleapis.com/2.20/chromedriver_linux64.zip and copy it to ~/selenium, also create python file for unittests:

$ wget http://chromedriver.storage.googleapis.com/2.20/chromedriver_linux64.zip
$ unzip chromedriver_linux64.zip
$ touch selenium_unittests.py

Create first unittest file and run it

1. Open selenium_unittests.py in your favourite text editor like:
   • $ gedit selenium_unittests.py & or
   • $ vim selenium_unittests.py
2. Edit your file to look like this:

from selenium import webdriver
from selenium.webdriver.common.keys import Keys

driver = webdriver.Firefox()
driver.get("https://github.com")

print(driver.title)
assert "GitHub" in driver.title

elem = driver.find_element_by_name("q")
elem.send_keys("dzitkowskik")
elem.send_keys(Keys.RETURN)
assert "No results found." not in driver.page_source

driver.close()

3. Save the file
4. Open another terminal and execute following instructions:

$ cd ~/selenium
$ java -jar selenium-server-standalone-2.49.0.jar

5. Open another terminal and run your python script:

$ cd ~/selenium
$ python selenium_unittests.py

See that nothing failed and the test opened Firefox driver, went to github and searched for dzitkowskik. This was only a simple example. Little explanation:

• First two lines imports webdriver bindings
• Next we instantiate driver for Firefox web browser and go to github web page
• Another two lines print driver title and checks if GitHub exists in page title
• Next four lines searches for element name "q" which is a text box for searching and type there "dzitkowskik" with ENTER and checks if we didn't get "No results"
• Last line closes a driver which closes also our web browser

Create unittest test case

1. Lets use unittest to create tests for our application and change our program to look like this:

import unittest
from selenium import webdriver
from selenium.webdriver.common.keys import Keys

class PythonOrgSearch(unittest.TestCase):

    def setUp(self):
        self.driver = webdriver.Firefox()

    def test_search_in_python_org(self):
        driver = self.driver
        driver.get("https://github.com")
        assert "GitHub" in driver.title
        elem = driver.find_element_by_name("q")
        elem.send_keys("dzitkowskik")
        elem.send_keys(Keys.RETURN)
        assert "No results found." not in driver.page_source

    def tearDown(self):
        self.driver.close()

if __name__ == "__main__":
    unittest.main()

2. Now lets run our program in the same way as before, it looks like normal unittest doesn't it?

Remote selenium server

However selenium server does not have to be on the same machine, so we need a way to run our test remotely. The only thing we have to change is instantiation of a driver. We change it to look like this:

driver = webdriver.Remote(
   command_executor='http://127.0.0.1:4444/wd/hub',
   desired_capabilities={'browserName': 'firefox', 'javascriptEnabled': True})

Webdriver API

An overview of WebDriver API. To see more detailed description go to selenium for python docs - http://selenium-python.readthedocs.org

Finding elements

We can find elements by their properties. To find <input type="text" name="passwd" id="passwd-id" /> we can use:

element = driver.find_element_by_id("passwd-id")
element = driver.find_element_by_name("passwd")
element = driver.find_element_by_xpath("//input[@id='passwd-id']")

Actually we can find any element in a page using:

• find_element_by_id
• find_element_by_name
• find_element_by_xpath
• find_element_by_link_text
• find_element_by_partial_link_text
• find_element_by_tag_name
• find_element_by_class_name
• find_element_by_css_selector

Or we can search for many elements at once using:

• find_elements_by_name
• find_elements_by_xpath
• find_elements_by_link_text
• find_elements_by_partial_link_text
• find_elements_by_tag_name
• find_elements_by_class_name
• find_elements_by_css_selector

To write sth into a textbox send with enter:

element.send_keys("dzitkowskik", Keys.ENTER)

To move backwards and forwards in your browser’s history:

driver.forward()
driver.back()

Now set the cookie.

This one's valid for the entire domain cookie = {‘name’ : ‘foo’, ‘value’ : ‘bar’} And output all the available cookies for the current URL

driver.add_cookie(cookie)
driver.get_cookies()

One can save the current cookies as a python object using pickle - for example

import pickle
import selenium.webdriver

driver = selenium.webdriver.Firefox()
driver.get("http://www.google.com")
pickle.dump( driver.get_cookies() , open("cookies.pkl","wb"))

And later to add them back:

import pickle
import selenium.webdriver

driver = selenium.webdriver.Firefox()
driver.get("http://www.google.com")
cookies = pickle.load(open("cookies.pkl", "rb"))
for cookie in cookies:
    driver.add_cookie(cookie)

Example

Here we will see a real life example of testing GitHub's repository creation, from the point of view of an end user. You will need your github account, to create it go to: https://github.com/join and register. It is very fast, simple and free to you should not have any problem with that. Remember your login and password!

Preparation

Start with a simple code we create before and edit it to look like this:

import unittest
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.support.ui import Select

class PythonOrgSearch(unittest.TestCase):

    def setUp(self):
        self.driver = webdriver.Remote(
           command_executor='http://127.0.0.1:4444/wd/hub',
           desired_capabilities={'browserName': 'firefox', 'javascriptEnabled': True})

    def sign_in_to_github(self):
        driver = self.driver
        # Here we will implement loggin into github (PART 1)

    def create_reporitory(self):
        driver = self.driver
        # Here we will implement creating repository (PART 2)


    def delete_repository(self):
        driver = self.driver
        # Here we will implement deleting repository (PART 3)

    def test_create_delete_repository(self):
        self.sign_in_to_github()
        self.create_reporitory()
        self.delete_repository()

    def tearDown(self):
        self.driver.close()

if __name__ == "__main__":
    unittest.main()

Now we will have 3 unittests which will be executed one after another, logging into our github account, creating new repository and deleting it.

Log in to github (PART 1)

Firstly go to GitHub main web page:

driver.get("https://github.com")

Then we must click element <a class="btn" href="/login" data-ga-click="(Logged out) Header, clicked Sign in, text:sign-in">Sign in</a> to do that we may use find_elements_by_xpath method:

login_in = driver.find_element_by_xpath("//a[@href='/login']")
login_in.click()

after that we can fill the login form and click submit:

user = driver.find_element_by_id("login_field")
password = driver.find_element_by_id("password")
user.clear()
password.clear()
user.send_keys("username")
password.send_keys("password")
driver.find_element_by_xpath("//input[@type='submit']").click()

Create a new repository

Find create repository button and click it:

new_repo_buttons = driver.find_elements_by_class_name("new-repo")
if len(new_repo_buttons) > 0:
    new_repo_buttons[0].click()
else:
    print("Cannot find new repository button")

Fill form of a new repository and submit:

driver.find_element_by_name("repository[name]").send_keys("name")
(driver.find_element_by_name("repository[description]")
    .send_keys("our new repository description"))
driver.find_element_by_id("repository_auto_init").click()
driver.find_element_by_xpath("//button[@type='submit']").click()

Now we are able to log in to our account in GitHub and create public repository. To be able to run this test multiple times we may want to clean after us, deleting newly created repository.

Delete repository

First we should enter "settings tab" but we have to wait some time since it is ajax asynchronous call:

driver.find_element_by_xpath("//a[@href='/dzitkowskik/name/settings']").click()
driver.implicitly_wait(5)

Next we should click on Delete Repository and fill danger popup and commit:

driver.find_element_by_link_text("Delete this repository").click()
(driver.find_element_by_css_selector("div.facebox-content.dangerzone > form.js-normalize-submit > p > input[name=\"verify\"]")
    .clear())
(driver.find_element_by_css_selector("div.facebox-content.dangerzone > form.js-normalize-submit > p > input[name=\"verify\"]")
    .send_keys("name"))
driver.find_element_by_xpath("(//button[@type='submit'])[5]").click()

Grid

Set up a grid with many nodes for different browsers and run many instances of our example in a distributed and concurrent manner.

Set up environment:

Run the standalone server in a hub mode (open new terminal first):

$ cd ~/selenium
$ java -jar selenium-server-standalone-2.49.1.jar -role hub

After that we will see where our nodes should register themselves: INFO - Nodes should register to http://25.165.67.204:4444/grid/register/ We can register now two nodes (do it in two different terminals):

java -jar selenium-server-standalone-2.49.0.jar -role node -hub http://25.165.67.204:4444/grid/register/ -port 3456
java -jar selenium-server-standalone-2.49.0.jar -role node -hub http://25.165.67.204:4444/grid/register/ -port 4567

Ok now we have two nodes registered to our grid.

Run example many times in a row using bash script

Create bash script grid.sh and edit it to contain:

#!/bin/bash

python selenium_unittests.py &
python selenium_unittests.py &
python selenium_unittests.py &
python selenium_unittests.py &
python selenium_unittests.py &

As you can see in selenium hub log, tests are distributed through nodes:

INFO - Trying to create a new session on node http://25.165.67.204:4567
INFO - Trying to create a new session on node http://25.165.67.204:3456
...

Z_example_selenium_unittests.py

import unittest 
from selenium import webdriver 
from selenium.webdriver.common.keys import Keys 
from selenium.webdriver.support.ui import Select

class PythonOrgSearch(unittest.TestCase):

	def setUp(self):
		self.driver = webdriver.Remote(
			command_executor = 'http://127.0.0.1:4444/wd/hub',
			desired_capabilities = {
			'browserName': 'firefox',
			'javascriptEnabled': True
			})

	def sign_in_to_github(self):
		driver = self.driver
		# Here we will implement loggin into github(PART 1)
		driver.get("https://github.com")
		login_in = driver.find_element_by_xpath("//a[@href='/login']")
		login_in.click()
		user = driver.find_element_by_id("login_field")
		password = driver.find_element_by_id("password")
		user.clear()
		password.clear()
		user.send_keys("username")
		password.send_keys("password")
		driver.find_element_by_xpath("//input[@type='submit']").click()

	def create_reporitory(self):
		driver = self.driver# Here we will implement creating repository(PART 2)
		new_repo_buttons = driver.find_elements_by_class_name("new-repo")
		if len(new_repo_buttons) > 0:
			new_repo_buttons[0].click()
		else :
			print("Cannot find new repository button")

		driver.find_element_by_name("repository[name]").send_keys("name")
		(driver.find_element_by_name("repository[description]")
		.send_keys("our new repository description"))
		driver.find_element_by_id("repository_auto_init").click()
		driver.find_element_by_xpath("//button[@type='submit']").click()


	def delete_repository(self):
		driver = self.driver# Here we will implement deleting repository(PART 3)

		driver.find_element_by_xpath("//a[@href='/dzitkowskik/name/settings']").click()
		driver.implicitly_wait(5)

		driver.find_element_by_link_text("Delete this repository").click()
		(driver.find_element_by_css_selector("div.facebox-content.dangerzone > form.js-normalize-submit > p > input[name=\"verify\"]")
			.clear())
		(driver.find_element_by_css_selector("div.facebox-content.dangerzone > form.js-normalize-submit > p > input[name=\"verify\"]")
			.send_keys("name"))
		driver.find_element_by_xpath("(//button[@type='submit'])[5]").click()

	def test_create_delete_repository(self):
		self.sign_in_to_github()
		self.create_reporitory()
		self.delete_repository()

	def tearDown(self):
		self.driver.close()

if __name__ == "__main__":
	unittest.main()