Skip to content

Instantly share code, notes, and snippets.

@dzitkowskik
Last active May 8, 2024 06:51
Show Gist options
  • Save dzitkowskik/0fc641cf59af0dc3de62 to your computer and use it in GitHub Desktop.
Save dzitkowskik/0fc641cf59af0dc3de62 to your computer and use it in GitHub Desktop.
A simple tutorial of using selenium grid in python unittests

Prepare files & libraries

  1. Download selenium standalone server from: http://goo.gl/rQhaxb, Create local folder at ~/selenium and copy or move jar to that path:
$ mkdir ~/selenium
$ cd ~/selenium
$ wget http://goo.gl/rQhaxb
$ mv selenium-server-standalone-2.49.1.jar ~/selenium/
  1. Download and install selenium WebDriver bindings for Python: https://pypi.python.org/pypi/selenium
$ wget https://pypi.python.org/packages/source/s/selenium/selenium-2.49.2.tar.gz#md5=17cfe7c6adb2cad1f64a61cf753f0738
$ tar -zxvf selenium-2.49.2.tar.gz
$ cd selenium-2.49.2
$ python2.7 setup.py install --user
$ cd ..
  1. Download chrome driver from http://chromedriver.storage.googleapis.com/2.20/chromedriver_linux64.zip and copy it to ~/selenium, also create python file for unittests:
$ wget http://chromedriver.storage.googleapis.com/2.20/chromedriver_linux64.zip
$ unzip chromedriver_linux64.zip
$ touch selenium_unittests.py

Create first unittest file and run it

  1. Open selenium_unittests.py in your favourite text editor like:
    • $ gedit selenium_unittests.py & or
    • $ vim selenium_unittests.py
  2. Edit your file to look like this:
from selenium import webdriver
from selenium.webdriver.common.keys import Keys

driver = webdriver.Firefox()
driver.get("https://github.com")

print(driver.title)
assert "GitHub" in driver.title

elem = driver.find_element_by_name("q")
elem.send_keys("dzitkowskik")
elem.send_keys(Keys.RETURN)
assert "No results found." not in driver.page_source

driver.close()
  1. Save the file
  2. Open another terminal and execute following instructions:
$ cd ~/selenium
$ java -jar selenium-server-standalone-2.49.0.jar
  1. Open another terminal and run your python script:
$ cd ~/selenium
$ python selenium_unittests.py

See that nothing failed and the test opened Firefox driver, went to github and searched for dzitkowskik. This was only a simple example. Little explanation:

  • First two lines imports webdriver bindings
  • Next we instantiate driver for Firefox web browser and go to github web page
  • Another two lines print driver title and checks if GitHub exists in page title
  • Next four lines searches for element name "q" which is a text box for searching and type there "dzitkowskik" with ENTER and checks if we didn't get "No results"
  • Last line closes a driver which closes also our web browser

Create unittest test case

  1. Lets use unittest to create tests for our application and change our program to look like this:
import unittest
from selenium import webdriver
from selenium.webdriver.common.keys import Keys

class PythonOrgSearch(unittest.TestCase):

    def setUp(self):
        self.driver = webdriver.Firefox()

    def test_search_in_python_org(self):
        driver = self.driver
        driver.get("https://github.com")
        assert "GitHub" in driver.title
        elem = driver.find_element_by_name("q")
        elem.send_keys("dzitkowskik")
        elem.send_keys(Keys.RETURN)
        assert "No results found." not in driver.page_source

    def tearDown(self):
        self.driver.close()

if __name__ == "__main__":
    unittest.main()
  1. Now lets run our program in the same way as before, it looks like normal unittest doesn't it?

Remote selenium server

However selenium server does not have to be on the same machine, so we need a way to run our test remotely. The only thing we have to change is instantiation of a driver. We change it to look like this:

driver = webdriver.Remote(
   command_executor='http://127.0.0.1:4444/wd/hub',
   desired_capabilities={'browserName': 'firefox', 'javascriptEnabled': True})

Webdriver API

An overview of WebDriver API. To see more detailed description go to selenium for python docs - http://selenium-python.readthedocs.org

Finding elements

We can find elements by their properties. To find <input type="text" name="passwd" id="passwd-id" /> we can use:

element = driver.find_element_by_id("passwd-id")
element = driver.find_element_by_name("passwd")
element = driver.find_element_by_xpath("//input[@id='passwd-id']")

Actually we can find any element in a page using:

  • find_element_by_id
  • find_element_by_name
  • find_element_by_xpath
  • find_element_by_link_text
  • find_element_by_partial_link_text
  • find_element_by_tag_name
  • find_element_by_class_name
  • find_element_by_css_selector

Or we can search for many elements at once using:

  • find_elements_by_name
  • find_elements_by_xpath
  • find_elements_by_link_text
  • find_elements_by_partial_link_text
  • find_elements_by_tag_name
  • find_elements_by_class_name
  • find_elements_by_css_selector

To write sth into a textbox send with enter:

element.send_keys("dzitkowskik", Keys.ENTER)

To move backwards and forwards in your browser’s history:

driver.forward()
driver.back()

Now set the cookie.

This one's valid for the entire domain cookie = {‘name’ : ‘foo’, ‘value’ : ‘bar’} And output all the available cookies for the current URL

driver.add_cookie(cookie)
driver.get_cookies()

One can save the current cookies as a python object using pickle - for example

import pickle
import selenium.webdriver

driver = selenium.webdriver.Firefox()
driver.get("http://www.google.com")
pickle.dump( driver.get_cookies() , open("cookies.pkl","wb"))

And later to add them back:

import pickle
import selenium.webdriver

driver = selenium.webdriver.Firefox()
driver.get("http://www.google.com")
cookies = pickle.load(open("cookies.pkl", "rb"))
for cookie in cookies:
    driver.add_cookie(cookie)

Example

Here we will see a real life example of testing GitHub's repository creation, from the point of view of an end user. You will need your github account, to create it go to: https://github.com/join and register. It is very fast, simple and free to you should not have any problem with that. Remember your login and password!

Preparation

Start with a simple code we create before and edit it to look like this:

import unittest
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.support.ui import Select

class PythonOrgSearch(unittest.TestCase):

    def setUp(self):
        self.driver = webdriver.Remote(
           command_executor='http://127.0.0.1:4444/wd/hub',
           desired_capabilities={'browserName': 'firefox', 'javascriptEnabled': True})

    def sign_in_to_github(self):
        driver = self.driver
        # Here we will implement loggin into github (PART 1)

    def create_reporitory(self):
        driver = self.driver
        # Here we will implement creating repository (PART 2)


    def delete_repository(self):
        driver = self.driver
        # Here we will implement deleting repository (PART 3)

    def test_create_delete_repository(self):
        self.sign_in_to_github()
        self.create_reporitory()
        self.delete_repository()

    def tearDown(self):
        self.driver.close()

if __name__ == "__main__":
    unittest.main()

Now we will have 3 unittests which will be executed one after another, logging into our github account, creating new repository and deleting it.

Log in to github (PART 1)

Firstly go to GitHub main web page:

driver.get("https://github.com")

Then we must click element <a class="btn" href="/login" data-ga-click="(Logged out) Header, clicked Sign in, text:sign-in">Sign in</a> to do that we may use find_elements_by_xpath method:

login_in = driver.find_element_by_xpath("//a[@href='/login']")
login_in.click()

after that we can fill the login form and click submit:

user = driver.find_element_by_id("login_field")
password = driver.find_element_by_id("password")
user.clear()
password.clear()
user.send_keys("username")
password.send_keys("password")
driver.find_element_by_xpath("//input[@type='submit']").click()

Create a new repository

Find create repository button and click it:

new_repo_buttons = driver.find_elements_by_class_name("new-repo")
if len(new_repo_buttons) > 0:
    new_repo_buttons[0].click()
else:
    print("Cannot find new repository button")

Fill form of a new repository and submit:

driver.find_element_by_name("repository[name]").send_keys("name")
(driver.find_element_by_name("repository[description]")
    .send_keys("our new repository description"))
driver.find_element_by_id("repository_auto_init").click()
driver.find_element_by_xpath("//button[@type='submit']").click()

Now we are able to log in to our account in GitHub and create public repository. To be able to run this test multiple times we may want to clean after us, deleting newly created repository.

Delete repository

First we should enter "settings tab" but we have to wait some time since it is ajax asynchronous call:

driver.find_element_by_xpath("//a[@href='/dzitkowskik/name/settings']").click()
driver.implicitly_wait(5)

Next we should click on Delete Repository and fill danger popup and commit:

driver.find_element_by_link_text("Delete this repository").click()
(driver.find_element_by_css_selector("div.facebox-content.dangerzone > form.js-normalize-submit > p > input[name=\"verify\"]")
    .clear())
(driver.find_element_by_css_selector("div.facebox-content.dangerzone > form.js-normalize-submit > p > input[name=\"verify\"]")
    .send_keys("name"))
driver.find_element_by_xpath("(//button[@type='submit'])[5]").click()

Grid

Set up a grid with many nodes for different browsers and run many instances of our example in a distributed and concurrent manner.

Set up environment:

Run the standalone server in a hub mode (open new terminal first):

$ cd ~/selenium
$ java -jar selenium-server-standalone-2.49.1.jar -role hub

After that we will see where our nodes should register themselves: INFO - Nodes should register to http://25.165.67.204:4444/grid/register/ We can register now two nodes (do it in two different terminals):

java -jar selenium-server-standalone-2.49.0.jar -role node -hub http://25.165.67.204:4444/grid/register/ -port 3456
java -jar selenium-server-standalone-2.49.0.jar -role node -hub http://25.165.67.204:4444/grid/register/ -port 4567

Ok now we have two nodes registered to our grid.

Run example many times in a row using bash script

Create bash script grid.sh and edit it to contain:

#!/bin/bash

python selenium_unittests.py &
python selenium_unittests.py &
python selenium_unittests.py &
python selenium_unittests.py &
python selenium_unittests.py &

As you can see in selenium hub log, tests are distributed through nodes:

INFO - Trying to create a new session on node http://25.165.67.204:4567
INFO - Trying to create a new session on node http://25.165.67.204:3456
...
import unittest
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.support.ui import Select
class PythonOrgSearch(unittest.TestCase):
def setUp(self):
self.driver = webdriver.Remote(
command_executor = 'http://127.0.0.1:4444/wd/hub',
desired_capabilities = {
'browserName': 'firefox',
'javascriptEnabled': True
})
def sign_in_to_github(self):
driver = self.driver
# Here we will implement loggin into github(PART 1)
driver.get("https://github.com")
login_in = driver.find_element_by_xpath("//a[@href='/login']")
login_in.click()
user = driver.find_element_by_id("login_field")
password = driver.find_element_by_id("password")
user.clear()
password.clear()
user.send_keys("username")
password.send_keys("password")
driver.find_element_by_xpath("//input[@type='submit']").click()
def create_reporitory(self):
driver = self.driver# Here we will implement creating repository(PART 2)
new_repo_buttons = driver.find_elements_by_class_name("new-repo")
if len(new_repo_buttons) > 0:
new_repo_buttons[0].click()
else :
print("Cannot find new repository button")
driver.find_element_by_name("repository[name]").send_keys("name")
(driver.find_element_by_name("repository[description]")
.send_keys("our new repository description"))
driver.find_element_by_id("repository_auto_init").click()
driver.find_element_by_xpath("//button[@type='submit']").click()
def delete_repository(self):
driver = self.driver# Here we will implement deleting repository(PART 3)
driver.find_element_by_xpath("//a[@href='/dzitkowskik/name/settings']").click()
driver.implicitly_wait(5)
driver.find_element_by_link_text("Delete this repository").click()
(driver.find_element_by_css_selector("div.facebox-content.dangerzone > form.js-normalize-submit > p > input[name=\"verify\"]")
.clear())
(driver.find_element_by_css_selector("div.facebox-content.dangerzone > form.js-normalize-submit > p > input[name=\"verify\"]")
.send_keys("name"))
driver.find_element_by_xpath("(//button[@type='submit'])[5]").click()
def test_create_delete_repository(self):
self.sign_in_to_github()
self.create_reporitory()
self.delete_repository()
def tearDown(self):
self.driver.close()
if __name__ == "__main__":
unittest.main()
@atique24
Copy link

atique24 commented Sep 2, 2020

Hello,

That was a very good explanation of selenium grid. I can perform the same steps for Firefox and it works fine.
However, I am unable to run for Chrome or IE.
I have created a framework using the POM technique, Pytest, Unittest.
I have created conftest.py file where I am getting the browser instance from user.
I have then created a webdriverfactory.py where I have my logic for driver initialization.
if self.browser == "IE" :
# driver = webdriver.Ie()
driver = webdriver.Remote(command_executor='http://192.168.0.59:4444/wd/hub',desired_capabilities={'browserName': 'internet explorer', 'javascriptEnabled': True})

    elif self.browser == "Chrome" :
        # driver = webdriver.Chrome()
        driver = webdriver.Remote(command_executor='http://192.168.0.59:4444/wd/hub',desired_capabilities={'browserName':'chrome','javascriptEnabled':True})

    elif self.browser == "FF":
        # driver = webdriver.Firefox()
        driver = webdriver.Remote(command_executor='http://192.168.0.59:4444/wd/hub',desired_capabilities={'browserName': 'firefox', 'javascriptEnabled': True})

I have created the hubconfig.json file on my hub machine
I have created the below nodeconfig.json file on my node.
{
"capabilities":
[
{
"browserName": "internet explorer",
"platform": "WINDOWS",
"maxInstances": 5,
"seleniumProtocol": "WebDriver",
"webdriver.ie.driver": "C:/Users/A610037/PycharmProjects/drivers/IEDriverServer.exe",
"binary":"C:/Program Files/Internet Explorer/iexplore.exe"
},
{
"browserName": "chrome",
"platform": "WINDOWS",
"maxInstances": 5,
"seleniumProtocol": "WebDriver",
"webdriver.chrome.driver": "C:/Users/A610037/PycharmProjects/drivers/chromedriver.exe",
"binary":"C:/Program Files/Google/Chrome/Application/chrome.exe"
},
{
"browserName": "firefox",
"platform": "WINDOWS",
"maxInstances": 5,
"seleniumProtocol": "WebDriver",
"webdriver.gecko.driver": "C:/Users/A610037/PycharmProjects/drivers/geckodriver.exe",
"binary":"C:/Program Files/Mozilla Firefox/firefox.exe"
}
],
"proxy": "org.openqa.grid.selenium.proxy.DefaultRemoteProxy",
"maxSession": 5,
"port": 5555,
"register": true,
"registerCycle": 5000,
"hub": "http://192.168.0.59:4444",
"nodeStatusCheckTimeout": 5000,
"nodePolling": 5000,
"role": "node",
"unregisterIfStillDownAfter": 60000,
"downPollingLimit": 2,
"debug": false,
"servlets" : [],
"withoutServlets": [],
"custom": {}
}

I can establish the connection between hub and node and run test successfully for Firefox browser. But it is failing for Chrome and IE browsers. Getting the below error.

selenium.common.exceptions.SessionNotCreatedException: Message: Unable to create new service: ChromeDriverService
E Build info: version: '3.141.59', revision: 'e82be7d358', time: '2018-11-14T08:25:53'
E System info: host: 'DESKTOP-7U5MEVE', ip: '192.168.0.77', os.name: 'Windows 10', os.arch: 'x86', os.version: '10.0', java.version: '1.8.0_161'
E Driver info: driver.version: unknown
E Stacktrace:
E at org.openqa.selenium.grid.session.remote.ServicedSession$Factory.lambda$get$0 (ServicedSession.java:135)
E at org.openqa.selenium.grid.session.remote.ServicedSession$Factory.apply (ServicedSession.java:152)
E at org.openqa.selenium.remote.server.ActiveSessionFactory.lambda$apply$12 (ActiveSessionFactory.java:180)
E at java.util.stream.ReferencePipeline$3$1.accept (None:-1)
E at java.util.stream.ReferencePipeline$11$1.accept (None:-1)
E at java.util.stream.ReferencePipeline$2$1.accept (None:-1)
E at java.util.Spliterators$ArraySpliterator.tryAdvance (None:-1)
E at java.util.stream.ReferencePipeline.forEachWithCancel (None:-1)
E at java.util.stream.AbstractPipeline.copyIntoWithCancel (None:-1)
E at java.util.stream.AbstractPipeline.copyInto (None:-1)
E at java.util.stream.AbstractPipeline.wrapAndCopyInto (None:-1)
E at java.util.stream.FindOps$FindOp.evaluateSequential (None:-1)
E at java.util.stream.AbstractPipeline.evaluate (None:-1)
E at java.util.stream.ReferencePipeline.findFirst (None:-1)
E at org.openqa.selenium.remote.server.ActiveSessionFactory.apply (ActiveSessionFactory.java:183)
E at org.openqa.selenium.remote.server.NewSessionPipeline.lambda$null$2 (NewSessionPipeline.java:66)
E at java.util.stream.ReferencePipeline$3$1.accept (None:-1)
E at java.util.stream.ReferencePipeline$2$1.accept (None:-1)
E at java.util.Collections$2.tryAdvance (None:-1)
E at java.util.stream.ReferencePipeline.forEachWithCancel (None:-1)
E at java.util.stream.AbstractPipeline.copyIntoWithCancel (None:-1)
E at java.util.stream.AbstractPipeline.copyInto (None:-1)
E at java.util.stream.AbstractPipeline.wrapAndCopyInto (None:-1)
E at java.util.stream.FindOps$FindOp.evaluateSequential (None:-1)
E at java.util.stream.AbstractPipeline.evaluate (None:-1)
E at java.util.stream.ReferencePipeline.findFirst (None:-1)
E at org.openqa.selenium.remote.server.NewSessionPipeline.lambda$createNewSession$3 (NewSessionPipeline.java:69)
E at java.util.stream.ReferencePipeline$3$1.accept (None:-1)
E at java.util.stream.ReferencePipeline$3$1.accept (None:-1)
E at java.util.stream.ReferencePipeline$3$1.accept (None:-1)
E at java.util.stream.DistinctOps$1$2.accept (None:-1)
E at java.util.stream.ReferencePipeline$2$1.accept (None:-1)
E at java.util.stream.ReferencePipeline$3$1.accept (None:-1)
E at java.util.stream.ReferencePipeline$2$1.accept (None:-1)
E at java.util.stream.Streams$StreamBuilderImpl.tryAdvance (None:-1)
E at java.util.stream.Streams$ConcatSpliterator.tryAdvance (None:-1)
E at java.util.stream.ReferencePipeline.forEachWithCancel (None:-1)
E at java.util.stream.AbstractPipeline.copyIntoWithCancel (None:-1)
E at java.util.stream.AbstractPipeline.copyInto (None:-1)
E at java.util.stream.AbstractPipeline.wrapAndCopyInto (None:-1)
E at java.util.stream.FindOps$FindOp.evaluateSequential (None:-1)
E at java.util.stream.AbstractPipeline.evaluate (None:-1)
E at java.util.stream.ReferencePipeline.findFirst (None:-1)
E at org.openqa.selenium.remote.server.NewSessionPipeline.createNewSession (NewSessionPipeline.java:72)
E at org.openqa.selenium.remote.server.commandhandler.BeginSession.execute (BeginSession.java:65)
E at org.openqa.selenium.remote.server.WebDriverServlet.lambda$handle$0 (WebDriverServlet.java:235)
E at java.util.concurrent.Executors$RunnableAdapter.call (None:-1)
E at java.util.concurrent.FutureTask.run (None:-1)
E at java.util.concurrent.ThreadPoolExecutor.runWorker (None:-1)
E at java.util.concurrent.ThreadPoolExecutor$Worker.run (None:-1)
E at java.lang.Thread.run (None:-1)

I am very new to Selenium automation.
Could you please help me with this?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment