Using robotframework with Plone

login.txt

*** Settings ***

Documentation  A test suite with a single test for valid login. This test has
...            a workflow that is created using keywords from the resource file.
Resource       resource.txt


*** Test Cases ***

Valid Login
    ${username} =  Get site owner name
    ${password} =  Get site owner password
    Open Browser To Login Page
    Input Username  ${username}
    Input Password  ${password}
    Submit Credentials
    Welcome Message is Being Shown
    [Teardown]  Close Browser

resource.txt

*** Settings ***

Documentation  A resource file containing the application specific keywords
...            that create our own domain specific language. This resource
...            implements keywords for testing HTML version of the test
...            application.
Library        Selenium2Library
Library        plone.app.theming.testing.Keywords


*** Variables ***

${SERVER}        localhost:55001
${BROWSER}       firefox
${DELAY}         0
${SITE}          plone
${LOGIN URL}     http://${SERVER}/${SITE}/login
${WELCOME URL}   http://${SERVER}/${SITE}/front-page


*** Keywords ***

Open Browser To Login Page
    Open Browser  ${LOGIN URL}  ${BROWSER}
    Maximize Browser Window
    Set Selenium Speed  ${DELAY}
    Page should contain element  css=#login-form

Go To Login Page
    Go To  ${LOGIN URL}
    Page should contain element  css=#login-form

Input Username  [Arguments]  ${username}
    Input Text  __ac_name  ${username}

Input Password  [Arguments]  ${password}
    Input Text  __ac_password  ${password}

Submit Credentials
    Click Button  Log in

Welcome Message is Being Shown
    Page should contain  You are now logged in

Login Should Have Failed
    Location Should Be  ${LOGIN URL}
    Title Should Be  Error Page

robots.cfg

#
# Step 2 - Set versions (and use some packages from source for now)
#

[buildout]
extends = buildout.cfg

[versions]
robotsuite = 0.5.0
robotframework = 2.7.3
robotframework-selenium2library = 1.0.1
docutils = 0.8.1
decorator = 3.3.3
selenium = 2.25.0

[omelette]
eggs += ${test:eggs}

setup.py

#
# Step 1 - Declare dependencies (setup.py)
#

      extras_require={
        'test': [
            'plone.app.testing',
            'robotsuite',
            'selenium', # XXX: This is here because of a missing dependency declaration in robotframework-selenium2library 1.0.1
            'decorator', # XXX: This is here because of a missing dependency declaration in robotframework-selenium2library 1.0.1
            'robotframework-selenium2library'
        ],
      },

test_robot.py

#
# Step 4 - Create a test suite using robotsuite
#


import unittest2 as unittest

from plone.testing import layered
from robotsuite import RobotTestSuite

from plone.app.theming.testing import THEMING_ACCEPTANCE_TESTING


def test_suite():
    return unittest.TestSuite([
        layered(RobotTestSuite('acceptance-tests'),
            layer=THEMING_ACCEPTANCE_TESTING),
    ])

testing.py

from plone.testing import z2
from plone.app.testing import PloneSandboxLayer
from plone.app.testing import PLONE_FIXTURE
from plone.app.testing import applyProfile

from zope.configuration import xmlconfig

from plone.app.testing.layers import IntegrationTesting
from plone.app.testing.layers import FunctionalTesting


class Theming(PloneSandboxLayer):
    defaultBases = (PLONE_FIXTURE,)

    def setUpZope(self, app, configurationContext):
        # load ZCML
        import plone.app.theming.tests
        xmlconfig.file('configure.zcml', plone.app.theming.tests, context=configurationContext)

        # Run the startup hook
        from plone.app.theming.plugins.hooks import onStartup
        onStartup(None)

    def setUpPloneSite(self, portal):
        # install into the Plone site
        applyProfile(portal, 'plone.app.theming:default')


class Keywords(object):
    """Robot Framework keyword library
    """

    def get_site_owner_name(self):
        import plone.app.testing
        return plone.app.testing.interfaces.SITE_OWNER_NAME

    def get_site_owner_password(self):
        import plone.app.testing
        return plone.app.testing.interfaces.SITE_OWNER_PASSWORD

THEMING_FIXTURE = Theming()
THEMING_INTEGRATION_TESTING = IntegrationTesting(bases=(THEMING_FIXTURE,), name="Theming:Integration")
THEMING_FUNCTIONAL_TESTING = FunctionalTesting(bases=(THEMING_FIXTURE,), name="Theming:Functional")
THEMING_ACCEPTANCE_TESTING = FunctionalTesting(bases=(THEMING_FIXTURE, z2.ZSERVER_FIXTURE,), name="Theming:Acceptance")

@datakurre: cool - I like your approach a lot. I didn't realise it was that easy to define custom keywords.

I think the robotsuite approach is the only one that makes sense, to be honest. Without it, we don't get proper test isolation or fixture management and tests need to use the UI to tear down state they set up. That's incredibly error-prone, and I've never seen it work well in other contexts.

If you look at the latest gist, we can still have a situation where you just dump files into a directory, so we can probably make this relatively easy to get started with. We could probably even create a buildout recipe that creates the relevant boilerplate. Something like the plone-devstart approach could also work, where you have a non-redistributable package for project-specific customisations.

corejet.robot is really cool. ;-)

Oh, I just realised why I used Selenium2Libarary from svn: there's an undeclared dependency on the 'decorator' module. It also doesn't pull in 'selenium' as far as I can tell. In fact, I think the released version isn't packaged with setuptools at all.

s/svn/github/ - old habits die hard ;)

I hope that @gotcha could comment about test isolation in plone.act. I guess, he's been using it without issues so far. I don't have enough low level experience with zope/plone.testing to review his isolation keywords.

I wonder, why I don't have similar issues with using the released version of selenium2library. It might be, because I've been running robot test only with Python 2.7.

A few more things to know about Robot Framework:

• Keywords are looked up case-insensitively.
• You can create BDD-style test clauses beginning with Given, When, Then or And. Those simple stripped before looking for the related keyword: http://robotframework.googlecode.com/hg/doc/userguide/RobotFrameworkUserGuide.html?r=2.7.2#behavior-driven-style
• You can use Cucumber-style embedded arguments in your custom keywords: http://robotframework.googlecode.com/hg/doc/userguide/RobotFrameworkUserGuide.html?r=2.7.2#behavior-driven-development-example

About:

Note that to filter with -t, use "bin/test -s -t robotsuite"

Would you have any objections against this behavior in the current master of robotsuite:

$ bin/test --list-tests
Listing my.app.testing.MyApp:Acceptance tests:
    Can_do_A (functional/activation.txt)
    Can_do_B (functional/activation.txt)
    Can_do_C (functional/payment.txt) #current

In summary, robotsuite.RobotTestCase is replaced with a relative path of the test (including RF-testsuite folder) and appended with Robot Framework tags.

Now you could filter tests with:

$ bin/test -t functional
$ bin/test -t activation.txt
$ bin/test -t \#current