jaredhirsch

how to run selenium tests inside the automation-tests directory in the 123done or myfavoritebeers.org repos

Node bindings don't exist for Selenium 2 API (webdriver), so we're using python bindings instead. This requires some python-centric setup, but it shouldn't take more than 15 minutes or so to get up and running.

check system-wide python requirements

You should have python 2.7 on your system (check python --version).

We have to install a bunch of python libraries. pip fetches packages; virtualenv sandboxes them. If pip and virtualenv aren't on your system already, you'll need to do this once (once per computer, not once per repo):

jaredhirsch

idea: let travis test pull requests without giving our secrets to travis

1. travis kicks off a script which talks to a test server by curling an endpoint.
    maybe something like this: curl: http://testy.personatest.org/45dc2a
  - where the curl request includes the SHA of the pull request
2. testy server kicks off deployment using some script like this: 
    - https://gist.github.com/3342964
    - which means testy server:
       - spins up an ephemeral instance
       - points sauce labs at it

jaredhirsch

#!/usr/bin/env python

# This Source Code Form is subject to the terms of the Mozilla Public
# License, v. 2.0. If a copy of the MPL was not distributed with this
# file, You can obtain one at http://mozilla.org/MPL/2.0/.

class RestmailInbox(object):
    def __init__(self, object):

        prod = True # just hard coding for the moment

jaredhirsch

<stability%> (<fail>/<success>) platform

Report of all errors with analysis by platform

100% (0/30) identity.browserid.dev.mac.firefox.14
100% (0/30) identity.browserid.dev.windows.7.firefox.13
97%  (1/30) identity.browserid.dev.windows.7.chrome
  * GIT CLONE ERROR: https://ci.mozilla.org/view/Persona/job/identity.browserid.dev.windows.7.chrome/1082/console
97%  (1/30) identity.browserid.dev.windows.xp.ie.8
  * INTERNAL SERVER ERROR (sauce labs selenium): https://ci.mozilla.org/view/Persona/job/identity.browserid.dev.windows.xp.ie.8/1067/

jaredhirsch

>               raise ConnectionError(e)
E               ConnectionError: HTTPSConnectionPool(host='restmail.net', port=443): Max retries exceeded with url: /mail/123donetest_1350930713.2574339

jaredhirsch

areds-partybus:automation-tests jhirsch$ ./node_modules/.bin/vows --xunit tests/remove-email.js 

Driving the web on session: d16c2cf63dbf47eda13bcd41405d5dcb


assert.js:102
  throw new assert.AssertionError({
        ^
      
      actual expected

jaredhirsch

==vista ff 16

1926
- returning-user
  - reason: clicking the 'useNewEmail' button didn't work. so the next element wasn't found.
  - https://saucelabs.com/jobs/48b85200cd114915bbfddc24597846f7/selenium-server.log
- remove-email
  - https://saucelabs.com/tests/345d214ed13d4a06aea41d0571878c36

1925

jaredhirsch

#!/usr/bin/env node

/* This Source Code Form is subject to the terms of the Mozilla Public
 * License, v. 2.0. If a copy of the MPL was not distributed with this
 * file, You can obtain one at http://mozilla.org/MPL/2.0/. */

var webdriver = require('wd');
browser = webdriver.remote();

browser.on('status', function(info){

jaredhirsch

ff 16/vista:

out of the last 5 failing tests,
4 failed because of failing to click "add another email address" button
1 failed because of weird exotic disgusting edge cases (ignore!)

11/26 19:03 - #2123
  - returning-user: failed to click "add another email address" button

jaredhirsch

vista/ff16: 10/10 :-)


vista/chrome: 8/10

11/27 13:01 - #2051
  - add-primary-to-secondary: setup failed. "something went wrong. someone should fix it."
  - sign-in-test: setup failed. "something went wrong. someone should fix it."

11/27 12:31 - #2045