Python Autotest - Automatically run tests upon detecting writes to source(s).

.gitignore

vids

README

Really need to add some tests!!

README.mkd

autotest.py

Automatically run tests upon detecting writes to source(s).

If you're developing on Linux, you've got no reason not to be using this. Just be careful not to set off your cats!

Features

• ASCII text colors to render green/red boxes
• Audio alerts
• Desktoppings, to change desktop background (aka wallpaper)
• Tweakable sounds, colors, pictures via a per-proj or $HOME config file
• Super-fast monitoring via inotify
• Dirt simple hackable script; just wraps nosetests or whatever suite you want
• Run-any-time control; accepts single SIGINT to re-run even if no changes detected
• All-in-one test runner; trivial to use
• Minimal output, with screen clearing
• Basic stats to log your change times and passes/fails (see your patterns/habits)
• It really purrs!
• Example video!

Anti-features:

• Only runs on platforms supporting inotify (which means just linux (AFAICT).

Installation

Dependencies (some could be substituted/disabled): gconftool-2 notify-send play pyinotify

Install packages with: apt-get install gconf2 libnotify-bin sox python-pyinotify

Uses pyinotify to address performance issues inherent in busy-wait looping (python-inotifyx might be alternative).

Usage

Pass a list of files to be watched. If a source file can be tested by: % nosetests hello.py then it should work with this.

Example:

% autotest.py **/*.py  # Auto-test all Python files

Collects whatever files you specify, as opposed to just monitoring a directory/tree. This makes it easier to only test exactly the sources you want to track. See usage example by running it.

More Details

Does flamboyant notification through three mechanisms:

• red/green text bar in console where it runs
• pop-up desktop notification
• desktop background color change

You may need to set your desktop background to be a non-image to get desktop coloring to work.

There are Color codes (for color, blinking, bold, underline, etc).

You'll have to use xterm or konsole if you want the lovely blinking errors (Eterm and gnome-terminal don't support) since they're the only ttys I can find that still support it. Kinda useful to have the red fail bar blinking at you. They has a cool (probably accidental) behavior where if you give the window focus and then unfocus it will stop blinking.

Since writing this a couple years ago (and keeping it under a rock), another nice-looking version of the autotest concept has surfaced. I haven't tried it yet, but I expect that this tiny script you're looking at is still a good way to go if you want nice simple output and a means to monitor specific file(s) in a light-weight fashion, using whatever testing mechanism you want.

Other Credits

Some ideas borrowed from Jeff Winkler's nosy, the TIP list, and Django's runserver.

The green/red bars are from Kent Beck's TDD book, but they're common.

Autotest for Ruby has been around for a while. I didn't know about it when I wrote this, but I've since contributed desktoppings to it. I got the "double-interrupt" idea from it, as well as run-at-startup. It has some features to re-run full suites vs single failing files that should be considered as additions here.

Some great notification ideas/extensions (alternative popups and a few sound/image ideas) were borrowed from a Ruby autotest add-on called autotest-notification.

TODO

• A testing tool should probably have some tests! But since results are mostly visual probably not worth attempting.
• Move global variables to separate file to be tweakable params (a la mailman), or maybe should just accept CLI options.
• Move all this verbiage into a proper README.
• Convert to proper project and put up as full repo on github.
• Stats tracking.
• Convert to Python3, but pyinotify might not be working with it yet. Just run 2to3 to create patch. Trying to make this support both version is tricky due to printing of colors.
• See what tweaks are needed to get working with rdoctests.

appearances-applet.png

autotest.py

#! /usr/bin/env python

""" autotest - Automatically run tests upon detecting writes to source(s).

See README for most details.
"""

__author__    = 'Micah Elliott http://MicahElliott.com'
__version__   = '0.3'
__license__   = 'WTFPL http://sam.zoy.org/wtfpl/'

#---------------------------------------------------------------------

import sys
import os
import time
import pyinotify

### Default to nose as test runner.
# Not accommodating for runner flexibility since it should be in its own
# conf file.
TESTER = os.environ.get('AUTOTEST', 'nosetests')
WATCHED_SOURCES = []

### Choose from these bar colors.
RED_ASCII     = '\033[30;41;5m'  # blink in xterm
GREEN_ASCII   = '\033[30;42;2m'
YELLOW_ASCII  = '\033[30;43;2m'
NOCOLOR_ASCII = '\033[0m'

### Notify and Desktop background colors.
RED_RGB   = '#ff0000'
#GREEN_RGB = '#00ff00'
# More pleasant green.
GREEN_RGB = '#074507'

path = os.path.abspath( os.path.dirname(__file__) )
# Beer image: http://www.alwaysdrinking.com/images/BeerPitcher.jpg
pass_icon = os.path.join(path, 'pass.png')
# Bomb image: http://alexrossmusic.typepad.com/photos/uncategorized/2008/02/15/bomb_2.jpg
fail_icon = os.path.join(path, 'fail.png')
# Cat purring: http://en.wikipedia.org/wiki/File:Purr.ogg
PASS_SOUND = os.path.join(path, 'pass.ogg')
# Cat screaming: http://www.pawsonline.info/feline_sounds.htm
FAIL_SOUND = os.path.join(path, 'fail.ogg')
# Would be much better to put the "expected" vs "got" values into the
# message, but not bothering for now.
pass_msg = "Tests looking good!"
NOTIFY_PASS = "notify-send 'YAY HOORAY!!'  '%s' --icon '%s'" % (pass_msg, pass_icon)
fail_msg = "A test just failed. Go fix it right now!"
NOTIFY_FAIL = "notify-send 'WHOA! OH NO!!' '%s' --icon '%s'" % (fail_msg, fail_icon)
# Use gconftool-2 to change desktop background color.
desktop = "gconftool-2 -t str --set /desktop/gnome/background/primary_color '%s'"
DESKTOP_PASS = desktop % GREEN_RGB
DESKTOP_FAIL = desktop % RED_RGB

STATS = 'autotest-stats.log'

#---------------------------------------------------------------------

class ModifyEvent(pyinotify.ProcessEvent):
    def process_IN_IGNORED(self, event):
        ### This is the event that gets triggered by writes in $EDITOR.
        # Eg, in vim, ":w" will run the suite.
        # Doesn't seem quite right, but hey IIABDFI.
        ##print 'IGNORED'
        run_tests()

    def process_IN_MODIFY(self, event):
        # Not sure why MODIFY is not triggered by $EDITOR. Doing a
        #   echo foo >>somesource.py
        # will trigger MODIFY, though. Since no one codes that way I
        # guess this event will not get used.
        print "MODIFY triggered. Hmm, that's funny!"
    def process_IN_OPEN(self, event):
        # Be silent. May want to use this for some editing tasks.
        # Eg, run test when you open a file for editing.
        ##print 'detected file opened, but not running tests.'
        pass

def run_tests():
    #global WATCHED_SOURCES
    global STATS_FILE
    ### Get rid of stale runs/bars. (Shift-PgUp will get it back.)
    os.system('clear')
    ### Show yellow bar to indicate waiting to see what will happen.
    print YELLOW_ASCII, '?'*5, 'autotest:', time.asctime(), '?'*5, NOCOLOR_ASCII
    #nose.run()
    tester_cmd = TESTER + ' ' + ' '.join(WATCHED_SOURCES)
    print '*** tester_cmd', tester_cmd
    # TODO: Would be better to use popen to capture output and save
    # to send into popup notification. But then would probably need to
    # be parsing output. Better sol'n would be to call nose API
    # directly, but then we're tied to nose.
    res = os.system(tester_cmd)
    #res = os.system( TESTER + ' ' + ' '.join(WATCHED_SOURCES) )
    ### Print new bar.
    if res == 0:
        ### PASS
        print GREEN_ASCII, '+'*5, 'GREEN BAR', '+'*5, NOCOLOR_ASCII
        ##print "desktopping with:", DESKTOP_PASS
        os.system(DESKTOP_PASS)
        ##print "notifying with:", NOTIFY_PASS
        os.system(NOTIFY_PASS)
        os.system('play -q "%s" &>/dev/null' % PASS_SOUND)
    else:
        ### FAIL
        print RED_ASCII, '!'*5, 'RED BAR', '!'*5, NOCOLOR_ASCII
        os.system(DESKTOP_FAIL)
        os.system(NOTIFY_FAIL)
        os.system('play -q "%s" &>/dev/null' % FAIL_SOUND)
    STATS_FILE.write( '%s -- %s\n' % (res, time.asctime()) )

def loop(sources):
    ### Start keeping stats.
    global STATS_FILE
    STATS_FILE = file(STATS, 'w')
    ### Do initial immediate run outside loop.
    run_tests()
    # Set up monitor.
    mask = pyinotify.IN_MODIFY | pyinotify.IN_OPEN
    wm = pyinotify.WatchManager()
    notifier = pyinotify.Notifier(wm, ModifyEvent())
    print ( 'Now continuously monitoring modifications of sources:\n  %s'
            % '\n  '.join(sources) )
    print ( 'Writing stats to:\n  %s' % STATS )
    ### Continuously look for mods.
    while True:
        try:
            wm.add_watch(sources, mask)
            notifier.process_events()
            if notifier.check_events():
                notifier.read_events()
        except KeyboardInterrupt:
            print "Interrupt again to really quit."
            try:
                time.sleep(2)
                run_tests()
            except KeyboardInterrupt:
                ### Outta here!
                print 'Done test monitoring.'
                notifier.stop()
                STATS_FILE.close()
                break
        except Exception, err:
            ### Inform of error, but keep watching.
            print 'ERROR:', err

def get_sources(srcs):
    """ Remove packages (__init__.py) since their inclusion causes all
    tests to be run twice.
    """
    return [s for s in srcs if not s.endswith('__init__.py')]
#---------------------------------------------------------------------

if __name__ == '__main__':
    if len(sys.argv) > 1:
        WATCHED_SOURCES = get_sources(sys.argv[1:])
        loop(WATCHED_SOURCES)
    else:
        print>>sys.stderr, (
            'usage:   autotest file...'
            "\nexample: autotest **/*.py"
        )

fail.ogg

fail.png

gconf.png

hello.py

#! /usr/bin/env python3

import sys

def saluton():
    """ Greet in Esperanto.
    >>> saluton()
    Bonan Tagon!
    """
    # A normal comment
    ##print("Good day!") # should be red
    ### This should be big and white.
    print("Bonan Tagon!")

######################################################################
#---------------------------------------------------------------------

def adiaŭ():
    """ Greet in English (not Australian!)
    >>> adiaŭ()
    bye
    """
    print("bye")

class Besto:
    def __init__(self):
        print("in ctor")
    def diru(self):
        print("Mi ne parolas.")

class Ĉevalo(Besto):
    def diru(self):
        print("nej")


if __name__ == '__main__':
    # Run doctest when run as main script.
    import doctest
    doctest.testmod(sys.modules[__name__])

    #saluton()
    ĉ = Ĉevalo()
    ĉ.diru()

pass.ogg

pass.png