One-line arg parse for flexible testing in Python.md

One-line arg parse for flexible testing in Python

Many Python scripts with parameters say M and N look like this:

# parameters --
M = 10
N = 10
...
myfunc( M, N ... )

To run this with different M and N, you can change the source file in a text editor, again and again. Here's a more flexible way:

import sys
...

# parameters --
M = 10
N = 20
...
    # run this.py  M=11  N=21 ...  in sh or ipython
for arg in sys.argv[1:]:  exec( arg )
...
myfunc( M, N ... )

Then in IPython you can run with 0, 1 or 2 args:

run my.py N=41  # default M
run my.py       # default all

or in a Unix shell:

python my.py N=41  |  tee my-N41.log

It's easy to run several M and N, in a shell script like this:

for M in 10 11 12
do
for N in 41
do
    time python my.py  M=$M  N=$N
done
done |
tee my.log

or similarly in Python.

This one-liner does NO error checking: Did you really ?

String, list, function args

exec() can parse any Python expression, for example

run my.py  a=1  b=None  c=[3,4]  d='"a string"'  'e = func(0)'

In Ipython, put single quotes '' around args that contain blanks or double quotes ""; in shells, around args with * () [] {} <> -- anything that the shell may interpret before the exec() loop. Shells differ in what characters may be expanded, and in error messages.

Important: echo the args

Always echo the arguments to a non-trivial program, e.g.

print "%s  args: M %d  N %d  x %s  y %.3g" % (
        __file__, M, N, x, y )

after the exec() loop. Otherwise, next week you'll be asking "how did I ever ..."; or you may have something to publish in the Annals of Improbable Research .

(In fact, I usually echo a bit more:

M = 10
...
plot = 1
seed = 0

for arg in sys.argv[1:]:
    exec( arg )
np.set_printoptions( 2, threshold=100, edgeitems=10, linewidth=100, suppress=True )
np.random.seed( seed )

print 80 * "-"
print nu.From( __file__ )  # file pwd time
print "args: M %s  N %s  seed %s " % (
            M, N, seed )
nu.versions()  # numpy scipy python ...

Alternatives

Python has getopt and argparse , which are certainly better than any one-liner in several ways. But they're necessarily bigger and more complex; see the 500-odd questions/tagged/argparse on stackoverflow.

See also

More could be said about basics of developing and testing software (plan, invest in scaffolding, keep a log book ...) -- and has been said. But most people learn by doing, whether cooking or programming. I recommend Software Carpentry .

Last change: 30 Jan 2014 denis