Skip to content

Instantly share code, notes, and snippets.

@weakish
Created April 12, 2016 14:20
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save weakish/676050e9374730d2e25a18fc486826d8 to your computer and use it in GitHub Desktop.
Save weakish/676050e9374730d2e25a18fc486826d8 to your computer and use it in GitHub Desktop.
#cli #parsering #python #review

Review of command line parsers in python

Warning

Unfinished and unmaintained.

Reviews

tl;tr

Use click.

optparse

Deprecated.

argparse

Built-in.

It supports sub-commands.

Example:

# Adapted from https://docs.python.org/2/library/argparse.html
# sub-command functions
def foo(args):
    print args.x * args.y

# create the top-level parser
parser = argparse.ArgumentParser()
subparsers = parser.add_subparsers()

# create the parser for the "foo" command
parser_foo = subparsers.add_parser('foo')
parser_foo.add_argument('-x', type=int, default=1)
parser_foo.add_argument('y', type=float)
parser_foo.set_defaults(func=foo)

We can see that the subcommand foo has its own function. And foo accepts -x y, but -x y is not specified in the definition of foo. But all subcommand function accepts args. We dislike this style.

argh

Rewrite the above example with argh:

import argh

app = EntryPoint('Example app')

@app
# Define arguments within the subcommand function.
@arg('-x', help='int')
@arg('y', help='float')
# In Python 3 we can use parameter annotations instead.
def foo(y, x=1):
    return x * y

if __name__ == '__main__':
    app()

argh supports global arguments, but we need to directly accesses argparse to add global arguments. And we also need to exploits the undocumented pre_call hook of the Argh dispatcher to load config, set up logging and such stuff.

argdeclare

argdeclare lacks support for nested commands.

argparse-cli

Its last commit is five years ago.

django-boss

django-boss is Django-specific.

entrypoint

Its last release on pypi five years ago.

opster

opster is based on the deprecated optparse library.

Example of nested subcommand:

from opster import Dispatcher

options = [('v', 'verbose', False, 'enable additional output'),
           ('q', 'quiet', False, 'suppress output')]i

d = Dispatcher()
nestedDispatcher = Dispatcher()

@d.command()
def info(host=('h', 'localhost', 'hostname'),
         port=('p', 8080, 'port')):
    '''Return some info'''
    print("INFO")


@nestedDispatcher.command(name='action')
def action(host=('h', 'localhost', 'hostname'),
          port=('p', 8080, 'port')):
    '''Make another action'''
    print("Action")

d.nest('nested', nestedDispatcher, 'some nested application commands')

if __name__ == "__main__":
    d.dispatch(globaloptions=options)

We dislike its style:

  • Additional initialization of Dispatcher() for every sub command.
  • Use tuple instead of dictionary for parameter, thus we need to fill in every thing.

finaloption

finaloption's pypi page is dead.

simpleopt

Its last release on pypi is seven years ago.

opterator

Again opterator is based on the deprecated optparse library and does not support subcommands.

Clap

[Clap][] ships with its own parser and its last release on pypi is five years ago.

plac

plac's last release on pypi is three years ago.

baker

baker is similar with argh:

import baker

# Instead of initialing an EntryPoint, we just use `@baker.command`.
@baker.command(params={'x': 'int', 'y': 'float'})
# Instead of separate `@arg` decorators, we just supply a dictionary
# to `params`.
# Also In Python 3 we can use parameter annotations instead.
def foo(x=1, y):
    """
    Besides, we can also use Sphinx-style `:param` blocks in docstring.

    :param x: int.
    :param y: float.
    """
    return args.x * args.y

if __name__ == '__main__':
    app()

However, unlike argh, baker dose not support global options and shell completion.

plumbum

plumbum uses additional class:

from plumbum import cli

class MyApp(cli.Application):
    verbose = cli.Flag(["v", "verbose"], help = "If given, I will be very talkative")

    def main(self, filename):
        print "I will now read", filename
        if self.verbose:
            print "Yadda " * 200

if __name__ == "__main__":
    MyApp.run()

We prefer decorator style instead.

docopt

docopt defines a description language for help message. And its option parser is generated from the help message.

First we write the help message as the module docstring:

"""Naval Fate.

Usage:
  naval_fate.py ship new <name>...
  naval_fate.py ship <name> move <x> <y> [--speed=<kn>]
  naval_fate.py ship shoot <x> <y>
  naval_fate.py mine (set|remove) <x> <y> [--moored | --drifting]
  naval_fate.py (-h | --help)
  naval_fate.py --version

Options:
  -h --help     Show this screen.
  --version     Show version.
  --speed=<kn>  Speed in knots [default: 10].
  --moored      Moored (anchored) mine.
  --drifting    Drifting mine.

"""

Then we call the docopt function:

from docopt import docopt


if __name__ == '__main__':
    arguments = docopt(__doc__, version='Naval Fate 2.0')
    print(arguments)

Then it will return a dictionary of parsed arguments, for example, if we run the following command:

naval_fate.py ship Guardian move 100 150 --speed=15

We will get:

{'--drifting': False,    'mine': False,
 '--help': False,        'move': True,
 '--moored': False,      'new': False,
 '--speed': '15',        'remove': False,
 '--version': False,     'set': False,
 '<name>': ['Guardian'], 'ship': True,
 '<x>': '100',           'shoot': False,
 '<y>': '150'}

It also supports subcommands:

#! /usr/bin/env python
"""
usage: git [--version] [--exec-path=<path>] [--html-path]
           [-p|--paginate|--no-pager] [--no-replace-objects]
           [--bare] [--git-dir=<path>] [--work-tree=<path>]
           [-c <name>=<value>] [--help]
           <command> [<args>...]

options:
   -c <name=value>
   -h, --help
   -p, --paginate

The most commonly used git commands are:
   add        Add file contents to the index
   branch     List, create, or delete branches
   checkout   Checkout a branch or paths to the working tree
   clone      Clone a repository into a new directory
   commit     Record changes to the repository
   push       Update remote refs along with associated objects
   remote     Manage set of tracked repositories

See 'git help <command>' for more information on a specific command.

"""
from subprocess import call

from docopt import docopt


if __name__ == '__main__':

    args = docopt(__doc__,
                  version='git version 1.7.4.4',
                  options_first=True)
    print('global arguments:')
    print(args)
    print('command arguments:')

    argv = [args['<command>']] + args['<args>']
    if args['<command>'] == 'add':
        # In case subcommand is implemented as python module:
        import git_add
        print(docopt(git_add.__doc__, argv=argv))
    elif args['<command>'] == 'branch':
        # In case subcommand is a script in some other programming language:
        exit(call(['python', 'git_branch.py'] + argv))
    elif args['<command>'] in 'checkout clone commit push remote'.split():
        # For the rest we'll just keep DRY:
        exit(call(['python', 'git_%s.py' % args['<command>']] + argv))
    elif args['<command>'] in ['help', None]:
        exit(call(['python', 'git.py', '--help']))
    else:
        exit("%r is not a git.py command. See 'git help'." % args['<command>'])

And it has been ported to C++11, Swift, Julia, Haskell, Rust, D, Nim, PHP, R, Go, CoffeeScript, C#, C, Java, Scala, Clojure, TCL, Ruby and Lua.

With docopt, we can write the usage info exactly, but it just do basic parsing. There is no argument dispatching and callback invocation or types. So we still need to write a lot of code in addition to the basic help page.

And docopt makes composability hard. While it does support dispatching to subcommands, it for instance does not directly support any kind of automatic subcommand enumeration based on what is available or it does not enforce subcommands to work in a consistent way.

aaargh

aaargh is no longer maintained!

cliff

cliff by openstack is powerful but a bit verbose:

# main.py
import sys

from cliff.app import App
from cliff.commandmanager import CommandManager


class DemoApp(App):

    def __init__(self):
        super(DemoApp, self).__init__(
            description='cliff demo app',
            version='0.1',
            command_manager=CommandManager('cliff.demo'),
            deferred_help=True,
            )

    def initialize_app(self, argv):
        self.LOG.debug('initialize_app')

    def prepare_to_run_command(self, cmd):
        self.LOG.debug('prepare_to_run_command %s', cmd.__class__.__name__)

    def clean_up(self, cmd, result, err):
        self.LOG.debug('clean_up %s', cmd.__class__.__name__)
        if err:
            self.LOG.debug('got an error: %s', err)


def main(argv=sys.argv[1:]):
    myapp = DemoApp()
    return myapp.run(argv)


if __name__ == '__main__':
    sys.exit(main(sys.argv[1:]))

And for subcommands:

# simple.py
import logging

from cliff.command import Command


class Simple(Command):
    "A simple command that prints a message."

    log = logging.getLogger(__name__)

    def take_action(self, parsed_args):
        self.log.info('sending greeting')
        self.log.debug('debugging')
        self.app.stdout.write('hi!\n')


class Error(Command):
    "Always raises an error"

    log = logging.getLogger(__name__)

    def take_action(self, parsed_args):
        self.log.info('causing error')
        raise RuntimeError('this is the expected exception')

cement

Cement CLI Application Framework is verbose:

from cement.core import backend, foundation, controller, handler

# define an application base controller
class MyAppBaseController(controller.CementBaseController):
    class Meta:
        label = 'base'
        description = "My Application does amazing things!"

        config_defaults = dict(
            foo='bar',
            some_other_option='my default value',
            )

        arguments = [
            (['-f', '--foo'], dict(action='store', help='the notorious foo option')),
            (['-C'], dict(action='store_true', help='the big C option'))
            ]

    @controller.expose(hide=True, aliases=['run'])
    def default(self):
        self.log.info('Inside base.default function.')
        if self.pargs.foo:
            self.log.info("Recieved option 'foo' with value '%s'." % \
                          self.pargs.foo)

    @controller.expose(help="this command does relatively nothing useful.")
    def command1(self):
        self.log.info("Inside base.command1 function.")

    @controller.expose(aliases=['cmd2'], help="more of nothing.")
    def command2(self):
        self.log.info("Inside base.command2 function.")

# define a second controller
class MySecondController(controller.CementBaseController):
    class Meta:
        label = 'secondary'
        stacked_on = 'base'

    @controller.expose(help='this is some command', aliases=['some-cmd'])
    def some_other_command(self):
        pass

class MyApp(foundation.CementApp):
    class Meta:
        label = 'helloworld'
        base_controller = MyAppBaseController

# create the app
app = MyApp()

# Register any handlers that aren't passed directly to CementApp
handler.register(MySecondController)

try:
    # setup the application
    app.setup()

    # run the application
    app.run()
finally:
    # close the app
    app.close()

And arguments definition is spited into different pieces of code.

click

click by pocoo is for creating beautiful command line interfaces in a composable way with as little code as necessary.

Example

import click

@click.command()
@click.option('--count', default=1, help='Number of greetings.')
@click.option('--name', prompt='Your name',
              help='The person to greet.')
def hello(count, name):
    """Simple program that greets NAME for a total of COUNT times."""
    for x in range(count):
        click.echo('Hello %s!' % name)

if __name__ == '__main__':
    hello()

Features

  • is lazily composable without restrictions
  • fully follows the Unix command line conventions
  • supports loading values from environment variables out of the box
  • supports for prompting of custom values
  • is fully nestable and composable
  • works the same in Python 2 and 3
  • supports file handling out of the box
  • comes with useful common helpers (getting terminal dimensions, ANSI colors, fetching direct keyboard input, screen clearing, finding config paths, launching apps and editors, etc.)

Click is internally based on optparse instead of argparse. This however is an implementation detail that a user does not have to be concerned with. The reason however Click is not using argparse is that it has some problematic behaviors that make handling arbitrary command line interfaces hard.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment