Command line: create dictionary from key-value pairs

key-pairs-cmdline.md

Converting an arbitrary list of key-value pairs from the command-line into a Python dictionary

Problem

You want to create a series of key-value pairs from the command line, using the argparse library, e.g.:

command par1 par2 --set foo=hello bar="hello world" baz=5

This is typically useful when you want to clearly distinguish":

1. Ordinary arguments for the command-line utility itself (output, input, format, etc.) from
2. A set of key-value pairs you want to pass to the python application. This is especially valid when you do not want that set of values to be predetermined, as this can save a lot of code.

This possibility changes somewhat the way we look at argparse.

Solution

Standard declaration with argparse:

import argparse
parser = argparse.ArgumentParser(description="...")
...
parser.add_argument("--set",
                        metavar="KEY=VALUE",
                        nargs='+',
                        help="Set a number of key-value pairs "
                             "(do not put spaces before or after the = sign). "
                             "If a value contains spaces, you should define "
                             "it with double quotes: "
                             'foo="this is a sentence". Note that '
                             "values are always treated as strings.")
args = parser.parse_args()

The argument is optional and multivalued, with a minimum of one occurrence (nargs='+').

The result is a list of strings e.g. ["foo=hello", "bar=hello world", "baz=5"] in args.set, which we now need to parse (note how the shell has processed and removed the quotes!).

Parsing the result

def parse_var(s):
    """
    Parse a key, value pair, separated by '='
    That's the reverse of ShellArgs.

    On the command line (argparse) a declaration will typically look like:
        foo=hello
    or
        foo="hello world"
    """
    items = s.split('=')
    key = items[0].strip() # we remove blanks around keys, as is logical
    if len(items) > 1:
        # rejoin the rest:
        value = '='.join(items[1:])
    return (key, value)


def parse_vars(items):
    """
    Parse a series of key-value pairs and return a dictionary
    """
    d = {}

    if items:
        for item in items:
            key, value = parse_var(item)
            d[key] = value
    return d

# parse the key-value pairs
values = parse_vars(args.set)

Now the variable valuescontains a dictionary with the key-value pairs defined on the command line:

values = {'foo':'hello', 'bar':'hello world', 'baz':'5'}

Note that the values are always returned as strings.

I found @jsolbrig idea more suitable in my case, just fixed some typos and put a bit more safeguards:

class store_dict(argparse.Action):
    def __call__(self, parser, namespace, value, option_string=None):
        d = getattr(namespace, self.dest) or {}
        if value:
            kv = value.split('=', 1)
            k = kv[0].strip() # we remove blanks around keys, as it's logical
            d[k] = kv[1] if len(kv) > 1 else None
        setattr(namespace, self.dest, d)

if __name__ == "__main__":
        parser = argparse.ArgumentParser()
        parser.add_argument('--label', '-l',
            action=store_dict,
            type=str,
            default={ "grafana_dashboard": "1" },
            dest="labels",
            metavar="LABEL=VALUE",
            help="Set a number of key-value pairs "
            "(do not put spaces before or after the = sign). "
            "If a value contains spaces, you should define "
            "it with double quotes: "
            'foo="this is a sentence". Note that '
            "values are always treated as strings.",
        )

It's questionable, what to do if the user-supplied --label foo without actually providing the value. I decided that I don't want to raise an exception here, so in such a case you'll get { "foo": None }. Alternatives could be { "foo": "" } or an exception.