Skip to content

Instantly share code, notes, and snippets.

@haydenflinner
Created September 23, 2018 23:00
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
Star You must be signed in to star a gist
Save haydenflinner/b1bcd4fc9d74abeb0f1061b86094f501 to your computer and use it in GitHub Desktop.
invoke-magic.md

This decorator is used to derive parameters to your function from the ctx argument to the function. This is best shown by example. Suppose this configuration:

    ctx = {
        "myfuncname" : {
            "param1" : 392,
            "namedparam1" : 199
        }
    }

And this typical task

   @task
   def myfuncname(ctx, namedparam1=None, namedparam2=None):
       namedparam1 = namedparam1 or ctx.myfuncname.namedparam1
       namedparam2 = namedparam2 or ctx.myfuncname.namedparam2
       print(param1, namedparam1)

This task can be invoked from the command line like so:

       $ invoke myfuncname
       (392, 199)

Other functions/tasks can re-use our task, but it's not clear from glancing at the function header what is required and what is truly optional. The semantics of the function are hidden behind this boilerplate at the beginning. Namedparam1 and namedparam2 are really required, we just can't reveal that in the function signature, or @invoke will force the user to give one, even though we have a default in the config.

Additionally, we repeated the parameter names a bit. Although it's nothing too bad yet, it could get out of hand with more params.

One solution is something like this:

    def myfuncname(ctx, namedparam1, namedparam2):
        print(param1, namedparam1)

    @task(name=myfuncname)
    def myfuncname_task(ctx, namedparam1=None, namedparam2=None)
        namedparam1 = namedparam1 or ctx.myfuncname.namedparam1
        namedparam2 = namedparam2 or ctx.myfuncname.namedparam2
        return myfuncname(ctx, namedparam1, namedparam2)

This solution decouples the core of your code from invoke, which could be seen as a plus. However, if we were going to write this much boiler-plate and passing stuff around, we could have just stuck with argparse. Also, notice that each parameter name appears 6 times. Maybe it's not the worst nightmare for maintainability, but it sure gives writing a new re-usable task quite a lot of friction, so most just won't do it. They'll write the task, and you'll either get runtime Nones because they forgot to load a newly added param from the ctx, or you'll have a cmd-line experience so painful that people generate calls to your task from their own configs and scripts.

Here's a better solution. It mirrors the logic of the above pair of functions, but with a simple decorator instead.

    @cascading_config_task
    def myfuncname(ctx, namedparam1, namedparam2)
        return print(namedparam1, namedparam2)

The semantics of the raw python function now match the task: Both params are required when calling from python or you will get a base Python error. However, when calling from the cmdline, defaults are loaded up by invoke from all of the various config files that it searches. This gives us nice features: users can maintain their own configs setting nice defaults for your system, and you can maintain one gargantuan config for a big complex system, that users can override portions of from the cmd line as-needed.

So, try it out! It should just work provided you follow the naming convention of ctx.funcname.paramname = paramvalue. If you'd like to change the config lookup path for your function, a patch would be welcome that lets users specify the lookup path in the @cascading_config call.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment