Skip to content

Instantly share code, notes, and snippets.

@samuell
Created June 4, 2014 16:11
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save samuell/c5a2a8842543d63a8fd5 to your computer and use it in GitHub Desktop.
Save samuell/c5a2a8842543d63a8fd5 to your computer and use it in GitHub Desktop.
Snippet showing how to define (and pass on) workflow-wide parameters in the luigi workflow system. Courtesy of Joe Ennever ("stolen" from https://groups.google.com/forum/#!msg/luigi-user/7zIS_qseNOM/TWcYtRe5SVoJ )
class FooParamsMixin(object):
param1 = luigi.Parameter()
param2 = luigi.Parameter()
...
def foo_params(self):
return { 'param1': self.param1, 'param2' : self.param2, ... }
class TaskA(FooParamsMixin, luigi.Task):
def requires(self):
return TaskB(**self.foo_params(), # plus any other params)
class TaskB(FooParamsMixin, luigi.Task):
pass
@jamescasbon
Copy link

I don't think you need foo_params.

Line 11 could be:

return TaskB(*self.param_args, **self.param_kwargs)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment