Skip to content

Instantly share code, notes, and snippets.

@ryantanaka
Last active July 22, 2020 16:03
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save ryantanaka/40d578df811049e3d798afd3690f6f58 to your computer and use it in GitHub Desktop.
Save ryantanaka/40d578df811049e3d798afd3690f6f58 to your computer and use it in GitHub Desktop.

Trigger Usage

  1. pegasus-em create <ensemble name>
  2. pegasus-em trigger <ensemble name> <trigger name> <prefix> <pattern> <interval> <workflow script>
  • At each interval, file names matching the given pattern(s) and which have a modification date s.t. (time now - interval) <= mod date < time now, will be given as arguments to the workflow script. The workflow will be planned and submitted as <ensemble name>.<prefix>_<time now or runXXX>.

  • Example:

    pegasus-em create casa
    pegasus-em trigger casa 10s_csv csv /inputs/*.csv 10m worfklow1.py
    pegasus-em trigger casa 10s_csv jpg /inputs/*.jpg 10m worfklow2.py
    .
    .
    
    #casa.csv_1000000
    #casa.csv_1000001
    #casa.jpg_1000000
    .
    pegasus-em trigger --stop casa csv
    pegasus-em trigger --stop casa jpg
    
  1. Stopping condition? Specify date?
  2. Ensemble manager supports priorities (10 being the highest). Should this be added in to the trigger command so that all workflows created as a result of that trigger receive the assigned priority?
  • pegasus-em trigger -p 10 <ensemble name> <trigger name> <prefix> <pattern> <interval> <workflow script>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment