Skip to content

Instantly share code, notes, and snippets.

@mlimotte
Created October 9, 2014 14:12
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save mlimotte/2693120f98adccbec655 to your computer and use it in GitHub Desktop.
Save mlimotte/2693120f98adccbec655 to your computer and use it in GitHub Desktop.
How to make step-name dynamic in a Lemur defstep
# Based on https://gist.github.com/gareth625/5d69cd883b3a154f0fa7
# Run it with `lemur run test_jobdef.clj`
(catch-args
[:run-step
"Set as the name of the step"
"lemur-is-awesome"])
(defcluster the-cluster
:app "AnApp"
:run-id "Unique"
:num-instances 2 ; Includes the master.
:master-instance-type "m1.small"
:slave-instance-type "m1.small"
:bucket "metail-s3-scratch"
:runtime-jar "s3://fake-location"
:keypair "rar"
:endpoint "elasticmapreduce.eu-west-1.amazonaws.com")
(defstep the-step
:main-class "example.main"
:step-name "x-${run-step}-x")
(fire! the-cluster the-step)
(println "---------Raw")
(println the-step)
(println "Raw form won't work, because defstep returns a regualr Map, not an evaluating map:"
(:step-name the-step))
(println "---------Evaluated 1")
(println "step-name from evaluating-map, won't work because the actual step name
comes form the command line args, which is part of 'the-cluster', and
not included here."
(:step-name
(lemur.evaluating-map/evaluating-map the-step)))
(println "---------Evaluated 2")
(println "step-name from evaluating-step. This works, because you now have an
evaluating-map and you are including the values from 'the-cluster', which
includes the catch-args stuff."
(:step-name (evaluating-step the-cluster the-step)))
; (wait-on-step the-step 864000)
; (wait-on-step (lemur.evaluating-map/evaluating-map the-step) 864000)
(wait-on-step (evaluating-step the-cluster the-step) 864000)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment