In this example we show how to adapt the example workflow so that we can run it on a cluster using the Slurm Workload Manager (Slurm).
We will write a workflow composed of two tasks, one which creates a file called foo.txt
and writes foo
in it, and one which reads foo.txt
and swaps every occurrence of foo
with the name of the cluster node on which the workflow is running.
To do this, we have to slightly change the workflow definition. In particular, we set up a runmode
parameter which will allow us to specify from the command line whether we want the workflow to be run locally or on the cluster.
When we define the tasks, we have to pass an additional SlurmInfo
object, which contains the specification of the resources we want to allocate to our tasks and other Slurm parameters.