New_Task_to_Rose-Cylc.md

Adding a new task to a suite

by Holger Wolff, CLEX CMS

Issue

In this example, we want to add an additional task before every instance of the coupler to update the sea surface temperature ancillary file.

Step 0: Make a copy of the ACCESS-CM2 suite

Run this command on accessdev to make a copy of the ACCESS-CM2 suite:

rosie copy u-br565

This command will create a copy of the original ACCESS-CM2 suite with a new suite-id that we can modify and track without interfering with the original suite.

"Suite Directory" from now on will mean the directory ${HOME}/roses/<suite-id>.

Step 1: Make a shell script to update the SST files:

Create a file called update_sst.sh and located in the directory apps/update_sst/bin under your rose suite directory:

#!/bin/bash

# Ensure that the relevant variables are set:

test -z ${FILE_TEMPLATE} && exit 1
test -z ${SST_FILE} && exit 2
test -z ${YEAR} && exit 3

# Obtain current model year f
export YEAR_FILE=${FILE_TEMPLATE/YEAR/${YEAR}}
echo ===============================================
echo Running script to update Sea Surface Temparatures
echo Input file: ${YEAR_FILE}
echo Output file: ${SST_FILE}
echo ===============================================

CMD="cp ${YEAR_FILE} ${SST_FILE}"
echo $CMD
$CMD || exit 4
echo ===============================================

Make sure that the script is executable by running the command: chmod +x update_sst.sh

Step 2: Create an option for running the task

Modify the file rose-suite.conf in your suite directory: Somewhere in the section [jinja2:suite.rc] (which should be the only section in this file), add the line:

UPDATE_SST=true

Step 2a: Create Metadata for the new option

Modify the file meta/rose-meta.conf and add this block:

[jinja2:suite.rc=UPDATE_SST]
compulsory=true
description=Run a script to update the SST files every run
help=
title=Update SST
ns=Build and Run
sort-key=3
type=boolean

More info on the metadata can be found on the Rose Documentation on Metadata

What's important is that this will add the option to switch the task on and off to the suite conf -> Build and Run window in rose.

Step 3: Add the configuration for the task

Create a new subdirectory app/update_sst and in this directory, create the file rose-app.conf:

[command]
default=update_sst.sh
[env]
FILE_TEMPLATE=/PATH/TO/SST_FILES/sst_YEAR_something.nc
SST_FILE=work/ocean/INPUT/temp_sfc_restore.nc

Note that these are the default values. There is an option to change these values in the rose editor.

Step 3a: Add the metadata for the task

Create a new subdirectory app/update_sst/meta. In this subdirectory, create the following file rose-meta.conf:

[env=FILE_TEMPLATE]
description=All SST Files
help=Template of the files that contain the sea surface temperatures by year.
    =the term YEAR will be replaced with the actual year.
compulsory=true
pattern=^.*YEAR.*$

[env=SST_FILE]
description=SST File
help=SST file expected by model
compulsory=true

Step 4: Create the environment for the new task

Edit the file suite.rc

Search for the entry [[coupled]], create a new section directly before that called [[update_sst]]:

    [[update_sst]]
        inherit = None, NCI, SHARE
        script = rose task-run --verbose
        [[[remote]]]
            host = {{ COMPUTE_HOST }}
        [[[job]]]
            batch system = background
        [[[environment]]]
            YEAR = $(cylc cyclepoint --print-year)

Notice how most of these settings are taken directly from other similar tasks.

Step 5: Add the task into the graph.

We now need to add the update_sst task to the task graph, if UPDATE_SST is true. The graph is described in two sections, [[[ R1 ]]] for the initial run, and [[[ {RESUB} ]]] for successive runs.

The second is easier to read, so we start there.

On line 86, we add the new part graph:

{% if UPDATE_SST %} filemove[-{{RESUB}}] => update_sst => coupled {% endif %}

The central location in the graph for [[[ R1 ]]] is harder to parse, manual folding of if and endifs is needed to deduce that the central graph for running the model is in line 72 and reads:

install_ancil => coupled

We add our task like this:

{% if UPDATE_SST %} install_ancil => update_sst => coupled {% endif %}

Step 6: Check in your changes.

svn add app/update_sst
svn ci -m 'include script to update SST files'

Hi Holger.

This looks good, just a couple of minor changes. In Step 1, the script should be placed in the app/update_sst/bin directory. This directory is on-path for the update_sst task. This means that [command] in app/update_sst/rose-app.conf can be set to update_sst.sh. You should also put a note in there about making sure update_sst.sh has executable permissions. That way the script is also in version control.

One other change I'd make is to add an {{{environment]]] section to the [[update_sst]] and define YEAR there instead of at the top of the script.

Thanks @dsroberts I've added your suggestions to the script.

No need to source the ancils file if you're not using it

    [[update_sst]]
        inherit = None, NCI, SHARE
        script = rose task-run --verbose

The graph only needs to be added once at each cycle, e.g.

[scheduling] 
    [[dependencies]] 
        [[[ R1 ]]] 
            graph = """
            # ...
            {% if UPDATE_SST %} update_sst => coupled {% endif %}
            """
        [[[ {{RESTART}} ]]] 
            graph = """
            # ...
            {% if UPDATE_SST %} update_sst => coupled {% endif %}
            """

assuming here it doesn't depend on some other task. All the lines from the graph section will be combined to form the task graph.

Hi @ScottWales
Thank you for your help. I have tried what you suggested, but I don't think it's working the way I expected.
The first update_sst runs before the install_ancil -- so I am worried that the sst file is overwritten immediately afterwards.
And the restart update_sst also starts before I even get to the first coupled, potentially overwriting the sst file with one for a future year.
So I left install_ancil and filemove[-{{RESUB}}] in the paths for update_sst for [[[ R1 ]]] and [[[ {{RESTART}} ]]] respectively.
Any other suggestions?