zshaheen

How to Add Diagnostic Sets to E3SM Diagnostics

Introduction

When planning to expand the current diagnostic sets, efforts have been made to structure the e3sm_diags code base in a more modularized and expandable fashion. The developers will be able to add new diagnostic sets easily and cleanly. This guide documents essential steps for guiding the users to add custom diagnostics by providing an example. If a user has a streamlined Python script for a complete analysis, from reading in files, data manipulation, computation, and visualization, it should be straightforward to take the below steps to add the analysis to e3sm_diags.

In this document, we will be explain by example the process of adding a diagnostics set into e3sm_diags. Some of the current diagnostics sets in e3sm_diags are the different kinds of plots seen here. If you have any questions or issues regarding this, please make a Github i

zshaheen

Creating a new E3SM Diagnostics Environment from Scratch

Every once in a while (fairly often), the yml environmental files in the conda folder break. It's either because:

• A dependency is removed.
• A dependency of a dependency is removed.
• ???

When this happens, you need to create a new environment from scratch. Here's how you do it.

1. On a Linux machine, use the command below to create an environment:

zshaheen

Running E3SM Diagnostics via the API

Getting the Environment Setup

We are running all of these tests on Cori. Here's how you setup the environment.

1. wget the environment yml file from the aprime_try1 branch.

   wget https://raw.githubusercontent.com/E3SM-Project/e3sm_diags/aprime_try1/conda/e3sm_diags_env_dev.yml
   

2. Create the environment.

zshaheen

E3SM Diags Parameters

The current diagnostics (zonal_mean_xy, zonal_mean_2d, lat_lon, polar, cosp_histogram, meridional_mean_2d) support the default parameters.

Some, like zonal_mean_2d have specific parameters that only pertain to that diagnostics, like zonal_mean_2d_plevs. As we add more diagnostic packages, the need for plot-set-specific parameter only increases. We need a way to deal with them.

The Brute-Force Solution

zshaheen

#! /bin/csh -fe

### run_e3sm: your one step script to running E3SMv1
### 
### This particular instance of run_e3sm has been configured to reproduce
### the standard resolution water cycle piControl simulation on NERSC
### Edison.
###
### The script will perform the following steps:
### - clone the 'v1.0.0' source code from the E3SM github repository

zshaheen

Quick guide for ANL ALCF Cooley using Singularity

Obtaining the container image

Singularity is the container runtime used at ALCF to run containers. It's an alternative to Docker, but supports Docker containers.

1. Set the SINGULARITY_PULLFOLDER environmental variable to /lus/theta-fs0/projects/ccsm/acme/containers/. This is where the containers are stored. If you're using bash, run the following.

   export SINGULARITY_PULLFOLDER="/lus/theta-fs0/projects/ccsm/acme/containers/"
   

zshaheen

Adding New Plotsets to E3SM Diagnostics

zshaheen

How to run using Docker

1. Make sure that Docker is installed and running on your machine. To check this, run the command below and you should see some output.

   docker info
   

2. View all of the e3sm_diags images.

   docker images | grep e3sm_diags
   

   If the version you want to use is already available, then please continue to step 4.

zshaheen

Quick guide for NERSC Edison using Shifter

Obtaining the container image

Shifter is the container runtime used at NERSC to run containers. It's an alternative to Docker, but supports Docker containers.

1. View the e3sm_diags images available on Edison.

   shifterimg images | grep e3sm_diags
   

If the version you want to use is already available, then please continue to step 3.

zshaheen

Testing Timeseries Input with E3SM Diagnostics on ACME1/AIMS4

This is a tentative guide. It will eventually be put on the official documentation site here.

You guys can probably figure out how to run this on Edison. Just use module load, create you env, and adjust the paths in the py accordingly.

Installation

1. Follow these instructions under Environment for development here, up to and including step 4 only. Don’t do step 5.

2. Remove cdp, we need the latest from master.