zshaheen

How to Setup Automatic Uploads to Anaconda from Travis CI in 15 minutes

TL;DR: Edit .travis.yaml to install Anaconda and to run conda_upload.sh after testing. Edit meta.yaml to take in the environmental variables $VERSION and $CONDA_BLD_PATH. Create conda_upload.sh which sets the needed environmental variables, builds the tar archive, and uploads it to Anaconda. Finally edit some stuff on your Anaconda and Travis CI account so they can talk.

Intro

The following steps will detail how to automatically trigger Anaconda builds and uploads from Travis CI. This will only upload successful builds in the master branch and if there are multiple commits in a single day, it'll only keep the latest one. Both of these settings can easily be changed.

Edit .travis.yaml

First, edit .travis.yml so that it installs Anaconda.

install:

zshaheen

How to Add Diagnostic Sets to E3SM Diagnostics

Introduction

When planning to expand the current diagnostic sets, efforts have been made to structure the e3sm_diags code base in a more modularized and expandable fashion. The developers will be able to add new diagnostic sets easily and cleanly. This guide documents essential steps for guiding the users to add custom diagnostics by providing an example. If a user has a streamlined Python script for a complete analysis, from reading in files, data manipulation, computation, and visualization, it should be straightforward to take the below steps to add the analysis to e3sm_diags.

In this document, we will be explain by example the process of adding a diagnostics set into e3sm_diags. Some of the current diagnostics sets in e3sm_diags are the different kinds of plots seen here. If you have any questions or issues regarding this, please make a Github i

zshaheen

Creating a new E3SM Diagnostics Environment from Scratch

Every once in a while (fairly often), the yml environmental files in the conda folder break. It's either because:

• A dependency is removed.
• A dependency of a dependency is removed.
• ???

When this happens, you need to create a new environment from scratch. Here's how you do it.

1. On a Linux machine, use the command below to create an environment:

zshaheen

zshaheen

Running E3SM Diagnostics via the API

Getting the Environment Setup

We are running all of these tests on Cori. Here's how you setup the environment.

1. wget the environment yml file from the aprime_try1 branch.

   wget https://raw.githubusercontent.com/E3SM-Project/e3sm_diags/aprime_try1/conda/e3sm_diags_env_dev.yml
   

2. Create the environment.

zshaheen

E3SM Diags Parameters

The current diagnostics (zonal_mean_xy, zonal_mean_2d, lat_lon, polar, cosp_histogram, meridional_mean_2d) support the default parameters.

Some, like zonal_mean_2d have specific parameters that only pertain to that diagnostics, like zonal_mean_2d_plevs. As we add more diagnostic packages, the need for plot-set-specific parameter only increases. We need a way to deal with them.

The Brute-Force Solution

zshaheen

Quick guide for ANL ALCF Cooley using Singularity

Obtaining the container image

Singularity is the container runtime used at ALCF to run containers. It's an alternative to Docker, but supports Docker containers.

1. Set the SINGULARITY_PULLFOLDER environmental variable to /lus/theta-fs0/projects/ccsm/acme/containers/. This is where the containers are stored. If you're using bash, run the following.

   export SINGULARITY_PULLFOLDER="/lus/theta-fs0/projects/ccsm/acme/containers/"
   

zshaheen

How to run using Docker

1. Make sure that Docker is installed and running on your machine. To check this, run the command below and you should see some output.

   docker info
   

2. View all of the e3sm_diags images.

   docker images | grep e3sm_diags
   

   If the version you want to use is already available, then please continue to step 4.

zshaheen

#! /bin/csh -fe

### run_e3sm: your one step script to running E3SMv1
### 
### This particular instance of run_e3sm has been configured to reproduce
### the standard resolution water cycle piControl simulation on NERSC
### Edison.
###
### The script will perform the following steps:
### - clone the 'v1.0.0' source code from the E3SM github repository

zshaheen

Installing and running Slurm on greyworm and other clusters

This page will specifically detail how to install and run Slurm on the greyworm cluster at LLNL. These instructions can be adapted to work on other clusters with other nodes.

In the current configuration, aims4.llnl.gov is the head, while greyworm[1-8].llnl.gov are the compute nodes. Note: The steps here are for the current configuration of aims4 and greyworm[1-8]. The configuration might change in the future.

A Brief Intro to Slurm

Slurm is a job scheduler used to run jobs on supercomputers and clusters.