Notes of WRF.md

Notes of compiling and running WRF-SUEWS on JASMIN

Some often used Linux commands:

ls list files
ls -l
which identify the location of a executable
whereis
mkdir create directory
wget download from url: e.g. $ wget http://example.com/file.tar -O ~/path/file.tar
cd change directory (/rootpath)
tar unpack
rm remove
mv move/rename
grep "ERROR" log.compile (search in the file)
tail -f follow changes in the file
pwd work path
ln -sf

options: usually --Version = -v

Use of vi:

i
insert mode
Esc
back to command mode
:
Bottom line command mode\

Transfer files between local and remote machines:

cp log.compile ~/ (need to firstly copy the file to home, and download it with jasmin-login
scp username@jasmin-login2.ceda.ac.uk:~/log.compile ~/Downloads

Login in to JASMIN:

1. For Windows: Use mobaxterm:
   https://help.jasmin.ac.uk/article/4832-mobaxterm
2. Generate key pair
   https://help.jasmin.ac.uk/article/185-generate-ssh-key-pair
3. Update public key
   go to profile -> SSH public key -> update key
   replace the key on the website with the generated local public key
   to open the lcoal key:
   more .ssh/id_rsa_jasmin.pub
4. Login
   ssh -A username@jasmin-login2.ceda.ac.uk
ssh sci2

Compile netcdf with intel:

https://software.intel.com/content/www/us/en/develop/articles/performance-tools-for-software-developers-building-netcdf-with-the-intel-compilers.html

module load intel/**
module load intel/mpi/**

export CC=icc
export CXX=icpc
export CFLAGS='-O3 -xHost -ip -no-prec-div -static-intel'
export CXXFLAGS='-O3 -xHost -ip -no-prec-div -static-intel'
export F77=ifort
export FC=ifort
export F90=ifort
export FFLAGS='-O3 -xHost -ip -no-prec-div -static-intel'
export CPP='icc -E'
export CXXCPP='icpc -E'

export NETCDF=/home/users/sunt05/LOCAL/netcdf-4.1.3 # set the env # this path needs to be changed accordingly
cd netcdf-4.1.3
./configure --prefix=$NETCDF --disable-dap --disable-netcdf-4 --disable-shared
make # determine the procedure of compilation
make check
make install

Compile WPS with intel:

If compilation fails,

1. ./clean
2. ./configure
3. make sure: in configure.wps,

DM_FC=mpifort
DM_CC=mpiicc

4. ./compile again

WRF-SUEWS compilation

1. Download the repo:
   May need to use: git clone https://github.com/Urban-Meteorology-Reading/WRF-SUEWS.git
2. set up key for git: (before cloning SUEWS)
   https://docs.github.com/en/free-pro-team@latest/github/authenticating-to-github/generating-a-new-ssh-key-and-adding-it-to-the-ssh-agent
3. May need to set up anaconda as well:\

which python # to check which python is being used\
vi ~/.bashrc
vi ~/.bash_profile
source ~/.bashrc # rerun bashrc which run when terminal shell starts
conda activate base
which python # should become anaconda one now

then go head with make

4. After make:
   Go to xx-text-xx/ (where WRF is going to be compiled),
   Compile WRF.

Running WPS:

1. Set up namelist.wps, change the path to the directory of saved terrestrial data
2. Download terrestrial data from: http://www2.mmm.ucar.edu/wrf/users/download/get_sources_wps_geog.html
3. Download forcing data from: https://rda.ucar.edu/
4. Make sure installed grib2 related libraries and compiled WPS with 17(intel series with grib2)
5. May need to change sf_surface_physics = 9, 9, 9, to 2, 2, 2, in namelist.input (may need to be changed back later to run WRF)
6. When using WRF to generate inputs, go with: mpirun -n 30 ./real.exe
7. Check output: tail rsl.out.0000 to see whether it is successful

WSPS:

1. create the environment:
   conda env create -f environment.yml\
2. Put all required files in the directory:
   wrfinput_d0* (link to wrfinput_d*.suews)
   wrfbdy
   namelist.suews
   namelist.input (may need to be changed)\
3. Run WSPS: python wsps.py\
4. Run site-specific (make sure required data is existing)

Run WRF on other directory

wrf.exe and related files can be linked to other directory to use. Copy the whole WRF-SUEWS directory to other locations and it can be used without any compilation.

Run WRF-SUEWS

Similar with running WRF.\

1. Put all required data in the

JASMIN SLURM

1. How to submit:
   https://help.jasmin.ac.uk/article/4890-how-to-submit-a-job-to-slurm example:

#!/bin/bash
#SBATCH -p par-multi
#SBATCH -n 30
#SBATCH -o %j.log
#SBATCH -e %j.err
#SBATCH --time=48:00

module load intel/19.0.0
mpirun ./wrf.exe

2. How to submit a job array:
   https://help.jasmin.ac.uk/article/4890-how-to-submit-a-job-to-slurm

example:\

3. How to monitor: https://help.jasmin.ac.uk/article/4892-how-to-monitor-slurm-jobs

example:
squeue -u username

4. How to cancel:
   scancel -u username

Rclone

Clone from JASMIN scratch to OneDrive
Follow procedures on: https://rclone.org/onedrive/ Note that for JASMIN, additional procedures need to be followed: https://rclone.org/remote_setup/

Salem

Used for analysing WRF outputs
https://salem.readthedocs.io/en/v0.3.0/wrf.html

Python environment

Set a python environment and activate it in the script
https://docs.conda.io/projects/conda/en/latest/user-guide/tasks/manage-environments.html