WRF 4.1.2 installation with netCDF4, HDF5 in ubuntu 18.04 LTS

install_wrf41.sh

#!/bin/bash
## WRF installation with parallel process.
# Download and install required library and data files for WRF.
# License: LGPL
# Jamal Khan <jamal.khan@legos.obs-mip.fr>
# Tested in Ubuntu 18.04 LTS

# basic package managment
sudo apt update
sudo apt upgrade
sudo apt install gcc gfortran g++ libtool automake autoconf make m4 grads default-jre csh

## Directory Listing
export HOME=`cd;pwd`
mkdir $HOME/WRF
cd $HOME/WRF
mkdir Downloads
mkdir Library

## Downloading Libraries
cd Downloads
wget -c https://www.zlib.net/zlib-1.2.13.tar.gz
wget -c https://support.hdfgroup.org/ftp/HDF5/releases/hdf5-1.10/hdf5-1.10.5/src/hdf5-1.10.5.tar.gz
wget -c https://downloads.unidata.ucar.edu/netcdf-c/4.9.0/netcdf-c-4.9.0.tar.gz
wget -c https://downloads.unidata.ucar.edu/netcdf-fortran/4.6.0/netcdf-fortran-4.6.0.tar.gz
wget -c http://www.mpich.org/static/downloads/3.3.1/mpich-3.3.1.tar.gz
wget -c https://download.sourceforge.net/libpng/libpng-1.6.37.tar.gz
wget -c https://www.ece.uvic.ca/~frodo/jasper/software/jasper-1.900.1.zip

# Compilers
export DIR=$HOME/WRF/Library
export CC=gcc
export CXX=g++
export FC=gfortran
export F77=gfortran

# zlib
cd $HOME/WRF/Downloads
tar -xvzf zlib-1.2.13.tar.gz
cd zlib-1.2.13/
./configure --prefix=$DIR
make
make install

# hdf5 library for netcdf4 functionality
cd $HOME/WRF/Downloads
tar -xvzf hdf5-1.10.5.tar.gz
cd hdf5-1.10.5
./configure --prefix=$DIR --with-zlib=$DIR --enable-hl --enable-fortran
make check
make install

export HDF5=$DIR
export LD_LIBRARY_PATH=$DIR/lib:$LD_LIBRARY_PATH

## Install NETCDF C Library
cd $HOME/WRF/Downloads
tar -xvzf netcdf-c-4.9.0.tar.gz
cd netcdf-c-4.9.0/
export CPPFLAGS=-I$DIR/include 
export LDFLAGS=-L$DIR/lib
./configure --prefix=$DIR --disable-dap
make check
make install

export PATH=$DIR/bin:$PATH
export NETCDF=$DIR

## NetCDF fortran library
cd $HOME/WRF/Downloads
tar -xvzf netcdf-fortran-4.6.0.tar.gz
cd netcdf-fortran-4.6.0/
export LD_LIBRARY_PATH=$DIR/lib:$LD_LIBRARY_PATH
export CPPFLAGS=-I$DIR/include 
export LDFLAGS=-L$DIR/lib
export LIBS="-lnetcdf -lhdf5_hl -lhdf5 -lz" 
./configure --prefix=$DIR --disable-shared
make check
make install

## MPICH
cd $HOME/WRF/Downloads
tar -xvzf mpich-3.3.1.tar.gz
cd mpich-3.3.1/
./configure --prefix=$DIR
make
make install

export PATH=$DIR/bin:$PATH

# libpng
cd $HOME/WRF/Downloads
export LDFLAGS=-L$DIR/lib
export CPPFLAGS=-I$DIR/include
tar -xvzf libpng-1.6.37.tar.gz
cd libpng-1.6.37/
./configure --prefix=$DIR
make
make install

# JasPer
cd $HOME/WRF/Downloads
unzip jasper-1.900.1.zip
cd jasper-1.900.1/
autoreconf -i
./configure --prefix=$DIR
make
make install
export JASPERLIB=$DIR/lib
export JASPERINC=$DIR/include

############################ WRF 4.1.2 #################################
## WRF v4.1.2
## Downloaded from git tagged releases
########################################################################
cd $HOME/WRF/Downloads
wget -c https://github.com/wrf-model/WRF/archive/v4.1.2.tar.gz
tar -xvzf v4.1.2.tar.gz -C $HOME/WRF
cd $HOME/WRF/WRF-4.1.2
./clean
./configure # 34, 1 for gfortran and distributed memory
./compile em_real

export WRF_DIR=$HOME/WRF/WRF-4.1.2

## WPSV4.1
cd $HOME/WRF/Downloads
wget -c https://github.com/wrf-model/WPS/archive/v4.1.tar.gz
tar -xvzf v4.1.tar.gz -C $HOME/WRF
cd $HOME/WRF/WPS-4.1
./configure #3
./compile

######################## Post-Processing Tools ####################
## ARWpost
cd $HOME/WRF/Downloads
wget -c http://www2.mmm.ucar.edu/wrf/src/ARWpost_V3.tar.gz
tar -xvzf ARWpost_V3.tar.gz -C $HOME/WRF
cd $HOME/WRF/ARWpost
./clean
sed -i -e 's/-lnetcdf/-lnetcdff -lnetcdf/g' $HOME/WRF/ARWpost/src/Makefile
./configure #3
sed -i -e 's/-C -P/-P/g' $HOME/WRF/ARWpost/configure.arwp
./compile

######################## Model Setup Tools ########################
## DomainWizard
cd $HOME/WRF/Downloads
wget -c http://esrl.noaa.gov/gsd/wrfportal/domainwizard/WRFDomainWizard.zip
mkdir $HOME/WRF/WRFDomainWizard
unzip WRFDomainWizard.zip -d $HOME/WRF/WRFDomainWizard
chmod +x $HOME/WRF/WRFDomainWizard/run_DomainWizard

######################## Static Geography Data ####################
# http://www2.mmm.ucar.edu/wrf/users/download/get_sources_wps_geog.html
cd $HOME/WRF/Downloads
wget -c https://www2.mmm.ucar.edu/wrf/src/wps_files/geog_high_res_mandatory.tar.gz
tar -xvzf geog_high_res_mandatory.tar.gz -C $HOME/WRF


## export PATH and LD_LIBRARY_PATH
echo "export PATH=$DIR/bin:$PATH" >> ~/.bashrc
echo "export LD_LIBRARY_PATH=$DIR/lib:$LD_LIBRARY_PATH" >> ~/.bashrc

You chose option 3 while installing WPS. What is the difference with option 1 and option 3? Does it change WRF run speed?

1. Linux x86_64, gfortran (serial) ***
2. Linux x86_64, gfortran (serial_NO_GRIB2)
3. Linux x86_64, gfortran (dmpar) ***

In simple terms depar uses all of your PC processors were serial utilizes one processor to do the tasks. https://forum.mmm.ucar.edu/phpBB3/viewtopic.php?t=65

…

On Tue, Oct 19, 2021, 8:31 AM İlayda Kurşun ***@***.***> wrote: ***@***.**** commented on this gist. ------------------------------ You chose option 3 while installing WPS. What is the difference with option 1 and option 3? Does it change WRF run speed? 1. Linux x86_64, gfortran (serial) *** 2. Linux x86_64, gfortran (serial_NO_GRIB2) 3. Linux x86_64, gfortran (dmpar) *** — You are receiving this because you commented. Reply to this email directly, view it on GitHub <https://gist.github.com/5498b868d34d5ec3920f306aaae7460a#gistcomment-3932293>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ATF3TC6NCESEZCSTXTEQV4DUHVXMDANCNFSM4LSKFXLA> .

Hi Jamal,

Thanks for the posting the script. I am getting far with compiling but still can't build the WRF executables.

I'd really appreciate you looking at my compile.log file. I've been trying for days to get this installed! Do you mind if I email it to you?

Thanks in advance,
Alyce

@alycest I might be able to help

@whatheway that would be amazing.... should I just post the start of the error code here?
if [ -f ../../frame/pack_utils.o -a -f ../../frame/clog.o ] ; then
time mpif90 -w -ffree-form -ffree-line-length-none -fconvert=big-endian -frecord-marker=4 -O2 -ftree-vectorize -funroll-loops -w -ffree-form -ffree-line-length-none -fconvert=big-endian -frecord-marker=4 -I../ioapi_share -o diffwrf diffwrf.f
../../frame/pack_utils.o ../../frame/module_internal_header_util.o ../../frame/module_driver_constants.o ../../frame/module_machine.o ../../frame/module_wrf_error.o ../../frame/wrf_debug.o libwrfio_int.a ;
fi

Thank you

Hi Will, Let me know if you receive this log. Thanks. Alyce

…

On Sun, Jan 30, 2022 at 10:21 PM Will H. ***@***.***> wrote: ***@***.**** commented on this gist. ------------------------------ I might be able to help if you can send me the log file On Sun, Jan 30, 2022, 7:44 AM Alyce ST ***@***.***> wrote: > ***@***.**** commented on this gist. > ------------------------------ > > Hi Jamal, > > Great work. I've been trying for days to get WRF compiled on Ubuntu. Can I > please send you my log file? > > — > Reply to this email directly, view it on GitHub > < https://gist.github.com/5498b868d34d5ec3920f306aaae7460a#gistcomment-4047308 >, > or unsubscribe > < https://github.com/notifications/unsubscribe-auth/ATF3TC3UWBBIXBW4X6QJU6TUYU6EXANCNFSM4LSKFXLA > > . > You are receiving this because you commented.Message ID: > ***@***.***> > — Reply to this email directly, view it on GitHub <https://gist.github.com/5498b868d34d5ec3920f306aaae7460a#gistcomment-4047329>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ADCCF66EIYPTVAITE2XMI53UYVCO3ANCNFSM4LSKFXLA> . Triage notifications on the go with GitHub Mobile for iOS <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android <https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>. You are receiving this because you commented.Message ID: ***@***.***>

@whatheway Thanks for giving a helping hand to @alycest. If you do not mind, I would be interested to know also if you had diagnosed and found a solution to the problem faced by @alycest

Cheers!

Hi @amirs2a, you can change the lines where JASPERLIB and JASPERINC environment variables are set.

export JASPERLIB=$DIR/lib
export JASPERINC=$DIR/include

Can you help me?
I installed NVIDIA HPC SDK to use pgf90. And I got the following error when installing WRF.
./compile em_real

Diffwrf io_int will be built later on in this compile. No need to rerun compile.
Diffwrf io_int will be built later on in this compile. No need to rerun compile.
Diffwrf io_int will be built later on in this compile. No need to rerun compile.
Diffwrf io_int will be built later on in this compile. No need to rerun compile.
Diffwrf io_int will be built later on in this compile. No need to rerun compile.
/lib/cpp -P -nostdinc -traditional-cpp -DUSE_NETCDF4_FEATURES -DWRFIO_NCD_LARGE_FILE_SUPPORT -DEM_CORE=1 -DNMM_CORE=0 -DNMM_MAX_DIM=2600 -DDA_CORE=0 -DWRFPLUS=0 -DIWORDSIZE=4 -DDWORDSIZE=8 -DRWORDSIZE=4 -DLWORDSIZE=4 -DNONSTANDARD_SYSTEM_SUBR -DWRF_USE_CLM -DUSE_NETCDF4_FEATURES -DWRFIO_NCD_LARGE_FILE_SUPPORT -DDM_PARALLEL -DNETCDF -DHDF5 -DLANDREAD_STUB=1 -DUSE_ALLOCATABLES -Dwrfmodel -DGRIB1 -DINTIO -DKEEP_INT_AROUND -DLIMIT_ARGS -DBUILD_RRTMG_FAST=1 -DSHOW_ALL_VARS_USED=0 -DCONFIG_BUF_LEN=65536 -DMAX_DOMAINS_F=21 -DMAX_HISTORY=25 -DNMM_NEST=0 test_io_idx.F90 > test_io_idx.f
Diffwrf io_int will be built later on in this compile. No need to rerun compile.
mpif90 -r4 -i4 -w -Mfree -byteswapio -mp -Minfo=mp -Mrecursive -mp -Minfo=mp -Mrecursive -o module_internal_header_util.o -c module_internal_header_util.f
mpicc -DFSEEKO64_OK -o io_int_idx.o -c -w -O3 io_int_idx.c
gfortran: error: unrecognized command line option '-r4'; did you mean '-r'?
gfortran: error: unrecognized command line option '-i4'
gfortran: error: unrecognized command line option '-Mfree'; did you mean '-free'?
gfortran: error: unrecognized command line option '-byteswapio'
gfortran: error: unrecognized command line option '-mp'; did you mean '-p'?
gfortran: error: unrecognized command line option '-Minfo=mp'
gfortran: error: unrecognized command line option '-Mrecursive'; did you mean '-frecursive'?
gfortran: error: unrecognized command line option '-mp'; did you mean '-p'?
gfortran: error: unrecognized command line option '-Minfo=mp'
gfortran: error: unrecognized command line option '-Mrecursive'; did you mean '-frecursive'?
makefile:62: recipe for target 'module_internal_header_util.o' failed
make[3]: [module_internal_header_util.o] Error 1 (ignored)
mpif90 -r4 -i4 -w -Mfree -byteswapio -mp -Minfo=mp -Mrecursi

Hi @kcj3784, sorry but I have no experience with pg compilers - only intel and gnu. You have to select carefully on the compiler choice screen. Goodluck.

@whatheway Thanks for giving a helping hand to @alycest. If you do not mind, I would be interested to know also if you had diagnosed and found a solution to the problem faced by @alycest

Cheers!

I believe it has to do with the libraries website path changing. NCAR and a lot of the other libraries website links changed. I had to modify my scripts to reflect that change. I recommend using github links as much as possible.

@jamal919 Hello Sir your Email please .

Hi, why do we want to add export LIBS="-lnetcdf -lhdf5_hl -lhdf5 -lz" in the script?

once we configure libraries using this, am getting an error while configuring flex library for compiling WRF chem as follows..
gcc -g -O -o flex ccl.o dfa.o ecs.o gen.o main.o misc.o nfa.o parse.o scan.o skel.o sym.o tblcmp.o yylex.o libfl.a -lnetcdf -lhdf5_hl -lhdf5 -lz
/usr/bin/ld: cannot find -lnetcdf: No such file or directory
/usr/bin/ld: cannot find -lhdf5_hl: No such file or directory
/usr/bin/ld: cannot find -lhdf5: No such file or directory
collect2: error: ld returned 1 exit status
make: *** [Makefile:97: flex] Error 1

@musaidpp in your command, something is not right, it should be prefixed with a -L/directory/to/libraries/lib. This is where you should have libnetcdf.so libnetcdff.so etc. e.g. your command should look something like the following -

gcc -g -O -o flex ccl.o dfa.o ecs.o gen.o main.o misc.o nfa.o parse.o scan.o skel.o sym.o tblcmp.o yylex.o libfl.a -L/path/to/lib -lnetcdf -lhdf5_hl -lhdf5 -lz

@jamal919 i have given all the path as it is in ur script. Bt I don't understand why it's not showing in the error message. Anyhow, without the LIBS="" , the chem was compiled and this can you please tell me, why should we need that line in our bashrc or in the script?

@jamal919 i have given all the path as it is in ur script. Bt I don't understand why it's not showing in the error message. Anyhow, without the LIBS="" , the chem was compiled and this can you please tell me, why should we need that line in our bashrc or in the script?

It was needed to compile netcdf-fortran, as netcdf-fortran links to netcdf-c, and it links to hdf. But you do not need to add this line to bashrc.

hello sir @jamal919 i need some information about WRF-Ocean . please

The zlib download link needs to be updated to https://www.zlib.net/zlib-1.2.13.tar.gz. https://www.zlib.net/zlib-1.2.12.tar.gz is not available any more.

The NETCDF download links are also broken. Replace with
wget -c https://downloads.unidata.ucar.edu/netcdf-c/4.9.0/netcdf-c-4.9.0.tar.gz
wget -c https://downloads.unidata.ucar.edu/netcdf-fortran/4.6.0/netcdf-fortran-4.6.0.tar.gz

Thanks @demiangomez. Updated.

Hello, Sir

Thank you for the script, but while installing the HDF5 i got the following errors:
***** 12 FAILURES! *****
Command exited with non-zero status 1
0.50user 0.10system 0:00.61elapsed 98%CPU (0avgtext+0avgdata 4068maxresident)k
0inputs+0outputs (0major+18193minor)pagefaults 0swaps
Makefile:3143: recipe for target 'dt_arith.chkexe_' failed
make[4]: *** [dt_arith.chkexe_] Error 1
make[4]: Leaving directory '/home/brillianzha/WRF/Downloads/hdf5-1.10.5/test'
Makefile:3128: recipe for target 'build-check-s' failed
make[3]: *** [build-check-s] Error 2
make[3]: Leaving directory '/home/brillianzha/WRF/Downloads/hdf5-1.10.5/test'
Makefile:3123: recipe for target 'test' failed
make[2]: *** [test] Error 2
make[2]: Leaving directory '/home/brillianzha/WRF/Downloads/hdf5-1.10.5/test'
Makefile:2905: recipe for target 'check-am' failed
make[1]: *** [check-am] Error 2
make[1]: Leaving directory '/home/brillianzha/WRF/Downloads/hdf5-1.10.5/test'
Makefile:654: recipe for target 'check-recursive' failed
make: *** [check-recursive] Error 1

Thanks in advance

@Brillianzha, You have to check the whole output and figure out where the failure happens. Please check if the output of the ./configure script L49 to see if all the dependencies are picked up correctly.

Sir, for setting paths for libraries do we need to type the export commands in bashrc sourcefile or can we type them in WRF/Downloads directory too. please clarify

Sir, for setting paths for libraries do we need to type the export commands in bashrc sourcefile or can we type them in WRF/Downloads directory too. please clarify

If you follow the script line-by-line, it is probably better to run the export commands in the terminal directly.

However, you will notice that at the end of the above script the PATH and LD_LIBRARY_PATH is updated in the .bashrc file.

I am thankful jamal919. It took me maybe 3 weeks struggeling trying to install it myself based on the website, and having problem with export and variables and now I just launched your script and it compiles, the ideal example worked pretty fine! I can start learning now! THANKS!

I am thankful jamal919. It took me maybe 3 weeks struggeling trying to install it myself based on the website, and having problem with export and variables and now I just launched your script and it compiles, the ideal example worked pretty fine! I can start learning now! THANKS!

Hi @kermarrecg , thanks for your feedback and I am glad to know that this helped you. Happy exploring WRF. Cheers!

Prio Jamal Bhai,
I use Ubuntu 20.04 LS version from USB. I am trying to compile the WRF in my system from several weeks but have been unfortunate every time. This time I tried your processes. I followed all of your codes serially. Compilation of LIBRARIES are okay. Configuration of WRF-4.1.2 is also okay. But whenever I tried to compile the WRF, every time it returns with errors that some module files were not found, some argument not matching, and the .exe files were not created. In fact, I repeatedly got the same results from my earlier efforts with instructions from other engineers and with other WRF versions (from 3.8 to 4.1.2). Can you please knock me a little to find where the problem is?

Thanks a lot for the information. I followed your protocols for installing WRF. I installed WRF on Ubuntu 22.04 Server edition (Server Name: HPE ProLiant DL385 Gen10 Plus )

I faced some issues while following your steps, but I resolved those issues.

[1] Along with sudo apt install gcc gfortran g++ libtool automake autoconf make m4 grads default-jre csh we need to install sudo apt install unzip also

[2] WRF 4.1.2 throws some physics errors, but WRF 4.4.2 will work.

[3]For ARW Post, we need to do the following steps:

cd $HOME/WRF/Downloads
wget -c http://www2.mmm.ucar.edu/wrf/src/ARWpost_V3.tar.gz
tar -xvzf ARWpost_V3.tar.gz -C $HOME/WRF_DIR
cd $HOME/WRF/ARWpost
./clean
sed -i -e 's/-lnetcdf/-lnetcdff -lnetcdf/g' $HOME/WRF/ARWpost/src/Makefile
export NETCDF=$DIR
./configure
export GCC_VERSION=$(/usr/bin/gcc -dumpfullversion | awk '{print$1}')
export GFORTRAN_VERSION=$(/usr/bin/gfortran -dumpfullversion | awk '{print$1}')
export GPLUSPLUS_VERSION=$(/usr/bin/g++ -dumpfullversion | awk '{print$1}')
export GCC_VERSION_MAJOR_VERSION=$(echo $GCC_VERSION | awk -F. '{print $1}')
export GFORTRAN_VERSION_MAJOR_VERSION=$(echo $GFORTRAN_VERSION | awk -F. '{print $1}')
export GPLUSPLUS_VERSION_MAJOR_VERSION=$(echo $GPLUSPLUS_VERSION | awk -F. '{print $1}')
export version_10="10"
if [ $GCC_VERSION_MAJOR_VERSION -ge $version_10 ] || [ $GFORTRAN_VERSION_MAJOR_VERSION -ge $version_10 ] || [ $GPLUSPLUS_VERSION_MAJOR_VERSION -ge $version_10 ]
then
sed -i '32s/-ffree-form -O -fno-second-underscore -fconvert=big-endian -frecord-marker=4/-ffree-form -O -fno-second-underscore -fconvert=big-endian -frecord-marker=4 -fallow-argument-mismatch /g' configure.arwp
fi
sed -i -e 's/-C -P -traditional/-P -traditional/g' $HOME/WRF/ARWpost/configure.arwp
./compile
export PATH=$HOME/WRF/ARWpost/ARWpost.exe:$PATH

Hi @jamal919 , do you have the updated list of libraries for installing the version 4.4 of WPS and WRF. Kindly share them if present, cause I am encountering errors while installing

I followed the same step as suggested by @jamal919 .. and installed the following WRF and WPS versions:

For WRF: git clone --recurse-submodule https://github.com/wrf-model/WRF.git
For WPS: git clone --recurse-submodule https://github.com/wrf-model/WPS.git