Continuing the theme of using conda to install all the libraries necessary for running Athena++ simulations in parallel
(see my previous gist on installing hdf5), here I'll walk through the steps I took for MPI, both for Ubuntu (20.04) and for OS X (10.11.6). First I installed mpi4py using
conda install mpi4py
Then in Ubuntu I encountered the error x86_64-conda_cos6-linux-gnu-c++ command not found, and the fix was to install gxx_linux-64:
conda install gxx_linux-64
For Mac, there's the similar error arm64-apple-darwin20.0.0-clang++: command not found (an older version was x86_64-apple-darwin13.4.0-clang++: command not found), the fix being
conda install -c conda-forge clangxx_osx-64
(for some reason, conda install clangxx_osx-64 no longer works)
At this point I was able to compile fine using
python configure.py --prob=kh -mpi --mpiccmd=mpic++
To be able to use HDF5 with MPI, you need a parallel-enabled HDF5 package. Unfortunately, I was only able to proceed with the simple method to handle the HDF5 install entirely using conda on a linux machine. On a Mac, we must build HDF5 from source (see below). For Ubuntu, all we need is one more library, and it is currently not available in the Conda-forge channel.
Thankfully, somebody has posted to Anaconda cloud a MPI-compatible HDF5 installation; install it with
conda install -c pjones hdf5-mpich
At this point, the following configure command should let you compile Athena++:
python configure.py --prob=kh -hdf5 --hdf5_path="${CONDA_PREFIX}" -mpi --mpiccmd=mpic++
There is no hdf5-mpich built for OS X, so we must install HDF5 from source instead. First, download the latest tarball (currently hdf5-1.12.0.tar.gz) from here.
There's a dependency on the compression package zlib. Install that with conda install zlib. Now find the paths to both mpicc and the zlib library. For me, they were /home/tim/anaconda3/bin and /home/tim/anaconda3/lib.
Then configure hdf5 using these paths and the --enable-parallel flag:
CC=/home/tim/anaconda3/bin/mpicc ./configure --with-zlib=/home/tim/anaconda3/lib --disable-fortran --prefix=/opt/local/ --enable-shared --enable-parallel
Finally, install HDF5 using
make
make check
sudo make install
Now Athena++ should be able to compile with MPI and parallel enabled HDF5 using
python configure.py --prob=kh -hdf5 --hdf5_path="/opt/local" -mpi --mpiccmd=mpic++
Please leave a comment below if additional steps were taken or if there are any further details that others may find helpful.
I encountered the same error today and solved it under your guidance. Thanks a lot!