Skip to content

Instantly share code, notes, and snippets.

@alexholehouse

alexholehouse/campari_install.md

Last active January 27, 2024 13:19
Show Gist options
  • Save alexholehouse/57feab526750fb1ec942 to your computer and use it in GitHub Desktop.
Save alexholehouse/57feab526750fb1ec942 to your computer and use it in GitHub Desktop.
Download ZIP
CAMPARI installation guide
Raw
campari_install.md

Installing CAMPARI from scratch

May 2014, Alex Holehouse - Pappulab

Last updated February 2016

CAMPARI can be challenging for users new to Linux/UNIX to install. To try and alleviate this, we provide here a more extended tutorial/guide for installing. This guide describes the process with both the Intel compilers (ifort, icc, and icpc) and GNU compilers too. Please note that your first port of call with respect to CAMPARI installation should be the CAMPARI documentation. This tutorial provides key information on correctly installing the dependencies (bits of software CAMPARI relies on to work). Although these dependencies are not part of CAMPARI, they can represent a barrier for getting CAMPARI running, so it seemed sensible to explicitly address their installation in the context of CAMPARI.

These steps have been tested successfully on several different Linux distros/setups (laptops, desktops, HPC cluster and Ubuntu, OpenSuse, RHEL, CentOS) using both Intel and GNU compilers. Additionally, the GNU compiler version worked without an issue on OSX 10.9.5. Because all the code is compiled from source, there is no distro-specific behavior, meaning this should be a highly portable set of instructions.

Table of contents

  1. Definitions
  • Dependencies (overview and dependency tree)
  • Shared libraries vs. static libraries
  • zlib installation
  • HDF5 installation
  • NetCDF C library installation
  • NetCDF FORTRAN library installation
  • FFTW installation
  • xdrf installation
  • CAMPARI installation
  • FAQs

Definitions

If you're not familiar with software development, I've included some definitions which should make things a little easier to understand

  • Binary (n) From a Windows perspective, this is an executable - the compiled code which runs. For most stand-alone programs, compilers takes the source code and converts it into a binary, which is what you might think of as the "software". Confusingly, as well as being a "file" (e.g. a binary file), binary is also a type of file - one written in machine code (and, also a type of encoding). So, you could say, "this file is a binary" [meaning its of the type binary] but that does not necessarily mean its a stand alone program which can be run.

  • Compile (v) The act of taking source code and, with the help of a special type of program (a compiler), converting it into a machine-readable format built specifically to run on your computer [a binary]

  • Compile time (n) the moment at which a program is compiled. Many things are required at compile time but then not thereafter. For example, the source code is needed at compile time, but can (if you really want) be deleted after. This is in contrast to run time (see below).

  • Environment variable (n) A variable which is generally available to programs running on your system. In Linux, type printenv in the shell to get a list of all the currently available environment variables. Cool, huh?

  • Library (n) A different type of software to a binary, libraries are often also compiled, but rather than being stand alone programs, they are called by other programs to complete some task. It means that if I'm really good at writing code to compute some mathematical function, I can release my work as a library and then many people can link (see below) against it when compiling. This way, you can piggy-back off other people's skill, and avoid having to write code yourself.

  • Linking (v) The act of combining source code and existing library files into one executable binary (for the purists, this actually involves first compiling the source code into object files, and then linking all the object files together with the shared or static object library files).

  • Run time (n) The moment that a program is actually executed. If you dynamically link in shared libraries, they are required at run time. On the other hand, if you use static libraries these are not then required at run time as their baked in to your binary.

  • Source code (n) AKA "source" - the software text files. These are written in a human readable/writable language. A compiler takes the source code and makes the most efficient machine code it can from that.

#1) CAMPARI dependencies CAMPARI requires a number of libraries (on top of standard compiler libraries) to be installed. In turn, each of those packages have their own dependencies. In the interest in writing this in a platform agnostic manner, the tutorial here explains how to install CAMPARI and all of its recursive dependencies from source.

CAMPARI dependency tree

The dependency tree below is meant to give you an idea of how CAMPARI is built with respect to its explicit dependencies. Included below that is a brief description of what these dependencies each do, and links for getting additional information.

CAMPARI (serial)
|
+----> FORTRAN, C, and C++ compilers
|
+----> FFTW
|
+----> xdrf
|
+----> NetCDF-FORTRAN
       |
       +----> NetCDF-C library
              |
              +----> zlib
              |
              +----> hdf5

CAMPARI_MPI (for replica exchange)
|
+----> Fortran, C, and C++ compilers (ifort, icc, icpc recommended)
|
+----> Fortran MPI-ready compiler (mpif90 recommended)
|
+----> OpenMPI 
|
+----> FFTW
|
+----> xdrf
|
+----> NetCDF-Fortran
       |
       +----> NetCDF-C library
              |
              +----> zlib
              |
              +----> hdf5

Unsurprisingly, we also need all the classic things like glibc, make etc, but if they're not already on your system you have bigger problems!

The way this guide is written is that you can follow through the same guide for both MPI and non-MPI builds - the MPI version of CAMPARI actually builds with almost identical requirements. If this is your first time building CAMPARI we strongly recommend you start with the non-MPI build before running the MPI build.

It is worth noting that the MPI build is only appropriate if you have some kind of cluster with OpenMPI installed and running. Standard CAMPARI jobs have no way to carry out domain decomposition for parallelization (primarily because for implicit solvent simulation, such decomposition is much less relevant). The upshot of this is that even on our cluster, for non replica exchange jobs we use the standard campari binary, as opposed to the campari_mpi binary. Parallelization is achieved by running multiple independent runs at the same time.

Role of dependencies

  • The compilers are required to convert FORTRAN and C code into machine code

  • zlib is a general compression/decompression library

  • HDF5 is a data model, library, and file format for storing and managing data broadly used in scientific and engineering

  • xdrf represents a portable FORTRAN library for writing XDR (External Data Representation) files. XDR is used extensively by many operating systems for a wide variety of purposes (NFS, NetCDF, NDMP, etc) and as a result is built in to nix and nix-like systems. xdrf provides a FORTRAN interface to that set of libraries which allows for the writing and reading of the high density .xtc file format, as used by CAMPARI but also other MD simulation trajectory packages such as GROMACS

  • NetCDF is an alternative binary format for storing high density numerical data, and is used by AMBER among others

  • FFTW (AKA fftw3) is a subroutine library for computing the discrete Fourier transform (DFT) in one or more dimensions, and is required for fast Particle Mesh Ewald (PME) summation

#2) Introduction to shared and static libraries

Before we begin its a good idea to introduce the idea of shared libraries and static libraries. This is especially important for compiling CAMPARI, given we're building everything from scratch, so is worth your attention.

Shared libraries (dynamic linking)

Shared libraries (.so) files are compiled code which a running program will call at runtime to execute some functionality. By default, any .so file located in the directories defined in the $LD_LIBRARY_PATH environmental variable will be accessible to any executed program at runtime. Why do this? Well say we have 100 programs, all of which want to accurately calculate the square root of a big number. They could all include the code to do this themselves, but a more efficient way would be to have one copy of that square root code and tell each program where that code is located. We now only need to maintain one copy, so if there's some new advance in the field of getting the square root of numbers, we only have to update one shared library instead of many different files.

The downside of shared libraries is you're basically promising your code, at compile time, that the shared library will be available at runtime in a specific version, as defined by your machine at compile time. Say I compile a binary on a computer that has version 1 of a shared library, and then I copy the binary I just created to another computer which only has version 2, there's no guarantee that my program is going to run. Worse - what if computer 2 doesn't even have the library at all?!

Shared libraries are often [basically always] the preferred means of providing standard code which is easily available, not least because it means I, as a software developer, don't need to worry about having my users recompile the code every time the library is updated. It also means the compiled binary code is much smaller, because I can rely on other libraries being located elsewhere. However, it can also introduce portability issues, dependency hell, and a bunch of other problems. That being said, often, this is a better option that static libraries.

Static libraries

Static libraries are, in many ways, the flip side of the coin. Here, rather than promising the compiler the code will be available in the future, you're saying, "Who knows if/where the code will be - take this library and directly incorporate it into the binary you're creating". The benefit of this is you can create a single binary file which should, in theory and if done correctly, work on other machines without dependencies.

The downsides here is that if the library changes you'll have to recompile, your binary is now potentially much bigger, and there may still be some dependencies you haven't covered. Moreover, there is the potential for massive security vulnerability due to the implicit roles of some of the types of libraries you could link in. As a result, GNU based compilers do not allow complete static linking at all. You can create a fully static binary with the Intel compilers. For new users, static libraries seem way more attractive, but, practically, they are generally bad practice.

Relating this to the CAMPARI build process

For all the dependency libraries here (NetCDF, zlib, etc.) please follow the instructions provided, which compiles them using shared libraries, but creating both shared and static libraries. However, for CAMPARI itself, you can choose if you want to compile with the shared or the static library - this will be covered how to do this later. Recall that for GNU compilers the most static you can make will only remove the dependencies we cover here (which may be enough if you're compiling on a workstation and uploading to a cluster).

The default OpenMPI compilation (not covered here) does not build with static libraries, and we recommend using the shared library approach for OpenMPI regardless of what you do for the other libraries (because MPI can be challenging to debug at the best of times, so adding potential library compatibility is not something you want!).

#3) Step-by-step guide

Do you have Intel compilers available?

Run

which ifort
which icc
which icpc

If these all return errors you will have to compile with the GNU compilers - gfort (Fortran), gcc (C) and g++ (C++), which is fine, but be aware that a number of these steps specify Intel compiler and Intel compiler flags. I've included the corresponding GNU-ready commands too. We prefer the Intel compilers as they tend to provide better performance.

  1. Installing zlib

1) Download zlib from http://www.zlib.net/

2) If using Intel compilers, set the compiler environmental variables to ensure we compile with the Intel compilers

The following works in BASH - CSH will use setenv

  export CC="icc"
  export CPP="icpc"

3) Run configure. As an example, on one of our clusters we configure zlib with the following command

 ./configure --prefix=/home/alex/campari/local_libs/local

The --prefix flag tells configure where the software should be installed. This is useful because the default location requires root access, which on a cluster you may not have. It's generally advisable to install CAMPARI in user space, though obviously installing in the default location is fine. 4) run make

make

5) run install

make install

Installing HDF5

1) Download HDF5 from https://support.hdfgroup.org/ftp/HDF5/releases/

2) Unpack

3) Run configure

For our local system we install with the following command

./configure --with-zlib=/home/alex/campari/local_libs/local --prefix=/home/alex/campari/local_libs/local --enable-fortran --enable-production --enable-filters=all CC=icc CXX=icpc FC=ifort

this configures how you want to compile - you can define the prefix= as where you want the program to be installed to, but its a good idea to install everything we're going to install to the same place.

For GNU compilers this becomes

./configure --with-zlib=/home/alex/campari/local_libs/local --prefix=/home/alex/campari/local_libs/local --enable-fortran --enable-production --enable-filters=all

4) Run make

make

NOTE that if you use the GNU compilers, the HDF5 make process will generate about a million warnings assuming the various warning flags are set on (which they are by default). Don't worry about this. For both the Intel and GNU compilers, this is a surprisingly long make process.

5) run install

make install

Installing NetCDF (2 parts - C library and Fortran library

Historically the C and Fortran libraries were combined. However, as of the 4.2 release you now have to install the C library first, followed by the Fortran library

###Part I - C LIBRARY INSTALLATION

1) Download the most recent stable NetCDF C-library from http://www.unidata.ucar.edu/downloads/netcdf/index.jsp (i.e. "The Latest Stable netCDF-C Release, tar.gz form ")

2) Unpack tar -vxf netcdf-4.3.2.tar.gz

3) Set environment variables for compilation (the preferred method for passing system specific information according to the NetCDF team). Crucially, there are three variables you must set according to where you install hdf5 and zlib

export CPPFLAGS=-I/home/alex/campari/local_libs/local/include
export LDFLAGS=-L/home/alex/campari/local_libs/local/lib
export LD_LIBRARY_PATH=/home/alex/campari/local_libs/local/lib:/home/alex/campari/local_libs/local:$LD_LIBRARY_PATH

Note that --with-hdf5 is now deprecated and does not work.

4) Run configure.

Figuring out the combination of environment and configure variables to set for this to work was a complete nightmare, as there is a massive amount of conflicting/wrong information online. The preceding export commands with the following configure work for us. If you're using GNU compilers just knock off the final three arguments (which tell configure to use Intel compilers)

As an example for our system we use

./configure --prefix=/home/alex/campari/local_libs/local --enable-netcdf4  CC=icc CXX=icpc FC=ifort

or for GNU compilers

./configure --prefix=/home/alex/campari/local_libs/local --enable-netcdf4

5) Run make

make

6) run install

make install

###Part II - FORTRAN LIBRARY INSTALLATION

7) We now have to download the Fotran library from https://www.unidata.ucar.edu/downloads/netcdf/netcdf-fortran/index.jsp

8) As before, unpack with

tar -vxf netcdf-fortran-4.2.tar.gz

9) Again, ensure the correct LD_LIBRARY_PATH, CPPFLAGS, and LDFLAGS variable is set (note, if you're still in the same terminal/shell as before you don't need to re-export this, as the variables are still set from when you installed the C library)

export LD_LIBRARY_PATH=/home/alex/campari/local_libs/local/lib:/home/alex/campari/local_libs/local:$LD_LIBRARY_PATH
export CPPFLAGS=-I/home/alex/campari/local_libs/local/include 
export LDFLAGS=-L/home/alex/campari/local_libs/local/lib

10) Run configure

Very similar to the C library process

./configure --prefix=/home/alex/campari/local_libs/local/ CC=icc CXX=icpc FC=ifort

or for GNU compilers

./configure --prefix=/home/alex/campari/local_libs/local

11) Run make

make

12) run install

make install

13) Run check

Once complete you should see something like;

+-------------------------------------------------------------+
| Congratulations! You have successfully installed the netCDF |
| Fortran libraries.                                          |
|                                                             |
| You can use script "nf-config" to find out the relevant     |
| compiler options to build your application. Enter           |
|                                                             |
|     nf-config --help                                        |
|                                                             |
| for additional information.                                 |
|                                                             |
| CAUTION:                                                    |
|                                                             |
| If you have not already run "make check", then we strongly  |
| recommend you do so. It does not take very long.            |
|                                                             |
| Before using netCDF to store important data, test your      |
| build with "make check".                                   |
|                                                             |
| NetCDF is tested nightly on many platforms at Unidata       |
| but your platform is probably different in some ways.       |
|                                                             |
| If any tests fail, please see the netCDF web site:          |
| http://www.unidata.ucar.edu/software/netcdf/                |
|                                                             |
| NetCDF is developed and maintained at the Unidata Program   |
| Center. Unidata provides a broad array of data and software |
| tools for use in geoscience education and research.         |
| http://www.unidata.ucar.edu                                 |
+-------------------------------------------------------------+`

As suggested, its a good idea to run make check once everything is installed. This just makes sure the various code works and can be linked against. While there are a whole bunch of tests, for a release candidate (stable release) its generally the case that if the first few tests run OK you're good to go. This check process will take a little time (20 minutes?) so grab a coffee and wait it out.

Installing FFTW

  1. Download from http://www.fftw.org/

  2. Run configure basically as we have before

    ./configure --prefix=/home/alex/campari/local_libs/local/ CC=icc CXX=icpc F77=ifort --enable-mpi

Note that FFTW doesn't depend on anything. Considering this, you could always install it first if you preferred. Again, we're specifying the Intel compilers here NOTE that the Fortran compiler is defined with the F77 flag here, not the FF. Additionally, note the --enable-mpi flag, which means the MPI libraries are to be compiled too. Note that, as of right now, the MPI libraries aren't, as far as I can tell, actually used in CAMPARI. However, its probably not a bad idea to just compile them and have done, given how widely used FFTW is. You're bound to need them at some point, maybe...

For GNU compilers

./configure --prefix=/home/alex/campari/local_libs/local/ --enable-mpi

3) Run make

make

4) run install

make install

5) Run check make check

Installing xdrf

1) CAMPARI used to provide the xdrf-basic tarball bundled with the main CAMPARI download. This could be obtained by downloading the full CAMPARI zip or tar file from http://campari.sourceforge.net/, and after extracting the CAMPARI archive, in the root directory there was an XDR tarball.

The CAMPARI zip zip file could be unpacked using:

unzip campari_v2_04172014.zip

The CAMPARI tarball can be unpackaged using:

tar -vxf xdr_basic.tar.bz2

NOTE In the most recent version of CAMPARI this tarball is no longer present. It can be downloaded from https://github.com/Pappulab/xdrf (hit the 'Download Zip' link) and unzip. Note the libxdrf_v1.2.tgz file (which is unpacked into the main directry) can be ignored - it is simply the repository in tarball format.

2) Set the CC and FZZ variables to Intel compilers if you're using them

  export CC="icc"
  export F77="ifort"

Alternativly you can set the compiler options in the Makefile.

3) Make

The xdrf static library can be compiled simply by moving into the unpacked directory which contains the Makefile and running make.

 make

4) Build and run tests

Run

make ftest
make ctest

To build the FORTRAN and C test programs, and then run

./ftest
./ctest

This will generate the test.xdr and test.out files. Check these exist and no error messages occurred during the test (no output is expected). The test.out file is a 5000 line text file of size 132K, while the test.xdr is that same data compressed and should be only 53K.

5) Copy the libxdrf.a file and xdrf.h files from this directory into the appropriate lib/ and include/ directories.

As an example, in our previous three pieces of softwares (zlib, HDF5 and NetCDF) these were installed to /home/alex/campari/local_libs/local. Within this directory are four folders;

bin/         # binary files
include/     # header files
lib/         # shared and static library files
share/       # other files (examples, man pages etc)

We want to move the xdrf.h header file into the /home/alex/campari/local_libs/local/include directory, and the libxdrf.a file in the /home/alex/campari/local_libs/local/lib directory.

Installing CAMPARI

OK - here we go with the main event!

In the interest of completeness, I've included a description of what is going. I always find it helpful to know what various components are doing, as it makes identifying obvious issues much more straight forward.

First thing to unzip the campari_v2_[DATE].zip file (NB: you have already done this to the the xdrf library)

unzip campari_v2_04172014.zip

And then move into the /source directory from that unzipped directory

cd campari/source/

This directory is full of the CAMPARI source code (.f90 files) and a few compilation files which are discussed below.

make_dependencies.sh, Makefile, and Makefile.local

Rather than use a configure script to define the Makefile, CAMPARI uses a shell script (make_dependencies.sh) and a customizable Makefile.local file to determine the system specific compilation behaviour in a highly configurable manner.

make_dependencies.sh constructs a makefile format file (DEPENDENCIES) which tells make which files depend on which other files for efficient compilation. By building this dynamically, adding additional files or modules involves simply adding the filename prefixes to the make_dependencies script (i.e. the the file name without the .f90 extension), and everything else is built and dealt with automatically.

The Makefile is the master Makefile which deals with this DEPENDENCIES file and carries out compilation. However, rather than use a configure script to define the compilers and libraries to use, a Makefile.local file acts as a user-configurable file which is sourced by the Makefile before compilation and linking. It's this file where things like library location and compiler choice are made. How to manipulate the Makefile.local is described below.

Compiling CAMPARI

1) Execute the make_dependencies.sh script

./make_dependencies.sh . # don't forget the period!

As described above, this builds a new DEPENDENCIES file

2) Ensure your dynamic linking library variables are set

export LD_LIBRARY_PATH=/home/alex/campari/local_libs/local/lib:$LD_LIBRARY_PATH
export LDFLAGS=-L/home/alex/campari/local_libs/local/lib

3) Edit (or create) the Makefile.local in the following way

3-A) The same FORTRAN compiler used previously should be used for CAMPARI compilation (this is defined by the FF keyword))

3-B) The COMPDEFAULTS should be set to to one of the following values.

COMPDEFAULTS=${INTELDEFAULTS}  # if ifort (Intel Fortran compiler) is used
COMPDEFAULTS=${SUNDEFAULTS}    # if the Sun FORTRAN compiler is used
COMPDEFAULTS=${GNUDEFAULTS}    # if gfortran (GNU Fortran compiler) is used

These three variables (INTELDEFAULTS, SUNDEFAULTS, and GNUDEFAULTS) are defined in the Makefile file and contain default compiler options to optimize CAMPARI compilation depending on which type of compiler you're using.

3-C) The FFLAGS variable should contain an argument in the form of

-I/location/of/the/include/folder

This -I argument tells the compiler to include relevant files found in the defined directory. If you installed the various programs above (netcdf/xdrf/fftw) such that their include folders are all in different locations you'll need an -I\LOCATION\ argument for each program's include/ folder. In our examples, we always set the prefix as /home/alex/campari/local_libs/local/, which means all the programs installed their include files to /home/alex/campari/local_libs/local/include.

The final step here defines how you do your linking - shared or static libraries (see introduction for a description of the pros/cons of both

3-D) The EXTRA_LIBS variable should include arguments which define the location of the static shared (.so) libraries built by xdrf, NetCDF, and FFTW3. There is some choice here as to how you proceed - campari can be compiled by linking in shared libraries, or by linking in static libraries.

SHARED LIBRARY COMPILATION

By defining a library location (-L/home/alex/campari/local_libs/local/lib) we can use the -l<LIBNAME> flag which tells the compiler to link in any library which begins lib<LIBNAME>.so or lib<LIBNAME>.a (with a preference for .so files). After compilation, for the binary to run, that .so file must be in the shells $LD_LIBRARY_PATH.

For our single core, Intel compiled binary this means that for our example, we just need

EXTRA_LIBS= -L/home/alex/campari/local_libs/local/lib  -lfftw3 -lnetcdff -lxdrf -lsvml

This means that where possible, each of these four libraries will be linked dynamically. Each of these libraries, in turn, depends on other libraries, but those other libraries are ALSO in the /home/alex/campari/local_libs/local/lib, which the compiler knows about because we set the $LD_LIBRARY_PATH variable earlier. In short, this makes compiling easy, and everything is resolved dynamically.

If we're doing this using gfortran then we have to remove the Intel specific flag -lsvml

STATIC LIBRARY COMPILATION

Alternatively, you can explicitly include each static library by specifying each and every library we compiled (thats 9 of them) and using the -static flag.

EXTRA_LIBS= -L/home/alex/campari/local_libs/local/lib -lnetcdff  -lnetcdf -lfftw3 -lxdrf -lhdf5_hl -lhdf5hl_fortran -lhdf5 -lhdf5_fortran -lz  -lsvml -static

The nice thing about this is the -static flag means we even link in things like glibc and other implicit libraries.

Another way you can achieve the same thing here is to leave off the -static flag but explicitly define the static library (.a) files. This would mean we'd have the following EXTRA_LIBS variable

LIBDIR="/home/alex/campari/local_libs/local/lib"
EXTRA_LIBS=${LIBDIR}/libxdrf.a ${LIBDIR}/libfftw3.a  ${LIBDIR}/libnetcdff.a  ${LIBDIR}/libnetcdf.a ${LIBDIR}/libhdf5_hl.a ${LIBDIR}/libhdf5hl_fortran.a ${LIBDIR}/libhdf5.a ${LIBDIR}/libhdf5_fortran.a  ${LIBDIR}/libz.a /export/intel/composer_xe_2013.1.117/compiler/lib/intel64/libsvml.a

Two things to note here. The first is we define a $LIBDIR variable which gives that /libso we don't have to enter that prefix every time. Secondly, I've included our system specific location of the Intel libsvml.a. If you wanted to include this, you'll have to find it yourself on your own system. Note that libsvml is not required for compilation. Unlike the use of the -static flag, this does not link in implicit libraries.

As with the shared libraries, for GNU/gfortran compilation you have to remove the libsvml related arguments.

Combination approach You can also link some libraries in as shared libraries, while specifying explict paths to others. This is the ONLY way to get a semi-static binary while using GNU based compilers, as they explicitly forbid the linking of things like libc (well, its very difficult - it is possible, but it's VERY BAD PRACTICE).

Intel Specific Flags The example file below includes the following Intel specific flags. They should be removed for GNU compilers.

  • -lsvml is the Intel short vector math library and is used for improved performance on Intel chips. This library may not be available if you're compiling with non Intel compilers, in which case it can be removed from the makefile.

  • -axSSE3 is an Intel specific flag to build optimized instructions for Intel chips (SSE3 instructions)

  • -xSSE1 is an Intel specific flag to build optimized instructions for Intel chips (Xeon and Pentium chips)

INTEL COMPILERS Makefile.local

#==========================================================================
# INTEL COMPILERS Makefile.local
# This Makefile.local will allow CAMPARI to compile assuming the dependencies were also
# compiled with the ifort, icc, and icpc compilers. 
#
# May 2014, for CAMPARI, v2.0
# 
# System specific information should be included in this file, NOT in Makefile
# 

PREFIX_LOCATION="/home/alex/campari/local_libs/local"
CAMPARI_HOME=..
SRC_DIR=${CAMPARI_HOME}/source

ARCH=x86_64
BIN_DIR=${CAMPARI_HOME}/bin
LIB_DIR=${CAMPARI_HOME}/lib
FF=ifort
FFLAGS=-DDISABLE_FLOAT -DLINK_XDR -DLINK_FFTW -DLINK_NETCDF -I${PREFIX_LOCATION}/include -I${LIB_DIR}/${ARCH} -axSSE3 -xSSE3 
    
# Fully static build (WILL NOT WORK FOR MPI COMPLIATION)
# EXTRA_LIBS= -L/home/alex/campari/local_libs/local/lib -lnetcdff  -lnetcdf -lfftw3 -lxdrf -lhdf5_hl -lhdf5hl_fortran -lhdf5 -lhdf5_fortran -lz  -lsvml -static

# Semi static build (WILL WORK FOR MPI COMPILATION)
LIBDIR="/home/alex/campari/local_libs/local/lib"
EXTRA_LIBS=${LIBDIR}/libxdrf.a ${LIBDIR}/libfftw3.a  ${LIBDIR}/libnetcdff.a  ${LIBDIR}/libnetcdf.a ${LIBDIR}/libhdf5_hl.a ${LIBDIR}/libhdf5hl_fortran.a ${LIBDIR}/libhdf5.a ${LIBDIR}/libhdf5_fortran.a  ${LIBDIR}/libz.a /export/intel/composer_xe_2013.1.117/compiler/lib/intel64/libsvml.a 


# shared library build
EXTRA_LIBS= -L${PREFIX_LOCATION}/lib -lnetcdff -lfftw3 -lxdrf -lsvml   

# MPI compilation things
MPIFF=mpif90
MPIFFLAGS=-fpp ${FFLAGS}
MPILFLAGS=-O3 
MPIEXTRA_LIBS=${EXTRA_LIBS}

COMPDEFAULTS=${INTELDEFAULTS}
#==========================================================================

GNU COMPILERS Makefile.local

#==========================================================================
# GNU COMPILERS Makefile.local
# This Makefile.local will allow CAMPARI to compile assuming the dependencies were also
# compiled with the gfortran, gcc, and g++ compilers (or, general GNU compilers)
#
# May 2014, for CAMPARI, v2.0
# 
# System specific information should be included in this file, NOT in Makefile
# 


PREFIX_LOCATION="/home/alex/campari/local_libs/local"
CAMPARI_HOME=..
SRC_DIR=${CAMPARI_HOME}/source

ARCH=x86_64
BIN_DIR=${CAMPARI_HOME}/bin
LIB_DIR=${CAMPARI_HOME}/lib
FF=gfortran
FFLAGS=-DDISABLE_FLOAT -DLINK_XDR -DLINK_FFTW -DLINK_NETCDF -I${PREFIX_LOCATION}/include -I${LIB_DIR}/${ARCH} 

# Semi static build 
# LIBDIR="/home/alex/campari/local_libs/local/lib"
# EXTRA_LIBS=${LIBDIR}/libxdrf.a ${LIBDIR}/libfftw3.a  ${LIBDIR}/libnetcdff.a  ${LIBDIR}/libnetcdf.a ${LIBDIR}/libhdf5_hl.a ${LIBDIR}/libhdf5hl_fortran.a ${LIBDIR}/libhdf5.a ${LIBDIR}/libhdf5_fortran.a  ${LIBDIR}/libz.a

EXTRA_LIBS= -L${PREFIX_LOCATION}/lib -lnetcdff -lfftw3 -lxdrf   

# MPI compilation things
MPIFF=mpif90

# note that the -fpp flag is removed (compared to Intel compilers)
MPIFFLAGS=${FFLAGS} 
MPILFLAGS=-O3 
MPIEXTRA_LIBS=${EXTRA_LIBS}

COMPDEFAULTS=${GNUDEFAULTS}
#==========================================================================

4) Create the library and binary directories To create the binary and library folders run (still from the campari/source directory)

mkdir -p ../bin/x86_64
mkdir -p ../lib/x86_64 
mkdir -p ../lib/x86_64/mpi # for mpi library files, we'll come back to this later

The campari/lib directory contains all the compiled object code generated during the CAMPARI compilation process, which is then linked with the various libraries you installed previously to create a binary (executable file) in the campari/bin directory.

5) Now you're read to compile campari!

Once you've edited the Makefile.local appropriately, you ready to compile

make campari

This will compile and link the CAMPARI binary to campari/bin/x86_64/campari

Running CAMPARI

Finally, assuming the installation complete OK and created your new file, you can run campari!

If you used shared libraries for compilation, you'll need to add the /home/alex/campari/local_libs/local/lib directory to LD_LIBRARY_PATH so that the CAMPARI binary knows, at runtime, where to find the various shared libraries.

This can be achieved by setting

export LD_LIBRARY_PATH=/home/alex/campari/local_libs/local/lib:$LD_LIBRARY_PATH

In your .bashrc file (for POSIX compliant shells) or use the SENTENV syntax for CSH et al. If you fail to do this executing the binary will generate an error of the type

./campari: error while loading shared libraries: libnetcdff.so.5: cannot open shared object file: No such file or directory

If you used static libraries for compiling, you're good to run the binary without setting any kind of dynamic library path

Compiling CAMPARI_MPI

CAMPARI_MPI should be compiled with shared libraries. You can, if you really want, use the explicit static format described above, but mpif90 (the MPI-FORTRAN wrapper compiler) links in the libmpi library which by default is NOT built with a static library. This means if you use the -static flag you'll get an error about being unable to find the libmpi file. This is actually a good sanity check - you don't want arbitrary MPI libraries built into a binary which is going to be distributed across some shared resources via MPI.

Anyway, assuming everything went well, and you have access to the mpif90 compiler, compiling the MPI version should just be a case of running

make campari_mpi

The campari_mpi is built in the same location as the campari binary. I suggest going back to the official campari documentation to examine the role of MPI in more detail.

FAQs

##campari_mpi crashes during compilation`

If during the campari_mpi compilation the process dies with the following error

Error: Keyword argument requires explicit interface for procedure 'wl_parallel_combine' at (1)
make: *** [../lib/x86_64/mpiwanglandau.o] Error 1

Just do

make clean
make campari_mpi # this will work now                   
make campari

For some reason this error appears when campari is (successfully) compiled before campari_mpi.

I get a whole bunch of CURL error messages

The compilation runs fine, but when trying to link you get a whole bunch of e.g. undefined reference to 'curl_global_init' messages. This error just happened on a new system I was running an install on. To solve this I added the -lcurl to the libraries flag such that

EXTRA_LIBS= -L${PREFIX_LOCATION}/lib -lnetcdff -lfftw3 -lxdrf

becomes

EXTRA_LIBS= -L${PREFIX_LOCATION}/lib -lnetcdff -lfftw3 -lxdrf  -lcurl

Or

EXTRA_LIBS=${LIBDIR}/libxdrf.a ${LIBDIR}/libfftw3.a  ${LIBDIR}/libnetcdff.a  ${LIBDIR}/libnetcdf.a ${LIBDIR}/libhdf5_hl.a ${LIBDIR}/libhdf5hl_fortran.a ${LIBDIR}/libhdf5.a ${LIBDIR}/libhdf5_fortran.a  ${LIBDIR}/libz.a

becomes

EXTRA_LIBS=${LIBDIR}/libxdrf.a ${LIBDIR}/libfftw3.a  ${LIBDIR}/libnetcdff.a  ${LIBDIR}/libnetcdf.a ${LIBDIR}/libhdf5_hl.a ${LIBDIR}/libhdf5hl_fortran.a ${LIBDIR}/libhdf5.a ${LIBDIR}/libhdf5_fortran.a  ${LIBDIR}/libz.a -lcurl

Questions, comments or concerns

If anything here was unclear, wrong, or there is extra detail missing, contact me at alex.holehouse@gmail.com and I'd be happy to update, OR submit a pull request - I'd be thrilled to get one!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment