Julia on Blue Waters

Julia on Blue Waters

The default `gcc` on Blue Waters is `gcc 4.8.2`. The built-in version of MPFR is too old to build Julia against. On other clusters this often means you have to build a custom `gcc` with a newer MPFR. Thankfully, this isn't necessary on Blue Waters. They have an alternate, newer `gcc` installed. This will set us up with the right `gcc`:

```bash
module swap PrgEnv-cray PrgEnv-gnu
module swap gcc/4.8.2 gcc/4.9.2
```

Then we have a version of `gcc` and its dependencies new enough to build Julia. If you want to just build Julia for yourself in your homedir, you can follow the directions on Julia's [README](https://github.com/JuliaLang/julia/blob/master/README.md#source-download-and-compilation). If you want to build Julia in a projects directory for everyone, it's pretty simple. Here's what I did:

```bash
cd /projects/sciteam/OUR_DIRECTORY
mkdir julia
cd julia
git clone https://github.com/JuliaLang/julia.git v0.4.0
cd v0.4.0
export JULIA_PKGDIR=/projects/sciteam/OUR_DIRECTORY/.julia
make
```
This will build latest Julia master and put the packages directory in `/projects/sciteam/OUR_DIRECTORY`, so everyone in the project can use the same build. You can build the latest stable `0.3` as well:
```bash
cd /projects/sciteam/OUR_DIRECTORY
mkdir julia
cd julia
git clone https://github.com/JuliaLang/julia.git v0.3.10 -b v0.3.10
cd v0.3.10
export JULIA_PKGDIR=/projects/sciteam/OUR_DIRECTORY/.julia
make
```
This will build Julia `v0.3.10` for you. You can then add whichever one you want to your `PATH` in `.bashrc` or `.bash_profile`. Start whichever version of Julia you want on the command line with `julia` and run `Pkg.init()` to create the packages folder in `.julia/v0.3` or `.julia/v0.4`. That will checkout `METADATA` as well. Then you are good to go and add packages.

# How To Build MPI

I have already built `MPI.jl` for both 0.4 and 0.3 in `/projects/sciteam/jpz` but if you need to build it yourself you will have to do several things:

1. Custom compile CMake. You will need a version newer than 2.8 - Cray does not provide one. Follow the instructions at CMake's [website](http://www.cmake.org/install/).
2. Put CMake on your `PATH`.
3. Fire up Julia and run `Pkg.add("MPI")` and then `Pkg.checkout("MPI")`. We have to checkout the newest version since the package hasn't been tagged in a while. The build will fail.
4. `cd JULIA_PKG_DIR/JULIA_VERSION/MPI/deps`
5. The build fails because CMake doesn't know how to find MPI on a Cray system. We will tell it how to do so: `cmake -DCMAKE_C_COMPILER=cc -DCMAKE_Fortran_COMPILER=ftn`
6. The CMake build should succeed. Now run `make && make install` and we should be good to go!