Quick Berkely UPC build scripts

README.md

Quick Berkeley UPC build scripts

This small gist contains the required steps to compile and install Berkeley's UPC implementation.

This script is far from being fail-safe, but worked on my end. Good luck!

Requirements

To build UPC, you will need to have autoconf and a set of C/C++ compilers. The script defaults to using gcc/g++ and will install UPC in ~/.berkeley. If you wish to change this behavior, you will have to edit the first lines of the script.

You will also need a version of perl installed, accessible through the perl command.

Running the script

upcc and upcrun

The main component of Berkeley's UPC toolset is upc-runtime. It provides the command upcc, to compile UPC code to bytecode, and upcrun, to create the threads and run the code.

The script compile-upc.sh included with this gist will download the sources, build and install upc-runtime. You can manually download, edit and run it, or run the following commands to do it from the terminal:

curl -o compile-upc.sh https://gist.githubusercontent.com/adri326/471d02e55bc541f8eb158e004cd1582d/raw/compile-upc.sh

chmod +x compile-upc.sh

# Optionally edit `compile-upc.sh` now!

./compile-upc.sh

# Edit your bashrc/zshrc to add the installation directory to your PATH if the build succeeded and if you haven't done so already.

Compiling from UPC to C

You can safely skip this step if you do not care about compilation speed and internet requirement.

By default, upcc (the included compiler) does not actually do the translation from UPC to C. This task is offloaded to one of Berkeley Lab's servers through HTTPS. Their argument is that the compilation step from C to bytecode will certainly take longer than the HTTPS request. The server runs an instance of upc-translator, which takes approximately 200MB of space on the disk (no idea why it's so big, especially since Intrepid's deprecated upc2c is much smaller).

The compile-translator.sh script aims to fetch upc-translator, compile it, install it and tweak upcc's config file to use it instead. For unknown reasons, make cannot be run in multithreaded mode for upc-translator (it inevitably fails at random points of the compilation process). The compilation step should only take approximately 10 minutes on modern hardware.

To compile upc-translator, you will need gcc <= 10 (as of writing this, their codebase relies on older C++ versions); on Arch Linux, you can install gcc9 from the AUR. The script defaults to gcc9's binary paths. You will also need csh (on Arch Linux, install tcsh). Sometimes, the build randomly fails because of a missing "YYLEX" variable; this happens because the build script modifies one of the source files which cannot be built with modern versions of yacc/bison. To counter this, when an error occur the affected file is restored and the compilation is resumed.

You may manually download, edit and run compile-translator.sh, or run the following commands:

curl -o compile-translator.sh https://gist.githubusercontent.com/adri326/471d02e55bc541f8eb158e004cd1582d/raw/compile-translator.sh

chmod +x compile-translator.sh

# Optionally edit `compile-translator.sh` now!

./compile-translator.sh

# Edit ~/.upccrc to point to the newly-installed translator if you haven't done that yet!

compile-translator.sh

#!/bin/sh
# This small scripts aims to compile the UPC to C translator on linux.

CC=/bin/gcc-9
CXX=/bin/g++-9
export YACC="bison -y" # Note that the actual yacc executable on most repos won't work; bison and byacc are known to work
PREFIX="${HOME}/.berkeley"

# Download upc-translator
git clone https://bitbucket.org/berkeleylab/upc-translator
cd upc-translator

# Setup the build environment (build dir is sadly not supported)
mkdir -p "$PREFIX"
./configure CC="${CC}" CXX="${CXX}" YACC="${YACC}" || exit 2

# Build and install
YACC="${YACC}" make ||
  (git checkout master -- open64/osprey1.0/gccfe/gnu/c-parse.c open64/osprey1.0/gccfe/gnu/c-parse.h; YACC="${YACC}" make) || exit 3
  # Sometimes the build script modifies these two files and thus fails because it requires an older version of yacc. In that case, we restore these files and restart the compilation.
  # This explains the random git checkout of the above lines. I have no idea why this issue happens, why the build script modifies its own source code, why their code is so out of date.
  # I gave up trying to understand and have resorted to overriding this stupid behavior. Sorry if it means that you'll have to wait twice as long before you can run the software/figure out the error.
make install PREFIX="${PREFIX}"

# Change upcc's config
echo "Berkeley's UPC translator has been installed to '$PREFIX'. To point upcc to it, you should add the following line to '${HOME}/.upccrc':"
echo
echo "translator=${PREFIX}/targ"
echo
echo "You can then run 'upcc --version' to verify that it is now using the local version of upc-translator!"

compile-upc.sh

#!/bin/sh
# This small script aims to compile Berkeley's UPC runtime on linux. It requires you to have standard build tools (autoconf, a C compiler and a C++ compiler) and perl (haven't tested without it but it appears to be necessary)

CC=/bin/gcc
CXX=/bin/g++
PREFIX="${HOME}/.berkeley"

# Download upc-runtime
git clone https://bitbucket.org/berkeleylab/upc-runtime/
cd upc-runtime

# Download GASNet dependency
git clone https://bitbucket.org/berkeleylab/gasnet

# Setup the build environment
./Bootstrap || exit 1
mkdir -p build
mkdir -p "$PREFIX"
cd build
../configure CC=$CC CXX=$CXX --prefix="${PREFIX}" || exit 2

# Build and install
make -j && make install || exit 3
# NOTE: `make install` seems to create files in the build/ directories; if you had run it with `sudo` once, then it'll fail the next times. You will have to clear build/ and re-build in that scenario!

# Show success message
echo "Berkeley's UPC runtime has been installed to '$PREFIX'. To add its directory to the PATH variable, add the following line to you bashrc/zshrc:"
echo
echo "export PATH=\"\${PATH}:$PREFIX/bin\""
echo
echo 'You will need to restart your shell or run `source ~/.bashrc` to apply the modification!'