build script for LLVM & Flang-f18 Fortran compiler

Readme.md

Build script for Flang-f18 + LLVM

This is a Bash script (macOS, Linux, ...) for building Flang-f18 and LLVM from source. It is adapted from Jeff Hammond

Ninja: recommended for best build efficiency and speed.

In general, a recent GCC would work. On macOS, system AppleClang compiler can be used as well.

bash build-flang-f18.sh

The source download is about 300 MB. LLVM takes about a half-hour to build on a powerful laptop or workstation.

To use the compiler after install:

source clang-latest.sh

LLVM projects

• lld LLVM linker
• mlir intermediate representation
• pstl Parallel STL (optional)
• polly Loop optimization (optional)

Troubleshooting

See ulimit settings if link failures occur.

Disk usage

The installed binaries take a few gigabytes of disk space.

du -sh ~/llvm-latest  # the install prefix

3.3G

The source directory can be deleted after the build / install is complete.

du -sh $TMPDIR/llvm-src 

1.8G

The build directory can be deleted after the build / install is complete.

du -sh $TMPDIR/llvm-build 

5.0G

build-flang-f18.sh

#!/bin/bash -xe

# -x: verbose
# -e: exit on error

# Ninja is recommended for best build efficiency and speed

# adapted from https://github.com/jeffhammond/HPCInfo/blob/master/buildscripts/llvm-git.sh

# use TMPDIR if not empty, else /tmp
TMPDIR=${TMPDIR:-/tmp}

llvm_src=${TMPDIR}/llvm-src
llvm_build=${TMPDIR}/llvm-build
prefix=$HOME/llvm-latest
cmake_root=${llvm_src}/llvm-project-main/llvm

llvm_projects="lld;mlir;clang;flang;openmp;pstl"

echo "LLVM projects: $llvm_projects"
echo "LLVM source: $llvm_src"
echo "LLVM build: $llvm_build"
echo "LLVM install: $prefix"

mkdir -p $prefix
mkdir -p $llvm_src
mkdir -p $llvm_build


REPO=https://github.com/llvm/llvm-project.git
# ~300 MB
remote=https://github.com/llvm/llvm-project/archive/refs/heads/main.zip
archive=${TMPDIR}/llvm_main.zip

# Download/update the source
[[ -f $archive ]] || curl -L -o $archive $remote
# git clone --recursive $REPO $llvm_src  # so slow

[[ -f ${cmake_root}/CMakeLists.txt ]] || unzip -d $llvm_src $archive

[[ $(which ninja) ]] && CMAKE_GENERATOR="Ninja" || CMAKE_GENERATOR="Unix Makefiles"

case "$(uname -m)" in
  arm64|aarch64)
    llvm_arch=AArch64
    ;;
  *)
    llvm_arch=X86
    ;;
esac

# helpful system parameters

case "$OSTYPE" in
darwin*)
    macos_sysroot=-DDEFAULT_SYSROOT="$(xcrun --show-sdk-path)"
    ;;
*)
    llvm_linker=-DLLVM_USE_LINKER=gold
    ;;
esac\

# lldb busted on MacOS
# libcxx requires libcxxabi
cmake \
  -G"$CMAKE_GENERATOR" \
  -DCMAKE_BUILD_TYPE=Release \
  -DLLVM_TARGETS_TO_BUILD=$llvm_arch \
  -DLLVM_ENABLE_RUNTIMES="libcxxabi;libcxx;libunwind" \
  -DLLVM_ENABLE_PROJECTS=${llvm_projects} \
  $macos_sysroot \
  $llvm_linker \
  --install-prefix=$prefix \
  -S${cmake_root} \
  -B${llvm_build}

cmake --build ${llvm_build}

cmake --install ${llvm_build}

clang-latest.sh

r=$HOME/llvm-latest
b=${r}/bin

export CC=${b}/clang CXX=${b}/clang++ FC=${b}/flang-new

# flags below not generally needed

#export LDFLAGS="-L${r}/lib"
#export CPPFLAGS="-I${r}/include"

#case "$OSTYPE" in
#darwin*)
# avoid missing -lSystem
#export LIBRARY_PATH="/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/lib"
# avoid missing libc++.1.dylib
#export DYLD_FALLBACK_LIBRARY_PATH=${r}/lib
#;;
#esac

Update that the script might need. When I tried running this I got:

[6692/6762] Linking CXX shared library lib/libclang-cpp.so.19.0git
FAILED: lib/libclang-cpp.so.19.0git
...
/usr/bin/ld.gold: internal error in open, at descriptors.cc:99
collect2: error: ld returned 1 exit status

when it tried to link together what seems to be a crazy number of files. If I got my awk right, somewhere over 1300.

Searching internet seems to show that this could be a ulimit issue and on my system:

@$ ulimit -n
1024

And, well, that is less than 1300!

So I then ran:

ulimit -n 65536

and I got past it.

Interesting. A linux system I used also had 1024 and macOS has 256. I will note it in the Readme.