Building libcudf with ninja

libcudf-ninja.md

Building cuDF with ninja

Install CUDA 10.0

Follow the instructions here to install CUDA 10.2.

CUDA will be installed under usr/local and the /usr/local/cuda symlink should now be pointing to version 10.2.

Reboot and then try running one of the examples to confirm CUDA is correctly installed.

Install CuDF

Clone the public CuDF repository and largely follow the instructions in the CONTRIBUTING.md document up until the part where you need to run cmake.

Don't forget.

git submodule update --init --remote --recursive

Set these environment variables:

export CUDA_HOME=/usr/local/cuda
export CUDACXX=/usr/local/cuda/bin/nvcc

Create conda env.

conda remove --name cudf_dev --all
conda env create --name cudf_dev --file conda/environments/cudf_dev_cuda10.2.yml
conda activate cudf_dev

We use ninja to build CuDF because it is faster than make.

sudo apt install ninja-build

Then from the root of the cloned CuDF project, run the following commands to generate the makefile. Use a value for PARALLEL_LEVEL that is appropriate for the server you are running on based on availability of cores and RAM. I was able to build CuDF on a laptop with 16 GB RAM but with PARALLEL_BUILD set to 1 and it took forever.

export CUDF_HOME=`pwd`
cd cpp
mkdir build
cd build
PARALLEL_LEVEL=6 cmake .. -GNinja -DCMAKE_INSTALL_PREFIX=$CONDA_PREFIX -DDISABLE_DEPRECATION_WARNING=ON -DARROW_STATIC_LIB=ON

If you get errors here it is likely that you forget to activate the conda environment. Also it is sometimes necessary to update the conda environment.

Run the following commands to build, test, and install. This takes a while (30 minutes for me).

ninja
ninja test
ninja install