I'm using Nvidia Jetson nano.
Quad-core ARM® Cortex®-A57 MPCore processor
NVIDIA Maxwell™ architecture with 128 NVIDIA CUDA® cores
4 GB 64-bit LPDDR4 1600MHz - 25.6 GB/s
Ubuntu 18.04 LTS
Python 3.6.9
export CUDACXX=/usr/local/cuda/bin/nvcc
export CUDA_HOME=/usr/local/cuda
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/cuda/lib64:/usr/local/cuda/extras/CUPTI/lib64
export PATH=$PATH:$CUDA_HOME/bin
$ pip install cmake-setuptools
Follow my earlier apache arrow installation guide for ARMv8 - https://gist.github.com/heavyinfo/04e1326bb9bed9cecb19c2d603c8d521.
$ git clone https://github.com/dmlc/dlpack.git
$ cd dlpack
$ mkdir build && cd build
$ cmake .. -DCMAKE_INSTALL_PREFIX=/usr/local/lib/dlpack
$ make -j4
$ make install
$ git clone --recurse-submodules https://github.com/rapidsai/rmm.git
$ cd rmm
$ mkdir build && cd build
$ cmake .. -DCMAKE_INSTALL_PREFIX=/usr/local/lib/dlpack
$ make -j4
$ sudo -E make install
cd python
sudo -E python3 setup.py build_ext --inplace
sudo -E python3 setup.py install
export LD_LIBRARY_PATH=/usr/local/lib:/usr/local/lib/rmm/lib:$LD_LIBRARY_PATH
sudo ldconfig
I'm checking out the commit to reflect the HEAD I used when building cudf. You can use later versions if there aren't any deviations from this install procedure.
git clone https://github.com/rapidsai/cudf.git
git checkout d6b4794b4a3ed7e6577042596a32732bd62fd074
Set the paths according to your own locations.
-DGPU_ARCHS=""
was needed for jetson nano as the cuda compute capability is 5.3 and cudf requires 6.0+. If you are building for a device is GPU with cuda compute 6.0+, this flag can be ommited.
export CUDF_HOME=/home/abishek/Downloads/cudf
export DLPACK_ROOT=/usr/local/lib/dlpack/
export RMM_ROOT=/usr/local/lib/rmm/
cd $CUDF_HOME/cpp/build/
cmake .. -DCMAKE_INSTALL_PREFIX=/usr/local/lib/cudf/ -DCMAKE_CXX11_ABI=ON -DRMM_INCLUDE=/usr/local/lib/rmm/include/ -DDLPACK_INCLUDE=/usr/local/lib/dlpack/include/ -DGPU_ARCHS=""
make #will take a day to build, more process will crash the build due to low memory.
sudo -E make install
cd $CUDF_HOME/python/nvstrings
export NVSTRINGS_ROOT=/usr/local/lib/cudf/
export NVSTRINGS_LIBRARY=/usr/local/lib/cudf/lib/
export NVTEXT_LIBRARY=/usr/local/lib/cudf/lib/
export NVCATEGORY_LIBRARY=/usr/local/lib/cudf/lib/
sudo -E python3 setup.py install
export LD_LIBRARY_PATH=/usr/local/lib:/usr/local/lib/cudf/lib:/usr/local/lib/rmm/lib:$LD_LIBRARY_PATH
sudo ldconfig
cd $CUDF_HOME/python/cudf
nano setup.py
To fix this issue - rapidsai/cudf#4121,
Add the following include paths to setup.py at this location - https://github.com/rapidsai/cudf/blob/d6b4794b4a3ed7e6577042596a32732bd62fd074/python/cudf/setup.py#L41-L50
Change the path to your appropirate location if you haven't followed the earlier paths for installation of these libraries.
"/usr/local/lib/",
"/usr/local/lib/cudf",
"/usr/local/lib/include",
"/usr/local/lib/rmm/include/",
"/usr/local/lib/dlpack/include/",
cd $CUDF_HOME/python/cudf
sudo -E python3 setup.py build_ext --inplace
sudo -E python3 setup.py install
- You could probably do away with
sudo
while installing python libraries if you don't face any permission errors. If you usesudo
for python library installations, understand the risks involved in it.
https://github.com/rapidsai/cudf/blob/branch-0.13/CONTRIBUTING.md#setting-up-your-build-environment rapidsai/cudf#2770 (comment) rapidsai/cudf#2505 (comment)
I have a problem with the proposed configuration. In my AGX Xavier gives an error when I import cudf from python. It doesn't give any error at compile time but at run time:
Python 3.6.9 (default, Apr 18 2020, 01:56:04)
[GCC 8.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
Running python, import rmm, import pyarrow, numba or dlpack does not cause problems. I'm several days into this problem.