Build and Install Ray on ARM (aarch64)

Ray_ARM64.md

Dependencies

Only dependencies which aren't highly likely to be featured in a ARM development environment are featured here, Obvious dependencies such as Python3, Python2, pip, wheel, GCC etc. aren't covered here.

Bazel

Note : Use bazel 1.x.x for Ray as 2.x.x is not supported.

sudo apt-get install build-essential openjdk-8-jdk unzip
export JAVA_HOME="/usr/bin/java"
wget https://github.com/bazelbuild/bazel/releases/download/1.0.0/bazel-1.0.0-dist.zip
unzip bazel-1.0.0-dist.zip
cd bazel-1.0.0
./compile.sh

py-spy

py-spy is a python profiling library written in rust. It's easier to install it via rust's package manager on ARM.

curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
cargo install py-spy

Download Ray from git

Download ray from git and checkout the commit without PyArrow dependency.

git clone https://github.com/ray-project/ray.git
git checkout fae99ecb8e8d750bddcb3674f720f068541dc15d

Patching BUILD.boost

Update: This patch has been added to the upstream - nelhage/rules_boost#168. So, one could skip this step. If you face the errors discussed in this issue - ray-project/ray#7184, then follow the patching process mentioned below.

The bazel build rules for installing boost library has assembly files which doesn't compile on ARM64 and missing some default conditions. I have prepared a patch to address it - https://gist.github.com/heavyinfo/25cf56fe0b5f8509dd0120257d008d3f

Download the patch and place it in ray/thirdparty/patches.

Add path to the patch here https://github.com/ray-project/ray/blob/fae99ecb8e8d750bddcb3674f720f068541dc15d/bazel/ray_deps_setup.bzl#L138.

Building ray

cd ray/python
python3 setup.py bdist_wheel

Installing ray

cd ray/python/dist
pip install ray-0.9.0.dev0-cp36-cp36m-linux_aarch64.whl

Troubleshooting

Permission errors

If there are any permission related errors, run the above commands with sudo after understanding the risks of doing so and that every installation command after that may require sudo.

@smorad thank you I will try it.

I have problem with error:

INFO: From Generating C++ proto_library //streaming:streaming_proto:
protobuf/streaming.proto:5:1: warning: Import google/protobuf/any.proto but not used.
ERROR: /home/pi/ray/BUILD.bazel:136:1: C++ compilation of rule '//:worker_rpc' failed (Exit 1) gcc failed: error executing command
  (cd /root/.cache/bazel/_bazel_root/27c0bb57e3de4f4c1e5336c83fd82d62/sandbox/linux-sandbox/1545/execroot/com_github_ray_project_ray && \
  exec env - \
    PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin \
    PWD=/proc/self/cwd \

Use --sandbox_debug to see verbose messages from the sandbox
In file included from bazel-out/arm-opt/bin/_virtual_includes/grpc_common_lib/ray/rpc/grpc_client.h:22,
                 from bazel-out/arm-opt/bin/_virtual_includes/worker_rpc/ray/rpc/worker/core_worker_client.h:33,
                 from bazel-out/arm-opt/bin/_virtual_includes/worker_rpc/ray/rpc/worker/core_worker_client_pool.h:21,
                 from src/ray/rpc/worker/core_worker_client_pool.cc:1:
bazel-out/arm-opt/bin/_virtual_includes/ray_common/ray/common/ray_config_def.h:200:75: error: integer overflow in expression of type 'long int' results in '-2147483648' [-Werror=overflow]
 RAY_CONFIG(uint64_t, object_manager_max_bytes_in_flight, 2L * 1024 * 1024 * 1024)
                                                          ~~~~~~~~~~~~~~~~~^~~~~~
bazel-out/arm-opt/bin/_virtual_includes/ray_common/ray/common/ray_config.h:35:18: note: in definition of macro 'RAY_CONFIG'
   type name##_ = default_value;               \
                  ^~~~~~~~~~~~~
cc1plus: all warnings being treated as errors
Target //:ray_pkg failed to build
INFO: Elapsed time: 2002.802s, Critical Path: 145.31s
INFO: 1546 processes: 1546 linux-sandbox.
FAILED: Build did NOT complete successfully
Traceback (most recent call last):
  File "setup.py", line 459, in <module

Does anybody have suggestion how to fix the error?

@heavyinfo for few days I have built packges in ray requirements files:
https://github.com/PeterPirog/Raspberry_armv7_builds
but now I’m confused which aproach is correct:

1. try use “python3 setup.py bdist_wheel” command in /ray/python directory
   2) try tu use npn according to Building Ray from Source
   3) try tu use Building Ray (Python Only)
   4) try tu use script build.sh from /ray/build.sh
   5) try tu use “bazel build -c fastbuild //:ray_pkg”
   6) try somehow ./configure and make

The goal is still the same build wheel ray file for python 3.8 armv7
I will be greatefull fot the suggestions, maybe some of options above have no sense so I will stop trying its

For arm64 I believe I used pip wheel, which calls bazel behind the scenes.

For the bazel build, I had to set JAVA_HOME to /usr/lib/jvm/java-8-openjdk-arm64 -- using the /usr/bin/java complains ERROR: JAVA_HOME (/usr/bin/java) is not a path to a working JDK.