instructions.rst

Ubuntu

> lsb_release -a
No LSB modules are available.
Distributor ID:  Ubuntu
Description:     Ubuntu 18.04.2 LTS
Release:         18.04
Codename:        bionic

> g++ --version
g++ (Ubuntu 7.3.0-27ubuntu1~18.04) 7.3.0
Copyright (C) 2017 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

Packages

Required system packages:

> sudo apt-get install -y \
apt-transport-https \
build-essential \
ca-certificates \
cmake \
curl \
git \
libatlas-base-dev \
libcurl4-openssl-dev \
libjemalloc-dev \
liblapack-dev \
libopenblas-dev \
libopencv-dev \
libzmq3-dev \
ninja-build \
software-properties-common \
sudo \
unzip \
wget \
libtinfo-dev \
zlib1g-dev \
libglfw3-dev

Conda

Install conda:

[/dl]> wget https://repo.anaconda.com/archive/Anaconda3-5.3.1-Linux-x86_64.sh
[/dl]> chmod +x Anaconda3-5.3.1-Linux-x86_64.sh

Run *.sh script and set /dl/anaconda as an install destination:

[/dl]> ./Anaconda3-5.3.1-Linux-x86_64.sh

Conda activation:

> source /dl/anaconda/etc/profile.d/conda.sh

Test installation (activate it first):

> conda list

Note

• https://conda.io/projects/conda/en/latest/user-guide/install/linux.html
• https://conda.io/projects/conda/en/latest/user-guide/install/test-installation.html

Conda environment

Load conda:

> source /dl/anaconda/etc/profile.d/conda.sh

Existent environments:

> conda info --envs

If dl environment present and you want to remove it:

> conda remove --name dl --all

Create environment:

> conda create --name dl python=3.6

Activate it:

> conda activate dl
(dl)>

Note

• https://conda.io/docs/user-guide/tasks/manage-environments.html

Pip dependencies

Install pip dependencies:

(dl)> pip install \
    cpplint==1.3.0 \
    h5py==2.8.0rc1 \
    nose \
    nose-timer \
    'numpy<=1.15.2,>=1.8.2' \
    pylint==1.8.3 \
    'requests<2.19.0,>=2.18.4' \
    scipy==1.0.1 \
    boto3 \
    decorator \
    Pillow \
    matplotlib

There should be no mxnet or nnvm package in system. Sanity check:

(dl)> python -c 'import mxnet'
Traceback (most recent call last):
  File "<string>", line 1, in <module>
ModuleNotFoundError: No module named 'mxnet'

(dl)> python -c 'import nnvm'
Traceback (most recent call last):
  File "<string>", line 1, in <module>
ModuleNotFoundError: No module named 'nnvm'

MXNet

Get mxnet sources:

(dl)> cd /dl
(dl) [/dl]> git clone https://github.com/apache/incubator-mxnet mxnet
(dl) [/dl]> cd mxnet
(dl) [/dl/mxnet]>

Lock version 7243806f3

(dl) [/dl/mxnet]> git branch test-7243806f3 7243806f3
(dl) [/dl/mxnet]> git checkout test-7243806f3
(dl) [/dl/mxnet]> git submodule update --init --recursive

Run mxnet build:

(dl) [/dl/mxnet]> cmake -H. -B_builds -DUSE_LAPACK=OFF -DCMAKE_BUILD_TYPE=Release -DCMAKE_VERBOSE_MAKEFILE=ON -DUSE_F16C=OFF
(dl) [/dl/mxnet]> cmake --build _builds -j $(nproc)

Check created libmxnet.so library:

(dl) [/dl/mxnet]> ls _builds/libmxnet.so

Have to create symbolic link because that's where Python code expects the library:

(dl) [/dl/mxnet]> mkdir -p lib
(dl) [/dl/mxnet]> ln -s /dl/mxnet/_builds/libmxnet.so /dl/mxnet/lib/libmxnet.so

Test import:

(dl) [/dl/mxnet] > (PYTHONPATH=/dl/mxnet/python python -c 'import mxnet')

Note

• https://mxnet.apache.org/install/ubuntu_setup.html#build-mxnet-from-source

LLVM

LLVM needed for TVM build:

(dl) > cd /dl
(dl) [/dl]> wget http://releases.llvm.org/6.0.1/llvm-6.0.1.src.tar.xz
(dl) [/dl]> tar xf llvm-6.0.1.src.tar.xz
(dl) [/dl]> cmake -H/dl/llvm-6.0.1.src -B/dl/llvm-6.0.1.src/_builds -DCMAKE_INSTALL_PREFIX=/dl/llvm -DCMAKE_VERBOSE_MAKEFILE=ON -DCMAKE_BUILD_TYPE=Release
(dl) [/dl]> cmake --build /dl/llvm-6.0.1.src/_builds --target install -- -j $(nproc)

TVM

Build TVM:

(dl) > cd /dl/mxnet/3rdparty/tvm
(dl) [/dl/mxnet/3rdparty/tvm]> cmake -H. -B_builds -DUSE_GRAPH_RUNTIME_DEBUG=ON -DUSE_CUDA=ON -DUSE_OPENCL=ON -DUSE_VULKAN=OFF -DUSE_OPENGL=ON -DUSE_LLVM=/dl/llvm/bin/llvm-config -DCMAKE_VERBOSE_MAKEFILE=ON -DCMAKE_BUILD_TYPE=Release
(dl) [/dl/mxnet/3rdparty/tvm]> cmake --build _builds -- -j $(nproc)

Check library libtvm.so:

(dl) [/dl/mxnet/3rdparty/tvm]> ls ./_builds/libtvm.so

Have to create symbolic link because that's where Python code expects the libraries:

(dl) [/dl/mxnet/3rdparty/tvm]> mkdir -p lib nnvm/lib
(dl) [/dl/mxnet/3rdparty/tvm]> ln -s /dl/mxnet/3rdparty/tvm/_builds/libtvm.so /dl/mxnet/3rdparty/tvm/lib/libtvm.so
(dl) [/dl/mxnet/3rdparty/tvm]> ln -s /dl/mxnet/3rdparty/tvm/_builds/libnnvm_compiler.so /dl/mxnet/3rdparty/tvm/nnvm/lib/libnnvm_compiler.so

Test import:

(dl) [/dl/mxnet/3rdparty/tvm]> (PYTHONPATH=/dl/mxnet/3rdparty/tvm/python:/dl/mxnet/3rdparty/tvm/topi/python:/dl/mxnet/3rdparty/tvm/nnvm/python python -c 'import nnvm')

Note

• https://docs.tvm.ai/install/from_source.html#build-the-shared-library
• https://docs.tvm.ai/install/from_source.html#python-package-installation

from_mxnet.py

Workaround for apache/mxnet#10856 :

(dl) > export OMP_NUM_THREADS=1

Test:

(dl) > cd /dl
(dl) [/dl]> mkdir __temp
(dl) [/dl]> cd __temp
(dl) [/dl/__temp]> (PYTHONPATH=/dl/mxnet/python:/dl/mxnet/3rdparty/tvm/python:/dl/mxnet/3rdparty/tvm/topi/python:/dl/mxnet/3rdparty/tvm/nnvm/python python /dl/mxnet/3rdparty/tvm/tutorials/nnvm/from_mxnet.py)