ZekunZh/installation_guide_Jetson.md

## installation_guide_Jetson.md

      
    Raw
  

              installation_guide_Jetson.md
            
          
    Docker Image Installation on Nvidia Jetson TX2


Requirements

Hardware


Nvidia Jetson TX2 card with 8GB meomory


Host x86_64 machine installed with Ubuntu 16.04


SD card with at least 128GB size


Micro-USB to USB cable


Wifi connection for Jetson TX2


Software


Nvidia SDK manager V1.3.1


Docker 19.03


Jetpack 4.4.1 (included in SDK manager)


\pagebreak

Prepare Jetson


Part 1: SDKManager: Flash & Install SDK components (CUDA etc.)


The way to install nvidia drivers and CUDA on Jetson is totally different from normal x86_64 machines. It is more similar to flashing your mobile phone (or your Nintendo Switch !) with another system.
The general idea is to flash a new OS with Nvidia drivers and CUDA components to Jetson TX2. In order to do that, we first need to put the device into reset / recovery mode and attach the TX2 card to host machine with Micro-USB to USB cable.


Press and hold down the Force Recovery button.


Press and hold down the Power button.


Release the Power button, then release the Force Recovery button.


To make sure the the device is in reset mode, you can observe the two green lights for power and recovery are both on, and the output HDMI has no signal.
Then follow the steps in installation guide.
If you don't have graphic interface for the host machine, we can also install SDKManager with command line. Here are some commands that we have used on gpu-1:
sudo apt install ./sdkmanager_1.3.1-7110_amd64.deb

sdkmanager --query noninteractive --license accept

sdkmanager --cli install --logintype devzone --product Jetson --version 4.4 --targetos Linux --host --target P3310-1000 --flash all --license accept

Here are the packages that we choose to install on Jetson TX2:
Host
    * CUDA
    * Computer Vision
    * Developer Tools

Target (Jetson TX2)
    * Jetson OS
    * Jetson SDK components (including CUDA and nvidia-container)

If we want to manually install CUDA
wget https://developer.download.nvidia.com/compute/cuda/repos/ubuntu1804/sbsa/cuda-ubuntu1804.pin
sudo mv cuda-ubuntu1804.pin /etc/apt/preferences.d/cuda-repository-pin-600
wget http://developer.download.nvidia.com/compute/cuda/11.0.2/local_installers/cuda-repo-ubuntu1804-11-0-local_11.0.2-450.51.05-1_arm64.deb
sudo dpkg -i cuda-repo-ubuntu1804-11-0-local_11.0.2-450.51.05-1_arm64.deb
sudo apt-key add /var/cuda-repo-ubuntu1804-11-0-local/7fa2af80.pub
sudo apt-get update
sudo apt-get -y install cuda

Comments


Make sure the Jetson TX2 can be detected by the SDK Manager. It seems that when connecting Jetson in reset mode by micro-USB to USB cable, the board is recognized as a remote machine assigned with a local IP address. The Jetson OS and other components are transferred to the device in total, then the board will restart itself to install the OS and other packages. Exactly like updating your mobile phones !


The SDK manager will run sudo apt update and fails at every error of update. The following problems are encountered:


public key not found. Try sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys your-public-key


invalid/outdated sources. Try to comment them in /etc/apt/sources.list


Might also need to check all the sources: grep ^ /etc/apt/sources.list /etc/apt/sources.list.d/*


Possible Errors

The installation of SDK components might fail during installation of CUDA or other components with errors like the following.
exit status 255
 ===== INSTALLATION FAILED =====
      - Device Mode Host Setup in Target SDK: Installed
      - DateTime Target Setup: Installed
      - CUDA Toolkit for L4T: Up-to-date
      - cuDNN on Target: Up-to-date
      - TensorRT on Target: Installed
      - OpenCV on Target: Up-to-date
      - VisionWorks on Target: Installed
      - VPI on Target: Installed
      - NVIDIA Container Runtime with Docker integration (Beta): Installed
      - Multimedia API: Installed
      - CUDA on Host: Up-to-date
      - CUDA Cross Compile Package on Host: Up-to-date
      - VisionWorks on Host: Failed
      - VPI on Host: Up-to-date
      - NVIDIA Nsight Graphics: Up-to-date
      - NVIDIA Nsight Systems: Up-to-date

  ===== Installation failed - Total 16 components =====
  ===== 7 succeeded, 1 failed, 8 up-to-date, 0 skipped =====


Make sure that you have already created a new user on Jetson & installed ssh-server in order to establish the connection between host machine and Jetson.
Select the installation via Ethernet cable, SSH connection via IPv4, and the IP address of eth0 when running ifconfig on Jetson.
Try to run multiple times of installation with this command sdkmanager --cli install --logintype devzone --product Jetson --version 4.4 --targetos Linux --host --target P3310-1000 --flash all --license accept. It seems that even the installation might fail for the first time, the second time it has greater chance to succeed with previously installed packages.
Only select the SDK components, no need to flash the OS image


Part 2: Instal Azure Cli


Install Pre-requirements

sudo apt update && sudo apt install -y \
            build-essential \
            git \
            curl \
            wget \
            yasm \
            pkg-config \
            vim \
            zip \
            unzip \
            libssl-dev libffi-dev python3-dev python3-pip

Install jtop

sudo -H pip3 install -U jetson-stats

Then restart the board to take effect
Install Azure Cli from script

Source
curl -L https://aka.ms/InstallAzureCli | bash


Part 3: Storage Expansion


The internal storage of Jetson TX2 is only 32GB. After installing Jetson OS, nvidia drivers and CUDA components, we only have ~15GB free space. Considering that the squashed docker image file is already ~10GB, and docker load needs more than 10GB to unzip the image, therefore, a storage expansion is absolutely necessary.
We use one SD card of 128GB as external storage for docker. First we need to format the SD card and create a new partition.
Create new partition
sudo fdisk /dev/mmcblk2
d: delete partition
p: show current partition
n: new partition
t: choose partition type (ext3/4 => 83)
w: write all modifications

Source
Mount new partition
Reference
# format partition
sudo mke2fs -j /dev/mmcblk2p1
# Create mount point
sudo mkdir /mnt/sd
sudo chown -R nvidia /mnt/sd
# Update filesystem table
sudo vim /etc/fstab
# Add following line
/dev/mmcblk2p1 /mnt/sd ext4 defaults 1 2
# Mount all partitions
# sudo mount -a
# df -H


Part 4: Install Docker & Change Storage Location


Install docker

Docker has been used to encapsulate our software to make it easily shareable. The installation guide can be found on the official documentation here. We summarize needed steps here:
sudo apt-get update
sudo apt-get install \
 apt-transport-https \
 ca-certificates \
 curl \
 gnupg-agent \
 software-properties-common
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -

sudo add-apt-repository \
 "deb [arch=arm64] https://download.docker.com/linux/ubuntu \

 $(lsb_release -cs) \
 stable"
sudo apt-get update
sudo apt-get install docker-ce docker-ce-cli containerd.io

Verify that Docker Engine is installed correctly by running the hello-world image.
sudo docker run hello-world

Change docker storage location

The default storage for docker images are in / where we only have ~15GB space. Therefore, we need to change the storage location for docker.
Modify /etc/docker/daemon.json as follows:
{
    //...
    "data-root": "/mnt/sd/docker",
    //...
}

Then reload configuration and restart Docker:
sudo systemctl daemon-reload
sudo systemctl restart docker

To verify:
docker info|grep "Docker Root Dir"

Add Nvidia Container Runtime

In order to be able to build & run docker with Nvidia support, we need to add one line in /etc/docker/daemon.json as follows:
{
    //...
    "default-runtime": "nvidia",
    //...
}

You can check that the NVIDIA Container Runtime is installed by running the following commands: Reference
$ sudo dpkg --get-selections | grep nvidia
libnvidia-container-tools			install
libnvidia-container0:arm64			install
nvidia-container-runtime			install
nvidia-container-toolkit    		install
nvidia-docker2				        install

$ sudo docker info | grep nvidia
+ Runtimes: nvidia runc


Part 5: Other Setup


Make space to hold docker images

We can install LibreOffice and Chrome to make more space.
sudo apt-get remove --purge libreoffice* chromium-browser*
sudo apt-get clean
sudo apt-get autoremove


Part 6: Hello-world !


Now we can build an ARM64 CUDA application to test whether Jetson is well configured.
$ mkdir /tmp/docker-build && cd /tmp/docker-build
$ cp -r /usr/local/cuda/samples/ ./

$ tee ./Dockerfile <<EOF
FROM nvcr.io/nvidia/l4t-base:r32.4.3

RUN apt-get update && apt-get install -y --no-install-recommends make g++
COPY ./samples /tmp/samples

WORKDIR /tmp/samples/1_Utilities/deviceQuery
RUN make clean && make

CMD ["./deviceQuery"]
EOF

$ sudo docker build -t test-demo .
$ sudo docker run -it --runtime nvidia test-demo

CUDA Device Query (Runtime API) version (CUDART static linking)

Detected 1 CUDA Capable device(s)

Device 0: "Xavier"
  CUDA Driver Version / Runtime Version          10.0 / 10.0
  CUDA Capability Major/Minor version number:    7.2
  Total amount of global memory:                 15692 MBytes (16454430720 bytes)
  Memory Bus Width:                              256-bit
  L2 Cache Size:                                 524288 bytes
...

deviceQuery, CUDA Driver = CUDART, CUDA Driver Version = 10.0, CUDA Runtime Version = 10.0, NumDevs = 1
Result = PASS


\pagebreak


Build Docker Image


In order to build an image that works on aarch64 (arm64) machine like Jetson TX2, there are A LOT of modifications in the Dockerfile. We will only mention some important changes here and more details can be found in the Dockerfiles.


Replace base image by aarch64-l4t (Linux for Tegra) base image.


Replace miniconda by miniforge, since conda only supports x86_64 machines.


Install specific pytorch versions built for JetPack 4.4


Build torchvison, OpenCV, GDCM locally from scratch since there's no pre-built package for aarch64 machines.


A lot of Ubuntu packages need to be added/replaced for aarch64 machines.


dcm2jpg is no longer supported since there's no aarch64 java binaries. Therefore, we no longer need to install Java on Jetson.


Not surprisingly, the obfuscation also needs also some special tweaks in order to work on Jetson, especially using PyInstaller for OpenCV and torchvision.

\pagebreak


Run Docker Image


Save the docker image as tar on another machine

docker save -o my_docker_image.tar docker_hash

Load the docker image from local drive

First we need to unzip and load the image to docker.
gunzip -c /path/to/docker/image.tar.gz | docker load

The loading can take a few minutes. Once the loading performed, list the available docker images and check that you have one with adequate name.
$ sudo docker images

REPOSITORY            TAG               IMAGE ID            CREATED             SIZE
fuji-on-prem          arm64             f9737034869f        4 hours ago         9.7GB

Load the docker image from external drive / URL

There are three ways to install image to Jetson’s internal storage:

store docker tar in external storage (like USB disk), load the tar directly to internal storage with docker load --input /path/to/docker_image.tar
store docker tar on the network (local or public) and generate a URL to download the tar. Then stream the tar and load on the fly with curl "https://url/to/docker/image" | docker load
store docker tar on another machine of the local network, use ssh to stream the tar and load on the fly. For example:
ssh ubuntu@192.168.1.1 "cat /path/to/docker_image.tar" | docker load

Run the docker image

In order to effectively see docker running, you have to prepare a folder containing dicom images. During its run, the program will prepare the AI-model, look at all dicom images in this folder, process them sequentially and write results (bounding boxes coordinates and confidence score) to a json file results.json. Once all images have been processed, the docker container stops.
Let's suppose input_dicom_folder is the absolute path to your custom folder containing dicom images, and output_folder is the absolute path of the folder where you want to receive the results.
You can run the docker image by entering in your terminal:
docker run --runtime nvidia --env CUDA_VISIBLE_DEVICES=0 --env SAVE_IMG_PREVIEW=1 -v output_folder:/data/output -v intput_dicom_folder:/data/input -it fuji-on-prem:arm64


\pagebreak


Additional information


Running time

On Jetson TX2, it takes approximately 15 sec to run inference on one dicom image. Note that loading the model can take some time (up to 3-4 min for extreme cases), but this is only done once when running the docker image. Thus, it is preferable to run docker image for processing a batch of images.