Notes on Setting up the Microsoft Azure Kinect on Ubuntu 18.04

setup-azure-kinect-on-jetson-x-nx.md

Azure Kinect on Ubuntu 18.04

Notes on Installing Microsoft Azure Kinect Sensor and Body Tracking SDKs on Linux PC and NVIDIA Jetson Xavier NX

06.12.2020

Jump to:

• Installing Sensor SDK on Linux PC
• Installing on Jetson Xavier NX
• Installing Body Tracking SDK
• Updating Firmware for Azure Kinect

---

Installing Sensor SDK on Linux PC

See Microsoft's Installation Guide

1. Add link to Microsoft's Linux Software Repository for AMD64

By adding microsoft's code repository to your system so that you can call sudo apt install xxx to install system wide.

Guide for Configuring the Repository

Here are Microsoft's Instructions:

curl https://packages.microsoft.com/keys/microsoft.asc | sudo apt-key add -
sudo apt-add-repository https://packages.microsoft.com/ubuntu/18.04/prod
sudo apt-get update

I ran into errors with these instructions. Here's how I solved them:

1. Install curl if you haven't already.

sudo apt update && upgrade
sudo apt install curl

2. Specify the architecture of the repository. Running the repository command, I got this error on runningsudo apt-get update: N: Skipping acquire of configured file 'main/binary-i386/Packages' as repository 'https://packages.microsoft.com/ubuntu/18.04/prod' doesn't support support architecture 'i386'.

Turns out you need specify the architecture of the repository to [arch=amd64]:

• Log in as root:

sudo -i

• Modify /etc/apt/sources.list. At the bottom of the file, change from:

deb https://packages.microsoft.com/ubuntu/18.04/prod bionic main
# deb-src https://packages.microsoft.com/ubuntu/18.04/prod bionic main

to:

deb [arch=amd64] https://packages.microsoft.com/ubuntu/18.04/prod bionic main
# deb-src [arch=amd64] https://packages.microsoft.com/ubuntu/18.04/prod bionic main

and save.

• Log out of root:

exit

3. Rerun sudo apt-get update

2. Install Kinect Packages

If there are no more errors, you should be able to install the Kinect packages install libk4a1.X libk4a1.X-dev k4a-tools

If you install k4a-tools first, it automatically downloads the latest version of libk4a1.X. Take note of the version for installing libk4a1.X-dev (it must be the same version as libk4a1.X).

sudo apt install k4a-tools
sudo apt install libk4a1.4-dev

If you mess up, you can always uninstall and try again using sudo apt remove k4a-tools

3. Finish Device Setup

Finish device setup by setting up udev rules:

• Copy 'scripts/99-k4a.rules' into '/etc/udev/rules.d/'.
• Detach and reattach Azure Kinect devices if attached during this process.

4. Run an App

To run an app, you just call it directly in the terminal:

k4aviewer

You may get a warning to upgrade your firmware when running k4aviewer the first time. See instructions below.

The k4aviewer, k4arecorder, and AzureKinectFirmwareTool downloaded through k4a-tools are all located in /usr/bin.

5. Download Samples

Microsoft provides code samples that demonstrate how to use Microsoft's Azure Kinect DK Sensor and Body Tracking SDKs.

git clone https://github.com/microsoft/Azure-Kinect-Samples.git

---

Installing Sensor SDK on Jetson Xavier NX

At the time of writing (June 2020), Microsoft has debian packages for Ubuntu 18.04 for AMD64 and ARM64. However, this is the first release for ARM: there are some bugs and the Body Tracking SDK is not yet supported. More details on that below.

1. Add Microsoft's product repository for ARM64

curl https://packages.microsoft.com/keys/microsoft.asc | sudo apt-key add -
sudo apt-add-repository https://packages.microsoft.com/ubuntu/18.04/multiarch/prod
sudo apt-get update

2. Install Kinect Package

Same as instructions above.

3. Finish Device Setup

Same as instructions above.

4. Increase default USB memory for Ubuntu

By default, Linux-based host computers allocate the USB controller only 16 MB of kernel memory to handle USB transfers. This amount is supposed to be enough to support a single Azure Kinect ... so this step might not be necessary. But I went ahead and did it anyway just to be safe.

Microsoft has instructions for this, however Jetson boards don't have grub. So instead, I used these instructions from FLIR for the Jetson TX2.

To acquire images greater than 2 MB in resolution, add the following to the APPEND line:

usbcore.usbfs_memory_mb=1000

to this file:

/boot/extlinux/extlinux.conf

If this method fails to set the memory limit, run the following command:

sudo sh -c 'echo 1000 > /sys/module/usbcore/parameters/usbfs_memory_mb'

Here's a good overview of why USBFS matters for usb camera on Linux, also from FLIR

Notes on Running on Jetson Xavier NX

1. The Jetson Xavier NX Developer Kit seems compatible with minimum system requirements of the Azure Kinect.
2. I'm running my X NX 15W 6CORE mode with 250 GB NVMe SSD added to the DK.
   • See this Jetson Hacks repo for a walkthrough of adding NVMe storage to the Jetson X NX.
3. Testing with k4aviewer, I got the following warnings and errors:

This warning is constant, but harmless:

[ warning  ] : depth_engine_thread(). Depth image processing is too slow at 1000ms (this may be transient).

This is a bug that crashes the data stream when the color camera is enabled:

[ error    ] : DecodeMJPEGtoBGRA32(). MJPEG decode failed: -1
[ warning  ] : capturesync_add_capture(). Capture Error Detected, Color 
[ error    ] : capturesync_get_capture(device->capturesync, capture_handle, timeout_in_ms) returned failure in k4a_device_get_capture()

Disabling the Color Camera avoids the errors, however there's a fix in the next release, 1.4.1. But you can download the beta with the fix now:

• Click this nuget download link https://www.nuget.org/api/v2/package/Microsoft.Azure.Kinect.Sensor/1.4.1-beta.0, rename from .nupkg to .zip, and unzip for access to the AMD64 and ARM64 libraries.

4. From from the k4aviewer app, I consistently got 30FPS from the 3 different camera streams and 60FPS.

   • 3-4 of the 6 CPUs were maxing out, with the other 2-3 running at 50% (with a few chrome tabs open).
   • You can max out all the CPUs just by opening a lot of tabs.
   • Disabling a camera frees up a CPU.
   • Microphone Array and IMU also worked reliably.

5. The Body Tracking SDK does not run on ARM yet ... supposedly that is coming soon though.

---

Installing Body Tracking SDK

See Microsoft's Instructions

The Body Tracking SDK is not currently supported on ARM ... supposedly that is coming soon though.

At the time of writing, the latest Body Tracking SDK (version 1.0.0) has a dependency on version 1.3 of the Sensor SDK. However, k4a-tools is dependent on version 1.4. It's currently not possible to have multiple SDKs installed, so you have to pick one.

To downgrade from SDK 1.4 to 1.3:

1. Remove the 1.4 SDK.

sudo apt remove k4a-tools
sudo apt remove libk4a1.4
sudo apt remove libk4a1.4-dev

2. Install the 1.3 Sensor and Body Tracking SDKs.

sudo apt install k4a-tools=1.3.0 k4a1.3-dev libk4abt1.0 libk4abt1.0-dev

3. Run the apps

k4abt_simple_3d_viewer

---

Updating Firmware for Azure Kinect

See Microsoft's Instructions

1. Download the latest firmware's .bin file (at the time it was here).

• You can check the firmware of your connected device by running $AzureKinectFirmwareTool -q in the terminal.

2. In the terminal, go to the directory where you downloaded the firmware and update with the AzureKinectFirmwareTool: AzureKinectFirmwareTool.exe -u <device_firmware_file.bin>

git clone https://github.com/microsoft/Azure-Kinect-Sensor-SDK.git
cd Azure-Kinect-Sensor-SDK/firmware/
AzureKinectFirmwareTool -u <device_firmware_file.bin>

I got the following error when running the firmware tool:

ERROR: The device failed to reset after an update. Please manually power cycle the device.
[2020-06-10 11:33:27.424] [error] [t=19238] /__w/1/s/extern/Azure-Kinect-Sensor-SDK/src/firmware/firmware.c (61): firmware_t_get_context(). Invalid firmware_t (nil)
[2020-06-10 11:33:27.425] [error] [t=19238] /__w/1/s/extern/Azure-Kinect-Sensor-SDK/src/firmware/firmware.c (313): Invalid argument to firmware_get_device_version(). firmware_handle ((nil)) is not a valid handle of type firmware_t
ERROR: Failed to get updated versions

After power cylcing and unplugging from USB, the firmware did in fact update.

❤️

Thank you!

Awesome, thanks a lot!

solution for:

Do you want to continue? [Y/n] Y
Preconfiguring packages ...
(Reading database ... 141839 files and directories currently installed.)
Preparing to unpack .../libk4a1.3_1.3.0_amd64.deb ...
Unpacking libk4a1.3 (1.3.0) ...
dpkg: error processing archive /var/cache/apt/archives/libk4a1.3_1.3.0_amd64.deb (--unpack):
 trying to overwrite '/usr/lib/x86_64-linux-gnu/libk4a.so', which is also in package libk4a1.1-dev 1.1.1
Errors were encountered while processing:
 /var/cache/apt/archives/libk4a1.3_1.3.0_amd64.deb
E: Sub-process /usr/bin/dpkg returned an error code (1)

@LogWell thanks for sharing :)

Have you tried to extract color, depth and mask with C++(not k4aviewer or k4abt_simple_3d_viewer) on Ubuntu18? I encountered some problems when extracting mask(body tracking), hope someone can discuss it together

As for 20.04 it seems probably possible but more involved as of now. See here.

Dear all,

does anyone know why after the first step, when executing "sudo apt install k4a-tools", the error of "can't locate k4a-tools" pops up?
Like this:

E: Unable to locate package k4a1.3-dev
E: Couldn't find any package by glob 'k4a1.3-dev'
E: Couldn't find any package by regex 'k4a1.3-dev'
E: Unable to locate package k4abt1.0
E: Couldn't find any package by glob 'k4abt1.0'
E: Couldn't find any package by regex 'k4abt1.0'
E: Unable to locate package k4abt1.0-dev
E: Couldn't find any package by glob 'k4abt1.0-dev'
E: Couldn't find any package by regex 'k4abt1.0-dev'

There is no error during the first step and I have checked the /vat/lib/apt/lists/, there is no k4a-tools in the list.
I work on jetson tx2 with Ubuntu 16.04, but I have also checked the 18.04. Has anyone encountered this problem?
Any advices would be greatly appreciated.

I had to jump through some hoops to get it to work (as noted in Step 1). Where any errors printed out along the way?

If you're on 16.04, did you modify Microsoft's repository key accordingly? Something like:

https://packages.microsoft.com/keys/microsoft.asc | sudo apt-key add -
sudo apt-add-repository https://packages.microsoft.com/ubuntu/18.04/prod
sudo apt-get update

incredibly helpful, thanks a lot!

I am not sure if the installation procedure for body tracking library works, when I search k4a-tools in microsoft repo, it seems the version 1.3.0 is gone for Ubuntu 20.04, only version 1.4.1 exists...

~$ sudo apt search k4a-tool*
Sorting... Done
Full Text Search... Done
k4a-tools/bionic 1.4.1 amd64
  Tools for Azure Kinect Development

I wonder if this means body tracking cannot work on Ubuntu 20.04 for now?

I believe the Body Tracking SDK is now fully compatible with latest Sensor SDK — meaning you don't need to downgrade. I tried this week, and was able to install on 18.04 Linux PC using Microsoft's instructions.
Let me know if that's the case for you too, and I'll update the instructions...

I am trying to run k4abt_simple_3d_viewer CPU but I get

Shader Error: 0:2(38): error: gl_FragColor' undeclared
0:2(38): error: value of type vec4 cannot be assigned to variable of type error`

What am i missing here?

I can't find the BodyTracking library in the Arm64 version (Jetson Xavier)!

$ sudo apt install libk4abt1.*

E: Unable to locate package libk4abt1*

sudo apt-add-repository https://packages.microsoft.com/ubuntu/20.04/prod hangs forever for me.

nice!