Notes on Installing Microsoft Azure Kinect Sensor and Body Tracking SDKs on Linux PC and NVIDIA Jetson Xavier NX
06.12.2020
Jump to:
- Installing Sensor SDK on Linux PC
- Installing on Jetson Xavier NX
- Installing Body Tracking SDK
- Updating Firmware for Azure Kinect
By adding microsoft's code repository to your system so that you can call sudo apt install xxx
to install system wide.
Here are Microsoft's Instructions:
curl https://packages.microsoft.com/keys/microsoft.asc | sudo apt-key add -
sudo apt-add-repository https://packages.microsoft.com/ubuntu/18.04/prod
sudo apt-get update
I ran into errors with these instructions. Here's how I solved them:
- Install
curl
if you haven't already.
sudo apt update && upgrade
sudo apt install curl
- Specify the architecture of the repository.
Running the repository command, I got this error on running
sudo apt-get update
:N: Skipping acquire of configured file 'main/binary-i386/Packages' as repository 'https://packages.microsoft.com/ubuntu/18.04/prod' doesn't support support architecture 'i386'
.
Turns out you need specify the architecture of the repository to [arch=amd64]
:
- Log in as root:
sudo -i
- Modify
/etc/apt/sources.list
. At the bottom of the file, change from:
deb https://packages.microsoft.com/ubuntu/18.04/prod bionic main
# deb-src https://packages.microsoft.com/ubuntu/18.04/prod bionic main
to:
deb [arch=amd64] https://packages.microsoft.com/ubuntu/18.04/prod bionic main
# deb-src [arch=amd64] https://packages.microsoft.com/ubuntu/18.04/prod bionic main
and save.
- Log out of root:
exit
- Rerun
sudo apt-get update
If there are no more errors, you should be able to install the Kinect packages install libk4a1.X libk4a1.X-dev k4a-tools
If you install
k4a-tools
first, it automatically downloads the latest version oflibk4a1.X
. Take note of the version for installinglibk4a1.X-dev
(it must be the same version aslibk4a1.X
).
sudo apt install k4a-tools
sudo apt install libk4a1.4-dev
If you mess up, you can always uninstall and try again using
sudo apt remove k4a-tools
Finish device setup by setting up udev rules:
- Copy 'scripts/99-k4a.rules' into '/etc/udev/rules.d/'.
- Detach and reattach Azure Kinect devices if attached during this process.
To run an app, you just call it directly in the terminal:
k4aviewer
You may get a warning to upgrade your firmware when running
k4aviewer
the first time. See instructions below.
The
k4aviewer
,k4arecorder
, andAzureKinectFirmwareTool
downloaded throughk4a-tools
are all located in/usr/bin
.
Microsoft provides code samples that demonstrate how to use Microsoft's Azure Kinect DK Sensor and Body Tracking SDKs.
git clone https://github.com/microsoft/Azure-Kinect-Samples.git
At the time of writing (June 2020), Microsoft has debian packages for Ubuntu 18.04 for AMD64 and ARM64. However, this is the first release for ARM: there are some bugs and the Body Tracking SDK is not yet supported. More details on that below.
curl https://packages.microsoft.com/keys/microsoft.asc | sudo apt-key add -
sudo apt-add-repository https://packages.microsoft.com/ubuntu/18.04/multiarch/prod
sudo apt-get update
Same as instructions above.
Same as instructions above.
By default, Linux-based host computers allocate the USB controller only 16 MB of kernel memory to handle USB transfers. This amount is supposed to be enough to support a single Azure Kinect ... so this step might not be necessary. But I went ahead and did it anyway just to be safe.
Microsoft has instructions for this, however Jetson boards don't have grub. So instead, I used these instructions from FLIR for the Jetson TX2.
To acquire images greater than 2 MB in resolution, add the following to the APPEND line:
usbcore.usbfs_memory_mb=1000
to this file:
/boot/extlinux/extlinux.conf
If this method fails to set the memory limit, run the following command:
sudo sh -c 'echo 1000 > /sys/module/usbcore/parameters/usbfs_memory_mb'
Here's a good overview of why USBFS matters for usb camera on Linux, also from FLIR
- The Jetson Xavier NX Developer Kit seems compatible with minimum system requirements of the Azure Kinect.
- I'm running my X NX
15W 6CORE
mode with 250 GB NVMe SSD added to the DK.- See this Jetson Hacks repo for a walkthrough of adding NVMe storage to the Jetson X NX.
- Testing with
k4aviewer
, I got the following warnings and errors:
This warning is constant, but harmless:
[ warning ] : depth_engine_thread(). Depth image processing is too slow at 1000ms (this may be transient).
This is a bug that crashes the data stream when the color camera is enabled:
[ error ] : DecodeMJPEGtoBGRA32(). MJPEG decode failed: -1
[ warning ] : capturesync_add_capture(). Capture Error Detected, Color
[ error ] : capturesync_get_capture(device->capturesync, capture_handle, timeout_in_ms) returned failure in k4a_device_get_capture()
Disabling the Color Camera avoids the errors, however there's a fix in the next release, 1.4.1. But you can download the beta with the fix now:
- Click this nuget download link https://www.nuget.org/api/v2/package/Microsoft.Azure.Kinect.Sensor/1.4.1-beta.0, rename from
.nupkg
to.zip
, and unzip for access to the AMD64 and ARM64 libraries.
-
From from the
k4aviewer
app, I consistently got 30FPS from the 3 different camera streams and 60FPS.- 3-4 of the 6 CPUs were maxing out, with the other 2-3 running at 50% (with a few chrome tabs open).
- You can max out all the CPUs just by opening a lot of tabs.
- Disabling a camera frees up a CPU.
- Microphone Array and IMU also worked reliably.
-
The Body Tracking SDK does not run on ARM yet ... supposedly that is coming soon though.
The Body Tracking SDK is not currently supported on ARM ... supposedly that is coming soon though.
At the time of writing, the latest Body Tracking SDK (version 1.0.0) has a dependency on version 1.3 of the Sensor SDK. However, k4a-tools
is dependent on version 1.4. It's currently not possible to have multiple SDKs installed, so you have to pick one.
To downgrade from SDK 1.4 to 1.3:
- Remove the 1.4 SDK.
sudo apt remove k4a-tools
sudo apt remove libk4a1.4
sudo apt remove libk4a1.4-dev
- Install the 1.3 Sensor and Body Tracking SDKs.
sudo apt install k4a-tools=1.3.0 k4a1.3-dev libk4abt1.0 libk4abt1.0-dev
- Run the apps
k4abt_simple_3d_viewer
- Download the latest firmware's .bin file (at the time it was here).
- You can check the firmware of your connected device by running
$AzureKinectFirmwareTool -q
in the terminal.
- In the terminal, go to the directory where you downloaded the firmware and update with the AzureKinectFirmwareTool:
AzureKinectFirmwareTool.exe -u <device_firmware_file.bin>
git clone https://github.com/microsoft/Azure-Kinect-Sensor-SDK.git
cd Azure-Kinect-Sensor-SDK/firmware/
AzureKinectFirmwareTool -u <device_firmware_file.bin>
I got the following error when running the firmware tool:
ERROR: The device failed to reset after an update. Please manually power cycle the device.
[2020-06-10 11:33:27.424] [error] [t=19238] /__w/1/s/extern/Azure-Kinect-Sensor-SDK/src/firmware/firmware.c (61): firmware_t_get_context(). Invalid firmware_t (nil)
[2020-06-10 11:33:27.425] [error] [t=19238] /__w/1/s/extern/Azure-Kinect-Sensor-SDK/src/firmware/firmware.c (313): Invalid argument to firmware_get_device_version(). firmware_handle ((nil)) is not a valid handle of type firmware_t
ERROR: Failed to get updated versions
After power cylcing and unplugging from USB, the firmware did in fact update.
Awsome