Skip to content

Instantly share code, notes, and snippets.

@madelinegannon
Last active November 3, 2024 19:40
Show Gist options
  • Save madelinegannon/c212dbf24fc42c1f36776342754d81bc to your computer and use it in GitHub Desktop.
Save madelinegannon/c212dbf24fc42c1f36776342754d81bc to your computer and use it in GitHub Desktop.
Notes on Setting up the Microsoft Azure Kinect on Ubuntu 18.04

Azure Kinect on Ubuntu 18.04

Notes on Installing Microsoft Azure Kinect Sensor and Body Tracking SDKs on Linux PC and NVIDIA Jetson Xavier NX

06.12.2020

Jump to:


Installing Sensor SDK on Linux PC

See Microsoft's Installation Guide

1. Add link to Microsoft's Linux Software Repository for AMD64

By adding microsoft's code repository to your system so that you can call sudo apt install xxx to install system wide.

Guide for Configuring the Repository

Here are Microsoft's Instructions:

curl https://packages.microsoft.com/keys/microsoft.asc | sudo apt-key add -
sudo apt-add-repository https://packages.microsoft.com/ubuntu/18.04/prod
sudo apt-get update

I ran into errors with these instructions. Here's how I solved them:

  1. Install curl if you haven't already.
sudo apt update && upgrade
sudo apt install curl
  1. Specify the architecture of the repository. Running the repository command, I got this error on runningsudo apt-get update: N: Skipping acquire of configured file 'main/binary-i386/Packages' as repository 'https://packages.microsoft.com/ubuntu/18.04/prod' doesn't support support architecture 'i386'.

Turns out you need specify the architecture of the repository to [arch=amd64]:

  • Log in as root:
sudo -i
  • Modify /etc/apt/sources.list. At the bottom of the file, change from:
deb https://packages.microsoft.com/ubuntu/18.04/prod bionic main
# deb-src https://packages.microsoft.com/ubuntu/18.04/prod bionic main

to:

deb [arch=amd64] https://packages.microsoft.com/ubuntu/18.04/prod bionic main
# deb-src [arch=amd64] https://packages.microsoft.com/ubuntu/18.04/prod bionic main

and save.

  • Log out of root:
exit
  1. Rerun sudo apt-get update

2. Install Kinect Packages

If there are no more errors, you should be able to install the Kinect packages install libk4a1.X libk4a1.X-dev k4a-tools

If you install k4a-tools first, it automatically downloads the latest version of libk4a1.X. Take note of the version for installing libk4a1.X-dev (it must be the same version as libk4a1.X).

sudo apt install k4a-tools
sudo apt install libk4a1.4-dev

If you mess up, you can always uninstall and try again using sudo apt remove k4a-tools

3. Finish Device Setup

Finish device setup by setting up udev rules:

  • Copy 'scripts/99-k4a.rules' into '/etc/udev/rules.d/'.
  • Detach and reattach Azure Kinect devices if attached during this process.

4. Run an App

To run an app, you just call it directly in the terminal:

k4aviewer

You may get a warning to upgrade your firmware when running k4aviewer the first time. See instructions below.

The k4aviewer, k4arecorder, and AzureKinectFirmwareTool downloaded through k4a-tools are all located in /usr/bin.

5. Download Samples

Microsoft provides code samples that demonstrate how to use Microsoft's Azure Kinect DK Sensor and Body Tracking SDKs.

git clone https://github.com/microsoft/Azure-Kinect-Samples.git

Installing Sensor SDK on Jetson Xavier NX

At the time of writing (June 2020), Microsoft has debian packages for Ubuntu 18.04 for AMD64 and ARM64. However, this is the first release for ARM: there are some bugs and the Body Tracking SDK is not yet supported. More details on that below.

1. Add Microsoft's product repository for ARM64

curl https://packages.microsoft.com/keys/microsoft.asc | sudo apt-key add -
sudo apt-add-repository https://packages.microsoft.com/ubuntu/18.04/multiarch/prod
sudo apt-get update

2. Install Kinect Package

Same as instructions above.

3. Finish Device Setup

Same as instructions above.

4. Increase default USB memory for Ubuntu

By default, Linux-based host computers allocate the USB controller only 16 MB of kernel memory to handle USB transfers. This amount is supposed to be enough to support a single Azure Kinect ... so this step might not be necessary. But I went ahead and did it anyway just to be safe.

Microsoft has instructions for this, however Jetson boards don't have grub. So instead, I used these instructions from FLIR for the Jetson TX2.

To acquire images greater than 2 MB in resolution, add the following to the APPEND line:

usbcore.usbfs_memory_mb=1000

to this file:

/boot/extlinux/extlinux.conf

If this method fails to set the memory limit, run the following command:

sudo sh -c 'echo 1000 > /sys/module/usbcore/parameters/usbfs_memory_mb'

Here's a good overview of why USBFS matters for usb camera on Linux, also from FLIR

Notes on Running on Jetson Xavier NX

  1. The Jetson Xavier NX Developer Kit seems compatible with minimum system requirements of the Azure Kinect.
  2. I'm running my X NX 15W 6CORE mode with 250 GB NVMe SSD added to the DK.
    • See this Jetson Hacks repo for a walkthrough of adding NVMe storage to the Jetson X NX.
  3. Testing with k4aviewer, I got the following warnings and errors:

This warning is constant, but harmless:

[ warning  ] : depth_engine_thread(). Depth image processing is too slow at 1000ms (this may be transient).

This is a bug that crashes the data stream when the color camera is enabled:

[ error    ] : DecodeMJPEGtoBGRA32(). MJPEG decode failed: -1
[ warning  ] : capturesync_add_capture(). Capture Error Detected, Color 
[ error    ] : capturesync_get_capture(device->capturesync, capture_handle, timeout_in_ms) returned failure in k4a_device_get_capture()

Disabling the Color Camera avoids the errors, however there's a fix in the next release, 1.4.1. But you can download the beta with the fix now:

  1. From from the k4aviewer app, I consistently got 30FPS from the 3 different camera streams and 60FPS.

    • 3-4 of the 6 CPUs were maxing out, with the other 2-3 running at 50% (with a few chrome tabs open).
    • You can max out all the CPUs just by opening a lot of tabs.
    • Disabling a camera frees up a CPU.
    • Microphone Array and IMU also worked reliably.
  2. The Body Tracking SDK does not run on ARM yet ... supposedly that is coming soon though.


Installing Body Tracking SDK

See Microsoft's Instructions

The Body Tracking SDK is not currently supported on ARM ... supposedly that is coming soon though.

At the time of writing, the latest Body Tracking SDK (version 1.0.0) has a dependency on version 1.3 of the Sensor SDK. However, k4a-tools is dependent on version 1.4. It's currently not possible to have multiple SDKs installed, so you have to pick one.

To downgrade from SDK 1.4 to 1.3:

  1. Remove the 1.4 SDK.
sudo apt remove k4a-tools
sudo apt remove libk4a1.4
sudo apt remove libk4a1.4-dev
  1. Install the 1.3 Sensor and Body Tracking SDKs.
sudo apt install k4a-tools=1.3.0 k4a1.3-dev libk4abt1.0 libk4abt1.0-dev
  1. Run the apps
k4abt_simple_3d_viewer

Updating Firmware for Azure Kinect

See Microsoft's Instructions

  1. Download the latest firmware's .bin file (at the time it was here).
  • You can check the firmware of your connected device by running $AzureKinectFirmwareTool -q in the terminal.
  1. In the terminal, go to the directory where you downloaded the firmware and update with the AzureKinectFirmwareTool: AzureKinectFirmwareTool.exe -u <device_firmware_file.bin>
git clone https://github.com/microsoft/Azure-Kinect-Sensor-SDK.git
cd Azure-Kinect-Sensor-SDK/firmware/
AzureKinectFirmwareTool -u <device_firmware_file.bin>

I got the following error when running the firmware tool:

ERROR: The device failed to reset after an update. Please manually power cycle the device.
[2020-06-10 11:33:27.424] [error] [t=19238] /__w/1/s/extern/Azure-Kinect-Sensor-SDK/src/firmware/firmware.c (61): firmware_t_get_context(). Invalid firmware_t (nil)
[2020-06-10 11:33:27.425] [error] [t=19238] /__w/1/s/extern/Azure-Kinect-Sensor-SDK/src/firmware/firmware.c (313): Invalid argument to firmware_get_device_version(). firmware_handle ((nil)) is not a valid handle of type firmware_t
ERROR: Failed to get updated versions

After power cylcing and unplugging from USB, the firmware did in fact update.

@valdivj
Copy link

valdivj commented Sep 29, 2020

Awsome

@madelinegannon
Copy link
Author

❤️

@ArturNiederfahrenhorst
Copy link

Thank you!

@NehilDanis
Copy link

Awesome, thanks a lot!

@LogWell
Copy link

LogWell commented Jan 8, 2021

solution for:

Do you want to continue? [Y/n] Y
Preconfiguring packages ...
(Reading database ... 141839 files and directories currently installed.)
Preparing to unpack .../libk4a1.3_1.3.0_amd64.deb ...
Unpacking libk4a1.3 (1.3.0) ...
dpkg: error processing archive /var/cache/apt/archives/libk4a1.3_1.3.0_amd64.deb (--unpack):
 trying to overwrite '/usr/lib/x86_64-linux-gnu/libk4a.so', which is also in package libk4a1.1-dev 1.1.1
Errors were encountered while processing:
 /var/cache/apt/archives/libk4a1.3_1.3.0_amd64.deb
E: Sub-process /usr/bin/dpkg returned an error code (1)

@madelinegannon
Copy link
Author

@LogWell thanks for sharing :)

@LogWell
Copy link

LogWell commented Jan 8, 2021

Have you tried to extract color, depth and mask with C++(not k4aviewer or k4abt_simple_3d_viewer) on Ubuntu18? I encountered some problems when extracting mask(body tracking), hope someone can discuss it together

@matanox
Copy link

matanox commented Feb 4, 2021

As for 20.04 it seems probably possible but more involved as of now. See here.

@JianxiangFENG
Copy link

JianxiangFENG commented Feb 7, 2021

Dear all,

does anyone know why after the first step, when executing "sudo apt install k4a-tools", the error of "can't locate k4a-tools" pops up?
Like this:

E: Unable to locate package k4a1.3-dev
E: Couldn't find any package by glob 'k4a1.3-dev'
E: Couldn't find any package by regex 'k4a1.3-dev'
E: Unable to locate package k4abt1.0
E: Couldn't find any package by glob 'k4abt1.0'
E: Couldn't find any package by regex 'k4abt1.0'
E: Unable to locate package k4abt1.0-dev
E: Couldn't find any package by glob 'k4abt1.0-dev'
E: Couldn't find any package by regex 'k4abt1.0-dev'

There is no error during the first step and I have checked the /vat/lib/apt/lists/, there is no k4a-tools in the list.
I work on jetson tx2 with Ubuntu 16.04, but I have also checked the 18.04. Has anyone encountered this problem?
Any advices would be greatly appreciated.

@madelinegannon
Copy link
Author

I had to jump through some hoops to get it to work (as noted in Step 1). Where any errors printed out along the way?

If you're on 16.04, did you modify Microsoft's repository key accordingly? Something like:

https://packages.microsoft.com/keys/microsoft.asc | sudo apt-key add -
sudo apt-add-repository https://packages.microsoft.com/ubuntu/18.04/prod
sudo apt-get update

@lschmidtke
Copy link

incredibly helpful, thanks a lot!

@ShaoxiongYao
Copy link

I am not sure if the installation procedure for body tracking library works, when I search k4a-tools in microsoft repo, it seems the version 1.3.0 is gone for Ubuntu 20.04, only version 1.4.1 exists...

~$ sudo apt search k4a-tool*
Sorting... Done
Full Text Search... Done
k4a-tools/bionic 1.4.1 amd64
  Tools for Azure Kinect Development

I wonder if this means body tracking cannot work on Ubuntu 20.04 for now?

@madelinegannon
Copy link
Author

I believe the Body Tracking SDK is now fully compatible with latest Sensor SDK — meaning you don't need to downgrade. I tried this week, and was able to install on 18.04 Linux PC using Microsoft's instructions.
Let me know if that's the case for you too, and I'll update the instructions...

@angelicaCruz
Copy link

I am trying to run k4abt_simple_3d_viewer CPU but I get

Shader Error: 0:2(38): error: gl_FragColor' undeclared
0:2(38): error: value of type vec4 cannot be assigned to variable of type error`

What am i missing here?

@bhomaidan1990
Copy link

I can't find the BodyTracking library in the Arm64 version (Jetson Xavier)!

$ sudo apt install libk4abt1.*
E: Unable to locate package libk4abt1*

@denniskb
Copy link

sudo apt-add-repository https://packages.microsoft.com/ubuntu/20.04/prod hangs forever for me.

@silencht
Copy link

nice!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment