madelinegannon/kinectV2_openFrameworks_linux.md

## kinectV2_openFrameworks_linux.md

      
    Raw
  

              kinectV2_openFrameworks_linux.md
            
          
    How to get the Kinect V2 working in openFrameworks on Linux

This tutorial walks through how to get the Microsoft Kinect One working in Linux and openFrameworks.
Tested On


Microsoft Kinect One (aka Kinect V2)
OS: Ubuntu 18.04
Arch: linux64
openFrameworks Nightly (of_v20190324_linux64gcc6_nightly)
libfreenect2 (@dfd4eaf)

1. Install libfreenect2

To work with the KinectV2 in openFrameworks for Linux, you need to first install libfreenect2. You can find their Linux Installation instructions here.
> cd ~
> git clone https://github.com/OpenKinect/libfreenect2.git
> cd libfreenect2
> sudo apt-get install build-essential cmake pkg-config
> sudo apt-get install libusb-1.0-0-dev
> sudo apt-get install libturbojpeg0-dev
> sudo apt-get install libglfw3-dev

You'll also need to have cuda. If it's not already on your system, see installation instructions here.
Build the library and install system-wide:
> mkdir build && cd build
> cmake -DBUILD_SHARED_LIBS=ON ..
> make
> sudo make install

Set up udev rules for device access:
> sudo cp ../platform/linux/udev/90-kinect2.rules /etc/udev/rules.d/

Plug in the v2 Kinect, and then run the libfreect2's example app:
> ./bin/Protonect

With the Protonect example, should now see RGB, Depth, and IR feeds streaming from the Kinect. If not, look through libfreenect2's Troubleshooting section.

This step needs to be working before moving forward.

2. Install openFrameworks

This tutorial uses nightly build found at the bottom of the openFrameworks Downloads page.

Get the package name that corresponds with of_v2019XXXX_linux64gcc6_nightly.tar.gz.
At the time of this tutorial, the package name was of_v20190324_linux64gcc6_nightly.tar.gz.

To download and unpack openFrameworks:
> cd ~
> wget https://openframeworks.cc/ci_server/versions/nightly/of_v20190324_linux64gcc6_nightly.tar.gz
> tar -zxvf of_v20190324_linux64gcc6_nightly.tar.gz

Follow openFrameworks's Installation Instructions for Linux:
> cd of_v20190324_linux64gcc6_release/scripts/linux/ubuntu
> sudo ./install_dependencies.sh
> cd ..
> ./compileOf.sh

Test your build by running one of openFramework's example apps:
> cd ../../../examples/graphics/polygonExample
> make
> make run

3. Get the ofxKinectV2 addon

There are a few addons out there for working with the KinectV2 in openFrameworks, but Linux support is rare. I modifed Theo Watson's ofxKinectV2 to work for Linux. Download my fork to start:
> cd ~/of/addons
> cd git clone https://github.com/madelinegannon/ofxKinectV2.git

By default, when an openFrameworks app builds, it links to the libraries in its local libs folder. But we want libfreenect2 and libusb-1.0 to link against our system wide installations. To resolve this, move or delete these folders in ofxKinectV2/libs:
> cd ofxKinectV2/libs
> mv libfreenect2 ~/Desktop
> mv libusb ~/Desktop

Now you should only see a protonect folder in libs.
Next, we want to make sure our LD_LIBRARY_PATH is set properly, so openFrameworks knows where to search for ofKinectV2's linked libraries.
Add CUDA paths to the system environment (you should also to add these lines to your ~/.bashrc):
> export LD_LIBRARY_PATH="/usr/local/cuda/lib64:${LD_LIBRARY_PATH}"
> export PATH="/usr/local/cuda/bin:${PATH}"

A system-wide configuration of the libary path can be created with the following commands:
> echo "/usr/local/cuda/lib64" | sudo tee /etc/ld.so.conf.d/cuda.conf
> sudo ldconfig

Also add the libfreenect2 path to the system environment (you should also add this path to your ~/.bashrc):
export LD_LIBRARY_PATH="/usr/local/lib:${$LD_LIBRARY_PATH}"

4. Build the example app

Everything should be set up to build and run the example app included of ofxKinectV2.
But first, just a quick note on how things get linked and built in openFrameworks on linux64:

The ofxKinectV2/addons_config.mk file is used by openFramework's Project Generator to add all the addon libraries and dependencies to a given IDE. If you're building from the command line, this file gets ignored.
To build the example app from the command line, I added a default openFrameworks Makefile and config.make inside the example/ directory.

The config.make is where you add all the compiler flags for a project.
You'll notice I've added libfreenect2 and cuda paths and libraries to the variable PROJECT_LDFLAGS.


Now we can make and run:
> make -j
> make run

Now you should see something very similar to libfreenect2 Protonect example, but with the addition of some sliders that let you easily do depth thresholding.
Troubleshooting

I had one runtime error that really tripped me up for a bit. The ofxKinectV2 example app would build and run, and in the console it would acknowledge that it detected and opened the Kinect. However, the stream wouldn't start, and the following error would print out:
[Error] [protocol::CommandTransaction] bulk transfer failed: LIBUSB_ERROR_TIMEOUT Operation timed out

What was happening was that openFrameworks wasn't actually linking to libfreenect2.so at runtime. I fixed this by making sure that I removed the local libfreenect2 directory in ofxKinectV2/libs, and that the system-wide path to libfreenect2.so (/usr/local/lib) was also added to the LD_LIBRARY_PATH in my ~/.bashrc.
Helpful Resources

Here are a few sites that helped me piece together how to get this running:

Building ofxKinectV2 on Linux: ofTheo/ofxKinectV2#27
ofxLibfreenect2: https://github.com/pierrep/ofxLibFreenect2
IAI Kinect2 for ROS: https://github.com/code-iai/iai_kinect2
openFrameworks Forum: SOLVED ofxKinectV2 on windows with libfreenect2