Skip to content

Instantly share code, notes, and snippets.

@mnyrop

mnyrop/nyu_ds_ar_sandbox.md

Last active November 6, 2019 16:27
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save mnyrop/918730683cfb5590d1bdc3e64a467072 to your computer and use it in GitHub Desktop.
Save mnyrop/918730683cfb5590d1bdc3e64a467072 to your computer and use it in GitHub Desktop.
Download ZIP
Raw
nyu_ds_ar_sandbox.md

Notes for Setting up NYU AR Sandbox

Here's what it will look like: https://www.youtube.com/watch?v=RetExD02kk&feature=youtu.be

sample ar sandbox

You will need

Hardware / materials

  • Oculus ready ASUS PC with NVIDIA GeForce GTX1060 3GB graphics
    • Power cord
    • HDMI to HDMI cable
    • USB mouse
    • USB keyboard
  • HDMI Monitor
    • Ethernet cable (for downloading software + patches; not necessary day-of)
  • Short-throw HDMI projector
    • Power cord
    • HDMI to HDMI cable
  • Kinect 3D camera
    • USB adapter + power cord (1 item)
  • Extension cords
  • Sand
  • Sandbox
  • Metal beams for projector
  • Metal Kinect mount
  • Clamps (x4)
  • Calibration disc
  • Zip ties / rubber bands
  • Tape
  • Felt spacers
  • Counter weight discs
  • Sand toys + brush

Software

Credentials

  • Use ds admin user. For pw, see NYU box.

Steps

Software installation

  1. Check for software from previous iterations by running ls ~/src/. If you see SARndbox-2.6 listed, skip to Physical installation steps.
  2. Check for NVIDIA drivers by running glxinfo | grep vendor. You can skip to step 4 if you see: 
    server glx vendor string: NVIDIA Corporation
client glx vendor string: NVIDIA Corporation
OpenGL vendor string: NVIDIA Corporation
  3. Install NVIDIA drivers: https://linoxide.com/debian/install-nvidia-driver-debian/.
  4. Follow steps 3-11 on this guide: http://idav.ucdavis.edu/~okreylos/ResDev/SARndbox/SoftwareInstallation.html.

Physical installation

diagram

  1. Put empty sandbox on 2 triangle rolling tables
  2. Lean 2 large metal beams against sandbox on the marked side
  3. Clamp mounts to box, 2 clamps per beam
  4. Fill box with sand and smooth it out as much as possible
  5. Attach Kinect mount to beams
  6. Add projector to holder at the top, using felt spacers if necessary, and secure.
  7. Attach Kinect 3D camera to holder using zip ties or rubber bands
  8. Use counter weight discs to balance weight of the Kinect camera; secure with rubber bands
  9. Plug in power cords for 3D camera, projector, and PC
  10. Connect 3D camera USB adapter to PC
  11. Connect projector and monitor both to PC with HDMI cables (x2)
  12. Connect USB mouse and keyboard to PC

Kinect 3D camera and projector calibration

  1. Mount the Kinect camera above the sandbox so that it is looking straight down. Turn it on and ensure it can see the entire sand surface.

  2. Mount the projector above the sand surface so that it projects its image perpendicularly onto the flattened sand surface, and so that the projector's field-of-projection and the Kinect camera's field-of- view overlap as much as possible. Focus the projector to the flattened average-height sand surface, then turn it off for now.

  3. Use RawKinectViewer from the Kinect 3D video capture project to line up the depth camera while ignoring the color camera. Measure the base plane equation of the sand surface relative to the Kinect camera's internal coordinate system using RawKinectViewer's plane extraction tool. (See: Step 8, Video@1:12)

  4. Use RawKinectViewer to measure the 3D extents of the sand surface. (See: Step 9, Video@4:05)

  5. Turn on and align the projector with Vrui's XBackground utility. (See: Step 10)

  6. Calculate a calibration matrix from the Kinect camera's camera space to projector space using the CalibrateProjector utility and a circular calibration target. (See: Step 11, Video@10:10)

Running the application

When calibration is complete, it is now possible to run the main Augmented Reality Sandbox application from inside the source code directory:

$ ./bin/SARndbox -uhm -fpv

The -fpv ("fix projector view") option tells the AR Sandbox to use the projector calibration matrix created in step 7. The -uhm ("Use Height Map") option tells the AR Sandbox to color-map the 3D surface by elevation, using the default height color map.

It is very important to run the application in full-screen mode on the projector, or at least with the exact same window position and size as CalibrateProjector in step 7. If this is not done correctly, the calibration will not work as desired. To manually switch SARndbox into full-screen mode after start-up, press the F11 function key.

To check the calibration, observe how the projected colors and topographic contour lines exactly match the physical features of the sand surface. If there are discrepancies between the two, repeat calibration step 7 until satisfied. On a typical 40"x30" sandbox, where the Kinect is mounted approximately 38" above the sandbox's center point, and using a perpendicularly projecting 1024x768 projector, alignment between the real sand surface and the projected features should be on the order of 1 mm.

SARndbox provides a plethora of configuration files and command line options to fine-tune the operation of the Augmented Reality Sandbox as desired. Run SARndbox -h to see the full list of options and their default values, or refer to external documentation on the project's web site.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment