leapmotion-vr-quickstart.md

Leap Motion + VR in Javascript

Awesome! You’re doing a hackathon. This is a quickstart guide for getting going.

Here are the components you need, after THREE.js and Leap.js:

• OculusRiftEffect - Renders the lense effect with THREE.js or Mozilla’s native API*.
• OculusControls.js - Takes position tracking data from your oculus to your web app
• The LeapJS optimizeHMD flag.
• leap.transform.js - Re-orients the hand for usage in top-down mode.

In addition, there are some libraries floating around which you may find useful in your app:

• leap.boneHand.js - Add a hand to your THREE.js scene, constructed from cylinders and spheres.
• leap.proximity.js - [alpha] detects collisions between either a points, line segments, and plane segments.

There is one example called Light Pinch VR, which demonstrates the VREffect, optimizeHMD, and leap.transform.js. You can view the source here: https://developer.leapmotion.com/gallery/light-pinch-vr

Let me point out three relevant pieces of code:

Rift Effect

// setting up
effect = new THREE.OculusRiftEffect(renderer, {
  worldScale: 2
});

effect.setSize(window.innerWidth, window.innerHeight);

// when resizing:
effect.setSize(window.innerWidth, window.innerHeight)

// when rendering:
effect.render(scene, camera);

Optimize HMD

// optimizeHMD optimizes the Leap Motion tracking for seeing the backs of hands from above.  It doesn’t require background mode, but the flag is seen here as a nice convenience.
window.controller = controller = new Leap.Controller({
  background: true,
  optimizeHMD: true
});

Transform

Without this, your hand would show up upside-down in the scene when in HMD mode. Here we specify a position offset and a rotation to be applied.

IMPORTANT - There’s a couple bugs in the released version of this plugin, which make it not work with the bone hand. Grab the latest from master, here. Changelog.

 controller.use('transform', {
  quaternion: (new THREE.Quaternion).setFromEuler(new THREE.Euler(Math.PI * -0.3, 0, Math.PI, 'ZXY')),
  position: new THREE.Vector3(0, 100, 0)
});

OculusControls

Used to get the Oculus position and orientation data in to your web site, over a user-installed REST server. Example is here: http://threejs.org/examples/#misc_controls_oculusrift

*Mozilla Native API

Mozilla is providing the Oculus Effect and Oculus Controls out of the box with a special VR Firefox build. It’s pretty sweet. To use, swap out OculusEffect.js and OculusControls.js with VREffect.js and VRControls.js, and get the VR build as seen here: leapmotion/javascript#4

Other Tools

Bone Hand

Add a hand to your scene. Pretty simple. Get the latest, here.

You can either give it an element, and it will add a scene and do all the THREE.js work, or you can pass in a scene of your own. Here’s some config:

// set up the hand
controller.use('boneHand', {
  scene: myScene, 
  opacity: 0.7,
  jointColor: new THREE.Color(0x222222),
  arm: true
})

If you don’t pass in your own scene, you can access the default one (as well as the renderer, camera, etc).

var scene = controller.plugins.boneHand.scene

If you want hands attached to the camera, here's a quote from one of our apps:

  //we don't actually want the leap hand as child of the camera, as we want the data itself properly transformed.
  // on every frame, we combine the base transformation with the camera transformation to the leap data
  // this is used in grab, proximity, etc.
  Leap.loopController.plugins.boneHand.scene = scene;
  Leap.loopController.plugins.transform.effectiveParent = camera;

Leap Proximity

This allows you to get the intersection point between a Line Segment and a Plane Segment. You’d best play with the example here (requires Leap): http://leapmotion.github.io/VRCollage/testProximity.html, and the source, here: view-source:http://leapmotion.github.io/VRCollage/testProximity.html

The API is like this:

// Create a “proximity”, which combines the object in the scene and a function which will be called on every frame with the hand and return either an array of points or an array of point-pairs to check against.  Note: this current'y doesn't support angled planes.

This can also be used to detect point intersecting spheres, by giving a `THREE.SphereMesh` as the first argument, and have an array of single points (not tuples) returned by the callback.

  var proximity = Leap.loopController.watch(
    plane, // a Plane mesh
    function(hand){
      return [
        lineGeometry.vertices // Two THREE.Vector3s
      ]
    }
  );

  proximity.in(function(){
    console.log('in');
    sphere.material.color.setHex(0x00cc04);
  });

  proximity.out(function(){
    console.log('out');
    sphere.material.color.setHex(0x003304);
  });

  Leap.loopController.on('frame', function(frame){

    var points = proximity.intersectionPoints, key;

    for (var key in points){

      if (points.hasOwnProperty(key)){
        // leap.proximity doubles the function for a per-hand basis.  so we only take the first.

        // only one sphere for now.
        sphere.position.copy(points[key]);

        break;
      }

    }


  });

That’s it!

In case you need more:

Some official plugins/libs: https://developer.leapmotion.com/downloads/javascript

Examples: https://developer.leapmotion.com/gallery/tags/javascript

Get in touch: pehrlich@leapmotion.com or @ehrlicp on twitter.