Kinect Wings

Howto

Kinect Wings is a gesture controller for Tiny Wings. Check out this video: https://vimeo.com/38280453

Howto

We used Synapse to get the coordinates of hands. Synapse is an open source program which tracks human gestures with kinect and sends the position of joints or bones through OSC messages. We then need another program to receive, and handle these messages. For that task we used Processing with OSCP5. This sketch will convert the hand position to a servo direction, and sending that data over serial port. An Arduino will then receive the data on serial port and send the values to the servo motor connected directly to the Arduino board.

You will need:
Hardware:
- Kinect
- Arduino
- a Servo 5VDC
- iPhone 4S (only 4S version is enabled to be connected to a projector)
- any kind of stylus for iPad/iPhone (can be also DIY like here: http://blog.makezine.com/2010/05/05/collins-lab-diy-ipad-stylus/)
- a Mac with OSX
- vga–dock connector cable
- a projector
- cardboard, gluegun, duct tape, wires

Software:
- Processing (http://processing.org)
- oscP5 library for Processing http://www.sojamo.de/libraries/oscP5/ - Follow the instructions to put the unzipped files to the right folder
- Arduino software (http://arduino.cc)
- Synapse for Kinect (http://synapsekinect.tumblr.com/)
- Tiny Wings for iPhone (http://itunes.apple.com/us/app/tiny-wings/id417817520?mt=8)

1. Get a little 5V servo motor from a hobby electronic shop, for example this one: http://www.parallax.com/StoreSearchResults/tabid/768/txtSearch/servo/List/0/SortField/4/ProductID/488/Default.aspx
2. Create a little square layout pillar from cardboard with about 20 cm height which will hold the servo motor.
3. Attach servo on the cardboard with gluegun to the cardboard.
4. Attach stylus to the arm of the servo using some duct tape
5. Place iPhone under the stylus, connect iPhone to a projector and a music stand
6. Start Tiny Wings on iPhone
7. Connect Arduino to the computer via USB and connect the servo to the Arduino board. You can find instructions here: http://arduino.cc/en/Reference/Servo
8. Upload the supplied code to the Arduino board
9. Connect the Kinect to another USB port on your computer
10. Run Synapse for Kinect.
11. Start Processing, and run the supplied Processing code.
12. Make a psi pose in front of the kinect. You can check the control window of Synapse when your body is recognized.
13. Have fun!

kinectwings.pde

import processing.serial.*;
import oscP5.*;
import netP5.*;
OscP5 oscP5;
NetAddress myRemoteLocation;
Serial myPort;
float rightHand;

void setup() {
  myPort = new Serial(this, Serial.list()[0], 9600);  
  oscP5 = new OscP5(this,12345);
  myRemoteLocation = new NetAddress("127.0.0.1",12346);
}

void draw() {  
  int val = 40+int(rightHand/6);
  if (val<0) {val =0; }
  myPort.write(val); // sending value to servo
  
  // send keepalive messages to Synapse
  OscMessage myMessage = new OscMessage("/righthand_trackjointpos");
  myMessage.add(3); 
  oscP5.send(myMessage, myRemoteLocation); 
}

void oscEvent(OscMessage theOscMessage) {
  String addrPattern = theOscMessage.addrPattern();
  if (addrPattern.equals("/righthand_pos_screen")) {
    rightHand = theOscMessage.get(1).floatValue();
  } else {
    rightHand = 0; // park the stylus in safe position
  } 
}

servo.ino

#include <Servo.h> 
Servo myservo;  
int serialGet;
 
void setup() 
{ 
  myservo.attach(9); 
  Serial.begin(9600);
} 
void loop() 
{ 
  if (Serial.available()>0) {
    // getting value from Processing on serial
    serialGet = Serial.read(); 
    // writing to servo
    myservo.write(serialGet);  
  }
}