Hello World

This article discusses the JavaScript sample application included with the Leap SDK. After reading this article, you should be ready to access Leap Motion hand tracking data from your own Web applications.

In the Leap SDK folder, you can find the following file used for this article:

  • LeapSDK/samples/Sample.html — JavaScript sample application

You will also need the Leap JavaScript client library available from Leap Motion’s LeapJS repository on GitHub.

Overview

In a nutshell, the Leap Motion tracking device detects and tracks hands and fingers placed within its field of view. The Leap captures this data one frame at a time. Web applications can use the Leap Motion JavaScript API to access this data. The Leap Motion controller sends tracking information through the socket connection as a JSON formatted message. The JavaScript API takes the JSON message and evaluates it into proper objects.

The sample application demonstrates how to use the Leap Motion JavaScript API. The example is contained in a single Web page, Sample.html. The application displays several properties from the key tracking data objects in the Leap Motion API, including:

  • Frame — contains a set of hand and pointable tracking data
  • Hand — contains tracking data for a detected hand
  • Pointable — contains tracking data for a detected finger or tool
  • Gesture — represents a recognized gesture

JavaScript client library

The Leap Motion JavaScript client library is available from Leap Motion’s LeapJS repository on GitHub. The library handles the connection to the Leap Motion service or daemon and passes frames of tracking data to your application. LeapJS is an open-source project and distributed separately from the main Leap SDK. We encourage community contributions to this library.

Setting up the page

To use the JavaScript API, you must include the leap.js file (or the minified version, leap.min.js). The Sample.html file uses a simple HTML structure to display the Leap data and include the leap.js file in the usual way using script tags:

<html>
  <head>
    <title>Leap JavaScript Sample</title>
    <script src="leap.js"></script>
    <script>// JavaScript code goes here</script>
  </head>
  <body onload="checkLibrary()">
    <h1>Leap JavaScript Sample</h1>
    <div id="main">
      <h3>Frame data:</h3>
      <div id="frameData"></div>
      <div style="clear:both;"></div>
      <h3>Hand data:</h3>
      <div id="handData"></div>
      <div style="clear:both;"></div>
      <h3>Finger and tool data:</h3>
      <div id="pointableData"></div>
      <div style="clear:both;"></div>
      <h3>Gesture data:</h3>
      <div id="gestureData"></div>
    </div>
  </body>
</html>

To keep things simple, all the example code in the example is in the main HTML file.

Setting up the event loop

Since the Leap Motion controller provides data continuously, you need to set up an event loop to handle individual frames of data. You can either set up a loop that handles the data as the Leap Motion controller produces each frame, or you can set up a loop that handles the data when your application is ready to do something with it. The Leap Motion JavaScript client library supports both options.

This example demonstrates the second method since there is no need in this app to process the data faster than the browser can draw it to the screen. The Leap Motion API provides the Leap.loop() function, which invokes a callback function you provide whenever the browser is ready to draw. In this sample, the callback is an anonymous function that prints out the tracking data to the body of the web page. The first parameter of the loop() function contains optional settings parameters that are passed to the Controller object. This is the skeleton of the function:

// Setup Leap loop with frame callback function
var controllerOptions = {enableGestures: true};

Leap.loop(controllerOptions, function(frame) {
  // Body of callback function
})

Internally, Leap.loop() uses the browser’s requestAnimationFrame() feature and passes the latest frame of Leap Motion data to the application on each browser animation cycle.

Accessing tracking data

The Frame object passed to the loop() callback function as the frame parameter is root of the tracking data model and provides access to all the tracked objects, like hands, fingers, and tools. To display the data, the example creates a string of HTML containing the object properties and assigns the string to a div element on the page.

Frame data

Displaying the basic frame data is a simple matter of constructing a string using the Frame object properties:

var frameString = "Frame ID: " + frame.id  + "<br />"
                + "Timestamp: " + frame.timestamp + " &micro;s<br />"
                + "Hands: " + frame.hands.length + "<br />"
                + "Fingers: " + frame.fingers.length + "<br />"
                + "Tools: " + frame.tools.length + "<br />"
                + "Gestures: " + frame.gestures.length + "<br />";

Motion data

Motion data, which the Leap Motion controller provides for the scene as a whole and for each hand individually, requires a frame to serve as the basis for comparison. This example stores a reference to the previous frame (in a variable named previousFrame) and passes that reference to each of the motion functions. The example displays the frame translation, rotation axis, rotation angle, and scale factor; there are a few other methods available as well. The motion functions of the Frame object describe the motion of the overall scene. The example displays the motion data as follows:

// Frame motion factors
if (previousFrame) {
  var translation = frame.translation(previousFrame);
  frameString += "Translation: " + vectorToString(translation) + " mm <br />";

  var rotationAxis = frame.rotationAxis(previousFrame);
  var rotationAngle = frame.rotationAngle(previousFrame);
  frameString += "Rotation axis: " + vectorToString(rotationAxis, 2) + "<br />";
  frameString += "Rotation angle: " + rotationAngle.toFixed(2) + " radians<br />";

  var scaleFactor = frame.scaleFactor(previousFrame);
  frameString += "Scale factor: " + scaleFactor.toFixed(2) + "<br />";
}

When you access a value that represents a vector, the Leap JavaScript API expresses the vector as a standard Array object containing three elements. This is the case with the result of the translation() function, above. The element, translation[0], for example represents the x-coordinate of the vector. For convenience, the vectorToString() function converts a vector array to a human-readable format.

At the end of the callback function, remember to assign the current frame to the previousFrame variable to serve as the point of reference for motions in the next frame. This provides frame-by-frame motions. You can also grab a frame at the start of an interaction and use that as the reference frame until the interaction ends.

Hand data

Hands can be accessed from the Frame object using the hands property, which is an array. The example code iterates through the list of hands and prints out the Hand object properties, IDs for associated fingers, and the motion data for each hand.

// Display Hand object data
var handString = "";
if (frame.hands.length > 0) {
  for (var i = 0; i < frame.hands.length; i++) {
    var hand = frame.hands[i];

    handString += "Hand ID: " + hand.id + "<br />";
    handString += "Direction: " + vectorToString(hand.direction, 2) + "<br />";
    handString += "Palm normal: " + vectorToString(hand.palmNormal, 2) + "<br />";
    handString += "Palm position: " + vectorToString(hand.palmPosition) + " mm<br />";
    handString += "Palm velocity: " + vectorToString(hand.palmVelocity) + " mm/s<br />";
    handString += "Sphere center: " + vectorToString(hand.sphereCenter) + " mm<br />";
    handString += "Sphere radius: " + hand.sphereRadius.toFixed(1) + " mm<br />";

    // And so on...
  }
}

Finger and Tool data

Fingers and tools are represented by Pointable objects in the JavaScript API. (You can use the Pointable.tool property to tell which type an object represents.) The Frame object provides a pointables property, which is an array containing all the fingers and tools detected in that frame. Hand objects also contain a pointables list, which contains only the pointable objects representing the fingers of that hand.

// Display Pointable (finger and tool) object data
var pointableString = "";
if (frame.pointables.length > 0) {
  for (var i = 0; i < frame.pointables.length; i++) {
    var pointable = frame.pointables[i];

    pointableString += "Pointable ID: " + pointable.id + "<br />";
    pointableString += "Belongs to hand with ID: " + pointable.handId + "<br />";

    if (pointable.tool) {
      pointableString += "Classified as a tool <br />";
      pointableString += "Length: " + pointable.length.toFixed(1) + " mm<br />";
      pointableString += "Width: "  + pointable.width.toFixed(1) + " mm<br />";
    }
    else {
      pointableString += "Classified as a finger<br />";
      pointableString += "Length: " + pointable.length.toFixed(1) + " mm<br />";
    }

    pointableString += "Direction: " + vectorToString(pointable.direction, 2) + "<br />";
    pointableString += "Tip position: " + vectorToString(pointable.tipPosition) + " mm<br />";
    pointableString += "Tip velocity: " + vectorToString(pointable.tipVelocity) + " mm/s<br />";
  }
}

Gesture data

To receive gestures from the Leap, you first have to enable gesture recognition. Enable gestures when you call the Leap.loop() function by passing in an options object:

// Setup Leap loop with frame callback function
var controllerOptions = {enableGestures: true};

Leap.loop(controllerOptions, function(frame) {
  // Body of callback function
})

The Leap Motion controller adds Gesture objects representing each recognized movement pattern to the gestures list in the Frame object. The sample application loops through the gesture list and prints information about each one to the standard output. This operation is performed with a standard for-loop and switch statement.

The full code for the gesture loop is:

// Display Gesture object data
var gestureString = "";
if (frame.gestures.length > 0) {
  for (var i = 0; i < frame.gestures.length; i++) {
    var gesture = frame.gestures[i];
    gestureString += "Gesture ID: " + gesture.id + ", "
                  + "type: " + gesture.type + ", "
                  + "state: " + gesture.state + ", "
                  + "hand IDs: " + gesture.handIds.join(", ") + ", "
                  + "pointable IDs: " + gesture.pointableIds.join(", ") + ", "
                  + "duration: " + gesture.duration + " &micro;s, ";

    switch (gesture.type) {
      case "circle":
        gestureString += "center: " + vectorToString(gesture.center) + " mm, "
                      + "normal: " + vectorToString(gesture.normal, 2) + ", "
                      + "radius: " + gesture.radius.toFixed(1) + " mm, "
                      + "progress: " + gesture.progress.toFixed(2) + " rotations";
        break;
      case "swipe":
        gestureString += "start position: " + vectorToString(gesture.startPosition) + " mm, "
                      + "current position: " + vectorToString(gesture.position) + " mm, "
                      + "direction: " + vectorToString(gesture.direction, 2) + ", "
                      + "speed: " + gesture.speed.toFixed(1) + " mm/s";
        break;
      case "screenTap":
      case "keyTap":
        gestureString += "position: " + vectorToString(gesture.position) + " mm, "
                      + "direction: " + vectorToString(gesture.direction, 2);
        break;
      default:
        gestureString += "unknown gesture type";
    }
    gestureString += "<br />";
  }
}

Running the sample

To run the sample application:

  1. Plug the Leap device into a USB port and place it in front of you.

  2. If you haven’t already, install the Leap Motion software.

  3. The Leap Motion software should start automatically.

    The Leap Motion control panel icon appears in the notification area of the task bar (on Windows) or finder bar (on Mac) and turns green when ready. A service or daemon runs in the background and provides data to client applications through the Leap Motion API. You can use the diagnostic visualizer to check whether the software is set up and running correctly.

  4. Open Sample.html in a browser that supports WebSockets.

When you place a hand above the Leap, you should see the Frame, Hand, and Finger information printed out on the page. When you make a gesture, you should also see the appropriate Gesture information.

Now that you have seen how to access motion tracking data with the Leap Motion API, you can begin developing your own JavaScript applications that integrate the Leap Motion controller.