Hello World¶

How the Leap Motion Controller Works¶

The Leap Motion controller encompasses both hardware and software components.

The Leap Motion hardware consists primarily of a pair of stereo infrared cameras and illumination LEDs. The camera sensors look upward (when the device is in its standard orientation). The following illustration shows how a user’s hands look from the perspective of the Leap Motion sensor:

The Leap Motion software receives the sensor data and analyzes this data specifically for hands, fingers, arms, and tools. (Tracking for other types of objects could be added in the future, but this is the current set of tracked entities). The software maintains an internal model of the human hand and compares that model to the sensor data to determine the best fit. Sensor data is analyzed frame-by-frame and the service sends each frame of data to Leap Motion-enabled applications. The Frame object received by your application contains all the known positions, velocities and identities of tracked entities, such as hands, fingers, and tools. Constructs such as gestures and motions that span multiple frames are also updated each frame. For an overview for the tracking data provided by the controller, read API Overview.

The Leap Motion software runs as a service (Windows) or daemon (Mac and Linux) on the client computer. Native Leap Motion-enabled applications can connect to this service using the API provide by the Leap Motion dynamic libraries (provided as part of the Leap Motion SDK). Web applications can connect to a WebSocket server hosted by the service. The WebSocket provides tracking data as a JSON-formatted message – one message per frame of data. A JavaScript library, LeapJS, provides an API wrapping this data. For more information read System Architecture.

Set Up the Files¶

This tutorial also uses command line compilers and linkers (where needed) in order to focus on the code rather than the environment.

For details on how to set up projects using the Leap Motion SDK in popular IDEs, see Setting Up a Project.

1. If you haven’t already, download and unzip the latest Leap Motion SDK from the developer site and install the latest Leap Motion service.

2. Open a terminal or console window and navigate to the SDK samples folder.

3. Sample.java contains the finished code for this tutorial, but to get the most out of this lesson, you can rename the existing file, and create a new, blank Sample.java file in this folder. Keep the existing file for reference.

4. In your new Sample.java file, add code to import the Leap Motion libraries:

   import com.leapmotion.leap.*;
   

5. Add the “structural” code to define a Java command-line program:

   class Sample {
       public static void main(String[] args) {
   
           // Keep this process running until Enter is pressed
           System.out.println("Press Enter to quit...");
           try {
               System.in.read();
           } catch (IOException e) {
               e.printStackTrace();
           }
       }
   }
   

This code simply prints a message and then waits for keyboard input before exiting. See Running the Sample for instructions on running the program.

Get Connected¶

The next step is to add a Controller object to the program – which serves as our connection to the Leap Motion service/daemon.

class Sample {
    public static void main(String[] args) {
        Controller controller = new Controller();

        // Keep this process running until Enter is pressed
        System.out.println("Press Enter to quit...");
        try {
            System.in.read();
        } catch (IOException e) {
            e.printStackTrace();
        }
    }
}

When you create a Controller object, it automatically connects to the Leap Motion service and, once the connection has been established, you can get tracking data from it using the Controller.frame() method.

The connection process is asynchronous, so you can’t create the Controller in one line and expect to get data in the next line. You have to wait for the connection to complete. But for how long?

To Listen or not to Listen?¶

You can add a Listener object to the Controller, which provides an event-based mechanism for responding to important Controller state changes. This is the approach used in this tutorial – but it is not always the best approach.

The Problem with Listeners: Listener objects use independent threads to invoke the code that you implement for each event. Thus, using the listener mechanism can introduce the complexities of threading into your program. It becomes your responsibility to make sure that the code you implement in your Listener subclass accesses other parts of your program in a thread-safe manner. For example, you might not be able to access variables related to GUI controls from anything except the main thread. There can also be additional overhead associated with the creation and cleanup of threads.

Avoiding Listeners: You can avoid using Listener objects by simply polling the Controller object for frames (or other state) when convenient for your program. Many programs already have a event- or animation-loop to drive user input and animation. If so, you can get the tracking data once per loop – which is often as fast as you can use the data anyway.

The Listener class in the API defines the signatures for a function that will be called when a Controller event occurs. You create a listener by creating a subclass of Listener and implementing the callback functions for the events you are interested in.

To continue this tutorial, add the SampleListener class to your program. For simplicity, add your SampleListener class to the same file as the Sample class:

class SampleListener extends Listener {

    public void onConnect(Controller controller) {
        System.out.println("Connected");
    }

    public void onFrame(Controller controller) {
        System.out.println("Frame available");
    }
}

If you have already taken a look at the finished file, you may have noticed that several more callback functions are present. You can add those too, if you wish, but to keep things simple, we will concentrate on onConnect() and onFrame().

Now create a SampleListener object using your new class and add it to your controller:

class Sample {
    public static void main(String[] args) {
        // Create a sample listener and controller
        SampleListener listener = new SampleListener();
        Controller controller = new Controller();

        // Have the sample listener receive events from the controller
        controller.addListener(listener);

        // Keep this process running until Enter is pressed
        System.out.println("Press Enter to quit...");
        try {
            System.in.read();
        } catch (IOException e) {
            e.printStackTrace();
        }

        // Remove the sample listener when done
        controller.removeListener(listener);
    }
}

Now is a good time to test your sample program. Follow the directions in: Running the Sample.

If everything is correct (and your Leap Motion hardware is plugged in), then you should see the string “Connected” printed to the terminal window followed by an rapid series of “Frame available”. If things go wrong and you cannot figure out why, you can get help on our developer forum at developer.leapmotion.com.

Whenever you run into trouble developing a Leap Motion application, try opening the diagnostic visualizer. This program displays a visualization of the Leap Motion tracking data. You can compare what you see in your program to what you see in the visualizer (which uses the same API) to isolate and identify many problems.

On Connect¶

When your Controller object successfully connects to the Leap Motion service/daemon AND the Leap Motion hardware is plugged in, The Controller object changes its isConnected() property to true and invokes your Listener.onConnect() callback (if there is one).

When the controller connects, you can set controller properties using the Controller.enableGesture() and Controller.setPolicy() methods. For example, you could enable the swipe gesture with the following onConnect() function:

public void onConnect(Controller controller) {
    System.out.println("Connected");
    controller.enableGesture(Gesture.Type.TYPE_SWIPE);
}

On Frame¶

All the tracking data in the Leap Motion system arrives through the Frame object. You can get Frame objects from your controller (after it has connected) by calling the Controller.frame() method. The onFrame() callback of your Listener subclass is called when a new frame of data becomes available. When you aren’t using a listener, you can compare the id() value to that of the last frame you processed to see if there is a new frame. Note that by setting the history parameter of the frame() function, you can get earlier frames than the current one (up to 60 frames are stored). Thus, even when polling at a slower rate than the Leap Motion frame rate, you can process every frame, if desired.

To get the frame, add this call to frame() to your onFrame() callback:

public void onFrame(Controller controller) {
    Frame frame = controller.frame();
}

Then, print out some properties of the Frame object:

public void onFrame(Controller controller) {
    Frame frame = controller.frame();

    System.out.println("Frame id: " + frame.id()
                   + ", timestamp: " + frame.timestamp()
                   + ", hands: " + frame.hands().count()
                   + ", fingers: " + frame.fingers().count()
                   + ", tools: " + frame.tools().count()
                   + ", gestures " + frame.gestures().count());
}

Run your sample again, put a hand or two over the Leap Motion device and you should now see the basic statistics of each frame printed to the console window.

I’ll end this tutorial here, but you can look at the rest of the code in the original sample program for examples on how to get all the Hand, Finger, Arm, Bone and Gesture objects in a frame.

Running the Sample¶

To run the sample application:

1. Compile the sample application:

   • On Windows, make sure that Sample.java and LeapJava.jar are in the current directory and run the following command in a command-line prompt:

     javac -classpath LeapJava.jar Sample.java
     

   • On Mac, make sure that Sample.java and LeapJava.jar are in the current directory and run the following command in a terminal window:

     javac -classpath ./LeapJava.jar Sample.java
     

2. Run the sample application:

   • On Windows, make sure that Sample.class, LeapJava.jar, LeapJava.dll, and Leap.dll are in the current directory. If you are using a 32-bit version of the Java Virtual Machine (JVM), use the .dll files from the lib\x86 folder of the SDK. Use the .dll files from lib\x64 with a 64-bit JVM. Run the following command in a command-line prompt:

     java -classpath "LeapJava.jar;." Sample
     

   • On Mac, make sure that Sample.class, LeapJava.jar, libLeapJava.dylib, and libLeap.dylib are in the current directory and run the following command in a terminal window:

     java -classpath "./LeapJava.jar:." Sample
     

   You should see the message “Connected” printed to standard output when the application initializes and connects to the Leap. You should then see frame information printed each time the Leap dispatches the onFrame event.