Hello World

This article discusses the Objective-C sample application included with the Leap SDK. After reading this article, you should be ready to access Leap hand tracking data from your own Objective-C applications.

In the Leap SDK folder, you can find the following files used for this article:

  • LeapSDK/samples/SampleObjectiveC.xcodeproj — Xcode project file.
  • LeapSDK/samples/ObjectiveC/Sample.h — Header file for the sample code which interacts with the Leap library.
  • LeapSDK/samples/ObjectiveC/Sample.m — Sample code that interacts with the Leap library to print out tracking data.
  • LeapSDK/samples/ObjectiveC/* — Miscellaneous standard OS X application files.
  • LeapSDK/include/LeapObjectiveC.h — Header file for the Leap Objective C++ wrapper.
  • LeapSDK/include/LeapObjectiveC.mm — Objective C++ wrapper for the Leap library.
  • LeapSDK/include/Leap.h — Leap C++ API class and struct definitions.
  • LeapSDK/include/LeapMath.h — Leap C++ API Vector and Matrix class and struct definitions.
  • LeapSDK/lib/libLeap.dylib — Leap library for Mac.

All the files needed for the sample application are already referenced in the SampleObjectiveC Xcode project file.

Overview

In a nutshell, the Leap motion tracking device detects and tracks hands and fingers placed within its field of view. The Leap captures this data one frame at a time. Your applications can use the Leap API to access this data.

The sample application demonstrates how to use the Leap API to listen for frame events dispatched by the Leap and how to access the hand and finger data in each frame. The application is a small program that prints information about detected hands and fingers to standard output.

The sample application uses most of the key classes in the Leap API, including:

  • LeapController — the interface between the Leap and your application
  • LeapFrame — contains a set of hand and finger tracking data
  • LeapHand — contains tracking data for a detected hand
  • LeapFinger — contains tracking data for a detected finger
  • LeapVector — represents a 3D position or directional vector

For more detailed information about these classes, pleases refer to the Leap API reference documentation.

Using the Leap Objective-C Wrapper

The Objective-C wrapper translates the classes in the standard Leap C++ API into classes that can be used directly from Objective-C.

The main difference between the Objective-C API and the Leap C++ API is the Listener callback mechanism. In Objective-C, you can use either an NSNotification-based listener or the LeapDelegate protocol to observe Leap events dispatched by a LeapController object. Another difference in the two APIs is that the class names of the Objective-C wrapper use the prefix Leap. Thus, the C++ Controller class is named LeapController in Objective-C.

Creating a LeapController object

The LeapController. class provides the main interface between the Leap and your application. When you create a LeapController object, it connects to the Leap Motion software running on the computer and makes hand tracking data available through LeapFrame objects. You can access these LeapFrame objects by instantiating a LeapController object and calling the object’s frame:history method.

If your application has a natural update loop or frame rate, then you can call the LeapController frame:history method as part of this update. Otherwise, you can either receive NSNotifications when a new LeapFrame of data is ready or add a delegate to the controller object.

The sample application creates a LeapController object in the run method of the Sample class. The Sample class adopts the |LeapListener|_ protocol and adds itself to the LeapController object so that it can receive the Leap Motion events:

- (void)run
{
    controller = [[LeapController alloc] init];
    [controller addListener:self];
    NSLog(@"running");
}

The LeapController class’ addlistener:listener method automatically subscribes the listener object to receive NSNotifications for the members of the |LeapListener| protocol which it implements.

Note that the Leap library is multi-threaded. Using the NSNotification-based listener avoids most threading issues since all of the Leap Motion callback functions are invoked on the main thread. If you use a delegate instead, be aware that the delegate functions are invoked from threads created by the Leap Motion library, not the main, UI thread.

Adopting the LeapListener protocol

To add a listener object to a controller, create an object adopting the |LeapListener|_ protocol. All the protocol methods are optional. The Sample class implements the methods in the LeapListener-p protocol to handle notifications dispatched by the Leap Motion. The methods include:

  • onInit:notification — dispatched once, when the controller to which the delegate is registered is initialized.
  • onConnect:notification — dispatched when the controller connects to the Leap and is ready to begin sending frames of motion tracking data.
  • onDisconnect:notification — dispatched if the controller disconnects from the Leap Motion controller (for example, if you unplug the device or shut down the Leap Motion software).
  • onExit:notification — Not implemented for notifications; will never be dispatched.
  • onFrame:notification — dispatched when a new frame of motion tracking data is available.

In the lifecycle notification handlers, onInit, onConnect, and onDisconnect, the sample application simply prints a message to the NSLog output. For the onFrame notification, the handler implementation does a bit more work. When the controller dispatches the onFrame:notification, the method gets the latest frame of motion tracking data and prints information about the detected objects to the NSLog output.

You must add a listener to the LeapController to use NSNotifications. While any object can subscribe to the Leap Motion notifications, the controller does not dispatch notifications unless there is at least one listener explicitly assigned.

Adopting the LeapDelgate Protocol

If you prefer to use a delegate rather than NSNotifications, you can create an object adopting the |LeapDelegate|_ protocol. The methods of the |LeapDelegate| protocol are the same as the |LeapListener| protocol except that they take a LeapController object as a parameter instead of an NSNotification object. The controller object invokes the methods defined in your |LeapDelegate| implementation whenever a new frame of tracking data is available (and also for a few other Leap events).

You can assign either a |LeapListener| object or a |LeapDelegate| object to a controller, but not both.

Getting a Frame of data

The LeapController object dispatches an NSNotification object when the Leap Motion generates a new frame of motion tracking data. The notification object includes a reference to the controller. You can access the new frame of data by calling the controller object’s frame:history method. The LeapFrame object returned by frame:history is the top level of the Leap Motion model hierarchy and provides such information as a frame ID, a timestamp, and lists containing the hands and fingers in view.

The following code defined in the Sample class’s implementation of the onFrame:notification method gets the most recent LeapFrame object from the controller, retrieves the list of hands from the LeapFrame and then logs the frame ID, timestamp, and the number of hands and fingers detected in the frame:

- (void)onFrame:(NSNotification *)notification;
{
    LeapController *aController = (LeapController *)[notification object];

    // Get the most recent frame and report some basic information
    LeapFrame *frame = [aController frame:0];
    NSLog(@"Frame id: %lld, timestamp: %lld, hands: %ld, fingers: %ld",
          [frame id], [frame timestamp], [[frame hands] count],
          [[frame fingers] count]);

The method goes on to examine the first hand in the list of hands:

if ([[frame hands] count] != 0) {
    // Get the first hand
    LeapHand *hand = [[frame hands] objectAtIndex:0];

A LeapHand object contains an ID, properties representing the hand’s physical characteristics, and a list of LeapFinger objects. Each LeapFinger object contains an ID and properties representing the characteristic of the finger.

Once it has retrieved a hand, the method checks it for fingers and then averages the finger tip positions, printing the result with the number of fingers:

// Check if the hand has any fingers
NSArray *fingers = [hand fingers];
if ([fingers count] != 0) {
    // Calculate the hand's average finger tip position
    LeapVector *avgPos = [[LeapVector alloc] init];
    for (int i = 0; i < [fingers count]; i++) {
        LeapFinger *finger = [fingers objectAtIndex:i];
        avgPos = [avgPos plus:[finger tipPosition]];
    }
    avgPos = [avgPos divide:[fingers count]];
    NSLog(@"Hand has %ld fingers, average finger tip position %@",
          [fingers count], avgPos);
}

Next, the method prints the radius of a sphere fit to the hand’s curvature, along with the hand’s palm position:

// Get the hand's sphere radius and palm position
NSLog(@"Hand sphere radius: %f mm, palm position: %@",
      [hand sphereRadius], [hand palmPosition]);

Finally, the onFrame:notification method uses LeapVector class methods to get the hand’s pitch, roll, and yaw angles from the hand’s normal vector and the direction vectors. The angles are converted from radians to degrees:

// Get the hand's normal vector and direction
const LeapVector *normal = [hand palmNormal];
const LeapVector *direction = [hand direction];

// Calculate the hand's pitch, roll, and yaw angles
NSLog(@"Hand pitch: %f degrees, roll: %f degrees, yaw: %f degrees\n",
      [direction pitch] * LEAP_RAD_TO_DEG,
      [normal roll] * LEAP_RAD_TO_DEG,
      [direction yaw] * LEAP_RAD_TO_DEG);

Running the sample

To run the sample application:

  1. Plug the Leap Motion device into a USB port and place it in front of you.

  2. If you haven’t already, install the Leap Motion software.

  3. The Leap Motion software should start automatically.

    The Leap Motion control panel icon appears in the notification area of the task bar (on Windows) or finder bar (on Mac) and turns green when ready. A service or daemon runs in the background and provides data to client applications through the Leap Motion API. You can use the diagnostic visualizer to check whether the software is set up and running correctly.

  4. Open the SampleObjectiveC project file in Xcode.

  5. Press the Xcode Run button.

You should see the messages “Initialized” and “Connected” printed to Debug Output area in Xcode when the application initializes and connects to the Leap Motion controller. You should then see frame information printed to the Debug Output area each time the controller dispatches the onFrame event. When you place a hand above the Leap Motion device, you should also see finger and palm position information printed. Note that the application window is intentionally left blank. All output appears in the Xcode Debug Output area.

Now that you have seen how to access motion tracking data with the Leap Motion API, you can begin developing your own Objective-C applications that integrate the Leap Motion controller.