Hands

Hands are the main entity tracked by the Leap Motion controller. The controller maintains an inner model of the human hand and validates the data from its sensors against this model. This allows the controller to track finger positions even when a finger is not completely visible. Note that it is possible for movement or changes in position to be lost when a finger is behind or directly in front of the hand (from the point of view of the controller). The Leap Motion software matches the internal model against the existing data. In some cases, the software can make an incorrect match – for example, identifying a right hand as a left hand.

The Hand class represents a physical hand detected by the Leap. A Hand object provides access to lists of its pointables as well as attributes describing the hand position, orientation, and movement.

Hands can be identified by left versus right handedness. In addition, each hand is assigned an ID value when the hand is first detected. If you remove a hand from the Leap Motion field of view and then re-insert it, a new ID is assigned (use the timeVisible attribute to tell whether a hand is newly detected or not). New hand IDs are also assigned if the Leap Motion software decides it has misclassified a hand and must change its type from right to left or vice versa.

Getting Hands

Get Hand objects from a Frame:

Leap::HandList hands = frame.hands();
Leap::Hand firstHand = hands[0];

Or, if you know the ID from a previous frame:

Leap::Hand knownHand = frame.hand(handID);

You can also get hands by their relative positions in the frame:

Leap::HandList hands = frame.hands();

Leap::Hand leftmost = hands.leftmost();
Leap::Hand rightmost = hands.rightmost();
Leap::Hand frontmost = hands.frontmost();

Note that the the leftmost() and rightmost() functions only identify which hand is farthest to the left or right. Use the Hand isLeft or isRight attribute to tell if a hand object represents a left or a right hand.

Getting the Hand Characteristics

A hand is described by its handedness, position, orientation, posture, and motion:

  • isRight, isLeft — Whether the hand is a left or a right hand.
  • Palm Position — The center of the palm measured in millimeters from the Leap Motion origin.
  • Palm Velocity — The speed and movement direction of the palm in millimeters per second.
  • Palm Normal — A vector perpendicular to the plane formed by the palm of the hand. The vector points downward out of the palm.
  • Direction — A vector pointing from the center of the palm toward the fingers.
  • grabStrength, pinchStrength — Describe the posture of the hand.
  • Motion factors — Provide relative scale, rotation, and translation factors for movement between two frames.

The hand’s position is given by its palm position attribute, which provides a vector containing the 3-dimensional coordinates of the palm center point in millimeters from the Leap Motion origin. The hand’s orientation is given by two vectors: the direction, which points from the palm center towards the fingers, and the palm normal, which points out of the palm, perpendicular to the plane of the hand.

The movement of the hand is given by the velocity attribute, which is a vector providing the instantaneous motion of the hand in mm/s. You can also get motion factors that translate how a hand has moved between two given frames into translation, rotation, and scaling values.

The following code snippet illustrates how to get a Hand object from a frame and access its basic attributes:

Leap::Hand hand = frame.hands().rightmost();
Leap::Vector position = hand.palmPosition();
Leap::Vector velocity = hand.palmVelocity();
Leap::Vector direction = hand.direction();

Getting the Fingers

You can get the fingers associated with a hand as a list or individually using an ID obtained in a previous frame.

By list:

// hand is a Leap::Hand object
Leap::PointableList pointables = hand.pointables(); // Both fingers and tools
Leap::FingerList fingers = hand.fingers();

By ID from a previous frame:

Leap::Pointable knownPointable = hand.pointable(pointableID);

To get a finger by relative position within the Leap field of view, use the right-, left- and frontmost functions of the matching list class:

// hand is a Leap::Hand object
Leap::Pointable leftPointable = hand.pointables().leftmost();
Leap::Finger rightFinger = hand.fingers().rightmost();

Note that these functions are relative to the Leap Motion origin, not to the hand itself. To get the fingers relative to the hand, you can use the Leap Matrix class to transform the finger positions into the hands frame of reference.

Computing the Hand Orientation

You can compute the hand orientation angles using the Hand direction and normal vectors.

https://di4564baj7skl.cloudfront.net/documentation/images/Leap_Palm_Vectors.png

The normal vector points perpendicularly out of the hand; the direction vector points forward.

The Vector class defines functions for getting the pitch (angle around the x-axis), yaw (angle around the y-axis), and roll (angle around the z-axis):

float pitch = hand.direction().pitch();
float yaw = hand.direction().yaw();
float roll = hand.palmNormal().roll();

Note that the roll function only provides the expected angle when used with a normal vector.

Transforming Finger Coordinates into the Hand’s Frame of Reference

Sometimes it is useful to obtain the coordinates of the fingers of a hand with respect to the hand’s frame of reference. This lets you sort the fingers spatially and can simplify analysis of finger positions. You can create a transform matrix using the Leap Matrix class to transform the finger position and direction coordinates. The hand frame of reference can be usefully defined by the hand’s basis() and palmPosition(). The basis() orients the x-axis sideways across the hand, the z-axis pointing forward, and the y-axis parallel with the palm normal. The origin of the transform is the palmPosition().

Leap::Hand hand = frame.hands()[0];

Leap::Matrix handTransform = hand.basis();
handTransform.origin = hand.palmPosition();
handTransform = handTransform.rigidInverse();

for( int f = 0; f < hand.fingers().count(); f++ )
{
    Leap::Finger finger = hand.fingers()[f];
    Leap::Vector transformedPosition = handTransform.transformPoint(finger.tipPosition());
    Leap::Vector transformedDirection = handTransform.transformDirection(finger.direction());
    // Do something with the transformed fingers
}

Hand Lists

The HandList class acts like a vector-style array and supports iterators. You cannot remove or alter the member objects of a hand lists received from the Leap Motion API, but you can combine lists of the same object type.

To use an iterator with a list class:

for(HandList::const_iterator hl = handList.begin(); hl != handList.end(); hl++)
    std::cout << *hl << std::endl;

The HandList class defines additional functions for getting a member of the list based on its relative position within the Leap coordinate system. These functions include leftmost(), rightmost(), and frontmost(). The following snippet illustrates how to get the hand object furthest to the right:

Leap::Hand furthestRight = frame.hands().rightmost();