Frame

Attributes:

Methods:

class Leap.Frame

The Frame class represents a set of hand and finger tracking data detected in a single frame.

The Leap Motion software detects hands, fingers and tools within the tracking area, reporting their positions, orientations and motions in frames at the Leap Motion frame rate.

Access Frame objects through an instance of the Controller class. You can poll for data at your applications natural frame rate or implement a Listener subclass to receive a callback event when a new Frame is available.

The following example shows how to get the current frame from a controller and access lists of tracking data:

controller = Leap.Controller()
# wait until Controller.isConnected() evaluates to true
#...

frame = controller.frame()
hands = frame.hands
pointables = frame.pointables
fingers = frame.fingers
tools = frame.tools

New in version 1.0.

classmethod Frame()

Constructs a Frame object.

Frame instances created with this constructor are invalid. Get valid Frame objects by calling the Controller.frame() function.

frame = controller.frame()

New in version 1.0.

current_frames_per_second
Type:float

The instantaneous framerate.

fps = frame.current_frames_per_second

The rate at which the Leap Motion software is providing frames of data (in frames per second). The framerate can fluctuate depending on available computing resources, activity within the device field of view, software tracking settings, and other factors.

New in version 1.0.

fingers
Type:FingerList

The list of Finger objects detected in this frame, given in arbitrary order. The list can be empty if no fingers are detected.

fingers_for_frame = frame.fingers

New in version 1.0.

hands
Type:HandList

The list of Hand objects detected in this frame, given in arbitrary order. The list can be empty if no hands are detected.

handlist = frame.hands
for hand in handlist:
    print str(hand)

New in version 1.0.

id
Type:long integer

A unique ID for this Frame.

frame_identifier = frame.id

Consecutive frames processed by the Leap Motion software have consecutive increasing values (except when the controller is compensating for poor lighting – in which case, two sensor frames are used to construct each data frame and the frame ID increases by 2).

You can use the id value to detect whether this frame has already been processed when you are polling the the controller for frames rather than using a listener:

lastFrameID = 0
def processFrame(self, frame):
    if(frame.id == self.lastFrameID):
        return
    #...
    self.lastFrameID = frame.id

New in version 1.0.

images
Type:ImageList

The list of images from the Leap Motion cameras.

Images received through the frame object are the images analyzed to create the tracking data for that frame.

image_list = frame.images
left_image = image_list[0]
right_image = image_list[1]

New in version 2.1.

interaction_box
Type:InteractionBox

The current InteractionBox for the frame. See the InteractionBox class documentation for more details on how this class should be used.

ibox = frame.interaction_box

front_finger = frame.fingers.frontmost
normalized_position = ibox.normalize_point( front_finger.tip_position)

x = window_width  * normalized_position.x
y = window_height * (1 - normalized_position.y)
draw_pointer(x,y)

New in version 1.0.

is_valid
Type:boolean

Reports whether this Frame instance is valid.

if frame.is_valid:
    #Process frame data...

A valid Frame is one generated by the Controller object that contains tracking data for all detected entities. An invalid Frame contains no actual tracking data, but you can call its functions without risk of a null pointer exception. The invalid Frame mechanism makes it more convenient to track individual data across the frame history. For example, you can invoke:

finger = controller.frame(n).finger(fingerID)

for an arbitrary Frame history value, “n”, without first checking whether frame(n) returned a null object. (You should still check that the returned Finger instance is valid.)

New in version 1.0.

pointables
Type:PointableList

The list of Pointable objects (fingers and tools) detected in this frame, given in arbitrary order. The list can be empty if no fingers or tools are detected.

pointables_for_frame = frame.pointables

New in version 1.0.

serialize
Type:tuple(Leap.byte_array, length)

A tuple containing this Frame object as a Leap.byte_array object and the length of the byte_array.

import ctypes

frame = controller.frame()

serialized_tuple = frame.serialize
serialized_data = serialized_tuple[0]
serialized_length = serialized_tuple[1]
data_address = serialized_data.cast().__long__()
buffer = (ctypes.c_ubyte * serialized_length).from_address(data_address)
with open(os.path.realpath('frame.data'), 'wb') as data_file:
    data_file.write(buffer)

The tuple can be fed directly to the Frame.deserialize() function. However, saving the data to disk is a bit more work because of the limitations of the byte_array class used as the data type. Before you can save the data using Python’s built-in file API, you must convert this byte_array into an object that exposes the Python buffer interface. See Serializing Tracking Data.

New in version 2.1.0.

timestamp
Type:long integer

The frame capture time in microseconds elapsed since the Leap started.

timestamp = frame.timestamp

New in version 1.0.

tools
Type:ToolList

The list of Tool objects detected in this frame, given in arbitrary order. The list can be empty if no tools are detected.

tools_for_frame = frame.tools

New in version 1.0.

deserialize(frame_data)

Decodes a byte array to replace the properties of this Frame.

A Controller object must be instantiated for this function to succeed, but it does not need to be connected. To extract gestures from the deserialized frame, you must enable the appropriate gestures first.

Any existing data in the frame is destroyed. If you have references to child objects (hands, fingers, etc.), these are preserved as long as the references remain in scope.

import ctypes, numpy

frame = Leap.Frame()
filename = "frame.data"
with open(os.path.realpath(filename), 'rb') as data_file:
    data = data_file.read()

leap_byte_array = Leap.byte_array(len(data))
address = leap_byte_array.cast().__long__()
ctypes.memmove(address, data, len(data))

frame.deserialize((leap_byte_array, len(data)))

When loading data from a file, you must convert the Python buffer into a Leap.byte_array object before passing it into deserialize. In addition, len() does not work with a byte_array object. You must calculate this separately, for example, from the length of the file buffer. The length is provided by the second item in the tuple given by the serialize attribute. See Serializing Tracking Data.

Note: The behavior when calling functions which take another Frame object as a parameter is undefined when either frame has been deserialized. For example, calling gestures() on a deserialized frame or with a deserialized frame as parameter (or both) does not necessarily return all gestures that occured between the two frames. Motion functions, like scale_factor(), are more likely to return reasonable results, but could return anomalous values in some cases.

Parameters:frame_data (tuple) – A tuple containing the Leap.byte_array buffer containing the serialized frame data, followed by the length of this byte_array buffer.

New in version 2.1.0.

finger(id)

The Finger object with the specified ID in this frame.

Use this function to retrieve the Finger object from this frame using an ID value obtained from a previous frame. This function always returns a Finger object, but if no finger with the specified ID is present, an invalid Finger object is returned.

finger_by_ID = frame.finger(finger_ID)

Note that ID values persist across frames, but only until tracking of a particular object is lost. If tracking of a finger is lost and subsequently regained, the new Finger object representing that physical finger may have a different ID than that representing the finger in an earlier frame.

Parameters:id (integer) – The ID value of a Finger object from a previous frame.
Returns:Finger The finger with the matching ID if one exists in this frame; otherwise, an invalid Finger object is returned.

New in version 1.0.

hand(id)

The Hand object with the specified ID in this frame.

Use the Frame.hand() function to retrieve the Hand object from this frame using an ID value obtained from a previous frame. This function always returns a Hand object, but if no hand with the specified ID is present, an invalid Hand object is returned.

hand = frame.hand(hand_ID)

Note that ID values persist across frames, but only until tracking of a particular object is lost. If tracking of a hand is lost and subsequently regained, the new Hand object representing that physical hand may have a different ID than that representing the physical hand in an earlier frame.

Parameters:id (integer) – The ID value of a Hand object from a previous frame.
Returns:Hand The hand with the matching ID if one exists in this frame; otherwise, an invalid Hand object is returned.

New in version 1.0.

gesture(id)

The Gesture object with the specified ID in this frame.

Use the gesture function to return a Gesture object in this frame using an ID obtained in an earlier frame. The function always returns a Gesture object, but if there was no update for the gesture in this frame, then an invalid Gesture object is returned.

gesture = frame.gesture(gesture_ID)

All Gesture objects representing the same recognized movement share the same ID.

Parameters:id (integer) – The ID value of a Gesture object from a previous frame.
Returns:Gesture The gesture with the matching ID if one exists in this frame; otherwise, an invalid Gesture object is returned.

New in version 1.0.

gestures([sinceFrame])

Returns a GestureList containing all gestures that have occured since the specified frame.

gesture_list = frame.gestures(start_frame)
Parameters:sinceFrame (Frame) – An earlier Frame object. The starting frame must still be in the frame history cache, which has a default length of 60 frames. If no sinceFrame parameter is supplied, only the gestures from ththis frame ar returned.
Returns:GestureList – The list of the Gesture objects that have occured since the specified frame.

New in version 1.0.

pointable(id)

The Pointable object with the specified ID in this frame.

Use the this function to retrieve the Pointable object from this frame using an ID value obtained from a previous frame. This function always returns a Pointable object, but if no finger or tool with the specified ID is present, an invalid Pointable object is returned.

pointable_of_interest = frame.pointable(pointable_ID)

Note that ID values persist across frames, but only until tracking of a particular object is lost. If tracking of a finger or tool is lost and subsequently regained, the new Pointable object representing that finger or tool may have a different ID than that representing the finger or tool in an earlier frame.

Parameters:id (integer) – The ID value of a Pointable object from a previous frame.
Returns:Pointable The pointable with the matching ID if one exists in this frame; otherwise, an invalid Pointable object is returned.

New in version 1.0.

rotation_angle(sinceFrame[, axis])

The angle of rotation derived from the overall rotational motion between the current frame and the specified frame.

If no axis parameter is specified, then the returned angle is expressed in radians measured clockwise around the computed rotation_axis() vector (using the right-hand rule) between the start and end frames. The value is always between 0 and pi radians (0 and 180 degrees).

If an axis parameter is specified, then the returned angle is expressed in radians measured clockwise around the supplied rotation axis (using the right-hand rule). In this case, the value is always between -pi and pi radians (-180 and 180 degrees).

The Leap Motion software derives frame rotation from the relative change in position and orientation of all objects detected in the field of view.

rotation_around_y_axis = frame.rotation_angle(start_frame, Vector.y_axis)

If either this frame or sinceFrame is an invalid Frame object, then the angle of rotation is zero.

Parameters:
  • sinceFrame (Frame) – The starting frame for computing the relative rotation.
  • axis (Vector) – The axis to measure rotation around.
Returns:

float – A positive value containing the heuristically determined rotational change between the current frame and that specified in the sinceFrame parameter, in radians.

New in version 1.0.

rotation_axis(sinceFrame)

The axis of rotation derived from the overall rotational motion between the current frame and the specified frame.

The returned direction vector is normalized.

The Leap Motion software derives frame rotation from the relative change in position and orientation of all objects detected in the field of view.

axis_of_frame_rotation = frame.rotation_axis(start_frame)

If either this frame or sinceFrame is an invalid Frame object, or if no rotation is detected between the two frames, a zero vector is returned.

Parameters:sinceFrame (Frame) – The starting frame for computing the relative rotation.
Returns:Vector A normalized direction vector representing the axis of the heuristically determined rotational change between the current frame and that specified in the sinceFrame parameter.

New in version 1.0.

rotation_matrix(sinceFrame)

The transform matrix expressing the rotation derived from the overall rotational motion between the current frame and the specified frame.

The Leap Motion software derives frame rotation from the relative change in position and orientation of all objects detected in the field of view.

frame_rotation_transform = frame.rotation_matrix(start_frame)

If either this frame or sinceFrame is an invalid Frame object, then this method returns an identity matrix.

Parameters:sinceFrame (Frame) – The starting frame for computing the relative rotation.
Returns:Matrix – A transformation Matrix containing the heuristically determined rotational change between the current frame and that specified in the sinceFrame parameter.

New in version 1.0.

rotation_probability(sinceFrame)

The estimated probability that the overall motion between the current frame and the specified frame is intended to be a rotating motion.

rotation_intent_factor = frame.rotation_probability(start_frame)

If either this frame or sinceFrame is an invalid Frame object, then this method returns zero.

Parameters:sinceFrame (Frame) – The starting frame for computing the relative rotation.
Returns:float – A value between 0 and 1 representing the estimated probability that the overall motion between the current frame and the specified frame is intended to be a rotating motion.

New in version 1.0.

scale_factor(sinceFrame)

The scale factor derived from the overall motion between the current frame and the specified frame.

The scale factor is always positive. A value of 1.0 indicates no scaling took place. Values between 0.0 and 1.0 indicate contraction and values greater than 1.0 indicate expansion.

The Leap Motion software derives scaling from the relative inward or outward motion of all objects detected in the field of view (independent of translation and rotation).

frame_scale_motion = frame.scale_factor(start_frame)

If either this frame or sinceFrame is an invalid Frame object, then this method returns 1.0.

Parameters:sinceFrame (Frame) – The starting frame for computing the relative scaling.
Returns:float – A positive value representing the heuristically determined scaling change ratio between the current frame and that specified in the sinceFrame parameter.

New in version 1.0.

scale_probability(sinceFrame)

The estimated probability that the overall motion between the current frame and the specified frame is intended to be a scaling motion.

scale_intent_factor = frame.scale_probability(start_frame)

If either this frame or sinceFrame is an invalid Frame object, then this method returns zero.

Parameters:sinceFrame (Frame) – The starting frame for computing the relative scaling.
Returns:float – A value between 0 and 1 representing the estimated probability that the overall motion between the current frame and the specified frame is intended to be a scaling motion.

New in version 1.0.

tool(id)

The Tool object with the specified ID in this frame.

Use this function to retrieve the Tool object from this frame using an ID value obtained from a previous frame. This function always returns a Tool object, but if no tool with the specified ID is present, an invalid Tool object is returned.

tool_of_interest = frame.tool(tool_ID)

Note that ID values persist across frames, but only until tracking of a particular object is lost. If tracking of a tool is lost and subsequently regained, the new Tool object representing that tool may have a different ID than that representing the tool in an earlier frame.

Parameters:id (integer) – The ID value of a Tool object from a previous frame.
Returns:Tool The tool with the matching ID if one exists in this frame; otherwise, an invalid Tool object is returned.

New in version 1.0.

translation(sinceFrame)

The change of position derived from the overall linear motion between the current frame and the specified frame.

The returned translation vector provides the magnitude and direction of the movement in millimeters.

The Leap Motion software derives frame translation from the linear motion of all objects detected in the field of view.

linear_frame_movement = frame.translation(start_frame)

If either this frame or sinceFrame is an invalid Frame object, then this method returns a zero vector.

Parameters:sinceFrame (Frame) – The starting frame for computing the relative translation.
Returns:Vector A vector representing the heuristically determined change in position of all objects between the current frame and that specified in the sinceFrame parameter.

New in version 1.0.

translation_probability(sinceFrame)

The estimated probability that the overall motion between the current frame and the specified frame is intended to be a translating motion.

translation_intent_factor = frame.translation_probability(start_frame)

If either this frame or sinceFrame is an invalid Frame object, then this method returns zero.

Parameters:sinceFrame (Frame) – The starting frame for computing the translation.
Returns:float A value between 0 and 1 representing the estimated probability that the overall motion between the current frame and the specified frame is intended to be a translating motion.

New in version 1.0.

invalid
Type:Frame

An invalid Frame object.

def find_frame(self, criteria):
    #search for frame meeting criteria...
    if found:
        return frame
    else:
        return Frame.invalid

New in version 1.0.

eq(a, b)

Compare Frame object equality.

Two Frame objects are equal if and only if both Frame objects represent the exact same frame of tracking data and both Frame objects are valid.

New in version 1.0.

ne(a, b)

Compare Frame object inequality.

Two Frame objects are equal if and only if both Frame objects represent the exact same frame of tracking data and both Frame objects are valid.

New in version 1.0.