Touch Emulation¶

Overview¶

The Leap defines an adaptive touch surface that you can use to orchestrate interaction with 2D elements of your application. This surface is oriented roughly parallel to the x-y plane, but adapts to the user’s finger and hand position. As the user reaches forward with a finger or tool, the Leap reports whether that pointable object is close-to or touching this imaginary surface. The API reports touches with respect to the surface with two values: the zone and the distance to the touch plane.

The virtual touch surface

The touch zone identifies whether the Leap Motion software considers a Pointable as hovering near the touch surface, as penetrating the touch surface, or as too far from the surface (or pointing in the wrong direction). The zones are “hovering,” “touching,” and “none.” The transition between zones tends to lag behind the touch distance. This lag is used to prevent abrupt and repeated transitions. If you are implementing touch interaction within an application, you may not need to consider the touch zone very often.

The touch distance is valid only when a Pointable is within the hovering or touching zones. The distance is a normalized value in the range [+1..-1]. When a Pointable first enters the hovering zone, the touch distance is +1.0 and the distance decreases toward 0 as the Pointable nears the touch surface. When the Pointable penetrates the surface, the distance is 0. As the Pointable pushes deeper into the touch zone, the distance approaches, but never exceeds, -1.

You can use the zone value to decide when to update UI elements based on hover or touch. You can use the distance to further modify UI elements based on proximity to the touch plane. For example, you can show the highlight state of a control when a finger is over the control and in the hovering zone and change the cursor based on distance to provide feedback about how close the user is to touching the control.

As part of the touch emulation API the Leap Motion provides a stabilized position for Pointable objects in addition to the standard position. The Leap Motion software stabilizes the position using an adaptive filter that smooths and slows the motion to make it easier for the user to interact with small regions on the screen (like buttons and links). The smoothing is greater when the movement is slow so that the user can zero in and touch a particular point more easily.

Getting the Touch Zone¶

The touch zone is reported by the touchZone attribute of the Pointable class. The zones are identified using the Zone enumeration, which defines the following states:

• NONE — the pointable is either too far from the touch surface to be considered touching, or it is pointing back toward the user.
• HOVERING — the pointable tip has crossed into the hovering zone, but isn’t considered touching.
• TOUCHING — the pointable has crossed the virtual touch surface.

The following code snippet illustrates how to retrieve the zone of the forward-most finger:

zone = pointable.touch_zone

Getting the Touch Distance¶

The touch distance is reported by the touchDistance attribute of the Pointable class. The distance ranges from +1 to -1 as the finger moves to and through the virtual touch surface. The distance does not represent a physical quantity, but rather how close to touching the Leap Software considers the pointable.

The following code snippet illustrates how to retrieve the touch distance of the forward-most finger:

distance = pointable.touch_distance

Getting the Stabilized Position of a Pointable¶

The stabilized position is reported by the stabilizedTipPosition attribute of the Pointable class. This position is reported in reference to the standard Leap Motion coordinate system, but has a context-sensitive amount of filtering and stabilization.

The following code snippet illustrates how to retrieve the stabilized position of the forward-most finger:

frame = controller.frame()
pointable = frame.pointables.frontmost
stabilizedPosition = pointable.stabilized_tip_position

Converting from Leap Motion Coordinates to Application Coordinates¶

When implementing touch emulation, you must map the Leap Motion coordinate space to the screen space of your application. To make this mapping easier, the Leap Motion API provides the InteractionBox class. The InteractionBox represents a rectilinear volume within the Leap Motion field of view. The class provides a function that normalizes positions within this volume to coordinates in the range [0..1]. You can normalize a position and then scale the resulting coordinate by the application dimensions to get a point in application coordinates.

For example, if you have a window with a client-area size represented by the variables windowWidth and windowHeight, you can get the 2D pixel coordinates of a touch point within this window using the following code:

frame = controller.frame()
finger = frame.fingers.frontmost
stabilizedPosition = finger.stabilized_tip_position

interactionBox = controller.frame().interaction_box
normalizedPosition = interactionBox.normalize_point(stabilizedPosition)
x = normalizedPosition.x * windowWidth
y = windowHeight - normalizedPosition.y * windowHeight

TouchPoints Example¶

The following example uses the touch emulation APIs to display the positions of all detected Pointable objects in an application window. The example uses the touch zone to set the color of the points and uses the touch distance to set the alpha value. The stabilized tip positions are mapped to the application window using the InteractionBox class.

from Tkinter import Frame, Canvas, YES, BOTH
import Leap

class TouchPointListener(Leap.Listener):
    def on_init(self, controller):
        print "Initialized"

    def on_connect(self, controller):
        print "Connected"

    def on_frame(self, controller):
        self.paintCanvas.delete("all")
        frame = controller.frame()

        interactionBox = frame.interaction_box
        
        for pointable in frame.pointables:
            normalizedPosition = interactionBox.normalize_point(pointable.tip_position)
            if(pointable.touch_distance > 0 and pointable.touch_zone != Leap.Pointable.ZONE_NONE):
                color = self.rgb_to_hex((0, 255 - 255 * pointable.touch_distance, 0))
                
            elif(pointable.touch_distance <= 0):
                color = self.rgb_to_hex((-255 * pointable.touch_distance, 0, 0))
                #color = self.rgb_to_hex((255,0,0))
                
            else:
                color = self.rgb_to_hex((0,0,200))
                
            self.draw(normalizedPosition.x * 800, 600 - normalizedPosition.y * 600, 40, 40, color)

    def draw(self, x, y, width, height, color):
        self.paintCanvas.create_oval( x, y, x + width, y + height, fill = color, outline = "")

    def set_canvas(self, canvas):
        self.paintCanvas = canvas
        
    def rgb_to_hex(self, rgb):
        return '#%02x%02x%02x' % rgb

class PaintBox(Frame):

    def __init__( self ):
        Frame.__init__( self )
        self.leap = Leap.Controller()
        self.painter = TouchPointListener()
        self.leap.add_listener(self.painter)
        self.pack( expand = YES, fill = BOTH )
        self.master.title( "Touch Points" )
        self.master.geometry( "800x600" )
      
        # create Canvas component
        self.paintCanvas = Canvas( self, width = "800", height = "600" )
        self.paintCanvas.pack()
        self.painter.set_canvas(self.paintCanvas)

def main():
    PaintBox().mainloop()

if __name__ == "__main__":
    main()

The example uses Tkinter, but the Leap Motion-related code is applicable to all Python projects.