The Leap Motion API provides information that you can use to implement touch emulation in your application. Touch information is provided by the Pointable class.
The Leap defines an adaptive touch surface that you can use to orchestrate interaction with 2D elements of your application. This surface is oriented roughly parallel to the x-y plane, but adapts to the user’s finger and hand position. As the user reaches forward with a finger or tool, the Leap reports whether that pointable object is close-to or touching this imaginary surface. The API reports touches with respect to the surface with two values: the zone and the distance to the touch plane.
The virtual touch surface
The touch zone identifies whether the Leap Motion software considers a Pointable as hovering near the touch surface, as penetrating the touch surface, or as too far from the surface (or pointing in the wrong direction). The zones are “hovering,” “touching,” and “none.” The transition between zones tends to lag behind the touch distance. This lag is used to prevent abrupt and repeated transitions. If you are implementing touch interaction within an application, you may not need to consider the touch zone very often.
The touch distance is valid only when a Pointable is within the hovering or touching zones. The distance is a normalized value in the range [+1..-1]. When a Pointable first enters the hovering zone, the touch distance is +1.0 and the distance decreases toward 0 as the Pointable nears the touch surface. When the Pointable penetrates the surface, the distance is 0. As the Pointable pushes deeper into the touch zone, the distance approaches, but never exceeds, -1.
You can use the zone value to decide when to update UI elements based on hover or touch. You can use the distance to further modify UI elements based on proximity to the touch plane. For example, you can show the highlight state of a control when a finger is over the control and in the hovering zone and change the cursor based on distance to provide feedback about how close the user is to touching the control.
As part of the touch emulation API the Leap Motion provides a stabilized position for Pointable objects in addition to the standard position. The Leap Motion software stabilizes the position using an adaptive filter that smooths and slows the motion to make it easier for the user to interact with small regions on the screen (like buttons and links). The smoothing is greater when the movement is slow so that the user can zero in and touch a particular point more easily.
The touch zone is reported by the touchZone attribute of the Pointable class. The zones are identified using the Zone enumeration, which defines the following states:
The following code snippet illustrates how to retrieve the zone of a finger or tool:
<p>Zone: <span id="zone">…</span></p>
<script>
var zoneDisplay = document.getElementById("zone");
var controller = new Leap.Controller();
controller.on('frame', function(frame){
if(frame.pointables.length > 0)
{
var touchZone = frame.pointables[0].touchZone;
zoneDisplay.innerText = touchZone;
}
});
controller.connect();
</script>
The touch distance is reported by the touchDistance attribute of the Pointable class. The distance ranges from +1 to -1 as the finger moves to and through the virtual touch surface. The distance does not represent a physical quantity, but rather how close to touching the Leap Software considers the pointable.
The following code snippet illustrates how to retrieve the touch distance of a finger:
<p>Distance: <span id="distance">…</span></p>
<script>
var distanceDisplay = document.getElementById("distance");
var controller = new Leap.Controller();
controller.on('frame', function(frame){
if(frame.pointables.length > 0)
{
var touchDistance = frame.pointables[0].touchDistance;
distanceDisplay.innerText = touchDistance;
}
});
controller.connect();
</script>
The stabilized position is reported by the stabilizedTipPosition attribute of the Pointable class. This position is reported in reference to the standard Leap Motion coordinate system, but has a context-sensitive amount of filtering and stabilization.
The following code snippet illustrates how to retrieve the stabilized position of a pointable:
<p>Stabilized Position: <span id="stabPosition">…</span></p>
<p>Difference from tip position: <span id="delta">…</span></p>
<script>
var stabilizedDisplay = document.getElementById("stabPosition");
var deltaDisplay = document.getElementById("delta");
var controller = new Leap.Controller();
controller.on('frame', function(frame){
if(frame.pointables.length > 0)
{
var pointable = frame.pointables[0];
var stabilizedPosition = pointable.stabilizedTipPosition;
var tipPosition = pointable.tipPosition;
stabilizedDisplay.innerText = "(" + stabilizedPosition[0] + ", "
+ stabilizedPosition[1] + ", "
+ stabilizedPosition[2] + ")";
deltaDisplay.innerText = "(" + (tipPosition[0] - stabilizedPosition[0]) + ", "
+ (tipPosition[1] - stabilizedPosition[1]) + ", "
+ (tipPosition[2] - stabilizedPosition[2]) + ")";
}
});
controller.connect();
</script>
When implementing touch emulation, you must map the Leap Motion coordinate space to the screen space of your application. To make this mapping easier, the Leap Motion API provides the InteractionBox class. The InteractionBox represents a rectilinear volume within the Leap Motion field of view. The class provides a function that normalizes positions within this volume to coordinates in the range [0..1].
<p>Normalized Position: <span id="normPosition">…</span></p>
<p>Tip Position: <span id="tipPosition">…</span></p>
<script>
var normalizedDisplay = document.getElementById("normPosition");
var tipDisplay = document.getElementById("tipPosition");
var controller = new Leap.Controller();
controller.on('frame', function(frame){
if(frame.pointables.length > 0)
{
var pointable = frame.pointables[0];
var interactionBox = frame.interactionBox;
var normalizedPosition = interactionBox.normalizePoint(pointable.tipPosition, true);
var tipPosition = pointable.tipPosition;
normalizedDisplay.innerText = "(" + normalizedPosition[0] + ", "
+ normalizedPosition[1] + ", "
+ normalizedPosition[2] + ")";
tipDisplay.innerText = "(" + tipPosition[0] + ", "
+ tipPosition[1] + ", "
+ tipPosition[2] + ")";
}
});
controller.connect();
</script>
You can normalize a position and then scale the resulting coordinate by the display area dimensions to get a point in application coordinates. For example, if you have a canvas element, you can map normalized coordinates to coordinates within the canvas as shown in the following code:
<canvas id="displayArea" width="200" height="100" style="background:#dddddd;"></canvas>
<script>
var canvasElement = document.getElementById("displayArea");
var displayArea = canvasElement.getContext("2d");
var controller = new Leap.Controller();
controller.on("frame", function(frame){
if(frame.pointables.length > 0)
{
canvasElement.width = canvasElement.width; //clear
//Get a pointable and normalize the tip position
var pointable = frame.pointables[0];
var interactionBox = frame.interactionBox;
var normalizedPosition = interactionBox.normalizePoint(pointable.tipPosition, true);
// Convert the normalized coordinates to span the canvas
var canvasX = canvasElement.width * normalizedPosition[0];
var canvasY = canvasElement.height * (1 - normalizedPosition[1]);
//we can ignore z for a 2D context
displayArea.strokeText("(" + canvasX.toFixed(1) + ", " + canvasY.toFixed(1) + ")", canvasX, canvasY);
}
});
controller.connect();
</script>
The following example uses the touch emulation APIs to display the positions of all detected Pointable objects in an application window. The example uses the touch zone to set the color of the points and uses the touch distance to set the alpha value. The stabilized tip positions are mapped to the application window using the InteractionBox class.
<div id="stage" style="background-color:rgb(250,235,200); width:800px; height:600"></div>
<script src="http://cdnjs.cloudflare.com/ajax/libs/kineticjs/5.0.6/kinetic.min.js"></script>
<script>
var stageWidth = 800;
var stageHeight = 600;
var tips = new Array(10);
var stage = new Kinetic.Stage({
container: 'stage',
width: stageWidth,
height: stageHeight
});
var leap = new Leap.Controller();
leap.connect();
var layer = new Kinetic.Layer();
//Make ten circles to use as finger tips
for (var t = 0; t < 10; t++) {
var tip = new Kinetic.Circle({
x: 239,
y: 75,
radius: 20,
fill: 'green',
stroke: 'black',
strokeWidth: 4,
opacity: .5,
visible: false
});
tips[t] = tip;
layer.add(tip);
}
// add the layer to the stage
stage.add(layer);
var anim = new Kinetic.Animation(function (frame) {
var time = frame.time,
timeDiff = frame.timeDiff,
frameRate = frame.frameRate;
// update finger tip display with data from latest frame
var tipPointer = 0;
var leapFrame = leap.frame();
if (leapFrame.valid) {
var iBox = leapFrame.interactionBox;
for (var p = 0; p < leapFrame.pointables.length; p++) {
var pointable = leapFrame.pointables[p];
var pos = iBox.normalizePoint(pointable.tipPosition, true);
tips[tipPointer].setX(pos[0] * stageWidth);
tips[tipPointer].setY(stageHeight - pos[1] * stageHeight);
tips[tipPointer].setVisible(true);
if (pointable.touchZone == "hovering") {
tips[tipPointer].setOpacity(.375 - pointable.touchDistance * .2);
tips[tipPointer].setFillRGB({r: 0, g: 128, b: 0});
} else if (pointable.touchZone == "touching") {
tips[tipPointer].setOpacity(.375 - pointable.touchDistance * .5);
tips[tipPointer].setFillRGB({r: 128, g: 0, b: 0});
} else {
tips[tipPointer].setOpacity(.1);
tips[tipPointer].setFillRGB({r: 0, g: 0, b: 128});
}
if (tipPointer < 9) tipPointer++;
}
while (tipPointer <= 9) tips[tipPointer++].setVisible(false);
}
}, layer);
anim.start();
</script>