Setting up a Unity3D project

The Leap Motion asset package includes plugin files for using the Leap Motion device on Windows computers. This package includes Hand prefabs, scripts, and demo scenes to help you develop Leap Motion apps quickly.

Get the Leap Motion asset package from: https://developer.leapmotion.com/downloads/unity.

Note that version 4.0+ of the Unity assets are currently available for Windows only and require the Orion Beta service. Earlier versions of the assets are available for both Mac and Windows, but require the earlier 2.3 version of the Leap Motion service. You cannot mix elements from Orion with earlier versions of the assets. They are not compatible.

Create a project

First, create a new Unity project in the usual way, i.e.:

  1. Open the Unity editor.
  2. Select File > New Project...
  3. Choose a name and save location.
  4. Click Create Project.

Then, import the Leap Motion asset package into the project:

  1. Download the assets.

  2. Select the Unity Assets > Import Package > Custom Package menu command.

  3. Locate the downloaded asset package and click Open.

    The assets are imported into your project.

If you are importing the assets into a project that already contains Leap Motion assets, it is recommended that you delete the old assets first (make a backup of your project). The Unity import process only adds new files and overwrites changed files. It does not remove obsolete files (and sometimes creates duplicates of existing files instead of overwriting them).

Import Modules

For Orion, we have divided our Unity assets into a core package supported by optional modules. The core package is required to use hand tracking, but you only need those modules containing features you want to use. None of the modules depend on each other. The currently available modules include:

  • Attachments – provides a utility hand model that makes it easy to attach game objects to various points on the hand.
  • Hands – provides additional graphics hand models and tools for using rigged 3D hand models.
  • Interaction Engine – provides natural interaction physics between hands and virtual 3D objects in a scene and extensible mechanisms for fine tuning interactions for particular shapes and circumstances.
  • UI Input – Enables using hands to control standard Unity UI elements.
  • Detection Examples – Provides examples for using the Detection utilities feature, which is part of the core assets package. This module does not add any new features (though some of the example scripts may be useful).

Import a module the same way you imported the Leap Motion asset package:

  1. Download the module.

  2. Select the Unity Assets > Import Package > Custom Package menu command.

  3. Locate the downloaded module asset package and click Open.

    The module is imported into your project.

Adding Hands to a VR Scene

The LMHeadMountedRig must replace any existing camera rigs in your scene.

Place the LMHeadMountedRig prefab at the position of the users’s initial viewpoint in the scene. The hands in the scene are positioned at the same position as your real hands relative to the real Leap Motion device.

To add hands to a scene:

  1. Remove any existing camera or camera rigs.

  2. Drag the prefab into the scene hierarchy.

  3. Position the prefab in the scene. Like any VR camera, the position and orientation of the rig will be the initial view of the scene (as modified by the HMD tracking system).

  4. Add your hand models to the Hand Pool in the LeapHandController component:

    1. Select the LeapHandController in the hierarchy.
    2. Set the Hand Pool’s Model Pool size to 2. This will create two elements in the pool.
    3. Set the name of one element to “gfx_hands” (or any suitable name).
    4. Drag the desired hand prefabs to the left and right model properties.
    5. Set Is Enabled and Can Duplicate to true.
    6. Repeat for the second element, using physics hand prefabs.

    You can use prefabs directly in the hand pool or place the hand models in the scene hierarchy first. If you place the hand models in the scene, note that we no longer recommend placing the models as children of the LeapHandController. The models should go outside the camera hierarchy. Otherwise stuttering can occur during head movement.

  5. Play the scene.

    You should see hands appear in the Game view when you put your hands in front of the Leap Motion device. If you don’t see hands, open the Visualizer from the Leap Motion control panel to make sure your Leap Motion device is connected and sending out tracking data. If the Unity hands are just out of view, you can pause the game and use the Scene view to locate them.

Adding Hands to a “Desktop” Scene

Place the LeapHandController prefab below the point at which you want the hands to appear. The hands in the scene are positioned at the same position as your real hands relative to the real Leap Motion device. You can change the Hand Movement Scale setting to allow the hands to move a within a larger volume.

To add hands to a scene:

  1. In the Project panel, locate the LeapHandController prefab in Assets/LeapMotion/Prefabs

  2. Drag the LeapHandController prefab into the Scene view.

  3. Set the transform position to the desired location.

    To see hands, the LeapHandController must be within the field of view of the camera.

  4. Create an empty GameObject and name it “HandModels”. This game object should be a sibling of the LeapHandController object, never a child.

  5. Drag the desired hand prefabs to the HandModels game object. You will need a separate prefab for right and left hands and will typically have at least two sets: one for visual dispaly and one for Unity physics interaction. You can have any number of sets.

  6. Select the LeapHandController in the hierarchy.

  7. Drag the HandModels GameObject to the ModelsParent propety of the HandPool component of the LeapHandController.

  8. Set the Hand Pool’s Model Pool size to 2. This will create two elements in the pool.

  9. Set the name of one element to “gfx_hands” (or any suitable name).

  10. Drag the graphics hand models from your HandModels game object to the left and right model properties.

  11. Set Is Enabled and Can Duplicate to true.

  12. Repeat for the second element, using physics hand prefabs.

    You can use prefabs directly in the hand pool.

    Note: The CapsuleHand prefabs are designed for 1:1 scale only.

  13. Play the scene.

    You should see hands appear in the Game view when you put your hands over the Leap Motion device. If you don’t see hands, open the Visualizer from the Leap Motion control panel to make sure your Leap Motion device is connected and sending out tracking data. If the Unity hands are just out of view, you can pause the game and use the Scene view to locate them.

Accessing the Leap Motion API directly

In addition to the prefabs and scripts included in the asset package, you can write your own scripts to access tracking data from the Leap Motion API. The Leap Motion classes are defined in the Leap namespace. A basic MonoBehavior class that accesses the Leap Motion API will look something like the following (this script moves the object to which it is attached to just underneath the palm of the left hand):

using UnityEngine;
using System.Collections.Generic;
using Leap;

public class LeapBehavior : MonoBehaviour {
    LeapProvider provider;

    void Start ()
    {
        provider = FindObjectOfType<LeapProvider>() as LeapProvider;
    }

    void Update ()
    {
        Frame frame = provider.CurrentFrame;
        foreach (Hand hand in frame.Hands)
        {
          if (hand.IsLeft)
          {
              transform.position = hand.PalmPosition.ToVector3() +
                                   hand.PalmNormal.ToVector3() *
                                  (transform.localScale.y * .5f + .02f);
              transform.rotation = hand.Basis.Rotation();
          }
       }
    }
}

The script, LeapUnityExtensions.cs in Assets/LeapMotion/Scripts/Utils, provides helpful extension functions for converting vectors and matrices from the Leap Motion API classes to the Unity API classes. The functions ToVector3() and Rotation() used above are defined in this script.

Note: In almost every case, you should get Frame objects from the scene’s LeapProvider object. The LeapProvider transforms the frame data into the proper frame of reference. If you get a frame from the Leap.Controller object, the data will still be in the Leap Motion frame of reference and will not match the hands displayed in the scene.