Using the Image Hand Assets in version 2.3.1

This documentation applies to version 2.3.1 of the Leap Motion Core Assets only. Currently, the Oculus headset is the only VR headset supported by both Leap Motion and Unity3D. Since Oculus only supports the Windows platform as of version 0.7, OS X developers can only use earlier versions of Unity3D, the Oculus runtime, and the Leap Motion Unity assets. For documentation about Image Hands and setting up a VR scene using version 3.0+ of the core assets, see VR Setup.

The Image Hand asset is a blend between Augmented Reality (AR) and Virtual Reality (VR). It displays images of your real hands from the camera sensors with the physicality and interactivity of motion-controlled 3D hands.

https://di4564baj7skl.cloudfront.net/documentation/images/unity/ImageHand_Balls.jpg

The image hands can add to the immersiveness of your VR or AR applications. The hands are properly depth-sorted so they appear behind closer objects and in front of those more distant. When the hands touch or penetrate an object, a highlight is applied to the intersection point to provide a depth cue. A glow effect is applied to the hand to indicate a high-confidence pose.

Trying Out The Image Hands

The Leap Motion Core Asset package includes several demo scenes to demonstrate the Image Hand assets.

https://di4564baj7skl.cloudfront.net/documentation/images/unity/ImageHands_DemoScene.png

You can find theVR demo scenes in the Assets folder LeapMotionVR/scenes. The 500Blocks, HandViewr, MovingDemo, and QuickSwitchDemo scenes have also been updated to use the Image Hands.

The LeapMotion/scenes folder also includes a desktop-mode demo, ImageHands. Note, however, that the Image Hand assets aren’t well suited to non-VR/AR use. This is because the viewpoint of the scene cameras must match those of the Leap Motion cameras; in desktop mode, the Leap Motion cameras are looking upward – not typically a useful viewpoint for an application.

To try out these scenes, create a new project and import the latest Leap Motion Core Asset package. Then open the desired scene.

Adding Image Hands to a Scene

To use the Image Hands, add the LeapOVRCameraRig or LeapOVRPlayerController prefab to the scene:

  1. Import the latest Leap Motion Core Asset package.

  2. Create a new Unity scene.

  3. In the Project tab, navigate to LeapMotion+OVR/Prefabs.

  4. Drag one of the LeapOVR prefabs into the scene:

    For a stationary camera, use LeapOVRCameraRig.

    For a camera that moves with a 1st-person avatar, use LeapOVRPlayerController.

    These rigs are based on the standard Oculus VR SDK prefabs.

    https://di4564baj7skl.cloudfront.net/documentation/images/unity/ImageHands_AddRig.png

    Doublecheck that the graphics hand models are set to ‘’ImageFullLeft’’ and ImageFullRight.

  5. Position the controller in your scene as appropriate.

  6. Turn off the Skybox:

    1. Open the Lighting settings panel with Window > Lighting menu command.
    2. Change the Skybox material from “Default-Skybox” to “None”.
  7. Play the scene. You should see something like:

    https://di4564baj7skl.cloudfront.net/documentation/images/unity/ImageHand_AddFinal.png

Now you can add your other objects to the scene. The Image Hand is life-sized to match the camera images. Thus your other objects should be scaled appropriately. For you to reach an object with a virtual hand, that object must be within an arm’s reach away from the controller (i.e. less than about .7 meters).

In general, we recommend using the LeapOVRCameraRig or LeapOVRPlayerController as is or with only simple, easily repeatable modifications. The structure of the game objects, scripts, and shaders may change in future releases.

Known Issues

There are few known issues with the Image Hand asset related to rendering:

  1. Use point lights for shadows. When casting shadows with the Image Hand, do not use directional lighting – use point lights instead. If dynamic directional lighting is used, shadows (not lighting) will be occluded in the vicinity of the hand.
  2. Avoid legacy Unity GUI Skins due to issues with intersection and occlusion of the hand by the GUI layers.
  3. The hand glow effect can bleed into bright areas of the scene near the hand.
  4. The linear color space is recommended for rendering. This is not the default setting of the Unity engine. (It can be changed by opening the Player Settings panel using the Edit>Project Settings>Player menu command and selecting the Linear option for Color Space under Other Settings.)