Image Class Reference

The Image class represents a single image from one of the Leap Motion cameras. More...

Inherits Interface.

Public Types
enum	FormatType
	Enumerates the possible image formats. More...

Public Member Functions
int	bytesPerPixel () const
	The number of bytes per pixel. More...
const unsigned char *	data () const
	The image data. More...
void	data (unsigned char *dst) const
void *	dataPointer () const
const float *	distortion () const
	The distortion calibration map for this image. More...
void	distortion (float *dst) const
int	distortionHeight () const
	The distortion map height. More...
void *	distortionPointer () const
int	distortionWidth () const
	The stride of the distortion map. More...
FormatType	format () const
	The image format. More...
int	height () const
	The image height. More...
int32_t	id () const
	The image ID. More...
	Image (ImageImplementation *)
	Image ()
	Constructs a Image object. More...
bool	isValid () const
	Reports whether this Image instance contains valid data. More...
bool	operator!= (const Image &) const
	Compare Image object inequality. More...
bool	operator== (const Image &) const
	Compare Image object equality. More...
float	rayOffsetX () const
	The horizontal ray offset. More...
float	rayOffsetY () const
	The vertical ray offset. More...
float	rayScaleX () const
	The horizontal ray scale factor. More...
float	rayScaleY () const
	The vertical ray scale factor. More...
Vector	rectify (const Vector &uv) const
	Provides the corrected camera ray intercepting the specified point on the image. More...
int64_t	sequenceId () const
	The image sequence ID. More...
int64_t	timestamp () const
	Returns a timestamp indicating when this frame began being captured on the device. More...
std::string	toString () const
	A string containing a brief, human readable description of the Image object. More...
Vector	warp (const Vector &xy) const
	Provides the point in the image corresponding to a ray projecting from the camera. More...
int	width () const
	The image width. More...

Static Public Member Functions
static const Image &	invalid ()
	Returns an invalid Image object. More...

Friends
std::ostream &	operator<< (std::ostream &, const Image &)
	Writes a brief, human readable description of the Image object. More...

Detailed Description

The Image class represents a single image from one of the Leap Motion cameras.

In addition to image data, the Image object provides a distortion map for correcting lens distortion.

//Uses Cinder OpenGL wrapper
Frame frame = controller.frame();

ImageList images = frame.images();
for(int i = 0; i < 2; i++){
    Image image = images[i];

    const unsigned char* image_buffer = image.data();

    //Draw the raw image data as a greyscale bitmap
    Surface surface(image.width(), image.height(), image.width() * 4, SurfaceChannelOrder::RGBA);
    int cursor = 0;
    Surface::Iter iter = surface.getIter();
    while( iter.line() ) {
        while( iter.pixel() ) {
            iter.r() = image_buffer[cursor];
            iter.g() = iter.b() = iter.r();
            iter.a() = 255;
            cursor++;
        }
    }

Note that Image objects can be invalid, which means that they do not contain valid image data. Get valid Image objects from Frame::frames(). Test for validity with the Image::isValid() function.

Since

2.1.0

Member Enumeration Documentation

enum FormatType

Enumerates the possible image formats.

The Image::format() function returns an item from the FormatType enumeration.

Since

2.2.0

Enumerator
INFRARED
IBRG

Constructor & Destructor Documentation

Image

(

ImageImplementation *

)

Image

(

)

Constructs a Image object.

An uninitialized image is considered invalid. Get valid Image objects from a ImageList object obtained from the Frame::images() method.

Since

2.1.0

Member Function Documentation

int bytesPerPixel

(

)

const

The number of bytes per pixel.

Use this value along with Image::width() and Image:::height() to calculate the size of the data buffer.

int bufferSize = image.bytesPerPixel() * image.width() * image.height();

Since

2.2.0

const unsigned char* data

(

)

const

The image data.

The image data is a set of 8-bit intensity values. The buffer is Image::width() * Image::height() * Image::bytesPerPixel() bytes long.

const unsigned char* image_buffer = image.data();

Returns

The array of unsigned char containing the sensor brightness values.

Since

2.1.0

void data ( unsigned char *  dst ) const

inline

void* dataPointer ( ) const

inline

const float* distortion

(

)

const

The distortion calibration map for this image.

The calibration map is a 64x64 grid of points. Each point is defined by a pair of 32-bit floating point values. Each point in the map represents a ray projected into the camera. The value of a grid point defines the pixel in the image data containing the brightness value produced by the light entering along the corresponding ray. By interpolating between grid data points, you can find the brightness value for any projected ray. Grid values that fall outside the range [0..1] do not correspond to a value in the image data and those points should be ignored.

const float* distortion_buffer = image.distortion();

The calibration map can be used to render an undistorted image as well as to find the true angle from the camera to a feature in the raw image. The distortion map itself is designed to be used with GLSL shader programs. In non-realtime contexts, it may be more convenient to use the Image::rectify() and Image::warp() functions.

If using shaders is not possible, you can use the distortion map directly. This can be faster than using the warp() function, if carefully optimized:

float destinationWidth = 320;
float destinationHeight = 120;
unsigned char destination[(int)destinationWidth][(int)destinationHeight];

//define needed variables outside the inner loop
float calibrationX, calibrationY;
float weightX, weightY;
float dX, dX1, dX2, dX3, dX4;
float dY, dY1, dY2, dY3, dY4;
int x1, x2, y1, y2;
int denormalizedX, denormalizedY;
int i, j;

const unsigned char* raw = image.data();
const float* distortion_buffer = image.distortion();

//Local variables for values needed in loop
const int distortionWidth = image.distortionWidth();
const int width = image.width();
const int height = image.height();

for (i = 0; i < destinationWidth; i += 1) {
    for (j = 0; j < destinationHeight; j += 1) {
        //Calculate the position in the calibration map (still with a fractional part)
        calibrationX = 63 * i/destinationWidth;
        calibrationY = 62 * (1 - j/destinationHeight); // The y origin is at the bottom
        //Save the fractional part to use as the weight for interpolation
        weightX = calibrationX - truncf(calibrationX);
        weightY = calibrationY - truncf(calibrationY);

        //Get the x,y coordinates of the closest calibration map points to the target pixel
        x1 = calibrationX; //Note truncation to int
        y1 = calibrationY;
        x2 = x1 + 1;
        y2 = y1 + 1;

        //Look up the x and y values for the 4 calibration map points around the target
        dX1 = distortion_buffer[x1 * 2 + y1 * distortionWidth];
        dX2 = distortion_buffer[x2 * 2 + y1 * distortionWidth];
        dX3 = distortion_buffer[x1 * 2 + y2 * distortionWidth];
        dX4 = distortion_buffer[x2 * 2 + y2 * distortionWidth];
        dY1 = distortion_buffer[x1 * 2 + y1 * distortionWidth + 1];
        dY2 = distortion_buffer[x2 * 2 + y1 * distortionWidth + 1];
        dY3 = distortion_buffer[x1 * 2 + y2 * distortionWidth + 1];
        dY4 = distortion_buffer[x2 * 2 + y2 * distortionWidth + 1];

         std::cout << i << ", " << j << " -- " << x1 << ", " << y1 << ", " << x2 << ", " << y2 << " -- "
                   << (x1 * 2 + y1 * distortionWidth) << ", "
                   << (x1 * 2 + y1 * distortionWidth + 1) << " -- "
                   << (x2 * 2 + y2 * distortionWidth) << ", "
                   << (x2 * 2 + y2 * distortionWidth + 1) << std::endl;
        //Bilinear interpolation of the looked-up values:
        // X value
        dX = dX1 * (1 - weightX) * (1 - weightY) +
             dX2 * weightX * (1 - weightY) +
             dX3 * (1 - weightX) * weightY +
             dX4 * weightX * weightY;

        // Y value
        dY = dY1 * (1 - weightX) * (1 - weightY) +
             dY2 * weightX * (1 - weightY) +
             dY3 * (1 - weightX) * weightY +
             dY4 * weightX * weightY;

        // Reject points outside the range [0..1]
        if((dX >= 0) && (dX <= 1) && (dY >= 0) && (dY <= 1)) {
            //Denormalize from [0..1] to [0..width] or [0..height]
            denormalizedX = dX * width;
            denormalizedY = dY * height;

            //look up the brightness value for the target pixel
            destination[i][j] = raw[denormalizedX + denormalizedY * width];
        } else {
            destination[i][j] = -1;
        }
    }
}

Distortion is caused by the lens geometry as well as imperfections in the lens and sensor window. The calibration map is created by the calibration process run for each device at the factory (and which can be rerun by the user).

Note, in a future release, there may be two distortion maps per image; one containing the horizontal values and the other containing the vertical values.

Returns

The float array containing the camera lens distortion map.

Since

2.1.0

void distortion ( float *  dst ) const

inline

int distortionHeight

(

)

const

The distortion map height.

Currently fixed at 64.

int correctionGridHeight = image.distortionHeight();

Since

2.1.0

void* distortionPointer ( ) const

inline

int distortionWidth

(

)

const

The stride of the distortion map.

Since each point on the 64x64 element distortion map has two values in the buffer, the stride is 2 times the size of the grid. (Stride is currently fixed at 2 * 64 = 128).

int correctionGridWidth = image.distortionWidth();

Since

2.1.0

FormatType format

(

)

const

The image format.

if(image.format() == Leap::Image::INFRARED){
    std::string openGL_format = "GL_RED";
    std::string openGL_type = "GL_UNSIGNED_BYTE";
}

Since

2.2.0

int height

(

)

const

The image height.

int height = image.height();

Since

2.1.0

int32_t id

(

)

const

The image ID.

Images with ID of 0 are from the left camera; those with an ID of 1 are from the right camera (with the device in its standard operating position with the green LED facing the operator).

Since

2.1.0

static const Image& invalid ( )

static

Returns an invalid Image object.

You can use the instance returned by this function in comparisons testing whether a given Image instance is valid or invalid. (You can also use the Image::isValid() function.)

Returns

The invalid Image instance.

Since

2.1.0

bool isValid

(

)

const

Reports whether this Image instance contains valid data.

Returns

true, if and only if the image is valid.

Since

2.1.0

bool operator!=

(

const Image &

)

const

Compare Image object inequality.

Two Image objects are equal if and only if both Image objects represent the exact same Image and both Images are valid.

Since

2.1.0

bool operator==

(

const Image &

)

const

Compare Image object equality.

Two Image objects are equal if and only if both Image objects represent the exact same Image and both Images are valid.

Since

2.1.0

float rayOffsetX

(

)

const

The horizontal ray offset.

Used to convert between normalized coordinates in the range [0..1] and the ray slope range [-4..4].

Leap::Vector raySlopes(-3.28, 1.76, 0);
Leap::Vector normRay =
    Leap::Vector(raySlopes.x * image.rayScaleX() + image.rayOffsetX(),
                 raySlopes.y * image.rayScaleY() + image.rayOffsetY(), 0);

Since

2.1.0

float rayOffsetY

(

)

const

The vertical ray offset.

Used to convert between normalized coordinates in the range [0..1] and the ray slope range [-4..4].

Leap::Vector normSlopes(.09, .72, 0);
Leap::Vector slope((normSlopes.x - image.rayOffsetX())/image.rayScaleX(),
                   (normSlopes.y - image.rayOffsetY())/image.rayScaleY(), 0);

Since

2.1.0

float rayScaleX

(

)

const

The horizontal ray scale factor.

Used to convert between normalized coordinates in the range [0..1] and the ray slope range [-4..4].

Leap::Vector raySlopes(-3.28, 1.76, 0);
Leap::Vector normRay =
    Leap::Vector(raySlopes.x * image.rayScaleX() + image.rayOffsetX(),
                 raySlopes.y * image.rayScaleY() + image.rayOffsetY(), 0);

Since

2.1.0

float rayScaleY

(

)

const

The vertical ray scale factor.

Used to convert between normalized coordinates in the range [0..1] and the ray slope range [-4..4].

Leap::Vector normSlopes(.09, .72, 0);
Leap::Vector slope((normSlopes.x - image.rayOffsetX())/image.rayScaleX(),
                   (normSlopes.y - image.rayOffsetY())/image.rayScaleY(), 0);

Since

2.1.0

Vector rectify

(

const Vector &

uv

)

const

Provides the corrected camera ray intercepting the specified point on the image.

Given a point on the image, rectify() corrects for camera distortion and returns the true direction from the camera to the source of that image point within the Leap Motion field of view.

This direction vector has an x and y component [x, y, 0], with the third element always zero. Note that this vector uses the 2D camera coordinate system where the x-axis parallels the longer (typically horizontal) dimension and the y-axis parallels the shorter (vertical) dimension. The camera coordinate system does not correlate to the 3D Leap Motion coordinate system.

Leap::Vector feature(127, 68, 0);
Leap::Vector slopes = image.rectify(feature);

Parameters

uv	A Vector containing the position of a pixel in the image.

Returns

A Vector containing the ray direction (the z-component of the vector is always 0).

Since

2.1.0

int64_t sequenceId

(

)

const

The image sequence ID.

long lastImage = 0;
while(!done){
    Leap::Image leftImage = controller.images()[0];
    if(leftImage.sequenceId() != lastImage){
        Leap::Image rightImage = controller.images()[1];
        lastImage = leftImage.sequenceId();
        // Use images...
    }
}

Since

2.2.1

int64_t timestamp

(

)

const

Returns a timestamp indicating when this frame began being captured on the device.

Since

2.2.7

std::string toString ( ) const

inline

A string containing a brief, human readable description of the Image object.

Returns

A description of the Image as a string.

Since

2.1.0

Vector warp

(

const Vector &

xy

)

const

Provides the point in the image corresponding to a ray projecting from the camera.

Given a ray projected from the camera in the specified direction, warp() corrects for camera distortion and returns the corresponding pixel coordinates in the image.

The ray direction is specified in relationship to the camera. The first vector element corresponds to the "horizontal" view angle; the second corresponds to the "vertical" view angle.

float horizontal_slope = tan(65 * Leap::PI/180);
float vertical_slope = tan(15 * Leap::PI/180);
Leap::Vector pixel = image.warp(Leap::Vector(horizontal_slope, vertical_slope, 0));
if(pixel.x >= 0 && pixel.y >= 0 && pixel.x <= image.width() && pixel.y <= image.height()){
    int data_index = floor(pixel.y) * image.width() + floor(pixel.x);
    unsigned char brightness = image.data()[data_index];
}

The warp() function returns pixel coordinates outside of the image bounds if you project a ray toward a point for which there is no recorded data.

warp() is typically not fast enough for realtime distortion correction. For better performance, use a shader program executed on a GPU.

Parameters

xy	A Vector containing the ray direction.

Returns

A Vector containing the pixel coordinates [x, y, 0] (with z always zero).

Since

2.1.0

int width

(

)

const

The image width.

int width = image.width();

Since

2.1.0

Friends And Related Function Documentation

std::ostream& operator<< ( std::ostream &  , const Image &    )

friend

Writes a brief, human readable description of the Image object.

Since

2.1.0