Data location in a buffer
- The origin is always located in the upper left corner of the image.
- Y axis is directed downward, X axis is directed to the right.
- Width and Height — image dimensions along the X and Y axes. The X axis is always scanned by the "fast" index, i.e. the image is stored in memory in rows along the X axis.
Pixel index in memory for its coordinates is:
index = width * y + x.
Camera orientation options
- Orientation — sets the counterclockwise image rotation. It has four possible values given in degrees: 0, 90, 180, 270.
- requireMirroring — when set to true, the image will be flipped along the X axis (applied after any rotation) before the recognition stage and effect drawing. If the device's frontal camera produces a mirrored image, you should pass it into recognizer with
requireMirroring = false.
- PixelFormat — the camera data binary format. E.g. YUV_420 or RGBA.
Default transformation fits by Height
The default transformation fits Input Frame into Output Surface by Height. It means the Input Frame will be displayed in the Result Frame padded (or cropped) by width.
Custom transformation fits input rectangle inside output rectangle
The custom transformation fits input
image_rect into the output surface (
viewport_rect). The Input image_rect will be inscribed into the viewport_rect by width or height.
The Output Surface may also be flipped by the X or Y axis with
You can also set the custom transformation with
To switch back to using the default transformation, call
set_render_transform with zero-size rectangles.
|image_rect||x, y, w, h||x, y - input rectangle coordinates, w, h - input rectangle size|
|viewport_rect||x, y, w, h||x, y - output rectangle coordinates, w, h - output rectangle size|
|flip_x||true, false||true - flip image by x axis, false - default value|
|flip_y||true, false||true - flip image by y axis, false - default value|
The transformation of coordinates from one image basis to another is an image processing task. Initially, Face Recognition Engine (FRX) detects the coordinates of facial features on the input image given by the camera. Then, these coordinates undergo a sequence of transformations to display visual effects on the screen.
During this process, the coordinates of the detected landmarks are converted to corresponding dimensions and stored as a part of the frame_data structure.
frame_data includes different data structures. Some of them transform coordinates to specific data structure dimensions. Such transformations are represented by Basis Transformations defined as a part of
frx_result structures. The Basis Transformation transfers coordinates from common space (common basis) to a specific space. The image below shows the relations between common basis and destination basis.
For all face recognition coordinate spaces, the axis Y has a downward direction. The center of the coordinate space may be located at top left corner or in the center of basis dimensions.
SDK defines several coordinate spaces:
Common basis: An intermediate space to perform convertions between other spaces. This space has the next parameters:
- The basis dimensions are defined by full_roi rectangle stored in full_image_t event data structure. The basis dimension depends on the size of the full image, e.g. if the full image is 1280x720, then the common basis space will be [-640:640; -360:360], and the center of the coordinate space is in (0,0);
- Rotation (orientation): 0 degrees;
- Axis Y downward directed;
- Mirroring disabled.
frx_y_frame: The coordinate space that enables the face analysis engine to recognize facial features. The face analysis engine accepts frame in a resolution of 640x480.
If the scaled image (one or both dimensions) is smaller than the destination image, then the scaled image is centered within the image borders. The rest part is filled by copying the edges of the frame.
frx_result: The space used to store face recognition results for further processing.
This space is a combination of frx_y_frame space rotated by 90 degrees counterclockwise and moved by FRX_WIDTH (480) pixels to compensate rotation offset (the head on the image is directed upward).
full_image: The coordinate space of the full frame acquired from the camera.
Transformations between these spaces are always performed via a common basis. In other words, when you need to transform coordinates from frx_result space to full_image space, you need to multiply the inversed transformation of frx_result space (3x3 matrix) by the Basis Transformation of full_image space (also 3x3 matrix). Schematically it will look like:
frx_result.basis_tranform.inverse() * full_image.basis_transform. Further, to perform transformation, you need to multiply coordinates in the source space with the destination transform matrix using a special method
transform. See examples below for more details.
Please note that transform stores the transformation matrix in row-based order, so the sequence of transformations reads from left to the right.
- Example: frx_result to full_frame
>> denotes multiplication of the source transform matrix to the destination one.
It is read: Transform frx_result to a common basis and then transform it to the full_image space.
- Example: frx_result to Open GL (OGL) space projection [-1:1; -1:1]
It is read: Transform OGL space dimensions to frx_result dimensions (no orientation fix) flipping axis y; transform the result to common basis taking into account the frx frame border size and orientation; then transform it to OGL space back without rotation, but flipping y axis; extend the matrix to support 3d space conversions.
- Example: OGL space to frx_result space.
It is read: Transform frx_result to OGL space dimensions (no orientation changes), fix orientation via common space transforming to FRX input frame dimensions. Get transformation from
frx_result space to OGL space and inverse this transformation to get the required OGL space to
frx_resultspace to normalized basis [0:1; 0:1], the space origin is at left top corner.
It is read: Transform
common basis to the OGL basis without flipping
y axis; transform
frx_result basis to
common basis, transform the result to OGL space and then OGL space to normalized basis.