Recognizer features allow you to implement real-time recoloring for lips, hair, eyes, skin and segment background or full body to change it in face filters. To enable some recognizer feature you need to add it into the field "recognizer" in config.json file within effect. This field must be the array with the recognizer features names.
Recognizer features with recoloring support:
Access to recognizer features results
Results of recognizer feature recognition you can find in your shaders. There are two types of recognition results:
Transformation is commonly a 2x4 transformation transposed matrix from global effect space into feature space. Read more about transformations.
Mask is a 2d image binary mask with recognition results.
To access transformation you need to declare a uniform block with
glfx_BASIS_DATA name in your shaders.
Commonly this block looks like this:
To access the recognition mask you need to declare sampler 2d uniform variable with a specific name (name of every feature variable will be specified in the next sections).
Process of implementing recognizer feature
The process of applying recognizer features consist of blending some color with the background image on a surface and cropping this surface with a binary mask.
First, you need to go to the vertex shader and transform this surface so that it will bind to the expected position. To achieve this, you need to use transformation, which will be delivered in shaders with the enabled feature. This transformation is the transformation matrix from effect space into local feature space. To transform your surface rightly you need to multiply the inverse feature transformation matrix with surface vertices.
The use case of transformation matrix feature depends on the surface you want to use. If you want to use the surface from the demo effect, first check the transformation use case from the effect shader.
Next, you need to go to the fragment shader and declare
glfx_LIPS_MASK uniform variables.
After that, you need to blend color, which you want to apply with the background color and crop your surface.
Next, you need to spawn the feature surface in config.js.Download example effect