Effect structure

Every Banuba effect is a folder containing files with special meaning. Typical effect will look the following:

effect_folder/
 |-- config.json
 |-- config.js
 |-- images/
 |    |
 |   ...
 |-- shaders/
 |    |
 |   ...
 |-- videos/
 |    |
 |   ...
...

Where only config.json is mandatory (see bellow). So, in order to start, create a folder with one blank file named like this.

Other subfolders have no special meaning and exist only to group related files, which usually referred from config.json or config.js.

config.json

This is the basic file which describes effect itself. It has he following format:

{
    "scene": "effect name",
    "version": "2.0.0",
    
    "camera": {},
    "background": {
        // ...
    },
    // ... other top-level prefabs

    "faces": [     
        { // first face
            "prefab1": {
                // "prefab1" parameters
            },
            "prefab2": {
                // "prefab2" parameters
            }
            // ...
        },
        { // second face
        }
        // ..   
    ]
}

Where

• scene -- this is the name of your effect.
• version -- version of this configuration file. Always set it to "2.0.0".
• camera -- tells that you will render camera feed on the screen.
• faces -- array of JSON objects describing features ("prefabs" in our terminology) you wish to place on each face.

So, for each face we will define a JSON object where keys are the names of prefabs and values are the parameters for each prefab.

There are also top-level prefabs, that affects the whole screen and do not attached to any face.

The basic complete example will look like this (it will render simple "makeup" on the face):

{
    "scene": "Retouch example",
    "version": "2.0.0",
    "camera": {},
    "faces": [     
        {
            "retouch": {}
        }
    ]
}

Another good example to start:

Download simple 3D model sample

Each prefab and possible parameters for them are described below.

note

Colors are represented as 3- or 4-component string, each component is a value in range [0, 1] or [0, 255] separated by a space. E.g.: "1 0 0 1" -- red color, etc. HTML-like hex strings are also accepted. E.g.: "#00FF00" -- green color, alpha FF is assumed if missing.

Top-level prefabs

background

"background": {
    "transparency": 0.5,
    "texture": "path/to/file",
    "rotation": 0,
    "scale": 1,
    "content_mode": "scale_to_fill",
    "blur": 0.5,
    "clear_color": "1 0 0 1"
}

Enable virtual background.

transparency (optional) -- Sets the background transparency from 0 to 1.

texture (optional) -- Sets the file (image or video) as background texture.

rotation (optional) -- Rotates the background texture clockwise in degrees.

scale (optional) -- Scales the background texture.

content_mode (optional) -- Fits the background texture inside frame. Possible values are: scale_to_fill, fill, fit.

blur (optional) -- Sets the background blur radius. Radius in [0, 1] range.

clear_color (optional) -- Specifies the color of the area not covered by background texture (e.g. content_mode fit). Black by default. Transparency (the last) is a conventional value only and currently ignored (use 1 for it).

foreground

Apply texture onto the whole screen.

"foreground": {
    "filename": "path/to/texture/file"
}

filename -- Path to a PNG texture file. It must have reasonable alpha channel or it will hide the underlying drawings.

lut

Apply color filter (aka LUT).

"lut": {
    "filename": "path/to/lut/file",
    "strength": 0.9
}

filename -- Path to the LUT file (usually PNG).

strength (optional) -- LUT strength. Value in [0, 1].

lights

Add up to 4 directional light sources to GLTF models in addition to IBL textures.

"lights":
{
    "radiance": ["10 0 0 0", "0 0 10 0"],
    "direction": ["-1 0 0", "1 0 0"]
}

radiance -- Array (or a single value for 1 light) of light sources radiances in R, G, B components and light wrap around factor in 4th component.

direction -- Array (or a single value for 1 light) of light sources directions in X, Y, Z.

Face prefabs

gltf

"faces": [
    {
        "gltf": {
            "@mesh": "path/to/gltf/model",
            "rotation": "0.0 0.0 0.0",
            "scale": "1.0 1.0 1.0",
            "translation": "0.0 0.0 0.0",
            "animation": {
                "name": "Animation 1",
                "mode": "loop",
                "seek_position": 100
            },
            "@use_physics": true,
            "gravity": "0.0 -1000.0 0.0",
            "bones": {
                "bone_1": 1.0,
                "bone_2": 0.0,
                "bone_3": 1.0,
                "bone_4": 0.0,
                "bone_5": 1.0
            },
            "colliders":[
                {"center": "0. 0. 0.", "radius": 100.0},
                {"center": "10. 110. 420.", "radius": 650.0},
                {"center": "14. 300. 156.", "radius": 10.0}
            ],
            "constraints": [
                {"from": "bone_1", "to": "bone_2", "distance": 10.0},
                {"from": "bone_2", "to": "bone_3", "distance": 50.0},
                {"from": "bone_3", "to": "bone_4", "distance": 30.0},
                {"from": "bone_4", "to": "bone_5", "distance": 60.0}
            ],
            "damping": 0.99
        }
        // ...
    }
    // ...
]

Place 3D model in GLTF format on the face.

@mesh -- Path to GLTF model. Note leading @ in parameter, it is mandatory.

rotation (optional) -- Rotation angles (in radians) over X, Y, Z axes.

scale (optional) -- Scale along X, Y, Z axes.

translation (optional) -- Translate the model along X, Y, Z axes (in millimetres).

animation (optional) -- Play animation from GLTF file. All keys are optional (in most cases just empty object is enough to play the default animation). "name" -- select animation from file by name; "mode" ("off" | "loop" | "once" | "once_reversed" | "fixed") -- how to play the animation selected by "name"; "seek_position" -- where to start the animation (in ms).

@use_physics (optional) -- Load GLTF file with physics simulation.

gravity (optional) -- Sets gravitation vector along X, Y, Z axes.

bones (optional) -- Sets bones inversed mass. Object, where key is a name of bone and parameter is an inversed mass.

colliders (optional) -- Add sphere colliders for physical bones. Array of objects, each object is a separate collider. "Center" - X, Y, Z coordinates of the center of sphere, "radius" - float radius of the sphere.

constraints (optional) -- Add constraint between two bones. Array of objects, each object is a one constraint. "from" - "from" bone name, "to" - "to" bone name, "distance" - constaint length.

damping (optional) -- Damping parameter for physics simulation, good values usually are in [0.9-1.0] range.

action_units

"faces": [
    {
        "action_units": {},
        // ...
    }
    // ...
]

Enable action units from GLTF model.

eyes_whitening

"faces": [
    {
        "eyes_whitening":{
            "strength": 1.0
        }
        // ...
    }
    // ...
]

Apply whitening on the eyes.

strength -- Eyes whitening. Float number in the range [0.0, 1.0].

eyes_flare

"faces": [
    {
        "eyes_flare":{
            "strength": 1.0
        }
        // ...
    }
    // ...
]

Apply flare on the eyes.

strength -- Flare brightness. Float number in the range [0.0, 1.0].

teeth_whitening

"faces": [
    {
        "teeth_whitening":{
            "strength": 1.0
        }
        // ...
    }
    // ...
]

Apply whitening on the teeth.

strength -- Teeth whitening. Float number in the range [0.0, 1.0].

softlight

"faces": [
    {
        "softlight":{
            "strength": 1.0,
            "texture": "path/to/file"
        }
        // ...
    }
    // ...
]

Apply softlight on the face.

strength -- Softlight strength. Float number in the range [0.0, 1.0].

texture (optional) -- Path to custom softlight texture.

eyelashes3d

"faces": [
    {
        "eyelashes3d":{
            "color": "1. 0. 0. 1.",
            "texture": "path/to/file"
        }
        // ...
    }
    // ...
]

Apply 3D eyelashes on the face.

color -- Color of the 3D eyelashes in RGBA string format.

texture (optional) -- Path to custom eyelashes texture.

eyelashes

"faces": [
    {
        "eyelashes":{
            "color": "1. 0. 0. 1.",
            "texture": "path/to/file"
        }
        // ...
    }
    // ...
]

Apply 2D eyelashes on the face.

color -- Color of the eyelashes in RGBA string format.

texture (optional) -- Path to custom eyelashes texture.

morphing

Morph (i.e. deform) certain parts of the face.

"faces": [
    {
        "morphing": {
            "eyebrows": {
                "spacing": 0.6, 
                "height": 0.3, 
                "bend": 1.0
            },
            "eyes": {
                "rounding": 0.6, 
                "enlargement": 0.3, 
                "height": 0.3, 
                "spacing": 0.3, 
                "squint": 0.3, 
                "lower_eyelid_pos": 0.3, 
                "lower_eyelid_size": 0.3,
                "down": 0.7, 
                "eyelid_upper": 0.5,    
                "eyelid_lower": 0.6 
            },
            "face": {
                "narrowing": 0.6, 
                "v_shape": 0.3, 
                "cheekbones_narrowing": 0.3, 
                "cheeks_narrowing": 0.3, 
                "jaw_narrowing": 0.7, 
                "chin_shortening": 0.3, 
                "chin_narrowing": 0.3, 
                "sunken_cheeks": 0.1, 
                "cheeks_jaw_narrowing": 0.2,
                "jaw_wide_thin": 0.6,
                "chin": 0.7,
                "forehead": 0.3
            },
            "nose": {
                "width": 0.3, 
                "length": 0.2, 
                "tip_width": 0.1,
                "down_up": 0.1,
                "sellion": 0.2
            },
            "lips": {
                "size": 0.4, 
                "height": 1.0, 
                "thickness": 0.1, 
                "mouth_size": 0.2, 
                "smile": 0.8, 
                "shape": 0.4,
                "sharp": 0.6 
            }
        }
        // ...
    }
    // ...
]

All settings are optional.

eyebrows

spacing -- Adjusting the space between the eyebrows [-1; 1].

height -- Raising/lowering the eyebrows [-1; 1].

bend -- Adjusting the bend of the eyebrows [-1; 1].

eyes

rounding -- Adjusting the roundness of the eyes [0; 1].

enlargement -- Enlarging the eyes [0; 1].

height -- Raising/lowering the eyes [-1; 1].

spacing -- Adjusting the space between the eyes [-1; 1].

squint -- Making the person squint by adjusting the eyelids [-1; 1].

lower_eyelid_pos -- Raising/lowering the lower eyelid [-1; 1].

lower_eyelid_size -- Enlarging/shrinking the lower eyelid [-1; 1].

down -- Eyes down [0; 1].

eyelid_upper -- Eyelid upper [0; 1].

eyelid_lower -- Eyelid lower [0; 1].

face

narrowing -- Narrowing the face [0; 1].

v_shape -- Shrinking the chin and narrowing the cheeks [0; 1].

cheekbones_narrowing -- Narrowing the cheekbones [-1; 1].

cheeks_narrowing -- Narrowing the cheeks [0; 1].

jaw_narrowing -- Narrowing the jaw [0; 1].

chin_shortening -- Decreasing the length of the chin [0; 1].

chin_narrowing -- Narrowing the chin [0; 1].

sunken_cheeks -- Sinking the cheeks and emphasizing the cheekbones [0; 1].

cheeks_and_jaw_narrowing -- Narrowing the cheeks and the jaw [0; 1].

jaw_wide_thin -- Jaw wide/thin [0; 1].

chin -- Face chin [0; 1].

forehead-- Forehead [0; 1].

nose

width -- Adjusting the nose width [-1; 1].

length -- Adjusting the nose length [-1; 1].

tip_width -- Adjusting the nose tip width [0; 1].

down_up -- Nose down/up [0; 1].

sellion -- Nose sellion [0; 1].

lips

size -- Adjusting the width and vertical size of the lips [-1; 1].

height -- Raising/lowering the lips [-1; 1].

thickness -- Adjusting the thickness of the lips [-1; 1].

mouth_size -- Adjusting the size of the mouth [-1; 1].

smile -- Making a person smile [0; 1].

shape -- Adjusting the shape of the lips [-1; 1].

sharp -- Lips sharp [0; 1].

eyes

Eyes recolor.

"faces": [
    {
        "eyes": {
            "eyes": "0.7 0 0 1",
            "corneosclera": "0.7 0 0 1",
            "pupil": "0.7 0 0 1"
        }
        // ...
    }
    // ...
]

eyes (optional) -- Eyes (i.e. iris) color. See note about color format above.

corneosclera (optional) -- Corneosclera ("sclera" in everyday language) color. See note about color format above.

pupil (optional) -- Pupil color. See note about color format above.

hair

Hair recoloring.

"faces": [
    {
        "hair": {
            "color": "0.7 0 0 1",
            "strands": ["0.7 0 0 1", "0.8 0.8 0 1"]
        }
        // ...
    }
    // ...
]

color (optional) -- Apply solid color if one element supplied or gradient recoloring if array (like stands in the sample). See note about color format above.

strands (optional) -- Hair strands recoloring. You can also provide one element (like color in the sample). See note about color format above.

Makeup

Apply makeup on the face.

Basic concepts:

Each makeup prefab represents a specific region of the face and has a similar style of settings:

// ...
{
    "color": "0.95 0.70 0.54",
    "finish": "natural",
    "coverage": "mid"
}
// ...

color -- Solid color of the region in RGB string format.

finish -- Type of the finish surface (e.g. natural, matte). See available options for the specific region below.

coverage -- Coverage intensity. Can be low, mid, high or float number in the range [0.0, 1.0].

makeup_base

"faces": [
    {
        "makeup_base":{
            "mode": "quality",
            "smooth": "0 1"
        }
        // ...
    }
    // ...
]

mode (optional) -- Can be speed or quality (to use heavy algorithms). Default value: speed

smooth (optional) -- Smooths face skin. Contains 2 values of the smoothing strength for corresponding regions: whole face and under eyes. Default value: 0 1.

makeup_eyebags

"faces": [
    {
        "makeup_eyebags": {
            "alpha": 0.8
        },
        // ...
    }
    // ...
]

alpha (optional) -- Transparency for eyes circles. Default value: 0.8.

makeup_foundation

"faces": [
    {
        "makeup_foundation": {
            "color": "0.95 0.70 0.54",
            "finish": "natural",
            "coverage": "mid"
        },
        // ...
    }
    // ...
]

finish -- One of: natural, matte, radiance.

makeup_concealer

"faces": [
    {
        "makeup_concealer": {
            "color": "0.94 0.73 0.66",
            "finish": "natural",
            "coverage": 1
        },
        // ...
    }
    // ...
]

finish -- One of: natural, matte.

makeup_contour

"faces": [
    {
        "makeup_contour": {
            "color": "1 0 0",
            "finish": "normal",
            "coverage": "mid"
        },
        // ...
    }
    // ...
]

finish -- One of: normal.

makeup_highlighter

"faces": [
    {
        "makeup_highlighter": {
            "color": "0.9 0.80 0.83",
            "finish": "shimmer",
            "coverage": "mid"
        },
        // ...
    }
    // ...
]

finish -- One of: shimmer.

makeup_blush

"faces": [
    {
        "makeup_blush": {
            "color": "0.88 0.65 0.75",
            "finish": "shimmer",
            "coverage": "mid"
        },
        // ...
    }
    // ...
]

finish -- One of: shimmer, matte, cream_shine.

makeup_lipstick

"faces": [
    {
        "makeup_lipstick": {
            "color": "0.88 0.47 0.61",
            "finish": "shimmer",
            "coverage": "hi"
        },
        // ...
    }
    // ...
]

finish -- One of: shimmer, matte_dry, matte_cream, matte_powder, matte_liquid, cream_shine, glossy_cream_plumping, balm, balm_light, glossy_cream_shimmer, cream_vividcolors, matte_velvet, cream_shine_glitter, matte_velvet_sparkling, matte_sheer_lightcolors, matte_cream_vividcolors, metallic_cream, matte_velvet_sparkling_lightcolors, matte_light, metallic_shine, metallic_dry_lightcolors, satin, shine, cream, clear, cream_darkcolors, clear_shimmer, metallic_sheer.

makeup_lipsliner

"faces": [
    {
        "makeup_lipsliner": {
            "color": "0.99 0.0 0.0",
            "finish": "shimmer",
            "coverage": "hi",
            "liner": "3 5"
        },
        // ...
    }
    // ...
]

finish -- One of: shimmer.

liner (optional) -- Liner options contain 2 values: liner width and softness. Default value: 3 5.

makeup_eyeshadow

"faces": [
    {
        "makeup_eyeshadow": [
                {
                    "color": "0.21 0.42 0.32",
                    "finish": "matte",
                    "coverage": "hi"
                },
                {
                    "color": "0.3 0.58 0.47",
                    "finish": "shimmer",
                    "coverage": "hi"
                },
                {
                    "color": "1.00 0.91 0.27",
                    "finish": "metallic",
                    "coverage": "hi"
                }
            ],
        // ...
    }
    // ...
]

Can be an object with one eyeshadow definition, or an array of the eyeshadows (max length 3) applied one after another.

finish -- One of: shimmer, matte, matte_powder, glitter_metallic, glitter, metallic, glitter_sheer, cream.

makeup_eyelashes

"faces": [
    {
        "makeup_eyelashes": {
            "color": "0 0 0",
            "finish": "volume",
            "coverage": "hi"
        },
        // ...
    }
    // ...
]

finish -- One of: volume, lengthening, lengthandvolume, natural, natural_bottom

makeup_eyeliner

"faces": [
    {
        "makeup_eyeliner": {
            "color": "0.0 0.0 0.0",
            "finish": "matte_liquid",
            "coverage": "hi"
        },
        // ...
    }
    // ...
]

finish -- One of: shimmer, cream, matte_liquid, matte_cream, matte_dark, metallic, shimmer, glitter, cream_lightcolors.

makeup_eyebrows

"faces": [
    {
        "makeup_eyebrows": {
            "color": "0.39 0.26 0.27",
            "finish": "matte",
            "coverage": "mid"
        },
        // ...
    }
    // ...
]

finish -- One of: matte, wet, clear.

makeup_lipsshine

"faces": [
    {
        "makeup_lipsshine": {
            "color": "1.0 0.0 0.0",
            "finish": "glitter",
            "coverage": "mid"
        },
        // ...
    }
    // ...
]

finish -- One of: shine, glitter.

sounds

This array contains audio effects bound to effect. You can enable several audio tracks at once.

audio

Simple audio effect on background, looped by default.

"sounds": [
    {
        "audio": {
            "filename": "sound.ogg",
            "start": 1,
            "end": 4,
            "loop": true,
            "volume": 0.7
        } 
        // ...
    }
    // ...
]

filename -- Path to audio track. Tracks in .ogg and .wav format are supported.

start (optional) -- Timepoint to start audio play (in seconds).

end (optional) -- Timepoint to end audio play (in seconds).

loop (optional) -- Loop the audio track.

volume (optional) -- This audio track volume. Value from [0, 1].

sprites

This array contains sprites based prefabs, like hint

hint

Text message on the screen.

"sprites": [
    {
        "hint": {
            "text": "Hello World!",
            "font": "path/to/font/file",
            "color": "1 1 1 1",
            "size": "100 100",
            "translation": "0 0",
            "rotation": "0 0 0"
        } 
        // ...
    }
    // ...
]

text -- Message will be displayed.

font (optional) -- Path to the font file.

color (optional) -- Color of the text.

size (optional) -- Size of the text area [X, Y].

translation (optional) -- Translate the model along X, Y axes.

rotation (optional) -- Rotation angles (in radians) over X, Y, Z axes.

config.js

This file may contain some business logic written in JavaScript. Refer other documentation pages in this section. The most important feature related to scripting probably Virtual background.

TODO: how to create GLTF