Creating effects from scratch

Effect structure

Every Banuba effect is a folder containing files with special meaning. Typical effect will look the following:

effect_folder/
 |-- config.json
 |-- config.js
 |-- images/
 |    |
 |   ...
 |-- shaders/
 |    |
 |   ...
 |-- videos/
 |    |
 |   ...
...

Where only config.json and config.js are mandatory (see bellow). So, in order to start create a folder with two blank files named like this.

Other folders have no special meaning and exist only to group related files, which usually referred from config.json or config.js.

config.json

This is the basic file which describes effect itself. It has he following format:

{
    "scene": "effect name",
    "version": "2.0.0",
    "camera": {},
    "faces": [     
        { // first face
            "prefab1": {
                // "prefab1" parameters
            },
            "prefab2": {
                // "prefab2" parameters
            }
            // ...
        },
        { // second face
        }
        // ..   
    ]
}

Where

• scene -- this is the name of your effect.
• version -- version of this configuration file. Always set it to "2.0.0".
• camera -- tells that you will render camera feed on the screen.
• faces -- array of JSON objects describing features ("prefabs" in our terminology) you wish to place on each face.

So, for each face we will define a JSON object where keys are the names of prefabs and values are the parameters for each prefab.

The basic complete example will look like this (it will render simple "makeup" on the face):

{
    "scene": "Retouch example",
    "version": "2.0.0",
    "camera": {},
    "faces": [     
        {
            "retouch": {}
        }
    ]
}

Another good example to start:

Download simple 3D model sample

Each prefab and possible parameters for them are described below.

action_units

"action_units": {}

Enable action units from GLTF model.

gltf

"gltf": {
    "mesh": "path/to/gltf/model",
    "rotation": [0.0, 0.0, 0.0],
    "scale": [1.0, 1.0, 1.0],
    "translation": [0.0, 0.0, 0.0]
}

Place 3D model in GLTF format on the face.

mesh -- path to GLTF model.

rotation (optional) -- rotation angles (in radians) over X, Y, Z axes.

scale (optional) -- scale along X, Y, Z axes.

translation (optional) -- translate the model along X, Y, Z axes (in millimetres).

morphing

"morphing": {
    "eyes_weight": 0.0,
    "nose_weight": 0.0,
    "face_weight": 0.0
}

Morph (i.e. deform) certain parts of the face.

eyes_weight (optional) -- positive values will make eyes bigger and negative values will make them smaller. Weights between 0 and 1 look more natural.

nose_weight (optional) -- positive values will make nose thinner and negative values will make it thicket. Weights between 0 and 1 look more natural.

face_weight (optional) -- just the same as nose_weight but applies to face.

retouch

"retouch": {
    "eyes_whitening": {
        "value": [0.5, 0.0, 0.0, 0.0],
        "type": "vec4"
    },
    "skin_soft": {
        "value": [0.7, 0.0, 0.0, 0.0],
        "type": "vec4"
    },
    "teeth_sharpen": {
        "value": [0.2, 0.0, 0.0, 0.0],
        "type": "vec4"
    },
    "eyes_sharpen": {
        "value": [0.3, 0.0, 0.0, 0.0],
        "type": "vec4"
    }
}

Apply retouch (i.e. basic "beautification") on the face. Each parameter is a 4-component vector but only first coordinate has the meaning (other are ignored).

eyes_whitening (optional) -- make eyes whiter.

skin_soft" (optional) -- level of skin smoothing.

teeth_sharpen (optional) -- put visual accent on teeth.

eyes_sharpen (optional) -- push visual accent on eyes.

config.js

This file may contain some business logic written in JavaScript. Refer other documentation pages in this section. The most important feature related to scripting probably Virtual background.

TODO: how to create GLTF