Shader interface

Each material has associated pair of vertex and fragment shaders. Those shaders are GLSL ES 3.0 and should start with #version 300 es (for Mac OS this string is patched to #version 410 core internally as Mac OS doesn't provide OpenGL 4.3 which is needed for GLSL ES 3.0 on desktop GL).

Vertex Attributes

There are two types of materials in effects:

  • FRX face mesh materials (onface=true in cfg.toml)
  • materials for skinned geometry from .bsm2 files

FRX face mesh materials receive 3 attributes:

// vertex position:
layout( location = 0 ) in vec3 attrib_pos;
// texture coordinates:
layout( location = 1 ) in vec2 attrib_uv;
// face segmentation mask, x=skin, y=mouth, z=eyes, w=1:
layout( location = 3 ) in vec4 attrib_red_mask;

.bsm2 materials receive 6 attributes:

// vertex position:
layout( location = 0 ) in vec3 attrib_pos;
// vertex normal:
layout( location = 1 ) in vec3 attrib_n;
// vertex tangent in xyz, and in w +1 or -1 multiplier for reconstruction on bitangent as
// attrib_t.w*cross( attrib_n, )
layout( location = 2 ) in vec4 attrib_t;
// texture coordinates:
layout( location = 3 ) in vec2 attrib_uv;
// skin bone indexes:
layout( location = 4 ) in uvec4 attrib_bones;
// skin bone weights (in sorted order, x >= y >= z >= w):
layout( location = 5 ) in vec4 attrib_weights;

Uniform parameters

Shaders can receive 4 uniform blocks with parameters:

  • glfx_GLOBAL
  • glfx_INSTANCES
  • glfx_BASIS_DATA
layout(std140) uniform glfx_GLOBAL
// modelview-projection matrices for each face:
mat4 glfx_MVPs[GLFX_MAX_FACES];
// projection matrix:
mat4 glfx_PROJ;
// modelview matrices for each face:
mat4 glfx_MVs[GLFX_MAX_FACES];
// camera orientation quaternion:
vec4 glfx_QUAT;
// additional parameters from effect script (passed with Api.meshfxMsg("shaderVec4",...))
vec4 script_params[];

glfx_MVPs and glfx_MVs arrays meant to be indexed with gl_InstanceID for .bsm2 shaders and with glfx_CURRENT_FACE uint uniform for FRX face mesh shaders.

For compatibility with single-face effects it is possible to declare glfx_GLOBAL block as:

layout(std140) uniform glfx_GLOBAL
mat4 glfx_MVP;
mat4 glfx_PROJ;
mat4 glfx_MV;
vec4 glfx_QUAT;
vec4 script_params[];

And shader will be patched automatically during loading.

uniform uint glfx_CURRENT_I;
layout(std140) uniform glfx_INSTANCES
// per-instance data, to be indexed with glfx_CURRENT_I
// x = time since instance spawn, in seconds
// y = time since last instance animation change, in seconds
// z = animation key frame, to be used as row index while sampling glfx_BONES texture
// w = base vertex index
vec4 glfx_IDATA[48];
layout(std140) uniform glfx_BASIS_DATA // also can be accessed as glfx_OCCLUSION_DATA
// reserved for future use
vec4 unused;
// framebuffer width, height, 1/width, 1/height
vec4 glfx_SCREEN;
// NN mask trnasformation info:
// *_T[0].xyz and *_T[1].xyz is a mat2x3 with transposed transform from clip space to mask texcoord
// _T[0].w is a channel index (0,1,2 or 3) in the NN mask texture
// _T[1].w is a 0 or 1 mask inverse flag
vec4 glfx_BG_MASK_T[2];
vec4 glfx_HAIR_MASK_T[2];
vec4 glfx_LIPS_MASK_T[2];
vec4 glfx_L_EYE_MASK_T[2];
vec4 glfx_R_EYE_MASK_T[2];
vec4 glfx_SKIN_MASK_T[2];
vec4 glfx_OCCLUSION_MASK_T[2];
layout(std140) uniform glfx_ACTION_UNITS
// action units weights, order is defined in config.json
// at most there are 51 AU, so 13 vec4 is enough for any case
vec4 glfx_AU[13];

Texture samplers

Shaders can use uniform samplers for textures declared in samplers table in cfg.toml

Additionaly, following built-in samplers are available:

  • uniform sampler2D glfx_BONES for .bsm2 materials this will contain bone transformations as mat3x4. Typical skinning transformation code look like:

    mat3x4 get_bone( uint bone_idx, int y )
    int b = int(bone_idx)*3;
    mat3x4 m = mat3x4(
    texelFetch( glfx_BONES, ivec2(b,y), 0 ),
    texelFetch( glfx_BONES, ivec2(b+1,y), 0 ),
    texelFetch( glfx_BONES, ivec2(b+2,y), 0 ) );
    return m;
    mat3x4 get_transform()
    int y = int(glfx_IDATA[glfx_CURRENT_I].z);
    mat3x4 m = get_bone( attrib_bones[0], y );
    if( attrib_weights[1] > 0. )
    m = m*attrib_weights[0] + get_bone( attrib_bones[1], y )*attrib_weights[1];
    if( attrib_weights[2] > 0. )
    m += get_bone( attrib_bones[2], y )*attrib_weights[2];
    if( attrib_weights[3] > 0. )
    m += get_bone( attrib_bones[3], y )*attrib_weights[3];
    return m;

    For physics meshes this texture have height = 1 and transformation matrices already include modelview transformation. For regular animated (non-physics) .bsm2 matrices in glfx_BONES doesn't include modelview transform.

  • uniform sampler2D glfx_STATICPOS texture with base face positions for Y-flipped face UVs. Typically should be sampled with vec2(attrib_uv.x,1.-attrib_uv.y) texcoords if mesh from .bsm2 have texcoords corresponding to face mesh texcoords.

  • uniform sampler2DArray glfx_UVMORPH textures with position offsets of FRX face meshes from the base face mesh for Y-flipped face UVs. Each layer of sampler2DArray corresponds to each found face. For compatibility with single-face effects it can be declared as uniform sampler2D glfx_UVMORPH and shader will be patched during loading.

  • uniform sampler2DArray glfx_UVMORPH_FISHEYE same as glfx_UVMORPH, but meant to be sampled with vec3(smoothstep(0.,1.,attrib_uv),float(gl_InstanceID)). This texture is smaller than glfx_UVMORPH and results in faster rendering.

  • uniform sampler2DArray glfx_MORPH textures with position offsets of FRX face meshes from the base face mesh for automatic cylindrical mapping:

    vec2 glfx_morph_coord( vec3 v )
    const float half_angle = radians(104.);
    const float y0 = -110.;
    const float y1 = 112.;
    float x = atan( v.x, v.z )/half_angle;
    float y = ((v.y-y0)/(y1-y0))*2. - 1.;
    return vec2(x,y);
  • uniform sampler2D glfx_VIDEO video frame .mp4, which can be declared in config.json

  • uniform sampler2D glfx_BACKGROUND texture with framebuffer copy, can be updated with '!glfx_UPDATE_BG' command in draw_order in cfg.toml. This texture can be created with mipmaps by setting bg_mips=true in cfg.toml

  • uniform sampler2D glfx_BLUR_BACKGROUND blurred version of glfx_BACKGROUND, blur strength can be configured from script with Api.meshfxMsg("blur", 0, strength)

  • uniform sampler2DShadow glfx_SHADOW dynamic shadows sampler with geometry with materials marked shadow=true in cfg.toml. In spherical projection imitating shadows from environment lighting. Meant to be sampled with following texcoords:

    vec3 spherical_proj( vec2 fovM, vec2 fovP, float zn, float zf, vec3 v )
    vec2 xy = (atan( v.xy, v.zz )-(fovP+fovM)*0.5)/((fovP-fovM)*0.5);
    float z = (length(v)-(zn+zf)*0.5)/((zf-zn)*0.5);
    return vec3( xy, z );
    vec2 shadow_texcoord = spherical_proj(
  • uniform sampler2D glfx_BG_MASK background separation mask texture from neural network

  • uniform sampler2D glfx_HAIR_MASK hair mask texture from neural network

  • uniform sampler2D glfx_OCCLUSION occlusion mask texture from neural network

  • uniform sampler2D glfx_L_EYE_MASK, glfx_R_EYE_MASK eyes segmentation mask textures from neural network

  • uniform sampler2D glfx_LIPS_MASK lips segmentation mask textures from neural network

  • uniform sampler2D glfx_SKIN_MASK skin segmentation mask texture from neural network

  • uniform sampler3D glfx_WLUT0, glfx_WLUT1, glfx_WLUT2, glfx_WLUT3 weighted lut as 3D texture which can be configured from script with Api.meshfxMsg("wlut",idx,weight,file)

Fragment shader output

Fragment shaders should write its result to 0 location, for example:

layout( location = 0 ) out vec4 frag_color;

This can be omitted for materials with colorwrite=false in cfg.toml.

Additionaly, on mobiles with EXT_shader_framebuffer_fetch this variable can be declared as inout to allow reading color from framebuffer.

Last updated on