Skip to main content

Shader interface

Each material has associated pair of vertex and fragment shaders. Those shaders are GLSL ES 3.0 and should start with #version 300 es (for Mac OS this string is patched to #version 410 core internally as Mac OS doesn't provide OpenGL 4.3 which is needed for GLSL ES 3.0 on desktop GL).

Vertex Attributes#

There are two types of materials in effects:

  • FRX face mesh materials (onface=true in cfg.toml)
  • materials for skinned geometry from .bsm2 files

FRX face mesh materials receive 3 attributes:

// vertex position:
layout( location = 0 ) in vec3 attrib_pos;
// texture coordinates:
layout( location = 1 ) in vec2 attrib_uv;
// face segmentation mask, x=skin, y=mouth, z=eyes, w=1:
layout( location = 3 ) in vec4 attrib_red_mask;

.bsm2 materials receive 6 attributes:

// vertex position:
layout( location = 0 ) in vec3 attrib_pos;
// vertex normal:
layout( location = 1 ) in vec3 attrib_n;
// vertex tangent in xyz, and in w +1 or -1 multiplier for reconstruction on bitangent as
// attrib_t.w*cross( attrib_n, )
layout( location = 2 ) in vec4 attrib_t;
// texture coordinates:
layout( location = 3 ) in vec2 attrib_uv;
// skin bone indexes:
layout( location = 4 ) in uvec4 attrib_bones;
// skin bone weights (in sorted order, x >= y >= z >= w):
layout( location = 5 ) in vec4 attrib_weights;

Uniform parameters#

Shaders can receive 4 uniform blocks with parameters:

  • glfx_GLOBAL
  • glfx_INSTANCES
  • glfx_BASIS_DATA
layout(std140) uniform glfx_GLOBAL
// modelview-projection matrices for each face:
mat4 glfx_MVPs[GLFX_MAX_FACES];
// projection matrix:
mat4 glfx_PROJ;
// modelview matrices for each face:
mat4 glfx_MVs[GLFX_MAX_FACES];
// camera orientation quaternion:
vec4 glfx_QUAT;
// additional parameters from effect script (passed with Api.meshfxMsg("shaderVec4",...))
vec4 script_params[];

glfx_MVPs and glfx_MVs arrays meant to be indexed with gl_InstanceID for .bsm2 shaders and with glfx_CURRENT_FACE uint uniform for FRX face mesh shaders.

For compatibility with single-face effects it is possible to declare glfx_GLOBAL block as:

layout(std140) uniform glfx_GLOBAL
mat4 glfx_MVP;
mat4 glfx_PROJ;
mat4 glfx_MV;
vec4 glfx_QUAT;
vec4 script_params[];

And shader will be patched automatically during loading.

uniform uint glfx_CURRENT_I;
layout(std140) uniform glfx_INSTANCES
// per-instance data, to be indexed with glfx_CURRENT_I
// x = time since instance spawn, in seconds
// y = time since last instance animation change, in seconds
// z = animation key frame, to be used as row index while sampling glfx_BONES texture
// w = base vertex index
vec4 glfx_IDATA[48];
layout(std140) uniform glfx_BASIS_DATA // also can be accessed as glfx_OCCLUSION_DATA
// reserved for future use
vec4 unused;
// framebuffer width, height, 1/width, 1/height
vec4 glfx_SCREEN;
// NN mask trnasformation info:
// *_T[0].xyz and *_T[1].xyz is a mat2x3 with transposed transform from clip space to mask texcoord
// _T[0].w is a channel index (0,1,2 or 3) in the NN mask texture
// _T[1].w is a 0 or 1 mask inverse flag
vec4 glfx_BG_MASK_T[2];
vec4 glfx_HAIR_MASK_T[2];
vec4 glfx_LIPS_MASK_T[2];
vec4 glfx_L_EYE_MASK_T[2];
vec4 glfx_R_EYE_MASK_T[2];
vec4 glfx_SKIN_MASK_T[2];
vec4 glfx_OCCLUSION_MASK_T[2];
layout(std140) uniform glfx_ACTION_UNITS
// action units weights, order is defined in config.json
// at most there are 51 AU, so 13 vec4 is enough for any case
vec4 glfx_AU[13];

Texture samplers#

Shaders can use uniform samplers for textures declared in samplers table in cfg.toml

Additionaly, following built-in samplers are available:

  • uniform sampler2D glfx_BONES for .bsm2 materials this will contain bone transformations as mat3x4. Typical skinning transformation code look like:

    mat3x4 get_bone( uint bone_idx, int y )
    int b = int(bone_idx)*3;
    mat3x4 m = mat3x4(
    texelFetch( glfx_BONES, ivec2(b,y), 0 ),
    texelFetch( glfx_BONES, ivec2(b+1,y), 0 ),
    texelFetch( glfx_BONES, ivec2(b+2,y), 0 ) );
    return m;
    mat3x4 get_transform()
    int y = int(glfx_IDATA[glfx_CURRENT_I].z);
    mat3x4 m = get_bone( attrib_bones[0], y );
    if( attrib_weights[1] > 0. )
    m = m*attrib_weights[0] + get_bone( attrib_bones[1], y )*attrib_weights[1];
    if( attrib_weights[2] > 0. )
    m += get_bone( attrib_bones[2], y )*attrib_weights[2];
    if( attrib_weights[3] > 0. )
    m += get_bone( attrib_bones[3], y )*attrib_weights[3];
    return m;

    For physics meshes this texture have height = 1 and transformation matrices already include modelview transformation. For regular animated (non-physics) .bsm2 matrices in glfx_BONES doesn't include modelview transform.

  • uniform sampler2D glfx_STATICPOS texture with base face positions for Y-flipped face UVs. Typically should be sampled with vec2(attrib_uv.x,1.-attrib_uv.y) texcoords if mesh from .bsm2 have texcoords corresponding to face mesh texcoords.

  • uniform sampler2DArray glfx_UVMORPH textures with position offsets of FRX face meshes from the base face mesh for Y-flipped face UVs. Each layer of sampler2DArray corresponds to each found face. For compatibility with single-face effects it can be declared as uniform sampler2D glfx_UVMORPH and shader will be patched during loading.

  • uniform sampler2DArray glfx_UVMORPH_FISHEYE same as glfx_UVMORPH, but meant to be sampled with vec3(smoothstep(0.,1.,attrib_uv),float(gl_InstanceID)). This texture is smaller than glfx_UVMORPH and results in faster rendering.

  • uniform sampler2DArray glfx_MORPH textures with position offsets of FRX face meshes from the base face mesh for automatic cylindrical mapping:

    vec2 glfx_morph_coord( vec3 v )
    const float half_angle = radians(104.);
    const float y0 = -110.;
    const float y1 = 112.;
    float x = atan( v.x, v.z )/half_angle;
    float y = ((v.y-y0)/(y1-y0))*2. - 1.;
    return vec2(x,y);
  • uniform sampler2D glfx_VIDEO video frame .mp4, which can be declared in config.json

  • uniform sampler2D glfx_BACKGROUND texture with framebuffer copy, can be updated with '!glfx_UPDATE_BG' command in draw_order in cfg.toml. This texture can be created with mipmaps by setting bg_mips=true in cfg.toml

  • uniform sampler2D glfx_BLUR_BACKGROUND blurred version of glfx_BACKGROUND, blur strength can be configured from script with Api.meshfxMsg("blur", 0, strength)

  • uniform sampler2DShadow glfx_SHADOW dynamic shadows sampler with geometry with materials marked shadow=true in cfg.toml. In spherical projection imitating shadows from environment lighting. Meant to be sampled with following texcoords:

    vec3 spherical_proj( vec2 fovM, vec2 fovP, float zn, float zf, vec3 v )
    vec2 xy = (atan( v.xy, v.zz )-(fovP+fovM)*0.5)/((fovP-fovM)*0.5);
    float z = (length(v)-(zn+zf)*0.5)/((zf-zn)*0.5);
    return vec3( xy, z );
    vec2 shadow_texcoord = spherical_proj(
  • uniform sampler2D glfx_BG_MASK background separation mask texture from neural network

  • uniform sampler2D glfx_HAIR_MASK hair mask texture from neural network

  • uniform sampler2D glfx_OCCLUSION occlusion mask texture from neural network

  • uniform sampler2D glfx_L_EYE_MASK, glfx_R_EYE_MASK eyes segmentation mask textures from neural network

  • uniform sampler2D glfx_LIPS_MASK lips segmentation mask textures from neural network

  • uniform sampler2D glfx_SKIN_MASK skin segmentation mask texture from neural network

  • uniform sampler3D glfx_WLUT0, glfx_WLUT1, glfx_WLUT2, glfx_WLUT3 weighted lut as 3D texture which can be configured from script with Api.meshfxMsg("wlut",idx,weight,file)

Fragment shader output#

Fragment shaders should write its result to 0 location, for example:

layout( location = 0 ) out vec4 frag_color;

This can be omitted for materials with colorwrite=false in cfg.toml.

Additionaly, on mobiles with EXT_shader_framebuffer_fetch this variable can be declared as inout to allow reading color from framebuffer.

Last updated on