# Banuba Face AR SDK Documentation - Version 1.18.5 > Docs provides full information about Banuba Face AR SDK, integration and customization guides. ## far-sdk Discover Banuba Face AR SDK and your possibilities to build AR apps - [Banuba Face AR SDK](/index.md): Discover Banuba Face AR SDK and your possibilities to build AR apps ### search - [Search the documentation](/search.md) ### support - [Contact Support](/support.md) ### api_docs API documentation iOS (Swift) - [API Documentation](/api_docs.md): API documentation iOS (Swift) ### effects #### guides ##### feature_params Since v1.6.1, Banuba SDK provides an oportunity to change some feature parameters using scripting engine: - [How to change feature parameters using scripting api](/effects/guides/feature_params.md): Since v1.6.1, Banuba SDK provides an oportunity to change some feature parameters using scripting engine: ##### hand_ar_hand_gestures Learn how to utilize hand gesture tracking with Banuba API. - [How to use the Hand gestures feature](/effects/guides/hand_ar_hand_gestures.md): Learn how to utilize hand gesture tracking with Banuba API. #### makeup_deprecated ##### face_beauty Banuba provides the Face Beauty API designed to help you integrate augmented reality beauty try-on features into iOS and Android apps - [Face Beauty API](/effects/makeup_deprecated/face_beauty.md): Banuba provides the Face Beauty API designed to help you integrate augmented reality beauty try-on features into iOS and Android apps ##### makeup Banuba provides the Virtual Makeup API designed to help developers integrate augmented reality beauty try-on features into their iOS and Android apps - [Virtual Makeup API](/effects/makeup_deprecated/makeup.md): Banuba provides the Virtual Makeup API designed to help developers integrate augmented reality beauty try-on features into their iOS and Android apps ##### makeup_usage Learn how to combine the Makeup API and Beauty API features and use them in your application. - [How to combine the Makeup API and Beauty API features](/effects/makeup_deprecated/makeup_usage.md): Learn how to combine the Makeup API and Beauty API features and use them in your application. #### overview You may also create your own effects with - [Effect structure](/effects/overview.md): You may also create your own effects with #### prefabs ##### face GLTF - [On Face Prefabs](/effects/prefabs/face.md): GLTF ##### hands Nails - [On Hands Prefabs](/effects/prefabs/hands.md): Nails ##### makeup Basic Concepts - [Makeup Prefabs](/effects/prefabs/makeup.md): Basic Concepts ##### overview A prefab is a high-level object that represents a set of rendering and SDK features. - [Prefabs Overview](/effects/prefabs/overview.md): A prefab is a high-level object that represents a set of rendering and SDK features. ##### sounds Sounds - [Sounds Prefabs](/effects/prefabs/sounds.md): Sounds ##### sprites Sprites - [Sprites Prefabs](/effects/prefabs/sprites.md): Sprites ##### top_level Background - [Top Level Prefabs](/effects/prefabs/top_level.md): Background #### virtual_background Banuba provides the Virtual Background API designed to help developers integrate augmented reality background separation into their apps - [Virtual Background API](/effects/virtual_background.md): Banuba provides the Virtual Background API designed to help developers integrate augmented reality background separation into their apps ### support_page - Dev Portal - [Support](/support_page.md): - Dev Portal ### tutorials #### capabilities ##### 3rd_licenses A list of third party libraries used within Banuba SDK - [Third parties library list](/tutorials/capabilities/3rd_licenses.md): A list of third party libraries used within Banuba SDK ##### demo_face_filters List of Demo Face Filters and technologies represented with them. - [Demo Face Filters](/tutorials/capabilities/demo_face_filters.md): List of Demo Face Filters and technologies represented with them. ##### glossary AR Technologies - [FaceAR Glossary](/tutorials/capabilities/glossary.md): AR Technologies ##### sdk_features A list of Banuba Face AR SDK features and platforms supported - [SDK Features](/tutorials/capabilities/sdk_features.md): A list of Banuba Face AR SDK features and platforms supported ##### system_requirements Supported Platforms - [System Requirements](/tutorials/capabilities/system_requirements.md): Supported Platforms ##### technical_specification Technical specification and minimal requirements of the Banuba Face AR SDK features. - [Technical Specification](/tutorials/capabilities/technical_specification.md): Technical specification and minimal requirements of the Banuba Face AR SDK features. ##### token_management This section will provide the reader with the answers from the user’s frequently asked questions related to the token management process. - [Token Management](/tutorials/capabilities/token_management.md): This section will provide the reader with the answers from the user’s frequently asked questions related to the token management process. #### changelog [1.18.5] - 2026-08-10 - [Changelog](/tutorials/changelog.md): [1.18.5] - 2026-08-10 #### development ##### api_overview API Overview - [API Overview](/tutorials/development/api_overview.md): API Overview ###### android image - [android](/tutorials/development/api_overview/android.md): image ###### desktop image - [desktop](/tutorials/development/api_overview/desktop.md): image ###### ios image - [ios](/tutorials/development/api_overview/ios.md): image ###### web image - [web](/tutorials/development/api_overview/web.md): image ##### basic_integration Getting Started guide for Banuba SDK - [Getting Started](/tutorials/development/basic_integration.md): Getting Started guide for Banuba SDK ###### android Installation - [android](/tutorials/development/basic_integration/android.md): Installation ###### desktop The steps below apply to desktop integration (Windows and/or macOS) with C++. - [desktop](/tutorials/development/basic_integration/desktop.md): The steps below apply to desktop integration (Windows and/or macOS) with C++. ###### flutter Banuba SDK for - [flutter](/tutorials/development/basic_integration/flutter.md): Banuba SDK for ###### ios Installation - [ios](/tutorials/development/basic_integration/ios.md): Installation ###### react_native Banuba SDK for - [react_native](/tutorials/development/basic_integration/react_native.md): Banuba SDK for ###### web Requirements - [web](/tutorials/development/basic_integration/web.md): Requirements ##### guides ###### ar_cloud A guide on how to use AR cloud in the SDK - [AR Cloud Guide](/tutorials/development/guides/ar_cloud.md): A guide on how to use AR cloud in the SDK ###### landmarks A guide on how to get face landmarks - [Face Landmarks Guide](/tutorials/development/guides/landmarks.md): A guide on how to get face landmarks ###### migration To version 1.17.0 - [Migration Guides](/tutorials/development/guides/migration.md): To version 1.17.0 ###### optimization Optimizing WebAR SDK bundle size - [Optimization Guides](/tutorials/development/guides/optimization.md): Optimizing WebAR SDK bundle size ###### watermark How to apply a watermark to a video - [Watermark Guide](/tutorials/development/guides/watermark.md): How to apply a watermark to a video ##### installation A getting started guide on how to add Banuba SDK to a project - [Adding Banuba SDK to your project](/tutorials/development/installation.md): A getting started guide on how to add Banuba SDK to a project ###### android Packages - [android](/tutorials/development/installation/android.md): Packages ###### desktop Banuba SDK for desktop platforms (i.e. Windows and MacOS) is distributed via - [desktop](/tutorials/development/installation/desktop.md): Banuba SDK for desktop platforms (i.e. Windows and MacOS) is distributed via ###### ios CocoaPods packages - [ios](/tutorials/development/installation/ios.md): CocoaPods packages ###### web NPM Package - [web](/tutorials/development/installation/web.md): NPM Package ##### known_issues Visit our FAQ or contact our support. - [Known Issues](/tutorials/development/known_issues.md): Visit our FAQ or contact our support. ###### web MediaStreamCapture stream freezes when a browser tab becomes inactive in Safari - [web](/tutorials/development/known_issues/web.md): MediaStreamCapture stream freezes when a browser tab becomes inactive in Safari ##### llms Use Banuba AI Skills and an AI coding agent like Claude Code to build a working Face AR Web demo in minutes. - [Build a Face AR App in Minutes with AI](/tutorials/development/llms.md): Use Banuba AI Skills and an AI coding agent like Claude Code to build a working Face AR Web demo in minutes. ##### samples A getting started guide for Banuba SDK - [Examples of using Banuba SDK](/tutorials/development/samples.md): A getting started guide for Banuba SDK ###### android Requirements - [android](/tutorials/development/samples/android.md): Requirements ###### desktop Examples bellow are written in C++ and will run both on Windows and macOS. - [desktop](/tutorials/development/samples/desktop.md): Examples bellow are written in C++ and will run both on Windows and macOS. ###### flutter Minimal sample - [flutter](/tutorials/development/samples/flutter.md): Minimal sample ###### ios iOS samples (Swift) - [ios](/tutorials/development/samples/ios.md): iOS samples (Swift) ###### macos macOS sample (Swift) - [macos](/tutorials/development/samples/macos.md): macOS sample (Swift) ###### react_native Minimal sample - [react_native](/tutorials/development/samples/react_native.md): Minimal sample ###### web Quickstart - [web](/tutorials/development/samples/web.md): Quickstart ##### videocall A guide on how to integrate video calling in a project with Banuba SDK - [Using video calls with the Banuba SDK](/tutorials/development/videocall.md): A guide on how to integrate video calling in a project with Banuba SDK ###### android Receive Banuba SDK camera frames as an RGBA pixel array on Android for use in a video call. - [android](/tutorials/development/videocall/android.md): Receive Banuba SDK camera frames as an RGBA pixel array on Android for use in a video call. ###### flutter Due to Flutter limitations, for every videocall solution you have to create a native Flutter plugin. We have developed one for integration with - [flutter](/tutorials/development/videocall/flutter.md): Due to Flutter limitations, for every videocall solution you have to create a native Flutter plugin. We have developed one for integration with ###### ios Install and integrate the Banuba SDK video-call components on iOS. - [ios](/tutorials/development/videocall/ios.md): Install and integrate the Banuba SDK video-call components on iOS. ###### react_native Due to React Native limitations, for every videocall solution you have to create a React native module. We developed one for integration with - [react_native](/tutorials/development/videocall/react_native.md): Due to React Native limitations, for every videocall solution you have to create a React native module. We developed one for integration with ###### web Agora - [web](/tutorials/development/videocall/web.md): Agora #### unity ##### basic_integration Discover how to launch Banuba Face AR SDK for Unity - [Banuba Face AR SDK for Unity](/tutorials/unity/basic_integration.md): Discover how to launch Banuba Face AR SDK for Unity ##### demo_scene Banuba SDK provides a Demo Scene for our Unity SDK. - [Unity Demo Scene](/tutorials/unity/demo_scene.md): Banuba SDK provides a Demo Scene for our Unity SDK. ##### overview Discover how to launch Banuba Face AR SDK for Unity - [Face AR SDK for Unity Overview](/tutorials/unity/overview.md): Discover how to launch Banuba Face AR SDK for Unity ##### videocall The example of Banuba SDK and Agora.io SDK integration to enable augmented reality filters in video calls for Unity. - [Using video calls with the Banuba SDK](/tutorials/unity/videocall.md): The example of Banuba SDK and Agora.io SDK integration to enable augmented reality filters in video calls for Unity. --- # Full Documentation Content [Skip to main content](#__docusaurus_skipToContent_fallback) [![Docs logo](/far-sdk/logo.png)![Docs logo](/far-sdk/logo.png)](/far-sdk/.md) [**Banuba SDK**](/far-sdk/.md)[v1.18.5](/far-sdk/.md) [Tutorials](/far-sdk/.md)[Effects](/far-sdk/effects/overview.md)[API Reference](/far-sdk/api_docs.md)[Support](/far-sdk/support_page.md) Search # Search the documentation Type your search here Powered by[](https://www.algolia.com/) --- # Contact Support --- # API Documentation [View as Markdown](https://docs.banuba.com/far-sdk/api_docs.md)[![](/far-sdk/img/ai-guide/chatgpt.svg)Open in ChatGPT](https://chatgpt.com/?q=Read%20https%3A%2F%2Fdocs.banuba.com%2Ffar-sdk%2Fapi_docs.md%20\(API%20Documentation\)%20and%20help%20me%20with%20it%20in%20my%20Banuba%20Face%20AR%20SDK%20project.)[![](/far-sdk/img/ai-guide/claude.svg)Open in Claude](https://claude.ai/new?q=Read%20https%3A%2F%2Fdocs.banuba.com%2Ffar-sdk%2Fapi_docs.md%20\(API%20Documentation\)%20and%20help%20me%20with%20it%20in%20my%20Banuba%20Face%20AR%20SDK%20project.)Install tools * iOS * Android * Web * macOS * Windows * Flutter * React Native [API documentation iOS (Swift)](/far-sdk/generated/jazzy/banuba_sdk/) [Low-level API (Objectve-C)](/far-sdk/generated/jazzy/objc/) [API Documentation for Android (Java)](/far-sdk/generated/javadoc/banuba_sdk/) [Low-level API](/far-sdk/generated/javadoc/core/) [API documentation for Web (Type Script)](/far-sdk/generated/typedoc/) [API (Objectve-C)](/far-sdk/generated/jazzy/objc/) [API (C++)](/far-sdk/generated/doxygen/html/) [API Documentation for Windows (C++)](/far-sdk/generated/doxygen/html/) [API Documentation for Flutter (Dart)](https://pub.dev/documentation/banuba_sdk/) React Native API is quite limited and described in [this file](https://github.com/Banuba/banuba-sdk-react-native/blob/master/src/NativeBanubaSdkManager.ts) --- # How to change feature parameters using scripting api [View as Markdown](https://docs.banuba.com/far-sdk/effects/guides/feature_params.md)[![](/far-sdk/img/ai-guide/chatgpt.svg)Open in ChatGPT](https://chatgpt.com/?q=Read%20https%3A%2F%2Fdocs.banuba.com%2Ffar-sdk%2Feffects%2Fguides%2Ffeature_params.md%20\(How%20to%20change%20feature%20parameters%20using%20scripting%20api\)%20and%20help%20me%20with%20it%20in%20my%20Banuba%20Face%20AR%20SDK%20project.)[![](/far-sdk/img/ai-guide/claude.svg)Open in Claude](https://claude.ai/new?q=Read%20https%3A%2F%2Fdocs.banuba.com%2Ffar-sdk%2Feffects%2Fguides%2Ffeature_params.md%20\(How%20to%20change%20feature%20parameters%20using%20scripting%20api\)%20and%20help%20me%20with%20it%20in%20my%20Banuba%20Face%20AR%20SDK%20project.)Install tools Since v1.6.1, Banuba SDK provides an oportunity to change some feature parameters using scripting engine: ``` bnb.scene.addFeatureParam(bnb.FeatureID.ID, []) ``` The first parameter is the feature id with a type bnb.FeatureID. The second parameter is the Array of bnb.FeatureParameter. bnb.FeatureParameter is similiar to vector4 type and contains x,y,z,w fields. ## List of all features with changing parameters[​](#list-of-all-features-with-changing-parameters "Direct link to List of all features with changing parameters") | feature | type name | | ------- | ------------------- | | ring | bnb.FeatureID.RING | | nails | bnb.FeatureID.NAILS | ## Ring[​](#ring "Direct link to Ring") As an input, the Ring feature takes only the first element of the array where x is the id of selected finger. At the the moment, this feature supports 4 fingers: * Index - 0 * Middle - 1 * Ring - 2 * Small - 3 Here is example of how to set the middle finger: * config.js * Java * Swift ``` let middle_finger_id = 1; let param = new bnb.FeatureParameter(middle_finger_id,0,0,0); bnb.scene.addFeatureParam(bnb.FeatureID.RING, [param]) ``` ``` // Effect mCurrentEffect = ... String script = "" " let middle_finger_id = 1; let param = new bnb.FeatureParameter(middle_finger_id, 0, 0, 0); bnb.scene.addFeatureParam(bnb.FeatureID.RING, [param]) "" "; mCurrentEffect.evalJs(script, null); ``` ``` // var currentEffect: BNBEffect = ... let script = """ let middle_finger_id = 1; let param = new bnb.FeatureParameter(middle_finger_id,0,0,0); bnb.scene.addFeatureParam(bnb.FeatureID.RING, [param]) """; currentEffect?.evalJs(script, resultCallback: nil) ``` --- # How to use the Hand gestures feature [View as Markdown](https://docs.banuba.com/far-sdk/effects/guides/hand_ar_hand_gestures.md)[![](/far-sdk/img/ai-guide/chatgpt.svg)Open in ChatGPT](https://chatgpt.com/?q=Read%20https%3A%2F%2Fdocs.banuba.com%2Ffar-sdk%2Feffects%2Fguides%2Fhand_ar_hand_gestures.md%20\(How%20to%20use%20the%20Hand%20gestures%20feature\)%20and%20help%20me%20with%20it%20in%20my%20Banuba%20Face%20AR%20SDK%20project.)[![](/far-sdk/img/ai-guide/claude.svg)Open in Claude](https://claude.ai/new?q=Read%20https%3A%2F%2Fdocs.banuba.com%2Ffar-sdk%2Feffects%2Fguides%2Fhand_ar_hand_gestures.md%20\(How%20to%20use%20the%20Hand%20gestures%20feature\)%20and%20help%20me%20with%20it%20in%20my%20Banuba%20Face%20AR%20SDK%20project.)Install tools This tutorial will guide you on how to get and use hand gesture triggers using Banuba API. The API provides ready-made documented methods making it easy for developers to call AR features in their apps. At the the moment, our algorithms are able to recognize 5 hand gestures: * Like πŸ‘ * Ok πŸ‘Œ * Palm βœ‹ * Rock 🀘 * Victory/Peace ✌️ [](pathname:///generated/effects/test_gestures.zip) [Download example](pathname:///generated/effects/test_gestures.zip) note Hand gesture tracking requires the [Hand gesture tracking neural network](/far-sdk/tutorials/capabilities/sdk_features.md). Please, make sure your licence plan includes one by contacting your sales manager or fill in the website form to request it. ## Integrating the hand gesture tracking feature into your app[​](#integrating-the-hand-gesture-tracking-feature-into-your-app "Direct link to Integrating the hand gesture tracking feature into your app") 1. [Build your project with Banuba SDK](/far-sdk/tutorials/development/basic_integration.md). 2. Use the `test_gestures` effect or your version of this effect to enable the hand gestures feature. If you are using the SDK demo app, just copy (if it's not already there) the `test_gestures` effect folder to `effects` folder and load it on demand using the code below: * Swift * Java ``` // private BanubaSdkManager mSdkManager; // ... mSdkManager.loadEffect("test_gestures") ``` ``` // private let sdkManager = BanubaSdkManager() // ... sdkManager.loadEffect("test_gestures") ``` note Hand gesture tracking with the trigger function being called in config.js may be added to any of your effects. It can be done by simply adding the "hand\_gestures" option to the "recognizer" property array in the config.json file. 3. Process the hand gestures tracking data we receive in the Effect. There is a trigger function being called in JS by the SDK every frame while any of the hand gestures are being recognized: * `setGesture(json)` - this function recieves a json with an idx of a currently recognized gesture. Its possible values are: * 0 - none, * 1 - like, * 2 - ok, * 3 - palm, * 4 - rock, * 5 - victory/peace. You may extend this trigger function behaviour by adding your custom logic to config.js from the "test\_gestures" effect folder. E.g. play a guitar solo audio every time someone shows the "rock": ``` function setGesture(json){ var gestureInfo = JSON.parse(json); switch (gestureInfo.idx) { case 4: Api.playSound("rockme.ogg", false, 1); break; default: break; } } ``` The only limit is your creativity! 4. Now you can run your application to test hand gestures tracking and your custom logic. --- # Face Beauty API [View as Markdown](https://docs.banuba.com/far-sdk/effects/makeup_deprecated/face_beauty.md)[![](/far-sdk/img/ai-guide/chatgpt.svg)Open in ChatGPT](https://chatgpt.com/?q=Read%20https%3A%2F%2Fdocs.banuba.com%2Ffar-sdk%2Feffects%2Fmakeup_deprecated%2Fface_beauty.md%20\(Face%20Beauty%20API\)%20and%20help%20me%20with%20it%20in%20my%20Banuba%20Face%20AR%20SDK%20project.)[![](/far-sdk/img/ai-guide/claude.svg)Open in Claude](https://claude.ai/new?q=Read%20https%3A%2F%2Fdocs.banuba.com%2Ffar-sdk%2Feffects%2Fmakeup_deprecated%2Fface_beauty.md%20\(Face%20Beauty%20API\)%20and%20help%20me%20with%20it%20in%20my%20Banuba%20Face%20AR%20SDK%20project.)Install tools danger **This feature is deprecated**. [We recommend you to use Prefabs](/far-sdk/effects/prefabs/overview.md) Banuba provides the Beauty API designed to help you integrate face modification and touch-up functionality into your app. The beautification features fit into a variety of use cases, e.g. live streaming, video chats and video conferencing apps, selfie editors, and portrait retouching software. It aims to make the user feel comfortable about the camera experience. [](pathname:///generated/effects/Makeup.zip) [Download example](pathname:///generated/effects/Makeup.zip) tip [Read more about how to use and combine](/far-sdk/effects/makeup_deprecated/makeup_usage.md) Makeup API and Face Beauty API features. note Please [contact us](/far-sdk/support/.md) if you wish to use Makeup API features in iOS version 14.x. The Beauty module allows to enhance the face via the following built-in features: ## Teeth Whitening[​](#teeth-whitening "Direct link to Teeth Whitening") Allows for a beautiful smile. * `Teeth.whitening(n)` - changes whitening texture intensity, where **n** - any value from 0 to 1 (including decimal). - config.js - Java - Swift - JavaScript ``` /* Feel free to add your custom code below */ // set teeth whitening strength Teeth.whitening(1) ``` ``` // Effect mCurrentEffect = ... // set teeth whitening strength mCurrentEffect.evalJs("Teeth.whitening(1)", null); ``` ``` // var currentEffect: BNBEffect = ... // set teeth whitening strength currentEffect?.evalJs("Teeth.whitening(1)", resultCallback: nil) ``` ``` // set teeth whitening strength await effect.evalJs("Teeth.whitening(1)") ``` **Preview** ![Right image compare](/far-sdk/assets/images/original-8823b66ca7c9e5a54656f6ee9b77c29f.jpg)![Left image compare](/far-sdk/assets/images/teeth-ad89bfabc20c76750c21b2348693680b.jpg) Drag ## Face morphing[​](#face-morphing "Direct link to Face morphing") Slims down the cheeks and nose to make it more delicate. * `FaceMorph.eyebrows({spacing: value, height: value, bend: value})` - set eyebrows morph params: spacing - Adjusting the space between the eyebrows \[-1;1], height - Raising/lowering the eyebrows \[-1;1], bend - Adjusting the bend of the eyebrows \[-1;1] * `FaceMorph.eyes({rounding: value, enlargement: value, height: value, spacing: value, squint: value, lower_eyelid_pos: value, lower_eyelid_size: value, down: value, eyelid_upper: value, eyelid_lower: value})` - set eyes morph params: rounding - Adjusting the roundness of the eyes \[0;1], enlargement - Enlarging the eyes \[0;1], height - Raising/lowering the eyes \[-1;1], spacing - Adjusting the space between the eyes \[-1;1], squint - Making the person squint by adjusting the eyelids \[-1;1], lower\_eyelid\_pos - Raising/lowering the lower eyelid \[-1;1], lower\_eyelid\_size - Enlarging/shrinking the lower eyelid \[-1;1], down - Eyes down \[0;1], eyelid\_upper - Eyelid upper \[0;1], eyelid\_lower - Eyelid lower \[0;1] * `FaceMorph.nose({width: value, length: value, tip_width: value, down_up: value, sellion: value})` - set nose morph params: width - Adjusting the nose width \[-1;1], length - Adjusting the nose length \[-1;1], tip\_width - Adjusting the nose tip width \[0;1], down\_up - Nose down/up \[0;1], sellion - Nose sellion \[0;1] * `FaceMorph.lips({size: value, height: value, thickness: value, mouth_size: value, smile: value, shape: value, sharp: value})` - set lips morph params: size - Adjusting the width and vertical size of the lips \[-1;1], height - Raising/lowering the lips \[-1;1], thickness - Adjusting the thickness of the lips \[-1;1], mouth\_size - Adjusting the size of the mouth \[-1;1], smile - Making a person smile \[0;1], shape - Adjusting the shape of the lips \[-1;1], sharp - Lips Sharp \[0;1] * `FaceMorph.face({narrowing: value, v_shape: value, cheekbones_narrowing: value, cheeks_narrowing: value, jaw_narrowing: value, chin_shortening: value, chin_narrowing: value, sunken_cheeks: value, cheeks_jaw_narrowing: value, jaw_wide_thin: value, chin: value, forehead: value})` - set face morph params: narrowing - Narrowing the face \[0;1], v\_shape - Shrinking the chin and narrowing the cheeks \[0;1], chekbones\_narrowing - Narrowing the cheekbones \[-1;1], cheeks\_narrowing - Narrowing the cheeks \[0;1], jaw\_narrowing - Narrowing the jaw \[0;1], chin\_shortening - Decreasing the length of the chin \[0;1], chin\_narrowing - Narrowing the chin \[0;1], sunken\_cheeks - Sinking the cheeks and emphasizing the cheekbones \[0;1], cheeks\_jaw\_narrowing - Narrowing the cheeks and the jaw \[0;1], jaw\_wide\_thin - Jaw wide/thin \[0;1], chin - Face Chin \[0;1], forehead - Forehead \[0;1] * `FaceMorph.clear()`- resets morph - config.js - Java - Swift - JavaScript ``` // Set FaceMorph effects FaceMorph.eyebrows({spacing: 0.6, height: 0.3, bend: 1.0}) FaceMorph.eyes({rounding: 0.6, enlargement: 0.3, height: 0.3, spacing: 0.3, squint: 0.3, lower_eyelid_pos: 0.3, lower_eyelid_size: 0.3}) FaceMorph.face({narrowing: 0.6, v_shape: 0.3, cheekbones_narrowing: 0.3, cheeks_narrowing: 0.3, jaw_narrowing: 0.7, chin_shortening: 0.3, chin_narrowing: 0.3, sunken_cheeks: 0.1, cheeks_jaw_narrowing: 0.2}) FaceMorph.nose({width: 0.3, length: 0.2, tip_width: 0.1}) FaceMorph.lips({size: 0.4, height: 1.0, thickness: 0.1, mouth_size: 0.2, smile: 0.8, shape: 0.4}) // Reset all the FaceMorph effects FaceMorph.clear() ``` ``` // Effect mCurrentEffect = ... // Set FaceMorph effects mCurrentEffect.evalJs("FaceMorph.eyebrows({spacing: 0.6, height: 0.3, bend: 1.0})", null); mCurrentEffect.evalJs("FaceMorph.eyes({rounding: 0.6, enlargement: 0.3, height: 0.3, spacing: 0.3, squint: 0.3, lower_eyelid_pos: 0.3, lower_eyelid_size: 0.3})", null); mCurrentEffect.evalJs("FaceMorph.face({narrowing: 0.6, v_shape: 0.3, cheekbones_narrowing: 0.3, cheeks_narrowing: 0.3, jaw_narrowing: 0.7, chin_shortening: 0.3, chin_narrowing: 0.3, sunken_cheeks: 0.1, cheeks_jaw_narrowing: 0.2})", null); mCurrentEffect.evalJs("FaceMorph.nose({width: 0.3, length: 0.2, tip_width: 0.1})", null); mCurrentEffect.evalJs("FaceMorph.lips({size: 0.4, height: 1.0, thickness: 0.1, mouth_size: 0.2, smile: 0.8, shape: 0.4})", null); // Reset FaceMorph effects mCurrentEffect.evalJs("FaceMorph.clear()", null); ``` ``` // var currentEffect: BNBEffect = ... // Set FaceMorph effects currentEffect?.evalJs("FaceMorph.eyebrows({spacing: 0.6, height: 0.3, bend: 1.0})", resultCallback: nil) currentEffect?.evalJs("FaceMorph.eyes({rounding: 0.6, enlargement: 0.3, height: 0.3, spacing: 0.3, squint: 0.3, lower_eyelid_pos: 0.3, lower_eyelid_size: 0.3})", resultCallback: nil) currentEffect?.evalJs("FaceMorph.face({narrowing: 0.6, v_shape: 0.3, cheekbones_narrowing: 0.3, cheeks_narrowing: 0.3, jaw_narrowing: 0.7, chin_shortening: 0.3, chin_narrowing: 0.3, sunken_cheeks: 0.1, cheeks_jaw_narrowing: 0.2})", resultCallback: nil) currentEffect?.evalJs("FaceMorph.nose({width: 0.3, length: 0.2, tip_width: 0.1})", resultCallback: nil) currentEffect?.evalJs("FaceMorph.lips({size: 0.4, height: 1.0, thickness: 0.1, mouth_size: 0.2, smile: 0.8, shape: 0.4})", resultCallback: nil) // Reset FaceMorph effects currentEffect?.evalJs("FaceMorph.clear()", resultCallback: nil) ``` ``` // Set FaceMorph effects await effect.evalJs("FaceMorph.eyebrows({spacing: 0.6, height: 0.3, bend: 1.0})") await effect.evalJs("FaceMorph.eyes({rounding: 0.6, enlargement: 0.3, height: 0.3, spacing: 0.3, squint: 0.3, lower_eyelid_pos: 0.3, lower_eyelid_size: 0.3})") await effect.evalJs("FaceMorph.face({narrowing: 0.6, v_shape: 0.3, cheekbones_narrowing: 0.3, cheeks_narrowing: 0.3, jaw_narrowing: 0.7, chin_shortening: 0.3, chin_narrowing: 0.3, sunken_cheeks: 0.1, cheeks_jaw_narrowing: 0.2})") await effect.evalJs("FaceMorph.nose({width: 0.3, length: 0.2, tip_width: 0.1})") await effect.evalJs("FaceMorph.lips({size: 0.4, height: 1.0, thickness: 0.1, mouth_size: 0.2, smile: 0.8, shape: 0.4})") // Reset FaceMorph effects await effect.evalJs("FaceMorph.clear()") ``` **Preview** ![Right image compare](/far-sdk/assets/images/original2-f74f044be90519e96c2b58b2db7ddce2.jpg)![Left image compare](/far-sdk/assets/images/morph-082d96aed26822a507fd45581b8b49f4.jpg) Drag ## Photo filters (LUTs)[​](#photo-filters-luts "Direct link to Photo filters (LUTs)") Applies color filters to the entire image. * `Filter.set("lut_texture.png")` - set lut filter texture, * `Filter.strength(n)` - set lut filter strength, where **n** - any value from 0 to 1 (including decimal), but also larger values may be passed, like 2, 3, etc; * `Filter.clear()`- clears filter. - config.js - Java - Swift - JavaScript ``` // Set lut filter characteristics Filter.set("lut_texture.png") Filter.strength(1) // Clear filter Filter.clear() ``` ``` // Effect mCurrentEffect = ... // Set lut filter characteristics mCurrentEffect.evalJs("Filter.set('lut_texture.png')", null); mCurrentEffect.evalJs("Filter.strength(1)", null); // Clear filter mCurrentEffect.evalJs("Filter.clear()", null); ``` ``` // var currentEffect: BNBEffect = ... // Set lut filter characteristics currentEffect?.evalJs("Filter.set('lut_texture.png')", resultCallback: nil) currentEffect?.evalJs("Filter.strength(1)", resultCallback: nil) // Clear filter currentEffect?.evalJs("Filter.clear()", resultCallback: nil) ``` ``` // Set lut filter characteristics await effect.evalJs("Filter.set('lut_texture.png')") await effect.evalJs("Filter.strength(1)") // Clear filter await effect.evalJs("Filter.clear()") ``` **Preview** ![Right image compare](/far-sdk/assets/images/original-8823b66ca7c9e5a54656f6ee9b77c29f.jpg)![Left image compare](/far-sdk/assets/images/Filter-db9e7fe33e0c0309c76a9abad0226df7.jpg) Drag ## Skin[​](#skin "Direct link to Skin") ### Skin smoothing (Skin softening)[​](#skin-smoothing-skin-softening "Direct link to Skin smoothing (Skin softening)") Makes the skin look younger by smoothing wrinkles. * `Skin.softening(n)` - set softening intensity, where **n** - any value from 0 to 1 (including decimal). - config.js - Java - Swift - JavaScript ``` /* Feel free to add your custom code below */ Skin.softening(1) ``` ``` // Effect mCurrentEffect = ... mCurrentEffect.evalJs("Skin.softening(1)", null); ``` ``` // var currentEffect: BNBEffect = ... currentEffect?.evalJs("Skin.softening(1)", resultCallback: nil) ``` ``` await effect.evalJs("Skin.softening(1)") ``` **Preview** ![Right image compare](/far-sdk/assets/images/original-8823b66ca7c9e5a54656f6ee9b77c29f.jpg)![Left image compare](/far-sdk/assets/images/SkinSoftening-e7800133c7d654b8372366266bdbf026.jpg) Drag ### Skin color[​](#skin-color "Direct link to Skin color") Changes the face and neck skin color. * `Skin.color("R G B A")` - set skin color in R G B A format (separated with space). Each value should be in a range from 0 to 1 (including decimal), * `Skin.clear()` - clears skin color and softening. info Requires [Skin segmentation](/far-sdk/tutorials/capabilities/sdk_features.md#face-ar-sdk-neural-network-features) Neural Network. * config.js * Java * Swift * JavaScript ``` // Set skin color Skin.color("0.8 0.6 0.1 0.4") // Reset skin color Skin.clear() ``` ``` // Effect mCurrentEffect = ... // Set skin color mCurrentEffect.evalJs("Skin.color('0.8 0.6 0.1 0.4')", null); // Reset skin color mCurrentEffect.evalJs("Skin.clear()", null); ``` ``` // var currentEffect: BNBEffect = ... // Set skin color currentEffect?.evalJs("Skin.color('0.8 0.6 0.1 0.4')", resultCallback: nil) // Reset skin color currentEffect?.evalJs("Skin.clear()", resultCallback: nil) ``` ``` // Set skin color await effect.evalJs("Skin.color('0.8 0.6 0.1 0.4')") // Reset skin color await effect.evalJs("Skin.clear()") ``` **Preview** ![Right image compare](/far-sdk/assets/images/original-8823b66ca7c9e5a54656f6ee9b77c29f.jpg)![Left image compare](/far-sdk/assets/images/SkinColor-34d47f813d6ab2280eaf6e3793f8d14d.jpg) Drag ## Background separation[​](#background-separation "Direct link to Background separation") info Requires [Background separation](/far-sdk/tutorials/capabilities/sdk_features.md#face-ar-sdk-neural-network-features) Neural Network. tip If you need only the background separation effect, see [Virtual Background API](/far-sdk/effects/virtual_background.md). ### Background texture[​](#background-texture "Direct link to Background texture") Sets the file as the background texture. * `Background.texture("bg_image.png")` - sets an image file as a background texture. * Supported formats: `.jpeg`, `.jpg`, `.png`, `.ktx`, `.gif`. * `Background.texture("bg_video.mp4")` - sets the video file as a background texture. Visit [technical specification](/far-sdk/tutorials/capabilities/technical_specification.md#video-formats-support) for supported video formats. - config.js - Java - Swift - JavaScript ``` /* Feel free to add your custom code below */ Background.texture("bg_colors_tile.png") ``` ``` // Effect mCurrentEffect = ... mCurrentEffect.evalJs("Background.texture('bg_colors_tile.png')", null); ``` ``` // var currentEffect: BNBEffect = ... currentEffect?.evalJs("Background.texture('bg_colors_tile.png')", resultCallback: nil) ``` ``` await effect.evalJs("Background.texture('bg_colors_tile.png')") ``` **Preview** ![Right image compare](/far-sdk/assets/images/original_wide-2f360425a0e0a779832de176b75c4354.jpg)![Left image compare](/far-sdk/assets/images/BackgroundTexture-fe6299595a550a13eec111677e4e2539.jpg) Drag ### Background transparency[​](#background-transparency "Direct link to Background transparency") Sets the background transparency. * `Background.transparency(n)` - set transperany value from 0 to 1 (including decimal). - config.js - Java - Swift - JavaScript ``` /* Feel free to add your custom code below */ Background.transparency(0.5) ``` ``` // Effect mCurrentEffect = ... mCurrentEffect.evalJs("Background.transparency(0.5)", null); ``` ``` // var currentEffect: BNBEffect = ... currentEffect?.evalJs("Background.transparency(0.5)", resultCallback: nil) ``` ``` await effect.evalJs("Background.transparency(0.5)") ``` **Preview** ![Right image compare](/far-sdk/assets/images/BackgroundTexture-fe6299595a550a13eec111677e4e2539.jpg)![Left image compare](/far-sdk/assets/images/BackgroundTransparent-2ee7764a986084bc8020b1342daef38c.jpg) Drag ### Background rotation[​](#background-rotation "Direct link to Background rotation") Rotates the background texture clockwise in degrees. * `Background.rotation(deg)` - set rotation value from 0 360 degrees. The value should be divisible by 90 degrees. - config.js - Java - Swift - JavaScript ``` /* Feel free to add your custom code below */ Background.rotation(90) ``` ``` // Effect mCurrentEffect = ... mCurrentEffect.evalJs("Background.rotation(90)", null); ``` ``` // var currentEffect: BNBEffect = ... currentEffect?.evalJs("Background.rotation(90)", resultCallback: nil) ``` ``` await effect.evalJs("Background.rotation(90)") ``` ### Background scale[​](#background-scale "Direct link to Background scale") Scales the background texture. * `Background.scale(n)` - multiplies the background texture size on given value. - config.js - Java - Swift - JavaScript ``` /* Feel free to add your custom code below */ Background.scale(2) ``` ``` // Effect mCurrentEffect = ... mCurrentEffect.evalJs("Background.scale(2)", null); ``` ``` // var currentEffect: BNBEffect = ... currentEffect?.evalJs("Background.scale(2)", resultCallback: nil) ``` ``` await effect.evalJs("Background.scale(2)") ``` ### Background contentMode[​](#background-contentmode "Direct link to Background contentMode") Sets the background texture content mode. * `Background.contentMode("mode")` - set mode type, possible values: `fill`, `fit`, `scale_to_fill`. - config.js - Java - Swift - JavaScript ``` /* Feel free to add your custom code below */ Background.contentMode("fill") ``` ``` // Effect mCurrentEffect = ... mCurrentEffect.evalJs("Background.contentMode('fill')", null); ``` ``` // var currentEffect: BNBEffect = ... currentEffect?.evalJs("Background.contentMode('fill')", resultCallback: nil) ``` ``` await effect.evalJs("Background.contentMode('fill')") ``` ### Background blur[​](#background-blur "Direct link to Background blur") Blurs the background behind the user. * `Background.blur(n)` - sets the background blur radius in \[0, 1] range. - config.js - Java - Swift - JavaScript ``` /* Feel free to add your custom code below */ Background.blur(0.2) ``` ``` // Effect mCurrentEffect = ... mCurrentEffect.evalJs("Background.blur(0.2)", null); ``` ``` // var currentEffect: BNBEffect = ... currentEffect?.evalJs("Background.blur(0.2)", resultCallback: nil) ``` ``` await effect.evalJs("Background.blur(0.2)") ``` **Preview** ![Right image compare](/far-sdk/assets/images/BackgroundTexture-fe6299595a550a13eec111677e4e2539.jpg)![Left image compare](/far-sdk/assets/images/BackgroundBlur-19465ffc736d1651b18c01ea8d99b1ef.jpg) Drag ### Background clear[​](#background-clear "Direct link to Background clear") Removes the background color and texture, resets any settings applied. * config.js * Java * Swift * JavaScript ``` /* Feel free to add your custom code below */ Background.clear() ``` ``` // Effect mCurrentEffect = ... mCurrentEffect.evalJs("Background.clear()", null); ``` ``` // var currentEffect: BNBEffect = ... currentEffect?.evalJs("Background.clear()", resultCallback: nil) ``` ``` await effect.evalJs("Background.clear()") ``` ## Hair coloring[​](#hair-coloring "Direct link to Hair coloring") info Requires [Hair segmentation](/far-sdk/tutorials/capabilities/sdk_features.md#face-ar-sdk-neural-network-features) Neural Network. ### Single color[​](#single-color "Direct link to Single color") Dyes hair with one color. * `Hair.color("R G B A")` - set hair color in R G B A format (separated with space). Each value should be in a rage from 0 to 1 (including decimal), * `Hair.clear()` - clears hair color (including gradient and hair strands). - config.js - Java - Swift - JavaScript ``` // Set hair color Hair.color("0.39 0.14 0.14 0.8") // Reset hair color Hair.clear() ``` ``` // Effect mCurrentEffect = ... // Set hair color mCurrentEffect.evalJs("Hair.color('0.39 0.14 0.14 0.8')", null); // Reset hair color mCurrentEffect.evalJs("Hair.clear()", null); ``` ``` // var currentEffect: BNBEffect = ... // Set hair color currentEffect?.evalJs("Hair.color('0.39 0.14 0.14 0.8')", resultCallback: nil // Reset hair color currentEffect?.evalJs("Hair.clear()", resultCallback: nil) ``` ``` // Set hair color await effect.evalJs("Hair.color('0.39 0.14 0.14 0.8')") // Reset hair color await effect.evalJs("Hair.clear()") ``` **Preview** ![Right image compare](/far-sdk/assets/images/original-8823b66ca7c9e5a54656f6ee9b77c29f.jpg)![Left image compare](/far-sdk/assets/images/HairColor-90c31b1560331c076aac589c7f741ce8.jpg) Drag ### Hair gradient[​](#hair-gradient "Direct link to Hair gradient") Dyes hair with 1 to 5 colors. * `Hair.color("start_color_rgba", "end_color_rgba")` - set hair gradient in R G B A format (separated with space). Each rgba value should be in a rage from 0 to 1 (including decimal), - config.js - Java - Swift - JavaScript ``` /* Feel free to add your custom code below */ Hair.color("0.19 0.06 0.25", "0.09 0.25 0.38") ``` ``` // Effect mCurrentEffect = ... mCurrentEffect.evalJs("Hair.color('0.19 0.06 0.25', '0.09 0.25 0.38')", null); ``` ``` // var currentEffect: BNBEffect = ... currentEffect?.evalJs("Hair.color('0.19 0.06 0.25', '0.09 0.25 0.38')", resultCallback: nil) ``` ``` await effect.evalJs("Hair.color('0.19 0.06 0.25', '0.09 0.25 0.38')") ``` **Preview** ![Right image compare](/far-sdk/assets/images/original-8823b66ca7c9e5a54656f6ee9b77c29f.jpg)![Left image compare](/far-sdk/assets/images/HairGradient-b253876cdcdbb46c8ae69f129ebcdbb5.jpg) Drag ### Hair strands painting[​](#hair-strands-painting "Direct link to Hair strands painting") Dyes hair strands with 1 to 5 colors. * `Hair.strands("R G B A", "R G B A", "R G B A", ...)` - set hair strands in R G B A format (separated with space) with 5 color maximum. info Requires [Hair strands painting](/far-sdk/tutorials/capabilities/sdk_features.md#face-ar-sdk-neural-network-features) Add-On. * config.js * Java * Swift * JavaScript ``` /* Feel free to add your custom code below */ Hair.strands("0.80 0.40 0.40 1.0", "0.83 0.40 0.40 1.0", "0.85 0.75 0.75 1.0", "0.87 0.60 0.60 1.0", "0.99 0.65 0.65 1.0") ``` ``` // Effect mCurrentEffect = ... mCurrentEffect.evalJs("Hair.strands('0.80 0.40 0.40 1.0', '0.83 0.40 0.40 1.0', '0.85 0.75 0.75 1.0', '0.87 0.60 0.60 1.0', '0.99 0.65 0.65 1.0')", null); ``` ``` // var currentEffect: BNBEffect = ... currentEffect?.evalJs("Hair.strands('0.80 0.40 0.40 1.0', '0.83 0.40 0.40 1.0', '0.85 0.75 0.75 1.0', '0.87 0.60 0.60 1.0', '0.99 0.65 0.65 1.0')", resultCallback: nil); ``` ``` await effect.evalJs("Hair.strands('0.80 0.40 0.40 1.0', '0.83 0.40 0.40 1.0', '0.85 0.75 0.75 1.0', '0.87 0.60 0.60 1.0', '0.99 0.65 0.65 1.0')") ``` **Preview** ![Right image compare](/far-sdk/assets/images/original-8823b66ca7c9e5a54656f6ee9b77c29f.jpg)![Left image compare](/far-sdk/assets/images/HairStrand-cd7d7af6e94d76429c69883f0ad7a201.jpg) Drag ## Eyes beautification[​](#eyes-beautification "Direct link to Eyes beautification") info Requires [Eye segmentation](/far-sdk/tutorials/capabilities/sdk_features.md#face-ar-sdk-neural-network-features) Neural Network. ### Eyes coloring[​](#eyes-coloring "Direct link to Eyes coloring") Changes the color of the iris as in virtual lens try on. * `Eyes.color("R G B A")` - set eyes color in R G B A format (separated with space). Each value should be in a rage from 0 to 1 (including decimal), * `Eyes.clear()` - clears eyes color including flare and whitening. - config.js - Java - Swift - JavaScript ``` // Set eyes color Eyes.color("0 0.2 0.8 0.64") // Reset eyes color Eyes.clear() ``` ``` // Effect mCurrentEffect = ... // Set eyes color mCurrentEffect.evalJs("Eyes.color('0 0.2 0.8 0.64')", null); // Reset eyes color mCurrentEffect.evalJs("Eyes.clear()", null); ``` ``` // var currentEffect: BNBEffect = ... // Set eyes color currentEffect?.evalJs("Eyes.color('0 0.2 0.8 0.64')", resultCallback: nil) // Reset eyes color currentEffect?.evalJs("Eyes.clear()", resultCallback: nil) ``` ``` // Set eyes color await effect.evalJs("Eyes.color('0 0.2 0.8 0.64')") // Reset eyes color await effect.evalJs("Eyes.clear()") ``` **Preview** ![Right image compare](/far-sdk/assets/images/original-8823b66ca7c9e5a54656f6ee9b77c29f.jpg)![Left image compare](/far-sdk/assets/images/eyescolor-fc2dcb06828e09ff46183aadd16ce026.jpg) Drag ### Eye flare[​](#eye-flare "Direct link to Eye flare") Makes eyes more expressive adding flare. * `Eyes.flare(n)` - sets the eyes flare strength from 0 to 1. - config.js - Java - Swift - JavaScript ``` /* Feel free to add your custom code below */ Eyes.flare("1") ``` ``` // Effect mCurrentEffect = ... mCurrentEffect.evalJs("Eyes.flare(1)", null); ``` ``` // var currentEffect: BNBEffect = ... currentEffect?.evalJs("Eyes.flare(1)", resultCallback: nil) ``` ``` await effect.evalJs("Eyes.flare(1)") ``` **Preview** ![Right image compare](/far-sdk/assets/images/original-8823b66ca7c9e5a54656f6ee9b77c29f.jpg)![Left image compare](/far-sdk/assets/images/EyesFlare-aaa9dfc2204ee98f6874e0556099d0c8.jpg) Drag ### Eyes whitening[​](#eyes-whitening "Direct link to Eyes whitening") Makes the look more expressive by whitening eyes. * `Eyes.whitening(n)` - sets the eyes sclera whitening strength from 0 to 1. - config.js - Java - Swift - JavaScript ``` /* Feel free to add your custom code below */ Eyes.whitening(1) ``` ``` // Effect mCurrentEffect = ... mCurrentEffect.evalJs("Eyes.whitening(1)", null); ``` ``` // var currentEffect: BNBEffect = ... currentEffect?.evalJs("Eyes.whitening(1)", resultCallback: nil) ``` ``` await effect.evalJs("Eyes.whitening(1)") ``` **Preview** ![Right image compare](/far-sdk/assets/images/original-8823b66ca7c9e5a54656f6ee9b77c29f.jpg)![Left image compare](/far-sdk/assets/images/EyesWhitening-53c06893890dff12356d60e292ee7b23.jpg) Drag --- # Virtual Makeup API [View as Markdown](https://docs.banuba.com/far-sdk/effects/makeup_deprecated/makeup.md)[![](/far-sdk/img/ai-guide/chatgpt.svg)Open in ChatGPT](https://chatgpt.com/?q=Read%20https%3A%2F%2Fdocs.banuba.com%2Ffar-sdk%2Feffects%2Fmakeup_deprecated%2Fmakeup.md%20\(Virtual%20Makeup%20API\)%20and%20help%20me%20with%20it%20in%20my%20Banuba%20Face%20AR%20SDK%20project.)[![](/far-sdk/img/ai-guide/claude.svg)Open in Claude](https://claude.ai/new?q=Read%20https%3A%2F%2Fdocs.banuba.com%2Ffar-sdk%2Feffects%2Fmakeup_deprecated%2Fmakeup.md%20\(Virtual%20Makeup%20API\)%20and%20help%20me%20with%20it%20in%20my%20Banuba%20Face%20AR%20SDK%20project.)Install tools danger **This is deprecated feature**. [We recommend you to use Prefabs](/far-sdk/effects/prefabs/overview.md) Banuba provides the Makeup API designed to help you integrate augmented reality beauty try-on features into your app. The AR makeup features fit into e-commerce try-on apps, makeovers, selfie editors and portrait retouching software. It aims to overlay realistic makeup onto the face to showcase the product or let users change their appearance. [](pathname:///generated/effects/Makeup.zip) [Download example](pathname:///generated/effects/Makeup.zip) tip [Read more about how to use and combine](/far-sdk/effects/makeup_deprecated/makeup_usage.md) Makeup API and Face Beauty API features. The Beauty module allows to enhance the face with the following built-in features: ## Face Makeup[​](#face-makeup "Direct link to Face Makeup") Sets texture as composite makeup (i.e. all-in-one: eyelashes, shadows, eyeliner, etc). * `Makeup.set("makeup_texture.png")` - sets an image-file as a makeup texture. The file should be placed into the effect's folder. * Supported formats: `.jpeg`, `.jpg`, `.png`, `.ktx`, `.gif`. - config.js - Java - Swift - JavaScript ``` /* Feel free to add your custom code below */ Makeup.set("example_makeup.png") ``` ``` // Effect mCurrentEffect = ... mCurrentEffect.evalJs("Makeup.set('example_makeup.png')", null); ``` ``` // var currentEffect: BNBEffect = ... currentEffect?.evalJs("Makeup.set('example_makeup.png')", resultCallback: nil) ``` ``` await effect.evalJs("Makeup.set('example_makeup.png')") ``` **Preview** ![Right image compare](/far-sdk/assets/images/original-8823b66ca7c9e5a54656f6ee9b77c29f.jpg)![Left image compare](/far-sdk/assets/images/Makeup-56cc767d4fda7de0c6c77d44b779ac40.jpg) Drag ### Highlighting[​](#highlighting "Direct link to Highlighting") Set highlighter color. * `Makeup.highlighter("R G B A")` - set highlighter color in R G B A format (separated with space). Each value should be in a range from 0 to 1 (including decimal). - config.js - Java - Swift - JavaScript ``` /* Feel free to add your custom code below */ Makeup.highlighter("0.75 0.74 0.74 0.4") ``` ``` // Effect mCurrentEffect = ... mCurrentEffect.evalJs("Makeup.highlighter('0.75 0.74 0.74 0.4')", null); ``` ``` // var currentEffect: BNBEffect = ... currentEffect?.evalJs("Makeup.highlighter('0.75 0.74 0.74 0.4')", resultCallback: nil) ``` ``` await effect.evalJs("Makeup.highlighter('0.75 0.74 0.74 0.4')") ``` **Preview** ![Right image compare](/far-sdk/assets/images/original-8823b66ca7c9e5a54656f6ee9b77c29f.jpg)![Left image compare](/far-sdk/assets/images/highlighter-a09bcfd376fe0db096ec2278c36bac0f.jpg) Drag **Set a custom highlighter texture** * `Makeup.highlighter("highlighter.png")` - sets an image-file as a highlighter texture. The file should be placed into the effect's folder. * Supported formats: `.jpeg`, `.jpg`, `.png`, `.ktx`, `.gif`. - config.js - Java - Swift - JavaScript ``` /* Feel free to add your custom code below */ Makeup.highlighter("highlighter.png") ``` ``` // Effect mCurrentEffect = ... mCurrentEffect.evalJs("Makeup.highlighter('highlighter.png')", null); ``` ``` // var currentEffect: BNBEffect = ... currentEffect?.evalJs("Makeup.highlighter('highlighter.png')", resultCallback: nil) ``` ``` await effect.evalJs("Makeup.highlighter('highlighter.png')") ``` ### Contouring[​](#contouring "Direct link to Contouring") Sets contour color. * `Makeup.contour("R G B A")` - set contour color in R G B A format (separated with space). Each value should be in a range from 0 to 1 (including decimal). - config.js - Java - Swift - JavaScript ``` /* Feel free to add your custom code below */ Makeup.contour("0.3 0.1 0.1 0.6") ``` ``` // Effect mCurrentEffect = ... mCurrentEffect.evalJs("Makeup.contour('0.3 0.1 0.1 0.6')", null); ``` ``` // var currentEffect: BNBEffect = ... currentEffect?.evalJs("Makeup.contour('0.3 0.1 0.1 0.6')", resultCallback: nil) ``` ``` await effect.evalJs("Makeup.contour('0.3 0.1 0.1 0.6')") ``` **Preview** ![Right image compare](/far-sdk/assets/images/original-8823b66ca7c9e5a54656f6ee9b77c29f.jpg)![Left image compare](/far-sdk/assets/images/contour-59178fb2d6a2d71b883e26139a6f65aa.jpg) Drag **Set a custom contour texture** * `Makeup.contour("contour.png")` - sets an image-file as a contour texture. The file should be placed into the effect's folder. * Supported formats: `.jpeg`, `.jpg`, `.png`, `.ktx`, `.gif`. - config.js - Java - Swift - JavaScript ``` /* Feel free to add your custom code below */ Makeup.contour("contour.png") ``` ``` // Effect mCurrentEffect = ... mCurrentEffect.evalJs("Makeup.contour('contour.png')", null); ``` ``` // var currentEffect: BNBEffect = ... currentEffect?.evalJs("Makeup.contour('contour.png')", resultCallback: nil) ``` ``` await effect.evalJs("Makeup.contour('contour.png')") ``` ### Foundation[​](#foundation "Direct link to Foundation") Foundation is a combination of two [Beauty API](/far-sdk/effects/makeup_deprecated/face_beauty.md) features: [`Skin.color()`](/far-sdk/effects/makeup_deprecated/face_beauty.md#skin-color) and [`Skin.softening()`](/far-sdk/effects/makeup_deprecated/face_beauty.md#skin-smoothing-skin-softening). info Requires [Skin segmentation](/far-sdk/tutorials/capabilities/sdk_features.md#face-ar-sdk-neural-network-features) Neural Network. * config.js * Java * Swift * JavaScript ``` /* Feel free to add your custom code below */ // set skin color Skin.color("0.73 0.39 0.08 0.3") // set softening strength (skin smoothing) Skin.softening(1) ``` ``` // Effect mCurrentEffect = ... // set skin color mCurrentEffect.evalJs("Skin.color('0.73 0.39 0.08 0.3')", null); // set softening strength (skin smoothing) mCurrentEffect.evalJs("Skin.softening(1)", null); ``` ``` // var currentEffect: BNBEffect = ... // set skin color currentEffect?.evalJs("Skin.color('0.73 0.39 0.08 0.3')", resultCallback: nil) // set softening strength (skin smoothing) currentEffect?.evalJs("Skin.softening(1)", resultCallback: nil) ``` ``` // set skin color await effect.evalJs("Skin.color('0.73 0.39 0.08 0.3')") // set softening strength (skin smoothing) await effect.evalJs("Skin.softening(1)") ``` **Preview** ![Right image compare](/far-sdk/assets/images/original-8823b66ca7c9e5a54656f6ee9b77c29f.jpg)![Left image compare](/far-sdk/assets/images/foundation-f79656a2dea0a0cb33ae9954459fc02c.jpg) Drag ### Blush[​](#blush "Direct link to Blush") Set blush color. * `Makeup.blushes("R G B A")` - set blushes color in R G B A format (separated with space). Each value should be in a range from 0 to 1 (including decimal). - config.js - Java - Swift - JavaScript ``` /* Feel free to add your custom code below */ Makeup.blushes("0.7 0.1 0.2 0.5") ``` ``` // Effect mCurrentEffect = ... mCurrentEffect.evalJs("Makeup.blushes('0.7 0.1 0.2 0.5')", null); ``` ``` // var currentEffect: BNBEffect = ... currentEffect?.evalJs("Makeup.blushes('0.7 0.1 0.2 0.5')", resultCallback: nil) ``` ``` await effect.evalJs("Makeup.blushes('0.7 0.1 0.2 0.5')") ``` **Preview** ![Right image compare](/far-sdk/assets/images/original-8823b66ca7c9e5a54656f6ee9b77c29f.jpg)![Left image compare](/far-sdk/assets/images/blush-0e2e26991bbfbf30ee310457cbc0fa3a.jpg) Drag **Set a custom blush texture** * `Makeup.blushes("blushes.png")` - sets an image-file as a blushes texture. The file should be placed into the effect's folder. * Supported formats: `.jpeg`, `.jpg`, `.png`, `.ktx`, `.gif`. - config.js - Java - Swift - JavaScript ``` /* Feel free to add your custom code below */ Makeup.blushes("blush.png") ``` ``` // Effect mCurrentEffect = ... mCurrentEffect.evalJs("Makeup.blushes('blush.png')", null); ``` ``` // var currentEffect: BNBEffect = ... currentEffect?.evalJs("Makeup.blushes('blush.png')", resultCallback: nil) ``` ``` await effect.evalJs("Makeup.blushes('blush.png')") ``` ### Softlight[​](#softlight "Direct link to Softlight") Highlights a face like a directional flashlight. * `Softlight.strength(n)` - changes softlight intensity, where **n** - any value from 0 to 1 (including decimal). Values > 1 may be also provided. * `Softlight.clear()` - reset softlight, equals to `Softlight.strength(0)`. - config.js - Java - Swift - JavaScript ``` // Set softlight strength Softlight.strength(1) // Reset softlight Softlight.clear() ``` ``` // Effect mCurrentEffect = ... // Set softlight strength mCurrentEffect.evalJs("Softlight.strength(1)", null); // Reset softlight mCurrentEffect.evalJs("Softlight.clear()", null); ``` ``` // var currentEffect: BNBEffect = ... // Set softlight strength currentEffect?.evalJs("Softlight.strength(1)", resultCallback: nil) // Reset softlight currentEffect?.evalJs("Softlight.clear()", resultCallback: nil) ``` ``` // Set softlight strength await effect.evalJs("Softlight.strength(1)") // Reset softlight await effect.evalJs("Softlight.clear()") ``` **Preview** ![Right image compare](/far-sdk/assets/images/original-8823b66ca7c9e5a54656f6ee9b77c29f.jpg)![Left image compare](/far-sdk/assets/images/Softlight-cbb459a901f91b9511924ae35b82dec7.jpg) Drag ## Brows makeup[​](#brows-makeup "Direct link to Brows makeup") Set brows color. * `Brows.color("R G B A")` - set brows color in R G B A format (separated with space). Each value should be in a range from 0 to 1 (including decimal). * `Brows.clear()`- clears brows color. - config.js - Java - Swift - JavaScript ``` // Set brows color Brows.color("0.172 0.125 0.105 0.732") // Reset brows color Brows.clear() ``` ``` // Effect mCurrentEffect = ... // Set brows color mCurrentEffect.evalJs("Brows.color('0.172 0.125 0.105 0.732')", null); // Reset brows color mCurrentEffect.evalJs("Brows.clear()", null); ``` ``` // var currentEffect: BNBEffect = ... // Set brows color currentEffect?.evalJs("Brows.color('0.172 0.125 0.105 0.732')", resultCallback: nil) // Reset brows color currentEffect?.evalJs("Brows.clear()", resultCallback: nil) ``` ``` // Set brows color await effect.evalJs("Brows.color('0.172 0.125 0.105 0.732')") // Reset brows color await effect.evalJs("Brows.clear()") ``` **Preview** ![Right image compare](/far-sdk/assets/images/original-8823b66ca7c9e5a54656f6ee9b77c29f.jpg)![Left image compare](/far-sdk/assets/images/brows-2703f25e2ff6933315668262ab5f9bd0.jpg) Drag ## Eye makeup[​](#eye-makeup "Direct link to Eye makeup") ### Eyeliner[​](#eyeliner "Direct link to Eyeliner") Set eyeliner color. * `Makeup.eyeliner("R G B A")` - set eyeliner color in R G B A format (separated with space). Each value should be in a range from 0 to 1 (including decimal). - config.js - Java - Swift - JavaScript ``` /* Feel free to add your custom code below */ Makeup.eyeliner("0 0 0") ``` ``` // Effect mCurrentEffect = ... mCurrentEffect.evalJs("Makeup.eyeliner('0 0 0')", null); ``` ``` // var currentEffect: BNBEffect = ... currentEffect?.evalJs("Makeup.eyeliner('0 0 0')", resultCallback: nil) ``` ``` await effect.evalJs("Makeup.eyeliner('0 0 0')") ``` **Preview** ![Right image compare](/far-sdk/assets/images/original-8823b66ca7c9e5a54656f6ee9b77c29f.jpg)![Left image compare](/far-sdk/assets/images/eyeliner-92fa652bb6068a9432490887ad80f046.jpg) Drag **Set a custom eyeliner texture** * `Makeup.eyeliner("eyeliner.png")` - sets an image-file as an eyeliner texture. The file should be placed into the effect's folder. * Supported formats: `.jpeg`, `.jpg`, `.png`, `.ktx`, `.gif`. - config.js - Java - Swift - JavaScript ``` /* Feel free to add your custom code below */ Makeup.eyeliner("eyeliner.png") ``` ``` // Effect mCurrentEffect = ... mCurrentEffect.evalJs("Makeup.eyeliner('eyeliner.png')", null); ``` ``` // var currentEffect: BNBEffect = ... currentEffect?.evalJs("Makeup.eyeliner('eyeliner.png')", resultCallback: nil) ``` ``` await effect.evalJs("Makeup.eyeliner('eyeliner.png')") ``` ### Eyeshadow[​](#eyeshadow "Direct link to Eyeshadow") Set eyeshadow color. * `Makeup.eyeshadow("R G B A")` - set eyeshadow color in R G B A format (separated with space). Each value should be in a range from 0 to 1 (including decimal). - config.js - Java - Swift - JavaScript ``` /* Feel free to add your custom code below */ Makeup.eyeshadow("0.6 0.5 1 0.6") ``` ``` // Effect mCurrentEffect = ... mCurrentEffect.evalJs("Makeup.eyeshadow('0.6 0.5 1 0.6')", null); ``` ``` // var currentEffect: BNBEffect = ... currentEffect?.evalJs("Makeup.eyeshadow('0.6 0.5 1 0.6')", resultCallback: nil) ``` ``` await effect.evalJs("Makeup.eyeshadow('0.6 0.5 1 0.6')") ``` **Preview** ![Right image compare](/far-sdk/assets/images/original-8823b66ca7c9e5a54656f6ee9b77c29f.jpg)![Left image compare](/far-sdk/assets/images/eyeshadow-2079f8ae8253e3db7c06a5d04b057109.jpg) Drag **Set a custom eyeshadow texture** * `Makeup.eyeshadow("eyeshadow.png")` - sets an image-file as an eyeshadow texture. The file should be placed into the effect's folder. * Supported formats: `.jpeg`, `.jpg`, `.png`, `.ktx`, `.gif`. - config.js - Java - Swift - JavaScript ``` /* Feel free to add your custom code below */ Makeup.eyeshadow("eyeshadow.png") ``` ``` // Effect mCurrentEffect = ... mCurrentEffect.evalJs("Makeup.eyeshadow('eyeshadow.png')", null); ``` ``` // var currentEffect: BNBEffect = ... currentEffect?.evalJs("Makeup.eyeshadow('eyeshadow.png')", resultCallback: nil) ``` ``` await effect.evalJs("Makeup.eyeshadow('eyeshadow.png')") ``` ### Eyelashes[​](#eyelashes "Direct link to Eyelashes") Set eyelashes color. * `Makeup.lashes("R G B A")` - set eyelashes color in R G B A format (separated with space). Each value should be in a rage from 0 to 1 (including decimal). - config.js - Java - Swift - JavaScript ``` /* Feel free to add your custom code below */ Makeup.lashes("0 0 0") ``` ``` // Effect mCurrentEffect = ... mCurrentEffect.evalJs("Makeup.lashes('0 0 0')", null); ``` ``` // var currentEffect: BNBEffect = ... currentEffect?.evalJs("Makeup.lashes('0 0 0')", resultCallback: nil) ``` ``` await effect.evalJs("Makeup.lashes('0 0 0')") ``` **Preview** ![Right image compare](/far-sdk/assets/images/original-8823b66ca7c9e5a54656f6ee9b77c29f.jpg)![Left image compare](/far-sdk/assets/images/eyelashes-5ebbf4d5d38bba778b494ff5d793a159.jpg) Drag **Set a custom eyelashes texture** * `Makeup.lashes("eyelashes.png")` - sets an image-file as an eyelashes texture. The file should be placed into the effect's folder. * Supported formats: `.jpeg`, `.jpg`, `.png`, `.ktx`, `.gif`. - config.js - Java - Swift - JavaScript ``` /* Feel free to add your custom code below */ Makeup.lashes("eyelashes.png") ``` ``` // Effect mCurrentEffect = ... mCurrentEffect.evalJs("Makeup.lashes('eyelashes.png')", null); ``` ``` // var currentEffect: BNBEffect = ... currentEffect?.evalJs("Makeup.lashes('eyelashes.png')", resultCallback: nil) ``` ``` await effect.evalJs("Makeup.lashes('eyelashes.png')") ``` ### Makeup.clear[​](#makeupclear "Direct link to Makeup.clear") Global method for Makeup. Clears all Makeup features that have been set. * config.js * Java * Swift * JavaScript ``` Makeup.clear() ``` ``` mCurrentEffect.evalJs("Makeup.clear()", null); ``` ``` currentEffect?.evalJs("Makeup.clear()", resultCallback: nil) ``` ``` await effect.evalJs("Makeup.clear()") ``` ## Lipstick[​](#lipstick "Direct link to Lipstick") ### Matt[​](#matt "Direct link to Matt") Set lips matte color. * `Lips.matt("R G B A")` - set lips matte color in R G B A format (separated with space). Each value should be in a range from 0 to 1 (including decimal). * `Lips.clear()`- clears lips color. - config.js - Java - Swift - JavaScript ``` // Set lips color Lips.matt("0.85 0.43 0.5 0.8") // Reset lips color Lips.clear() ``` ``` // Effect mCurrentEffect = ... // Set lips color mCurrentEffect.evalJs("Lips.matt('0.85 0.43 0.5 0.8')", null); // Reset lips color mCurrentEffect.evalJs("Lips.clear()", null); ``` ``` // var currentEffect: BNBEffect = ... // Set lips color currentEffect?.evalJs("Lips.matt('0.85 0.43 0.5 0.8')", resultCallback: nil) // Reset lips color currentEffect?.evalJs("Lips.clear()", resultCallback: nil) ``` ``` // Set lips color await effect.evalJs("Lips.matt('0.85 0.43 0.5 0.8')") // Reset lips color await effect.evalJs("Lips.clear()") ``` **Preview** ![Right image compare](/far-sdk/assets/images/original-8823b66ca7c9e5a54656f6ee9b77c29f.jpg)![Left image compare](/far-sdk/assets/images/lips_matt-01491f0b700df825a8a4fd2170a82261.jpg) Drag ### Shiny[​](#shiny "Direct link to Shiny") Set lips shiny color. * `Lips.shiny("R G B A")` - set shiny lips color in R G B A format (separated with space). Each value should be in a range from 0 to 1 (including decimal). * `Lips.clear()`- clears lips color. - config.js - Java - Swift - JavaScript ``` // Set lips color Lips.shiny("1 0 0.49 1") // Reset lips color Lips.clear() ``` ``` // Effect mCurrentEffect = ... // Set lips color mCurrentEffect.evalJs("Lips.shiny('1 0 0.49 1')", null); // Reset lips color mCurrentEffect.evalJs("Lips.clear()", null); ``` ``` // var currentEffect: BNBEffect = ... // Set lips color currentEffect?.evalJs("Lips.shiny('1 0 0.49 1')", resultCallback: nil) // Reset lips color currentEffect?.evalJs("Lips.clear()", resultCallback: nil) ``` ``` // Set lips color await effect.evalJs("Lips.shiny('1 0 0.49 1')") // Reset lips color await effect.evalJs("Lips.clear()") ``` **Preview** ![Right image compare](/far-sdk/assets/images/original-8823b66ca7c9e5a54656f6ee9b77c29f.jpg)![Left image compare](/far-sdk/assets/images/lips_shiny-815cce33a3d095ae1ebbb034b6bfd7f9.jpg) Drag ### Glitter[​](#glitter "Direct link to Glitter") Set lips glitter color. * `Lips.glitter("R G B A")` - set lips glitter color in R G B A format (separated with space). Each value should be in a range from 0 to 1 (including decimal). * `Lips.clear()`- clears lips color. - config.js - Java - Swift - JavaScript ``` // Set lips color Lips.glitter("0.552 0 0 1") // Reset lips color Lips.clear() ``` ``` // Effect mCurrentEffect = ... // Set lips color mCurrentEffect.evalJs("Lips.glitter('0.552 0 0 1')", null); // Reset lips color mCurrentEffect.evalJs("Lips.clear()", null); ``` ``` // var currentEffect: BNBEffect = ... // Set lips color currentEffect?.evalJs("Lips.glitter('0.552 0 0 1')", resultCallback: nil) // Reset lips color currentEffect?.evalJs("Lips.clear()", resultCallback: nil) ``` ``` // Set lips color await effect.evalJs("Lips.glitter('0.552 0 0 1')") // Reset lips color await effect.evalJs("Lips.clear()") ``` **Preview** ![Right image compare](/far-sdk/assets/images/original-8823b66ca7c9e5a54656f6ee9b77c29f.jpg)![Left image compare](/far-sdk/assets/images/lips_glitter-3a5f273825b856d74fa6567e3a99b915.jpg) Drag ### Extended lips options[​](#extended-lips-options "Direct link to Extended lips options") It is possible to set up extended lips parameters which are usually pre-defined in Matte, Shiny or Glitter lips. **Common options** * `Lips.color("R G B A")` - set lips color in R G B A format (separated with space). Each value should be in a range from 0 to 1 (including decimal). * `Lips.brightness(n)` - changes lips brightness intensity, where n - any value from 0 to 2 (including decimal). 0 stands for the minimal brightness (black color), 1 stands for standard brightness. Values > 2 may be also provided. **Shine options** * `Lips.saturation(n)` - changes shine saturation intensity, where n - any value from 0 to 1 (including decimal). * `Lips.shineIntensity(n)` - changes shine intensity, where n - any value from 0 to 2 (including decimal). Values > 2 may be also provided. * `Lips.shineBleeding(n)` - changes shine blending strength, where n - any value from 0 to 1 (including decimal). Values > 1 may be also provided. * `Lips.shineScale(n)` - changes shine scale, where n - any value from 0 to 1 (including decimal). 0 stands for the minimal scale (shine disabled), 1 stands for standard scale. Values > 1 may be also provided. **Glitter options** * `Lips.glitterGrain(n)` - changes glitter grain strength, where n - any value from 0 to 2 (including decimal). Values > 2 may be also provided. * `Lips.glitterIntensity(n)` - changes glitter intensity, where n - any value from 0 to 2 (including decimal). Values > 2 may be also provided. * `Lips.glitterBleeding(n)` - changes glitter blending strength, where n - any value from 0 to 2 (including decimal). Values > 2 may be also provided. - config.js - Java - Swift - JavaScript ``` /* Feel free to add your custom code below */ Lips.color("1 0 0 1") Lips.brightness(1) Lips.saturation(1) Lips.shineIntensity(2) Lips.shineBleeding(1) Lips.shineScale(1) Lips.glitterGrain(1) Lips.glitterIntensity(1) Lips.glitterBleeding(1) ``` ``` // Effect mCurrentEffect = ... mCurrentEffect.evalJs("Lips.color('1 0 0 1')", null); mCurrentEffect.evalJs("Lips.brightness(1)", null); mCurrentEffect.evalJs("Lips.saturation(1)", null); mCurrentEffect.evalJs("Lips.shineIntensity(2)", null); mCurrentEffect.evalJs("Lips.shineBleeding(1)", null); mCurrentEffect.evalJs("Lips.shineScale(1)", null); mCurrentEffect.evalJs("Lips.glitterGrain(1)", null); mCurrentEffect.evalJs("Lips.glitterIntensity(1)", null); mCurrentEffect.evalJs("Lips.glitterBleeding(1)", null); ``` ``` // var currentEffect: BNBEffect = ... currentEffect?.evalJs("Lips.color('1 0 0 1')", resultCallback: nil) currentEffect?.evalJs("Lips.brightness(1)", resultCallback: nil) currentEffect?.evalJs("Lips.saturation(1)", resultCallback: nil) currentEffect?.evalJs("Lips.shineIntensity(2)", resultCallback: nil) currentEffect?.evalJs("Lips.shineBleeding(1)", resultCallback: nil) currentEffect?.evalJs("Lips.shineScale(1)", resultCallback: nil) currentEffect?.evalJs("Lips.glitterGrain(1)", resultCallback: nil) currentEffect?.evalJs("Lips.glitterIntensity(1)", resultCallback: nil) currentEffect?.evalJs("Lips.glitterBleeding(1)", resultCallback: nil) ``` ``` await effect.evalJs("Lips.color('1 0 0 1')") await effect.evalJs("Lips.brightness(1)") await effect.evalJs("Lips.saturation(1)") await effect.evalJs("Lips.shineIntensity(2)") await effect.evalJs("Lips.shineBleeding(1)") await effect.evalJs("Lips.shineScale(1)") await effect.evalJs("Lips.glitterGrain(1)") await effect.evalJs("Lips.glitterIntensity(1)") await effect.evalJs("Lips.glitterBleeding(1)") ``` ## Lips liner[​](#lips-liner "Direct link to Lips liner") Set lips liner color. * `LipsLiner.color("R G B A")` - set lips liner color in R G B A format (separated with space). Each value should be in a range from 0 to 1 (including decimal). * `LipsLiner.clear()`- clears lips liner color. - config.js - Java - Swift - JavaScript ``` // Set lips color LipsLiner.color("1.0 0.7 0.8 0.8") // Reset lips color LipsLiner.clear() ``` ``` // Effect mCurrentEffect = ... // Set lips color mCurrentEffect.evalJs("LipsLiner.color('1.0 0.7 0.8 0.8')", null); // Reset lips color mCurrentEffect.evalJs("LipsLiner.clear()", null); ``` ``` // var currentEffect: BNBEffect = ... // Set lips color currentEffect?.evalJs("LipsLiner.color('1.0 0.7 0.8 0.8')", resultCallback: nil) // Reset lips color currentEffect?.evalJs("LipsLiner.clear()", resultCallback: nil) ``` ``` // Set lips color await effect.evalJs("LipsLiner.color('1.0 0.7 0.8 0.8')") // Reset lips color await effect.evalJs("LipsLiner.clear()") ``` **Preview** ![Right image compare](/far-sdk/assets/images/original-8823b66ca7c9e5a54656f6ee9b77c29f.jpg)![Left image compare](/far-sdk/assets/images/LipsLiner-bd67dbd8f788ec675fc8d6d0185170b0.jpg) Drag --- # How to combine the Makeup API and Beauty API features [View as Markdown](https://docs.banuba.com/far-sdk/effects/makeup_deprecated/makeup_usage.md)[![](/far-sdk/img/ai-guide/chatgpt.svg)Open in ChatGPT](https://chatgpt.com/?q=Read%20https%3A%2F%2Fdocs.banuba.com%2Ffar-sdk%2Feffects%2Fmakeup_deprecated%2Fmakeup_usage.md%20\(How%20to%20combine%20the%20Makeup%20API%20and%20Beauty%20API%20features\)%20and%20help%20me%20with%20it%20in%20my%20Banuba%20Face%20AR%20SDK%20project.)[![](/far-sdk/img/ai-guide/claude.svg)Open in Claude](https://claude.ai/new?q=Read%20https%3A%2F%2Fdocs.banuba.com%2Ffar-sdk%2Feffects%2Fmakeup_deprecated%2Fmakeup_usage.md%20\(How%20to%20combine%20the%20Makeup%20API%20and%20Beauty%20API%20features\)%20and%20help%20me%20with%20it%20in%20my%20Banuba%20Face%20AR%20SDK%20project.)Install tools danger **This feature is deprecated**. [We recommend you to use Prefabs](/far-sdk/effects/prefabs/overview.md) As you can notice, both the **Makeup API** and [**Beauty API**](/far-sdk/effects/makeup_deprecated/face_beauty.md) are using the same Face Filter. As a result, it is possible to combine their features in your application. You can consume the effect API in several ways. ## Via effect's *config.js*[​](#via-effects-configjs "Direct link to via-effects-configjs") Add to the bottom of the `Makeup/config.js` file: ``` /* Feel free to add your custom code below */ Lips.matt("0.85 0.23 0.2 0.8") ``` ## From application[​](#from-application "Direct link to From application") In your app code use `evalJs` from Banuba SDK: * Java * Swift * JavaScript ``` // Effect mCurrentEffect = ... // set lips matt color mCurrentEffect.evalJs("Lips.matt('0.85 0.23 0.2 0.8')", null); ``` ``` // var currentEffect: BNBEffect = ... // set lips matt color currentEffect?.evalJs("Lips.matt('0.85 0.23 0.2 0.8')", resultCallback: nil); ``` ``` // set lips matt color await effect.evalJs("Lips.matt('0.85 0.23 0.2 0.8')") ``` ## Combine features[​](#combine-features "Direct link to Combine features") To combine the Makeup API features, call the desired methods in your app or in `config.js` as shown above. **Example:** * config.js * Java * Swift * JavaScript ``` /* Feel free to add your custom code below */ Lips.matt("0.85 0.23 0.2 0.8") Makeup.eyeshadow("0.6 0.5 1 0.6") Makeup.contour("0.3 0.1 0.1 0.2") ``` ``` // Effect mCurrentEffect = ... mCurrentEffect.evalJs("Lips.matt('0.85 0.23 0.2 0.8')", null); mCurrentEffect.evalJs("Makeup.eyeshadow('0.6 0.5 1 0.6')", null); mCurrentEffect.evalJs("Makeup.contour('0.3 0.1 0.1 0.2')", null); ``` ``` // var currentEffect: BNBEffect = ... currentEffect?.evalJs("Lips.matt('0.85 0.23 0.2 0.8')", resultCallback: nil); currentEffect?.evalJs("Makeup.eyeshadow('0.6 0.5 1 0.6')", resultCallback: nil); currentEffect?.evalJs("Makeup.contour('0.3 0.1 0.1 0.2')", resultCallback: nil); ``` ``` await effect.evalJs("Lips.matt('0.85 0.23 0.2 0.8')") await effect.evalJs("Makeup.eyeshadow('0.6 0.5 1 0.6')") await effect.evalJs("Makeup.contour('0.3 0.1 0.1 0.2')") ``` **Preview** ![Right image compare](/far-sdk/assets/images/original-8823b66ca7c9e5a54656f6ee9b77c29f.jpg)![Left image compare](/far-sdk/assets/images/multiple_features-9024a99b4558c5965d099d96f0b3fdb4.jpg) Drag --- [View as Markdown](https://docs.banuba.com/far-sdk/effects/overview.md)[![](/far-sdk/img/ai-guide/chatgpt.svg)Open in ChatGPT](https://chatgpt.com/?q=Read%20https%3A%2F%2Fdocs.banuba.com%2Ffar-sdk%2Feffects%2Foverview.md%20\(Effects\)%20and%20help%20me%20with%20it%20in%20my%20Banuba%20Face%20AR%20SDK%20project.)[![](/far-sdk/img/ai-guide/claude.svg)Open in Claude](https://claude.ai/new?q=Read%20https%3A%2F%2Fdocs.banuba.com%2Ffar-sdk%2Feffects%2Foverview.md%20\(Effects\)%20and%20help%20me%20with%20it%20in%20my%20Banuba%20Face%20AR%20SDK%20project.)Install tools tip You may also create your own effects with [**Banuba Studio**](https://studio.banuba.com/) or buy some from the [**Banuba Asset Store**](https://assetstore.banuba.net/). # Effect structure Every Banuba effect is a folder containing files with special meaning. This is what a typical effect looks like: ``` effect_folder/ |-- config.json |-- config.js |-- images/ | | | ... |-- shaders/ | | | ... |-- videos/ | | | ... ... ``` Only `config.json` is mandatory (see below). So, in order to start, create a folder with one blank file named like this. Other subfolders have no special meaning and exist only to group related files, which are usually referred from `config.json` or `config.js`. ## config.json[​](#configjson "Direct link to config.json") This is the basic file that describes the effect itself. It has the following format: ``` { "scene": "effect name", "version": "2.0.0", "camera": {} } ``` Where * `scene` - this is the name of your effect. * `version` - version of this configuration file. Always set it to `"2.0.0"`. The previous version is designed for the complex legacy effects. * `camera` - tells that you will render camera feed on the screen. Each effect can be complemented by other features ("prefabs" in our terminology). tip **Learn more about prefabs** [here](/far-sdk/effects/prefabs/overview.md) The basic complete example will look like this (it will render black eyeliner on a face): ``` { "scene": "Retouch example", "version": "2.0.0", "camera": {}, "faces": [ { "makeup_eyeliner": { "color": "0.0 0.0 0.0", "finish": "matte_liquid", "coverage": "hi" } } ] } ``` Another good example to start: [](pathname:///generated/effects/simple_hat_v2.zip) [Download simple 3D model sample](pathname:///generated/effects/simple_hat_v2.zip) ## config.js[​](#configjs "Direct link to config.js") This file may contain some business logic written in JavaScript. Refer to other documentation pages in this section. The most important feature related to scripting is probably [Virtual background](/far-sdk/effects/virtual_background.md). --- # On Face Prefabs [View as Markdown](https://docs.banuba.com/far-sdk/effects/prefabs/face.md)[![](/far-sdk/img/ai-guide/chatgpt.svg)Open in ChatGPT](https://chatgpt.com/?q=Read%20https%3A%2F%2Fdocs.banuba.com%2Ffar-sdk%2Feffects%2Fprefabs%2Fface.md%20\(On%20Face%20Prefabs\)%20and%20help%20me%20with%20it%20in%20my%20Banuba%20Face%20AR%20SDK%20project.)[![](/far-sdk/img/ai-guide/claude.svg)Open in Claude](https://claude.ai/new?q=Read%20https%3A%2F%2Fdocs.banuba.com%2Ffar-sdk%2Feffects%2Fprefabs%2Fface.md%20\(On%20Face%20Prefabs\)%20and%20help%20me%20with%20it%20in%20my%20Banuba%20Face%20AR%20SDK%20project.)Install tools ## GLTF[​](#gltf "Direct link to GLTF") ``` "faces": [ { "gltf": { "@mesh": "path/to/gltf/model", "rotation": "0 0 0", "scale": "1.0 1.0 1.0", "translation": "0.0 0.0 0.0", "animation": { "name": "Animation 1", "mode": "loop", "seek_position": 100 }, "@use_physics": false, "gravity": "0.0 -1000.0 0.0", "bones": { "bone_1": 1.0, "bone_2": 0.0, "bone_3": 1.0, "bone_4": 0.0, "bone_5": 1.0 }, "colliders": [ { "center": "0. 0. 0.", "radius": 100.0 }, { "center": "10. 110. 420.", "radius": 650.0 }, { "center": "14. 300. 156.", "radius": 10.0 } ], "constraints": [ { "from": "bone_1", "to": "bone_2", "distance": 10.0 }, { "from": "bone_2", "to": "bone_3", "distance": 50.0 }, { "from": "bone_3", "to": "bone_4", "distance": 30.0 }, { "from": "bone_4", "to": "bone_5", "distance": 60.0 } ], "damping": 0.99 } // ... } // ... ] ``` Place a 3D model in GLTF format on the face. | Parameter | Description | Optional | Default Value | | :------------------ | :--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | :------------------------------------------------------------------: | :-----------: | | `@mesh` | Path to a GLTF model. The leading `@` in the parameter name is required. Supported formats are `.glb` and `.gltf`. The model and all associated files, such as shaders, textures, and sounds, must be located in the same folder. | *+* | *+* | | `rotation` | Rotation angles, in degrees, around the *X*, *Y*, and *Z* axes. Note the default value. | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | `"-90 0 0"` | | `scale` | Scale along the *X*, *Y*, and *Z* axes. | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | `"1 1 1"` | | `translation` | Translates the model along the *X*, *Y*, and *Z* axes, in millimetres. | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | `"0 0 0"` | | `animation` | Plays an animation from the GLTF file. **All keys are optional**; in most cases, an empty object is enough to play the default animation.*Parameters:*`name` - Selects an animation from the file by name.`seek_position` - Playback start position relative to the beginning of the animation, in milliseconds.`mode` - Determines how to play the animation selected by `name`. Possible values are `off`, `loop`, `once`, `once_reversed`, and `fixed`. | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | `{}` | | `@use_physics` | Loads the GLTF model with physics simulation. The leading `@` in the parameter name is required. | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | `false` | | `cut` | Enables head occlusion geometry. Allowed values are `head` and `head_with_ears`; omit the parameter to disable cutting. | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | `disabled` | | `gravity` | Sets the gravity vector along the *X*, *Y*, and *Z* axes. | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | `"0 0 0"` | | `bones` | Sets bone inverse masses. The object keys are bone names and the values are inverse masses. | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | `{}` | | `colliders` | Adds sphere colliders for physical bones. Each collider requires both `center` and `radius`.*Parameters:*`center` - *X*, *Y*, and *Z* coordinates of the center of the sphere.`radius` - Sphere radius. | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | `[]` | | `constraints` | Adds constraints between bones. Each constraint requires `from` and `to`; the legacy-named `distance` value is optional.*Parameters:*`from` - Name of the `from` bone.`to` - Name of the destination bone.`distance` - Optional constraint strength (the actual length is calculated from the bones). Values below `1` create a flexible constraint; values greater than or equal to `1`, or omission of this parameter, create a rigid constraint. | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | `[]` | | `bones_in_mv_space` | Performs physical-bone calculations in model-view space. | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | `false` | | `damping` | Controls damping for the physics simulation. Recommended values are in the range `[0.9, 1.0]`. | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | `0.99` | note When creating a 3D model in GLTF format for an effect, we recommend using our [head geometry](/far-sdk/assets/files/head-a7e3b48b08e13d5c85ae83246dbe64cb.glb) as a template. If you create the model without our head geometry, set its scale to `0.1` and rotate it `-90` degrees around the `X` axis in the prefab. ## Video Texture[​](#video-texture "Direct link to Video Texture") ``` "faces": [ { "video_texture": { "@mesh": "path/to/gltf/model", "use_separate_alpha": true, "video": "path/to/video/texture", "alpha": "path/to/video/texture/alpha", "rotation": "0 0 0", "scale": "1.0 1.0 1.0", "translation": "0.0 0.0 0.0" } // ... } // ... ] ``` | Parameter | Description | Optional | Default Value | | :------------------- | :----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | :------------------------------------------------------------------: | :--------------: | | `@mesh` | Path to a GLTF model. The leading `@` in the parameter name is required. Supported formats are `.glb` and `.gltf`. A built-in plane is used by default. | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | `built-in plane` | | `use_separate_alpha` | Set this parameter to `false` for a combined video with color on the left and alpha on the right. Set it to `true` when color and alpha are provided as separate videos. | *+* | *+* | | `video` | Path to a video texture file. When `use_separate_alpha` is `false`, the video must be split in half, with color on the left and alpha on the right. For more information, see the supported [video formats](https://docs.banuba.com/far-sdk/tutorials/capabilities/technical_specification#video-formats-support). | *+* | *+* | | `alpha` | Path to the alpha video. This parameter is required when `use_separate_alpha` is `true` and ignored in combined mode. For more information, see the supported [video formats](https://docs.banuba.com/far-sdk/tutorials/capabilities/technical_specification#video-formats-support). | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | \`\` | | `rotation` | Rotation angles, in degrees, around the *X*, *Y*, and *Z* axes. | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | `"0 0 0"` | | `scale` | Scale along the *X*, *Y*, and *Z* axes. | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | `"1 1 1"` | | `translation` | Translates the model along the *X*, *Y*, and *Z* axes, in millimetres. | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | `"0 0 0"` | `use_separate_alpha` is required because playback does not start until the alpha mode is selected. Set it to `false` for a side-by-side video (color on the left, alpha on the right), or to `true` and provide `alpha` as a second video. Videos loop automatically. If `@mesh` is omitted, the built-in plane is used. ## Earrings[​](#earrings "Direct link to Earrings") ``` "faces": [ { "earrings": { "@mesh_left": "path/to/left/gltf/model", "@mesh_right": "path/to/right/gltf/model", "@use_physics": true, "left": { "scale": "1 1 1", "rotation": "0 0 0", "translation": "0 0 0", "animation": { "name": "static", "mode": "fixed" }, "gravity": "0.0 -1800.0 0.0", "damping": 0.99, "bones": { "Bone_L_1": 0.0, "Bone_L_2": 1.0, "Bone_L_3": 1.0, "Bone_L_4": 1.0, "Bone_L_5": 1.0 } }, "right": { "scale": "1 1 1", "rotation": "0 0 0", "translation": "0 0 0", "animation": { "name": "static", "mode": "fixed" }, "gravity": "0.0 -1800.0 0.0", "damping": 0.99, "bones": { "Bone_R_1": 0.0, "Bone_R_2": 1.0, "Bone_R_3": 1.0, "Bone_R_4": 1.0, "Bone_R_5": 1.0 } } } // ... } // ... ] ``` Place two GLTF earring models, one on each ear. | Parameter | Description | Optional | Default Value | | :------------- | :-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | :------------------------------------------------------------------: | :-----------: | | `@mesh_left` | Path to the GLTF model for the left earring. The leading `@` in the parameter name is required. Supported formats are `.glb` and `.gltf`. The model and all associated files, such as shaders, textures, and sounds, must be located in the same folder. | *+* | *+* | | `@mesh_right` | Path to the GLTF model for the right earring. The leading `@` in the parameter name is required. Supported formats are `.glb` and `.gltf`. The model and all associated files, such as shaders, textures, and sounds, must be located in the same folder. | *+* | *+* | | `@use_physics` | Loads the GLTF models with physics simulation. The leading `@` in the parameter name is required. | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | `true` | | `left` | Settings for the left earring. All nested parameters are optional.*Parameters:*`scale` - Scale as `X Y Z`. Default: `1 1 1`.`rotation` - Rotation in degrees as `X Y Z`. Default: `0 0 0`.`translation` - Translation in millimetres as `X Y Z`. Default: `0 0 0`.`animation` - Animation object with optional `name`, `mode`, and `seek_position`; `mode` is one of `off`, `loop`, `once`, `once_reversed`, or `fixed`.`gravity` - Gravity vector as `X Y Z`. Default: `0 0 0`.`damping` - Physics damping. Default: `0.99`.`bones` - Object mapping bone names to inverse masses. Default: `{}`. | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | `{}` | | `right` | Settings for the right earring. All nested parameters are optional.*Parameters:*`scale` - Scale as `X Y Z`. Default: `1 1 1`.`rotation` - Rotation in degrees as `X Y Z`. Default: `0 0 0`.`translation` - Translation in millimetres as `X Y Z`. Default: `0 0 0`.`animation` - Animation object with optional `name`, `mode`, and `seek_position`; `mode` is one of `off`, `loop`, `once`, `once_reversed`, or `fixed`.`gravity` - Gravity vector as `X Y Z`. Default: `0 0 0`.`damping` - Physics damping. Default: `0.99`.`bones` - Object mapping bone names to inverse masses. Default: `{}`. | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | `{}` | Transform, animation, and physics settings belong inside `left` or `right`; they are not top-level earring parameters. ## Action Units[​](#action-units "Direct link to Action Units") ``` "faces": [ { "action_units": {}, // ... } // ... ] ``` Expose tracked facial action-unit values to GLTF models. This prefab has no parameters. ## Eyes Whitening[​](#eyes-whitening "Direct link to Eyes Whitening") **Usage** ``` "faces": [ { "eyes_whitening": { "strength": 1.0 } // ... } // ... ] ``` Makes the eyes look more expressive by whitening them. | Parameter | Description | Optional | Default Value | | :--------- | :--------------------------------------------------------------------------- | :------: | :-----------: | | `strength` | Eye-whitening strength as a floating-point number in the range `[0.0, 1.0]`. | *+* | *+* | **Preview** ![Right image compare](/far-sdk/assets/images/original-8823b66ca7c9e5a54656f6ee9b77c29f.jpg)![Left image compare](/far-sdk/assets/images/EyesWhitening-53c06893890dff12356d60e292ee7b23.jpg) Drag ## Eyes Flare[​](#eyes-flare "Direct link to Eyes Flare") **Usage** ``` "faces": [ { "eyes_flare": { "strength": 1.0 } // ... } // ... ] ``` Apply flare to the eyes. | Parameter | Description | Optional | Default Value | | :--------- | :--------------------------------------------------------------------- | :------: | :-----------: | | `strength` | Flare brightness as a floating-point number in the range `[0.0, 1.0]`. | *+* | *+* | **Preview** ![Right image compare](/far-sdk/assets/images/original-8823b66ca7c9e5a54656f6ee9b77c29f.jpg)![Left image compare](/far-sdk/assets/images/EyesFlare-aaa9dfc2204ee98f6874e0556099d0c8.jpg) Drag ## Teeth Whitening[​](#teeth-whitening "Direct link to Teeth Whitening") **Usage** ``` "faces": [ { "teeth_whitening": { "strength": 1.0 } // ... } // ... ] ``` Apply whitening to the teeth. | Parameter | Description | Optional | Default Value | | :--------- | :----------------------------------------------------------------------------- | :------: | :-----------: | | `strength` | Teeth-whitening strength as a floating-point number in the range `[0.0, 1.0]`. | *+* | *+* | **Preview** ![Right image compare](/far-sdk/assets/images/original-8823b66ca7c9e5a54656f6ee9b77c29f.jpg)![Left image compare](/far-sdk/assets/images/teeth-ad89bfabc20c76750c21b2348693680b.jpg) Drag ## Softlight[​](#softlight "Direct link to Softlight") **Usage** ``` "faces": [ { "softlight": { "strength": 1.0, "texture": "path/to/file" } // ... } // ... ] ``` Apply softlight to the face. | Parameter | Description | Optional | Default Value | | :--------- | :----------------------------------------------------------------------- | :------------------------------------------------------------------: | :--------------------------: | | `strength` | Softlight strength as a floating-point number in the range `[0.0, 1.0]`. | *+* | *+* | | `texture` | Path to a custom softlight texture. | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | `built-in softlight texture` | **Preview** ![Right image compare](/far-sdk/assets/images/original-8823b66ca7c9e5a54656f6ee9b77c29f.jpg)![Left image compare](/far-sdk/assets/images/SkinSoftening-e7800133c7d654b8372366266bdbf026.jpg) Drag ## Morphing[​](#morphing "Direct link to Morphing") Morph (i.e., deform) specific parts of the face. **Usage** ``` "faces": [ { "morphing": { "eyebrows": { "spacing": 0.6, "height": 0.1, "bend": 1.0 }, "eyes": { "rounding": 0.6, "enlargement": 0.3, "height": 0, "spacing": 0.3, "squint": 0.3, "lower_eyelid_pos": 0, "lower_eyelid_size": 0, "down": 0, "eyelid_upper": 0, "eyelid_lower": 0 }, "face": { "narrowing": 0, "v_shape": 0, "cheekbones_narrowing": 0, "cheeks_narrowing": 0, "jaw_narrowing": 0, "chin_shortening": 0.3, "chin_narrowing": 0, "sunken_cheeks": 0.0, "cheeks_jaw_narrowing": 0, "jaw_wide_thin": 0, "chin": 0, "forehead": 0.3 }, "nose": { "width": 0.3, "length": 0.2, "tip_width": 0.1, "down_up": 0.1, "sellion": 0.2 }, "lips": { "size": 0.4, "height": 1.0, "thickness": 0.1, "mouth_size": 0.2, "smile": 0.8, "shape": 0.4, "sharp": 0.6 } } // ... } // ... ] ``` All settings are optional. Each group also accepts a numeric shorthand that controls its primary parameter: `eyebrows` controls `spacing`, `eyes` controls `rounding`, `face` controls `narrowing`, `nose` controls `width`, and `lips` controls `size`. For low-level control, `weights` accepts exactly 37 numbers in this order: eyebrow spacing, height, bend; eye enlargement, rounding, height, spacing, squint, lower-eyelid position, lower-eyelid size; nose length, width, tip width; lip height, size, thickness, mouth size, smile, shape; face narrowing, V-shape, cheekbone narrowing, cheek narrowing, jaw narrowing, chin shortening, chin narrowing, sunken cheeks, cheek-and-jaw narrowing, jaw wide/thin; nose down/up; eyes down, upper eyelid, lower eyelid; chin, forehead, nose sellion, and lip sharpness. The named groups are less error-prone and are preferred. ### Eyebrows[​](#eyebrows "Direct link to Eyebrows") | Parameter | Description | Optional | Default Value | | :-------- | :---------------------------------------------------------------- | :------------------------------------------------------------------: | :-----------: | | `spacing` | Adjusts the spacing between the eyebrows in the range *\[-1, 1]*. | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | `0.0` | | `height` | Raises or lowers the eyebrows in the range *\[-1, 1]*. | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | `0.0` | | `bend` | Adjusts the eyebrow bend in the range *\[-1, 1]*. | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | `0.0` | ### Eyes[​](#eyes "Direct link to Eyes") | Parameter | Description | Optional | Default Value | | :------------------ | :---------------------------------------------------------------------- | :------------------------------------------------------------------: | :-----------: | | `rounding` | Adjusts eye roundness in the range *\[0, 1]*. | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | `0.0` | | `enlargement` | Enlarges the eyes in the range *\[0, 1]*. | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | `0.0` | | `height` | Raises or lowers the eyes in the range *\[-1, 1]*. | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | `0.0` | | `spacing` | Adjusts the spacing between the eyes in the range *\[-1, 1]*. | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | `0.0` | | `squint` | Adjusts the eyelids to make the person squint, in the range *\[-1, 1]*. | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | `0.0` | | `lower_eyelid_pos` | Raises or lowers the lower eyelid in the range *\[-1, 1]*. | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | `0.0` | | `lower_eyelid_size` | Enlarges or shrinks the lower eyelid in the range *\[-1, 1]*. | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | `0.0` | | `down` | Moves the eyes downward in the range *\[0, 1]*. | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | `0.0` | | `eyelid_upper` | Adjusts the upper eyelid in the range *\[0, 1]*. | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | `0.0` | | `eyelid_lower` | Adjusts the lower eyelid in the range *\[0, 1]*. | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | `0.0` | ### Face[​](#face "Direct link to Face") | Parameter | Description | Optional | Default Value | | :--------------------- | :--------------------------------------------------------------------- | :------------------------------------------------------------------: | :-----------: | | `narrowing` | Narrows the face in the range *\[0, 1]*. | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | `0.0` | | `v_shape` | Shrinks the chin and narrows the cheeks in the range *\[0, 1]*. | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | `0.0` | | `cheekbones_narrowing` | Narrows the cheekbones in the range *\[-1, 1]*. | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | `0.0` | | `cheeks_narrowing` | Narrows the cheeks in the range *\[0, 1]*. | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | `0.0` | | `jaw_narrowing` | Narrows the jaw in the range *\[0, 1]*. | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | `0.0` | | `chin_shortening` | Shortens the chin in the range *\[0, 1]*. | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | `0.0` | | `chin_narrowing` | Narrows the chin in the range *\[0, 1]*. | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | `0.0` | | `sunken_cheeks` | Sinks the cheeks and emphasizes the cheekbones in the range *\[0, 1]*. | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | `0.0` | | `cheeks_jaw_narrowing` | Narrows the cheeks and jaw in the range *\[0, 1]*. | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | `0.0` | | `jaw_wide_thin` | Adjusts the jaw between wide and thin in the range *\[0, 1]*. | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | `0.0` | | `chin` | Adjusts the chin in the range *\[0, 1]*. | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | `0.0` | | `forehead` | Adjusts the forehead in the range *\[0, 1]*. | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | `0.0` | ### Nose[​](#nose "Direct link to Nose") | Parameter | Description | Optional | Default Value | | :---------- | :------------------------------------------------ | :------------------------------------------------------------------: | :-----------: | | `width` | Adjusts nose width in the range *\[-1, 1]*. | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | `0.0` | | `length` | Adjusts nose length in the range *\[-1, 1]*. | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | `0.0` | | `tip_width` | Adjusts nose-tip width in the range *\[0, 1]*. | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | `0.0` | | `down_up` | Moves the nose down or up in the range *\[0, 1]*. | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | `0.0` | | `sellion` | Adjusts the nose sellion in the range *\[0, 1]*. | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | `0.0` | ### Lips[​](#lips "Direct link to Lips") | Parameter | Description | Optional | Default Value | | :----------- | :----------------------------------------------------------------------- | :------------------------------------------------------------------: | :-----------: | | `size` | Adjusts the width and vertical size of the lips in the range *\[-1, 1]*. | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | `0.0` | | `height` | Raises or lowers the lips in the range *\[-1, 1]*. | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | `0.0` | | `thickness` | Adjusts lip thickness in the range *\[-1, 1]*. | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | `0.0` | | `mouth_size` | Adjusts mouth size in the range *\[-1, 1]*. | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | `0.0` | | `smile` | Adjusts the smile in the range *\[0, 1]*. | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | `0.0` | | `shape` | Adjusts lip shape in the range *\[-1, 1]*. | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | `0.0` | | `sharp` | Adjusts lip sharpness in the range *\[0, 1]*. | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | `0.0` | **Preview** ![Right image compare](/far-sdk/assets/images/original2-f74f044be90519e96c2b58b2db7ddce2.jpg)![Left image compare](/far-sdk/assets/images/morphing-92a63912987e8ddc904b6bf20cbea396.jpg) Drag ## Eyes[​](#eyes-1 "Direct link to Eyes") Recolors the eyes. **Usage** ``` "faces": [ { "eyes": { "eyes": "0 0.2 0.8 0.64", "corneosclera": "1 1 1 1", "pupil": "0 0 0 1" } // ... } // ... ] ``` | Parameter | Description | Optional | Default Value | | :------------- | :-------------------------------------------------------------- | :------------------------------------------------------------------: | :-----------: | | `eyes` | Iris color. See the color-format note in the prefabs overview. | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | `"0 0 0 0"` | | `corneosclera` | Corneosclera color, commonly called the sclera. | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | `"0 0 0 0"` | | `pupil` | Pupil color. See the color-format note in the prefabs overview. | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | `"0 0 0 0"` | **Preview** ![Right image compare](/far-sdk/assets/images/original-8823b66ca7c9e5a54656f6ee9b77c29f.jpg)![Left image compare](/far-sdk/assets/images/eyes-c3613cadd6bb4ebe4eccbf7084c669f5.jpg) Drag ## Hair[​](#hair "Direct link to Hair") Recolors the hair. A single color is typically used to set the hair color: ``` "faces": [ { "hair": { "color": [ "0.19 0.06 0.25 1.0" ] } // ... } // ... ] ``` Hair recoloring also supports two to five colors to create a vertical gradient. The following example uses two colors: ``` "faces": [ { "hair": { "color": [ "0.19 0.06 0.25 1.0", "0.09 0.25 0.38 1.0" ] } // ... } // ... ] ``` | Parameter | Description | Optional | Default Value | | :-------- | :----------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | :------------------------------------------------------------------: | :-----------: | | `color` | A single color string applies a solid color. An array of one to five colors is also accepted: one element applies a solid color, while two to five create a vertical gradient. | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | `"0 0 0"` | **Preview** ![Right image compare](/far-sdk/assets/images/original-8823b66ca7c9e5a54656f6ee9b77c29f.jpg)![Left image compare](/far-sdk/assets/images/hair_colors-e5dc1a08bb938c142a4519d44206d47f.jpg) Drag ## Hair Strands[​](#hair-strands "Direct link to Hair Strands") Recolors individual hair strands. This prefab supports one to five colors for recoloring different strands. ``` "faces": [ { "hair_strands": { "color": [ "0.80 0.40 0.40 1.0", "0.83 0.40 0.40 1.0", "0.85 0.75 0.75 1.0", "0.87 0.60 0.60 1.0", "0.99 0.65 0.65 1.0" ] } // ... } // ... ] ``` | Parameter | Description | Optional | Default Value | | :-------- | :------------------------------------------------------------------------------------------------------------------ | :------------------------------------------------------------------: | :-----------: | | `color` | Applies one to five colors to hair strands. Provide a single color string or an array containing up to five colors. | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | `"0 0 0 0"` | **Preview** ![Right image compare](/far-sdk/assets/images/original-8823b66ca7c9e5a54656f6ee9b77c29f.jpg)![Left image compare](/far-sdk/assets/images/hair_strands-0a1cc04eb7f5482cef64a0567d00716f.jpg) Drag --- # On Hands Prefabs [View as Markdown](https://docs.banuba.com/far-sdk/effects/prefabs/hands.md)[![](/far-sdk/img/ai-guide/chatgpt.svg)Open in ChatGPT](https://chatgpt.com/?q=Read%20https%3A%2F%2Fdocs.banuba.com%2Ffar-sdk%2Feffects%2Fprefabs%2Fhands.md%20\(On%20Hands%20Prefabs\)%20and%20help%20me%20with%20it%20in%20my%20Banuba%20Face%20AR%20SDK%20project.)[![](/far-sdk/img/ai-guide/claude.svg)Open in Claude](https://claude.ai/new?q=Read%20https%3A%2F%2Fdocs.banuba.com%2Ffar-sdk%2Feffects%2Fprefabs%2Fhands.md%20\(On%20Hands%20Prefabs\)%20and%20help%20me%20with%20it%20in%20my%20Banuba%20Face%20AR%20SDK%20project.)Install tools ### Nails[​](#nails "Direct link to Nails") **Usage** You can choose a solid color and a gloss value to recolor the nails. You can also apply textures over the selected color. ``` { "nails": { "color": "#FFFF49", "gloss": 40, "textures": [ "tex1.png", "tex2.png", "tex3.png", "tex4.png", "tex5.png" ] } } ``` | Parameter | Description | Optional | Default Value | | :--------- | :------------------------------------------------------------------------------------------------------------------- | :------------------------------------------------------------------: | :-----------: | | `color` | Sets the nail color. See the color-format note in the prefabs overview. | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | `"0 0 0 0"` | | `gloss` | Controls the glossiness of the nail color. The recommended range is `[0, 60]`; the runtime does not clamp the value. | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | `40` | | `textures` | An array of up to five texture filenames, one per nail. Textures are applied over the selected color. | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | `[]` | **Preview** ![Right image compare](/far-sdk/assets/images/Right_before-4fb8bc03b86de05223f45d7c0bebc10f.png)![Left image compare](/far-sdk/assets/images/Right_after-0742345b61bc4398c8c0ff3e5c83f3d3.png) Drag note The nail-coloring feature requires the [nail segmentation neural network](/far-sdk/tutorials/capabilities/sdk_features.md). Please make sure your licence plan includes this feature by contacting your sales manager or filling out the website form to request it. --- # Makeup Prefabs [View as Markdown](https://docs.banuba.com/far-sdk/effects/prefabs/makeup.md)[![](/far-sdk/img/ai-guide/chatgpt.svg)Open in ChatGPT](https://chatgpt.com/?q=Read%20https%3A%2F%2Fdocs.banuba.com%2Ffar-sdk%2Feffects%2Fprefabs%2Fmakeup.md%20\(Makeup%20Prefabs\)%20and%20help%20me%20with%20it%20in%20my%20Banuba%20Face%20AR%20SDK%20project.)[![](/far-sdk/img/ai-guide/claude.svg)Open in Claude](https://claude.ai/new?q=Read%20https%3A%2F%2Fdocs.banuba.com%2Ffar-sdk%2Feffects%2Fprefabs%2Fmakeup.md%20\(Makeup%20Prefabs\)%20and%20help%20me%20with%20it%20in%20my%20Banuba%20Face%20AR%20SDK%20project.)Install tools ### Basic Concepts[​](#basic-concepts "Direct link to Basic Concepts") Each makeup prefab represents a specific region of the face and uses a similar set of settings: ``` // ... { "color": "0.95 0.70 0.54", "finish": "natural", "coverage": "mid" } // ... ``` `color` - A required solid color for the region in RGB string format. `finish` - A required finish preset (for example, `natural` or `matte`). See available options for the specific region below. `coverage` - A required coverage intensity. It can be `low`, `mid`, `high`, or a number in `[0.0, 1.0]`. The string shorthands `l`/`lo`, `m`/`mi`, and `h`/`hi`/`hig` are also accepted. `makeup_eyelashes` supports only the `mid` and `high` presets (or a numeric value). Generally, makeup doesn't work with multiple faces, but some makeup effects may support 2 and more faces. Check out the description. ### Makeup Base[​](#makeup-base "Direct link to Makeup Base") ``` "faces": [ { "makeup_base": { "mode": "quality", "smooth": "0 1", "smokey": 0 } // ... } // ... ] ``` | Parameter | Description | Optional | Default Value | | :-------- | :-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | :------------------------------------------------------------------: | :-----------: | | `mode` | Processing mode. Allowed values are `speed` and `quality`; `quality` uses more computationally intensive algorithms. 'quality' mode is not available with multiple faces makeup config. | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | `"speed"` | | `smooth` | Smooths facial skin. The value contains two strengths in the range `[0, 1]`: `whole face` and `under eyes`. | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | `"0 1"` | | `smokey` | Smoky-eye index. A value of `0` disables the smoky-eye effect; `1`, `2`, `3`, or `4` selects a different set of smoky-eye textures for the first two eyeshadow layers. | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | `0` | note To enable multifaces feature for `makeup_base`, create another array in `faces` list and add `makeup_base` config to it. Parameters for faces with indexes 1,2... must correspond to the parameters for the face with index 0. ``` "faces": [ { "makeup_base": { "mode": "speed", "smooth": "0 1", "smokey": 0 } // ... }, { "makeup_base": { "mode": "speed", "smooth": "0 1", "smokey": 0 } // ... } // ... ] ``` ### Eye Bags[​](#eyebags "Direct link to Eye Bags") **Usage** ``` "faces": [ { "makeup_eyebags": { "alpha": 0.8 }, // ... } // ... ] ``` `alpha` *(optional)* - The opacity of under-eye circles, in the range `[0, 1]`. The default value is `0.8`. **Preview** ![Right image compare](/far-sdk/assets/images/original-8823b66ca7c9e5a54656f6ee9b77c29f.jpg)![Left image compare](/far-sdk/assets/images/eyebags-9cbd40d2add99adcaa5df79b82b4b3cf.jpg) Drag ### Foundation[​](#foundation "Direct link to Foundation") **Usage** ``` "faces": [ { "makeup_foundation": { "color": "0.95 0.70 0.54", "finish": "natural", "coverage": "mid" }, // ... } // ... ] ``` `finish` - One of: `natural`, `matte`, `radiance`. **Preview** ![Right image compare](/far-sdk/assets/images/original-8823b66ca7c9e5a54656f6ee9b77c29f.jpg)![Left image compare](/far-sdk/assets/images/foundation-722812ce8d57e2adc8fd9492c47689f1.jpg) Drag ### Concealer[​](#concealer "Direct link to Concealer") **Usage** ``` "faces": [ { "makeup_concealer": { "color": "0.94 0.73 0.66", "finish": "natural", "coverage": "mid" }, // ... } // ... ] ``` `finish` - One of: `natural`, `matte`. note To enable multifaces feature for `makeup_concealer`, create another array in `faces` list and add `makeup_concealer` config to it. Parameters for faces with indexes 1,2... must correspond to the parameters for the face with index 0. ``` "faces": [ { "makeup_concealer": { "color": "0.94 0.73 0.66", "finish": "natural", "coverage": "mid" }, // ... }, { "makeup_concealer": { "color": "0.94 0.73 0.66", "finish": "natural", "coverage": "mid" }, // ... } // ... ] ``` **Preview** ![Right image compare](/far-sdk/assets/images/original-8823b66ca7c9e5a54656f6ee9b77c29f.jpg)![Left image compare](/far-sdk/assets/images/concealer-9f1a17f7cb038d705dce5b41732d744c.jpg) Drag ### Contour[​](#contour "Direct link to Contour") **Usage** ``` "faces": [ { "makeup_contour": { "color": "1 0 0", "finish": "normal", "coverage": "mid" }, // ... } // ... ] ``` `finish` - One of: `normal`. note To enable multifaces feature for `makeup_contour`, create another array in `faces` list and add `makeup_contour` config to it. Parameters for faces with indexes 1,2... must correspond to the parameters for the face with index 0. ``` "faces": [ { "makeup_contour": { "color": "1 0 0", "finish": "normal", "coverage": "mid" }, // ... }, { "makeup_contour": { "color": "1 0 0", "finish": "normal", "coverage": "mid" }, // ... } // ... ] ``` **Preview** ![Right image compare](/far-sdk/assets/images/original-8823b66ca7c9e5a54656f6ee9b77c29f.jpg)![Left image compare](/far-sdk/assets/images/contour-5841a00c527aea7d50ed6210be0c5705.jpg) Drag ### Highlighter[​](#highlighter "Direct link to Highlighter") **Usage** ``` "faces": [ { "makeup_highlighter": { "color": "0.9 0.80 0.83", "finish": "shimmer", "coverage": "mid" }, // ... } // ... ] ``` `finish` - One of: `shimmer`. note To enable multifaces feature for `makeup_highlighter`, create another array in `faces` list and add `makeup_highlighter` config to it. Parameters for faces with indexes 1,2... must correspond to the parameters for the face with index 0. ``` "faces": [ { "makeup_highlighter": { "color": "0.9 0.80 0.83", "finish": "shimmer", "coverage": "mid" }, // ... }, { "makeup_highlighter": { "color": "0.9 0.80 0.83", "finish": "shimmer", "coverage": "mid" }, // ... } // ... ] ``` **Preview** ![Right image compare](/far-sdk/assets/images/original-8823b66ca7c9e5a54656f6ee9b77c29f.jpg)![Left image compare](/far-sdk/assets/images/highlighter-bd43a4e867e535aad0d65353762c63a8.jpg) Drag ### Blush[​](#blush "Direct link to Blush") **Usage** ``` "faces": [ { "makeup_blush": { "color": "0.88 0.65 0.75", "finish": "shimmer", "coverage": "mid" }, // ... } // ... ] ``` `finish` - One of: `shimmer`, `matte`, `cream_shine`. note To enable multifaces feature for `makeup_blush`, create another array in `faces` list and add `makeup_blush` config to it. Parameters for faces with indexes 1,2... must correspond to the parameters for the face with index 0. ``` "faces": [ { "makeup_blush": { "color": "0.88 0.65 0.75", "finish": "shimmer", "coverage": "mid" }, // ... }, { "makeup_blush": { "color": "0.88 0.65 0.75", "finish": "shimmer", "coverage": "mid" }, // ... } // ... ] ``` **Preview** ![Right image compare](/far-sdk/assets/images/original-8823b66ca7c9e5a54656f6ee9b77c29f.jpg)![Left image compare](/far-sdk/assets/images/blush-af461e4c729818d407a7233758243c03.jpg) Drag ### Lipstick[​](#lipstick "Direct link to Lipstick") **Usage** ``` "faces": [ { "makeup_lipstick": { "color": "0.88 0.47 0.61", "finish": "shimmer", "coverage": "high" }, // ... } // ... ] ``` `finish` - One of: `shimmer`, `matte_dry`, `matte_cream`, `matte_powder`, `matte_liquid`, `cream_shine`, `glossy_cream_plumping`, `balm`, `balm_light`, `glossy_cream_shimmer`, `cream_vividcolors`, `matte_velvet`, `cream_shine_glitter`, `matte_velvet_sparkling`, `matte_sheer_lightcolors`, `matte_cream_vividcolors`, `metallic_cream`, `matte_velvet_sparkling_lightcolors`, `matte_light`, `metallic_shine`, `metallic_dry_lightcolors`, `satin`, `shine`, `cream`, `clear`, `cream_darkcolors`, `clear_shimmer`, `metallic_sheer`. **Preview** ![Right image compare](/far-sdk/assets/images/original2-f74f044be90519e96c2b58b2db7ddce2.jpg)![Left image compare](/far-sdk/assets/images/lipstick-4b6bc4d5377a5fb262ad948e7488b9bf.jpg) Drag ### Lip Liner[​](#lipsliner "Direct link to Lip Liner") **Usage** ``` "faces": [ { "makeup_lipsliner": { "color": "0.99 0.0 0.0", "finish": "shimmer", "coverage": "high", "liner": "3 5" }, // ... } // ... ] ``` `finish` - One of: `shimmer`. `liner` *(optional)* - Contains two values: `liner width` and `softness`. The default value is `3 5`. **Preview** ![Right image compare](/far-sdk/assets/images/original2-f74f044be90519e96c2b58b2db7ddce2.jpg)![Left image compare](/far-sdk/assets/images/lipsliner-c446fc27d392e9716288209b30a47e58.jpg) Drag ### Eyeshadow[​](#eyeshadow "Direct link to Eyeshadow") **Usage** ``` "faces": [ { "makeup_eyeshadow": [ { "color": "0.21 0.42 0.32", "finish": "matte", "coverage": "high" }, { "color": "0.3 0.58 0.47", "finish": "shimmer", "coverage": "high" }, { "color": "1.00 0.91 0.27", "finish": "metallic", "coverage": "high" } ], // ... } // ... ] ``` The value can be an object containing one eyeshadow definition or an array of up to three eyeshadow definitions applied in sequence. `finish` - One of: `shimmer`, `matte`, `matte_powder`, `glitter_metallic`, `glitter`, `metallic`, `glitter_sheer`, `cream`. note To enable multifaces feature for `makeup_eyeshadow`, create another array in `faces` list and add `makeup_eyeshadow` config to it. Parameters for faces with indexes 1,2... must correspond to the parameters for the face with index 0. ``` "faces": [ { "makeup_eyeshadow": [ { "color": "0.21 0.42 0.32", "finish": "matte", "coverage": "high" }, { "color": "0.3 0.58 0.47", "finish": "shimmer", "coverage": "high" }, { "color": "1.00 0.91 0.27", "finish": "metallic", "coverage": "high" } ], // ... }, { "makeup_eyeshadow": [ { "color": "0.21 0.42 0.32", "finish": "matte", "coverage": "high" }, { "color": "0.3 0.58 0.47", "finish": "shimmer", "coverage": "high" }, { "color": "1.00 0.91 0.27", "finish": "metallic", "coverage": "high" } ], // ... } // ... ] ``` **Preview** ![Right image compare](/far-sdk/assets/images/original-8823b66ca7c9e5a54656f6ee9b77c29f.jpg)![Left image compare](/far-sdk/assets/images/eyeshadow-ebee6f8b0da993382f550ad5872534a8.jpg) Drag ### Eyelashes[​](#eyelashes "Direct link to Eyelashes") **Usage** ``` "faces": [ { "makeup_eyelashes": { "color": "0 0 0", "finish": "volume", "coverage": "high" }, // ... } // ... ] ``` `finish` - One of: `volume`, `lengthening`, `lengthandvolume`, `natural`, `natural_bottom`. note To enable multifaces feature for `makeup_eyelashes`, create another array in `faces` list and add `makeup_eyelashes` config to it. Parameters for faces with indexes 1,2... must correspond to the parameters for the face with index 0. ``` "faces": [ { "makeup_eyelashes": { "color": "0 0 0", "finish": "volume", "coverage": "high" }, // ... }, { "makeup_eyelashes": { "color": "0 0 0", "finish": "volume", "coverage": "high" }, // ... } // ... ] ``` **Preview** ![Right image compare](/far-sdk/assets/images/original-8823b66ca7c9e5a54656f6ee9b77c29f.jpg)![Left image compare](/far-sdk/assets/images/eyelashes-d10119917263609bb8a19279278ebd76.jpg) Drag ### Eyeliner[​](#eyeliner "Direct link to Eyeliner") **Usage** ``` "faces": [ { "makeup_eyeliner": { "color": "0.0 0.0 0.0", "finish": "matte_liquid", "coverage": "high" }, // ... } // ... ] ``` `finish` - One of: `shimmer`, `cream`, `matte_liquid`, `matte_cream`, `matte_dark`, `metallic`, `glitter`, `cream_lightcolors`. note To enable multifaces feature for `makeup_eyeliner`, create another array in `faces` list and add `makeup_eyeliner` config to it. Parameters for faces with indexes 1,2... must correspond to the parameters for the face with index 0. ``` "faces": [ { "makeup_eyeliner": { "color": "0.0 0.0 0.0", "finish": "matte_liquid", "coverage": "high" }, // ... }, { "makeup_eyeliner": { "color": "0.0 0.0 0.0", "finish": "matte_liquid", "coverage": "high" }, // ... } // ... ] ``` **Preview** ![Right image compare](/far-sdk/assets/images/original-8823b66ca7c9e5a54656f6ee9b77c29f.jpg)![Left image compare](/far-sdk/assets/images/eyeliner-4d7a9a99580d76059b9d2fbcac49b5b9.jpg) Drag ### Eyebrows[​](#eyebrows "Direct link to Eyebrows") **Usage** ``` "faces": [ { "makeup_eyebrows": { "color": "0.39 0.26 0.27", "finish": "matte", "coverage": "mid" }, // ... } // ... ] ``` `finish` - One of: `matte`, `wet`, `clear`. **Preview** ![Right image compare](/far-sdk/assets/images/original-8823b66ca7c9e5a54656f6ee9b77c29f.jpg)![Left image compare](/far-sdk/assets/images/eyebrows-80609d099b6ebe704df37dc3d7203c39.jpg) Drag ### Lip Shine[​](#lipsshine "Direct link to Lip Shine") **Usage** ``` "faces": [ { "makeup_lipsshine": { "color": "1.0 0.0 0.0", "finish": "glitter", "coverage": "mid" }, // ... } // ... ] ``` `finish` - One of: `shine`, `glitter`. **Preview** ![Right image compare](/far-sdk/assets/images/original2-f74f044be90519e96c2b58b2db7ddce2.jpg)![Left image compare](/far-sdk/assets/images/lipshine-71bf9d4d51904c26b6347475215ee75b.jpg) Drag ### Lip Gloss[​](#lips-gloss "Direct link to Lip Gloss") **Usage** ``` "faces": [ { "makeup_lipsgloss": { "threshold": 0.96, "contour": 0.45, "weakness": 0.5, "multiplier": 1.5, "saturation": 0.1, "alpha": 0.8 }, // ... } // ... ] ``` | Parameter | Description | Optional | Default Value | | :----------- | :------------------------------------------------------------------ | :------------------------------------------------------------------: | :-----------: | | `threshold` | Gloss threshold. Higher values produce a smaller highlight. | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | `0.95` | | `contour` | Highlight contour softness. | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | `0.5` | | `weakness` | Highlight-mask weakness. Higher values produce a smaller highlight. | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | `0.5` | | `multiplier` | Highlight-mask multiplier. | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | `1.5` | | `alpha` | Highlight visibility. | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | `0.0` | | `saturation` | Gloss saturation. | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | `0.1` | **Preview** ![Right image compare](/far-sdk/assets/images/lipsgloss1-3552ded6972501314cbd7ec57bf8a502.jpg)![Left image compare](/far-sdk/assets/images/lipsgloss2-81fa5f99c282bfe870937e83c7435bd6.jpg) Drag ### Lip Chameleon[​](#lips-chameleon "Direct link to Lip Chameleon") **Usage** ``` "faces": [ { "makeup_lipschameleon": { "colors": [ { "color": "#3342b0", "finish": "metallic_cream", "coverage": "high" }, { "color": "#a028b0", "finish": "metallic_cream", "coverage": "high" } ], "threshold": 0.92, "contour": 0.3, "weakness": 0.25 } // ... } // ... ] ``` | Parameter | Description | Optional | Default Value | | :---------- | :------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | :------------------------------------------------------------------: | :-----------: | | `colors` | An array containing exactly two lipstick objects.*Parameters:*`color` - Required solid color in RGB string format.`finish` - Required lipstick finish; accepts the same preset values as `makeup_lipstick`.`coverage` - Required coverage preset or a number in `[0, 1]`; `strength` is accepted as a legacy alias. | *+* | *+* | | `threshold` | Chameleon-mask threshold. Higher values produce a smaller mask. | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | `0.92` | | `contour` | Chameleon-mask contour softness. | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | `0.3` | | `weakness` | Chameleon-mask weakness. Higher values produce a smaller mask. | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | `0.25` | `colors` is required. `threshold`, `contour`, and `weakness` are optional and use the defaults shown in the table. The runtime applies exactly the two entries in `colors`. **Preview** ![Right image compare](/far-sdk/assets/images/original2-f74f044be90519e96c2b58b2db7ddce2.jpg)![Left image compare](/far-sdk/assets/images/lipschameleon-ba7ea6e56fe0a7e7ecf62493feec9d29.jpg) Drag ### Eyelid Gloss[​](#eyelids-gloss "Direct link to Eyelid Gloss") **Usage** ``` "faces": [ { "makeup_eyelidsgloss": { "threshold": 0.95, "contour": 0.6, "weakness": 0.6, "multiplier": 1.5, "saturation": 0.1, "alpha": 1 } // ... } // ... ] ``` | Parameter | Description | Optional | Default Value | | :----------- | :------------------------------------------------------------------ | :------------------------------------------------------------------: | :-----------: | | `threshold` | Gloss threshold. Higher values produce a smaller highlight. | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | `0.98` | | `contour` | Highlight contour softness. | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | `0.5` | | `weakness` | Highlight-mask weakness. Higher values produce a smaller highlight. | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | `0.5` | | `multiplier` | Highlight-mask multiplier. | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | `1.5` | | `alpha` | Highlight visibility. | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | `0.0` | | `saturation` | Gloss saturation. | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | `0.1` | **Preview** ![Right image compare](/far-sdk/assets/images/eyelidsgloss1-88a2d06dcc4e82d1fe9212e9a54c3b0c.jpg)![Left image compare](/far-sdk/assets/images/eyelidsgloss2-d0fbc506fb3f1e7024ca6b3e34572a05.jpg) Drag ### Eyelid Chameleon[​](#eyelids-chameleon "Direct link to Eyelid Chameleon") **Usage** ``` "faces": [ { "makeup_eyelids_chameleon": { "colors": { "bottom": [ { "color": "#3342b0", "finish": "shimmer", "coverage": "high" }, { "color": "#3342b0", "finish": "shimmer", "coverage": "high" }, { "color": "#3342b0", "finish": "shimmer", "coverage": "high" } ], "upper": [ { "color": "#a028b0", "finish": "shimmer", "coverage": "high" }, { "color": "#a028b0", "finish": "shimmer", "coverage": "high" }, { "color": "#a028b0", "finish": "shimmer", "coverage": "high" } ] }, "threshold": 0.92, "contour": 0.3, "weakness": 0.25 } // ... } // ... ] ``` | Parameter | Description | Optional | Default Value | | :---------- | :-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | :------------------------------------------------------------------: | :-----------: | | `colors` | An object containing `bottom` and/or `upper` arrays. At least one array is required, and each array accepts one to three color definitions.*Parameters:*`bottom` - Optional array of one to three `{color, finish, coverage}` definitions.`upper` - Optional array of one to three `{color, finish, coverage}` definitions. | *+* | *+* | | `threshold` | Chameleon-mask threshold. Higher values produce a smaller mask. | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | `0.92` | | `contour` | Chameleon-mask contour softness. | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | `0.3` | | `weakness` | Chameleon-mask weakness. Higher values produce a smaller mask. | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | `0.25` | `colors` must contain `bottom`, `upper`, or both. Each array accepts one to three definitions. `threshold`, `contour`, and `weakness` are optional and use the defaults shown in the table. **Preview** ![Right image compare](/far-sdk/assets/images/original2-f74f044be90519e96c2b58b2db7ddce2.jpg)![Left image compare](/far-sdk/assets/images/eyelidschameleon-2a37c1d6e58bfad1fb2fd1470591e1e4.jpg) Drag ### Lip Glitter[​](#lips-glitter "Direct link to Lip Glitter") **Usage** ``` "faces": [ { "makeup_lips_glitter": { "color": "0.7 0.3 0.7", "alpha": 0.75, "light_direction": "0.5 0.3 0.8" } // ... } // ... ] ``` | Parameter | Description | Optional | Default Value | | :---------------- | :----------------------------------------------------------------------------------------------------------------------------- | :------------------------------------------------------------------: | :-------------: | | `color` | Glitter color in RGB format. | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | `"0 0 0"` | | `alpha` | Glitter effect visibility. | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | `0.0` | | `light_direction` | Light-direction vector. The default is used when the squared magnitude of the supplied vector is less than or equal to `0.01`. | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | `"0.5 0.3 0.8"` | **Preview** ![Right image compare](/far-sdk/assets/images/lips_glitter_right-220262837fd3b237c9a5e7b8fd06eec0.jpg)![Left image compare](/far-sdk/assets/images/lips_glitter_left-15bb0ecbc4cbc280253f24d1ff3581a7.jpg) Drag ### Eyelid Glitter[​](#eyelids-glitter "Direct link to Eyelid Glitter") **Usage** ``` "faces": [ { "makeup_eyelids_glitter": { "color": "0.7 0.3 0.7", "alpha": 0.75, "light_direction": "0.5 -0.6 0.7" } // ... } // ... ] ``` | Parameter | Description | Optional | Default Value | | :---------------- | :----------------------------------------------------------------------------------------------------------------------------- | :------------------------------------------------------------------: | :--------------: | | `color` | Glitter color in RGB format. | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | `"0 0 0"` | | `alpha` | Glitter effect visibility. | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | `0.0` | | `light_direction` | Light-direction vector. The default is used when the squared magnitude of the supplied vector is less than or equal to `0.01`. | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | `"0.5 -0.6 0.7"` | **Preview** ![Right image compare](/far-sdk/assets/images/eyelids_glitter_right-dbeb8a7e5fcff9d94c5c3f8ee7a757ca.jpg)![Left image compare](/far-sdk/assets/images/eyelids_glitter_left-48738db41cdc246b01d5b596b8780133.jpg) Drag ## All Features[​](#all-features "Direct link to All Features") Let's combine all features: **Example** ``` { "faces": [ { "makeup_base": { "smooth": "0 1", "mode": "quality" }, "makeup_eyebags": { "alpha": 0.8 }, "makeup_foundation": { "color": "0.95 0.70 0.54", "finish": "natural", "coverage": "mid" }, "makeup_concealer": { "color": "0.94 0.73 0.66", "finish": "natural", "coverage": 1 }, "makeup_contour": { "color": "1 0 0", "finish": "normal", "coverage": "mid" }, "makeup_highlighter": { "color": "0.9 0.80 0.83", "finish": "shimmer", "coverage": "mid" }, "makeup_blush": { "color": "0.88 0.65 0.75", "finish": "shimmer", "coverage": "mid" }, "makeup_lipstick": { "color": "0.88 0.47 0.61", "finish": "shimmer", "coverage": "high" }, "makeup_lipsliner": { "color": "0.99 0.0 0.0", "finish": "shimmer", "coverage": "high", "liner": "3 5" }, "makeup_eyeshadow": [ { "color": "0.21 0.42 0.32", "finish": "matte", "coverage": "high" }, { "color": "0.3 0.58 0.47", "finish": "shimmer", "coverage": "high" }, { "color": "1.00 0.91 0.27", "finish": "metallic", "coverage": "high" } ], "makeup_eyeliner": { "color": "0.0 0.0 0.0", "finish": "matte_liquid", "coverage": "high" }, "makeup_eyebrows": { "color": "0.39 0.26 0.27", "finish": "matte", "coverage": "mid" }, "makeup_lipsshine": { "color": "1.0 0.0 0.0", "finish": "glitter", "coverage": "mid" }, "makeup_eyelashes": { "color": "1 0 0", "finish": "volume", "coverage": "high" }, "makeup_lipsgloss": { "threshold": 0.96, "contour": 0.45, "weakness": 0.5, "multiplier": 1.5, "saturation": 0.1, "alpha": 0.8 }, "makeup_eyelidsgloss": { "threshold": 0.95, "contour": 0.6, "weakness": 0.6, "multiplier": 1.5, "saturation": 0.1, "alpha": 1 } } ], "scene": "test_makeup", "version": "2.0.0" } ``` **Preview** ![Right image compare](/far-sdk/assets/images/original2-f74f044be90519e96c2b58b2db7ddce2.jpg)![Left image compare](/far-sdk/assets/images/all-fa62ce77e0126c7c957ad612dfb34859.jpg) Drag --- # Prefabs Overview [View as Markdown](https://docs.banuba.com/far-sdk/effects/prefabs/overview.md)[![](/far-sdk/img/ai-guide/chatgpt.svg)Open in ChatGPT](https://chatgpt.com/?q=Read%20https%3A%2F%2Fdocs.banuba.com%2Ffar-sdk%2Feffects%2Fprefabs%2Foverview.md%20\(Prefabs%20Overview\)%20and%20help%20me%20with%20it%20in%20my%20Banuba%20Face%20AR%20SDK%20project.)[![](/far-sdk/img/ai-guide/claude.svg)Open in Claude](https://claude.ai/new?q=Read%20https%3A%2F%2Fdocs.banuba.com%2Ffar-sdk%2Feffects%2Fprefabs%2Foverview.md%20\(Prefabs%20Overview\)%20and%20help%20me%20with%20it%20in%20my%20Banuba%20Face%20AR%20SDK%20project.)Install tools A prefab is a high-level object that represents a set of rendering and SDK features. Prefabs are divided into several types: * [On Face](/far-sdk/effects/prefabs/face.md) - prefabs that depend on a face. * [Makeup](/far-sdk/effects/prefabs/makeup.md) - prefabs that represent makeup features and also depend on a face. * [Top Level](/far-sdk/effects/prefabs/top_level.md) - prefabs that affect the entire screen and are not attached to a face. * [On Hands](/far-sdk/effects/prefabs/hands.md) - prefabs that apply effects to hands. * [Sprites](/far-sdk/effects/prefabs/sprites.md#sprites) - prefabs that represent simple 2D sprites. * [Sounds](/far-sdk/effects/prefabs/sounds.md) - prefabs that represent audio. Example prefab configuration: ``` { "scene": "effect name", "version": "2.0.0", "camera": {}, // Top level prefabs "background": { // ... }, "foreground": { // ... }, "lut": { // ... }, "lights": { // ... }, "msaa": { // ... }, // On Face "faces": [ { // first face "face_prefab1": { //... }, "makeup_prefab1": { //... } // ... }, { // second face } // .. ], // Sounds "sounds": [ { "sound_prefab1": { //... }, "sound_prefab2": { //... } // ... } ], // Sprites "sprites": [ { "sprite_prefab1": { //... }, "sprite_prefab2": { //... } // ... } // ... ] } ``` Where: * `scene` - the name of your effect. * `version` - the version of this configuration file. Always set it to `2.0.0`. Earlier versions are intended for complex legacy effects. * `camera` - indicates that the camera feed will be rendered on the screen. * `faces` - an array of JSON objects describing the features to place on each face. For each face, define a JSON object whose keys are prefab names and whose values are the parameters for those prefabs. * `top_level_prefab` - one of the top-level prefabs. * `sprites` - an array of JSON objects describing sprite features. * `sounds` - an array of JSON objects describing sound features. tip You can create an effect with any set of prefabs. tip You can change an effect at runtime by calling the `reload_config()` or `reloadConfig()` method: * C++ * Java * Swift * JavaScript ``` constexpr auto new_config = R"( { "camera" : {}, "background" : { // ... } } )"; effect_player->effect_manager()->reload_config(new_config); ``` ``` import com.banuba.sdk.player.Player // ... val newConfig = "" " { "camera" : {}, "background" : { // ... } } "" "; player.effectPlayer.effectManager() .reloadConfig(newConfig) ``` ``` import BanubaEffectPlayer // ... let newConfig = """ { "camera": {}, "background": { // ... } } """; player.effectPlayer?.effectManager().reloadConfig(newConfig) ``` ``` const new_config = `{ "camera": {}, "background": { // ... } }` player._effectManager.reloadConfig(new_config) ``` note Colors are represented as three- or four-component strings. Each component is a value in the range *\[0, 1]* or *\[0, 255]*, and components are separated by spaces. For example, `1 0 0 1` represents red. HTML-style hex strings are also accepted. For example, `#00FF00` represents green; an alpha value of `FF` is assumed when omitted. --- # Sounds Prefabs [View as Markdown](https://docs.banuba.com/far-sdk/effects/prefabs/sounds.md)[![](/far-sdk/img/ai-guide/chatgpt.svg)Open in ChatGPT](https://chatgpt.com/?q=Read%20https%3A%2F%2Fdocs.banuba.com%2Ffar-sdk%2Feffects%2Fprefabs%2Fsounds.md%20\(Sounds%20Prefabs\)%20and%20help%20me%20with%20it%20in%20my%20Banuba%20Face%20AR%20SDK%20project.)[![](/far-sdk/img/ai-guide/claude.svg)Open in Claude](https://claude.ai/new?q=Read%20https%3A%2F%2Fdocs.banuba.com%2Ffar-sdk%2Feffects%2Fprefabs%2Fsounds.md%20\(Sounds%20Prefabs\)%20and%20help%20me%20with%20it%20in%20my%20Banuba%20Face%20AR%20SDK%20project.)Install tools ## Sounds[​](#sounds "Direct link to Sounds") This array contains audio tracks associated with the effect. You can enable several audio tracks at once. ### Audio[​](#audio "Direct link to Audio") A simple background audio track that loops by default. **Usage** ``` "sounds": [ { "audio": { "filename": "sound.ogg", "start": 0, "end": 10, "loop": true, "volume": 0.7 } // ... } // ... ] ``` | Parameter | Description | Optional | Default Value | | :--------- | :------------------------------------------------------------------------------------------ | :------------------------------------------------------------------: | :-----------: | | `filename` | Path to an audio track. The supported formats are `.ogg` and `.wav`. | *+* | *+* | | `start` | Playback start position in seconds. | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | `0` | | `end` | Playback end position in seconds. When omitted, playback continues to the end of the track. | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | `null` | | `loop` | Loops the audio track. | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | `true` | | `volume` | Audio-track volume in the range `[0, 1]`. | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | `1` | note To use the audio prefab on the web, you must call `player.setVolume(1);` in your code. --- # Sprites Prefabs [View as Markdown](https://docs.banuba.com/far-sdk/effects/prefabs/sprites.md)[![](/far-sdk/img/ai-guide/chatgpt.svg)Open in ChatGPT](https://chatgpt.com/?q=Read%20https%3A%2F%2Fdocs.banuba.com%2Ffar-sdk%2Feffects%2Fprefabs%2Fsprites.md%20\(Sprites%20Prefabs\)%20and%20help%20me%20with%20it%20in%20my%20Banuba%20Face%20AR%20SDK%20project.)[![](/far-sdk/img/ai-guide/claude.svg)Open in Claude](https://claude.ai/new?q=Read%20https%3A%2F%2Fdocs.banuba.com%2Ffar-sdk%2Feffects%2Fprefabs%2Fsprites.md%20\(Sprites%20Prefabs\)%20and%20help%20me%20with%20it%20in%20my%20Banuba%20Face%20AR%20SDK%20project.)Install tools ## Sprites[​](#sprites "Direct link to Sprites") This array contains sprite-based prefabs such as the hint. ### Hint[​](#hint "Direct link to Hint") Displays a text message on the screen. **Usage** ``` "sprites": [ { "hint": { "text": "Simple Text!", "font": "path/to/font/file", "color": "1 0 0 1", "size": "30 10", "translation": "0 0", "rotation": "0 0 0" } // ... } // ... ] ``` | Parameter | Description | Optional | Default Value | | :------------ | :--------------------------------------------------------------------------------------------------------------------------------------------- | :------------------------------------------------------------------: | :------------: | | `text` | The message to display. | *+* | *+* | | `font` | Path to the font file. Supported formats are `.otf`, `.otc`, `.ttf`, and `.ttc`. | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | `default font` | | `color` | Text color. When using multiple hints, set the color explicitly for each hint; each hint supports only one color. | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | `"1 1 1 1"` | | `size` | Size of the text area, expressed as `X Y` percentages of the geometric mean of the screen dimensions. Each component is clamped to `[0, 150]`. | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | `"30 10"` | | `translation` | Translates the center of the hint relative to the center of the screen along the *X* and *Y* axes, as percentages of the screen dimensions. | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | `"0 0"` | | `rotation` | Rotation angles in degrees. Only rotation around the *Z* axis is supported. | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | `"0 0 0"` | **Preview** ![Right image compare](/far-sdk/assets/images/original_wide-2f360425a0e0a779832de176b75c4354.jpg)![Left image compare](/far-sdk/assets/images/hint-e7610a1cad3f0facb2590c99cda3058f.jpg) Drag --- # Top Level Prefabs [View as Markdown](https://docs.banuba.com/far-sdk/effects/prefabs/top_level.md)[![](/far-sdk/img/ai-guide/chatgpt.svg)Open in ChatGPT](https://chatgpt.com/?q=Read%20https%3A%2F%2Fdocs.banuba.com%2Ffar-sdk%2Feffects%2Fprefabs%2Ftop_level.md%20\(Top%20Level%20Prefabs\)%20and%20help%20me%20with%20it%20in%20my%20Banuba%20Face%20AR%20SDK%20project.)[![](/far-sdk/img/ai-guide/claude.svg)Open in Claude](https://claude.ai/new?q=Read%20https%3A%2F%2Fdocs.banuba.com%2Ffar-sdk%2Feffects%2Fprefabs%2Ftop_level.md%20\(Top%20Level%20Prefabs\)%20and%20help%20me%20with%20it%20in%20my%20Banuba%20Face%20AR%20SDK%20project.)Install tools ### Background[​](#background "Direct link to Background") **Usage** You can use a solid color: ``` { "background": { "color": "1 0 0 1" } } ``` Alternatively, use a texture: ``` { "background": { "texture": "capy.jpeg", "rotation": 0, "scale": 1, "content_mode": "scale_to_fill", "blend_mode": "default", "clear_color": "1 0 0 1", "use_filter": true } } ``` To blur the camera background: ``` { "background": { "blur": 0.5 } } ``` To make the background transparent: ``` { "background": { "transparency": 0.5 } } ``` Use this prefab to create a virtual background. | Parameter | Description | Optional | Default Value | | :-------------------- | :--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | :------------------------------------------------------------------: | :-----------: | | `color` | Sets a solid background color. See the color-format note in the prefabs overview. | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | `"0 0 0 0"` | | `transparency` | Sets the background transparency to a value from 0 to 1. | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | `0.0` | | `texture` | Sets an image or video file as the background texture. For best performance, use `.jpeg`, `.jpg`, `.png`, or `.mp4` (H.264 video with AAC audio). You can also use formats such as `.heic`, `.webp`, and `.webm`, but performance may vary by device. For more information, see the supported [image formats](https://docs.banuba.com/far-sdk/tutorials/capabilities/technical_specification#image-formats-support) and [video formats](https://docs.banuba.com/far-sdk/tutorials/capabilities/technical_specification#video-formats-support). | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | `null` | | `rotation` | Rotates the background texture clockwise in degrees. | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | `0.0` | | `scale` | Scales the background texture. | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | `1.0` | | `content_mode` | Fits the background texture within the frame. Possible values are `scale_to_fill`, `fill`, and `fit`. | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | `"fill"` | | `blend_mode` | Sets the texture blending mode. Possible values are `default`, `screen`, `split_alpha`, and `multiply`. `default` uses traditional alpha blending; `split_alpha` expects the alpha channel on the right side of the input texture. | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | `"default"` | | `blur` | Sets the background blur radius in the range `[0, 1]`. | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | `0.0` | | `clear_color` | Specifies the color of the area not covered by the background texture, such as when `content_mode` is `fit`. The default is black. The final alpha component is currently ignored; use `1`. | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | `"0 0 0 1"` | | `camera_video_scale` | \[Experimental] Sets the scale factor for the camera image relative to the center of the image. | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | `"1 1"` | | `camera_video_origin` | \[Experimental] Sets the offset of the camera image relative to the center of the image. | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | `"0 0"` | | `use_filter` | Enables or disables a filter that smooths segmentation-mask contours. | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | `true` | **Preview** ![Right image compare](/far-sdk/assets/images/original_wide-2f360425a0e0a779832de176b75c4354.jpg)![Left image compare](/far-sdk/assets/images/background-470533cb99448493c845be22788ccd68.jpg) Drag ### Foreground[​](#foreground "Direct link to Foreground") **Usage** Apply a texture or a video to the whole screen. ``` { "foreground": { "filename": "path/to/texture/file", "@blend": "multiply", "rotation": 0, "content_mode": "scale_to_fill" } } ``` | Parameter | Description | Optional | Default Value | | :------------- | :-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | :------------------------------------------------------------------: | :---------------: | | `filename` | Path to a texture or video file. A texture must have a suitable alpha channel; otherwise, like a video, it hides the underlying content. For more information, see the supported [image formats](https://docs.banuba.com/far-sdk/tutorials/capabilities/technical_specification#image-formats-support) and [video formats](https://docs.banuba.com/far-sdk/tutorials/capabilities/technical_specification#video-formats-support). | *+* | *+* | | `@blend` | Applies the selected blending mode. The leading `@` in the parameter name is required. Possible values are `off`, `alpha`, `premul_alpha`, `alpha_rgba`, `screen`, `add`, `multiply`, `min`, and `max`. | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | `"alpha"` | | `rotation` | Rotates the foreground clockwise in degrees. | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | `0.0` | | `content_mode` | Fits the foreground texture within the frame. Possible values are `scale_to_fill`, `fill`, and `fit`. | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | `"scale_to_fill"` | ### LUT[​](#lut "Direct link to LUT") **Usage** Apply a color filter, also known as a LUT. ``` { "lut": { "filename": "path/to/lut/file", "strength": 0.9 } } ``` | Parameter | Description | Optional | Default Value | | :--------- | :---------------------------------- | :------------------------------------------------------------------: | :-----------: | | `filename` | Path to the LUT file (usually PNG). | *+* | *+* | | `strength` | LUT strength in the range `[0, 1]`. | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | `1.0` | **Preview** ![Right image compare](/far-sdk/assets/images/original_wide-2f360425a0e0a779832de176b75c4354.jpg)![Left image compare](/far-sdk/assets/images/lut-d90746e32baa84b6f5b8027228b9f563.jpg) Drag ### Lights[​](#lights "Direct link to Lights") **Usage** Add up to four directional light sources to GLTF models in addition to IBL textures. A GLTF model is required. ``` { "lights": { "radiance": [ "10 0 0 0", "0 0 10 0" ], "direction": [ "-1 0 0", "1 0 0" ] } } ``` | Parameter | Description | Optional | Default Value | | :---------- | :----------------------------------------------------------------------------------------------------------------------------------------------------------------- | :------------------------------------------------------------------: | :--------------------------------------------: | | `radiance` | An array of light-source radiance values, or a single value for one light. Each value contains *R*, *G*, and *B* components followed by a light wraparound factor. | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | `["0 0 0 0", "0 0 0 0", "0 0 0 0", "0 0 0 0"]` | | `direction` | An array of light-source directions, or a single direction for one light. Each direction contains *X*, *Y*, and *Z* components. | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | `["0 0 0", "0 0 0", "0 0 0", "0 0 0"]` | ### MSAA[​](#msaa "Direct link to MSAA") **Usage** Apply multisample anti-aliasing to the effect. MSAA makes the image look cleaner by smoothing jagged edges. A GLTF model is required. The `@samples_count` parameter is required. ``` { "msaa": { "@samples_count": 4 } } ``` | Parameter | Description | Optional | Default Value | | :--------------- | :-------------------------------------------------------------------------------------------- | :------: | :-----------: | | `@samples_count` | Required MSAA sample count. The value must be `1`, `2`, or `4`. A value of `1` disables MSAA. | *+* | *+* | ### Bokeh[​](#bokeh "Direct link to Bokeh") **Usage** Enable the bokeh effect for the background. ``` { "bokeh": { "samples": 16 } } ``` | Parameter | Description | Optional | Default Value | | :-------- | :---------------------------------------------------------------------------------- | :------------------------------------------------------------------: | :-----------: | | `samples` | Sample count in the range `[8, 24]`. Higher values produce a stronger bokeh effect. | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | `16` | **Preview** ![Right image compare](/far-sdk/assets/images/bokeh1-d7e98efa92c812542789f664a113638d.jpg)![Left image compare](/far-sdk/assets/images/bokeh2-7a57a4a3469764e1337ab4dc0f704eba.jpg) Drag --- # Virtual Background API [View as Markdown](https://docs.banuba.com/far-sdk/effects/virtual_background.md)[![](/far-sdk/img/ai-guide/chatgpt.svg)Open in ChatGPT](https://chatgpt.com/?q=Read%20https%3A%2F%2Fdocs.banuba.com%2Ffar-sdk%2Feffects%2Fvirtual_background.md%20\(Virtual%20Background%20API\)%20and%20help%20me%20with%20it%20in%20my%20Banuba%20Face%20AR%20SDK%20project.)[![](/far-sdk/img/ai-guide/claude.svg)Open in Claude](https://claude.ai/new?q=Read%20https%3A%2F%2Fdocs.banuba.com%2Ffar-sdk%2Feffects%2Fvirtual_background.md%20\(Virtual%20Background%20API\)%20and%20help%20me%20with%20it%20in%20my%20Banuba%20Face%20AR%20SDK%20project.)Install tools Banuba provides the Virtual Background API designed to help you integrate augmented reality background separation into your app. It aims to change your background to hide everything behind you. [](pathname:///generated/effects/Background_doc.zip) [Download example](pathname:///generated/effects/Background_doc.zip) ## How to add a background to an effect[​](#how-to-add-a-background-to-an-effect "Direct link to How to add a background to an effect") Assume you have an effect, say [Afro](pathname:///generated/effects/Afro.zip), and want to add a background image to it, say [beach.png](pathname:///img/effect/combine_effect_vbg/beach.png) . To accomplish this, connect the built-in Background module to the effect via `evalJs`. Now, you can set the image as background texture: * Java * Swift ``` // Effect mCurrentEffect = ... // Connect the built-in background module (once per effect) mCurrentEffect.evalJs("Background = require('bnb_js/background')", null); // Then, set the background texture mCurrentEffect.evalJs("Background.texture('/absolute/path/to/beach.png')", null); ``` ``` // var currentEffect: BNBEffect = ... // Connect the built-in background module (once per effect) currentEffect?.evalJs("Background = require('bnb_js/background')", resultCallback: nil); // Then, set the background texture currentEffect?.evalJs("Background.texture('/absolute/path/to/beach.png')", resultCallback: nil); ``` ### Combine VBG with a WebAR effect (AR 3D Mask)[​](#combine-vbg-with-a-webar-effect-ar-3d-mask "Direct link to Combine VBG with a WebAR effect (AR 3D Mask)") On the Web platform, the file system is represented by the effect itself. It means you should put the desired image inside the effect folder before the `evalJs` call. One way to accomplish this is to put the image directly into the effect archive: 1. Unpack the effect archive 2. Put the image into the effect folder, say as `images/beach.png` 3. Compress the effect folder and use the new archive instead of the original one This way you should be able to set the image using the relative path: ``` // const player = await Player.create(...) // const effect = new Effect(...) // await player.applyEffect(effect) // Connect the built-in background module (once per effect) await effect.evalJs("Background = require('bnb_js/background')") // Then, set the background texture await effect.evalJs("Background.texture('images/beach.png')") ``` Another way is to upload an image to the effect's file system on demand. You can leverage the `Effect.writeFile()` API to accomplish this: ``` // const player = await Player.create(...) // const effect = new Effect(...) // await player.applyEffect(effect) // Load the image file however you like, e.g from a remote server const image = await fetch("/path/to/beach.png").then(r => r.arrayBuffer()) await effect.writeFile("images/beach.png", image) // Connect the built-in background module (once per effect) await effect.evalJs("Background = require('bnb_js/background')") // Then, set the background texture await effect.evalJs("Background.texture('images/beach.png')") ``` **Preview** ![Right image compare](/far-sdk/assets/images/original-7af4345d086ee096ce3656413c269b22.jpg)![Left image compare](/far-sdk/assets/images/vbg-e9db7c61e2572f49df1134d25b033cd3.jpg) Drag The Virtual Background API allows to change background with the following built-in features: ## Background texture[​](#background-texture "Direct link to Background texture") Sets the background behind the user to a texture. * `Background.texture('image.png')` - sets an image or a video file as a background texture. The file should be placed into the effect's folder. * [Supported formats.](/far-sdk/tutorials/capabilities/technical_specification.md#video-formats-support) - config.js - Java - Swift - JavaScript ``` /* Feel free to add your custom code below */ Background.texture('image.png') ``` ``` // Effect mCurrentEffect = ... mCurrentEffect.evalJs("Background.texture('image.png')", null); ``` ``` // var currentEffect: BNBEffect = ... currentEffect?.evalJs("Background.texture('image.png')", resultCallback: nil) ``` ``` await effect.evalJs("Background.texture('image.png')") ``` **Preview** ![Right image compare](/far-sdk/assets/images/original_wide-2f360425a0e0a779832de176b75c4354.jpg)![Left image compare](/far-sdk/assets/images/BackgroundTexture-fe6299595a550a13eec111677e4e2539.jpg) Drag ### Background texture content mode[​](#background-texture-content-mode "Direct link to Background texture content mode") Scales the background content mode. * `Background.contentMode(mode)` - sets a content mode of background texture. * Available mods: `scale_to_fill`, `fill`, `fit`. - config.js - Java - Swift - JavaScript ``` /* Feel free to add your custom code below */ Background.texture('image.png') Background.contentMode('fit') ``` ``` // Effect mCurrentEffect = ... mCurrentEffect.evalJs("Background.texture('image.png')", null); mCurrentEffect.evalJs("Background.contentMode('fit')", null); ``` ``` // var currentEffect: BNBEffect = ... currentEffect?.evalJs("Background.texture('image.png')", resultCallback: nil) currentEffect?.evalJs("Background.contentMode('fit')", resultCallback: nil) ``` ``` await effect.evalJs("Background.texture('image.png')") await effect.evalJs("Background.contentMode('fit')") ``` ![Right image compare](/far-sdk/assets/images/original_wide-2f360425a0e0a779832de176b75c4354.jpg)![Left image compare](/far-sdk/assets/images/BackgroundContentModeFit-b2b07f464dccc49e433763ee6d1a99b7.jpg) Drag ### Background texture rotation[​](#background-texture-rotation "Direct link to Background texture rotation") Rotates the background texture clockwise in degrees. * `Background.rotation(angle)` - sets background image rotation angle. Angle should be provided in degrees. - config.js - Java - Swift - JavaScript ``` /* Feel free to add your custom code below */ Background.texture('image.png') Background.rotation(90) ``` ``` // Effect mCurrentEffect = ... mCurrentEffect.evalJs("Background.texture('image.png')", null); mCurrentEffect.evalJs("Background.rotation(90)", null); ``` ``` // var currentEffect: BNBEffect = ... currentEffect?.evalJs("Background.texture('image.png')", resultCallback: nil) currentEffect?.evalJs("Background.rotation(90)", resultCallback: nil) ``` ``` await effect.evalJs("Background.texture('image.png')") await effect.evalJs("Background.rotation(90)") ``` **Preview** ![Right image compare](/far-sdk/assets/images/original_wide-2f360425a0e0a779832de176b75c4354.jpg)![Left image compare](/far-sdk/assets/images/BackgroundRotation-bab2be0102d207bd6ae9427b96e95ece.jpg) Drag ### Background texture scale[​](#background-texture-scale "Direct link to Background texture scale") Scales the background texture. * `Background.scale(factor)` - sets the scale factor of background texture. - config.js - Java - Swift - JavaScript ``` /* Feel free to add your custom code below */ Background.texture('image.png') Background.scale(2) ``` ``` // Effect mCurrentEffect = ... mCurrentEffect.evalJs("Background.texture('image.png')", null); mCurrentEffect.evalJs("Background.scale(2)", null); ``` ``` // var currentEffect: BNBEffect = ... currentEffect?.evalJs("Background.texture('image.png')", resultCallback: nil) currentEffect?.evalJs("Background.scale(2)", resultCallback: nil) ``` ``` await effect.evalJs("Background.texture('image.png')") await effect.evalJs("Background.scale(2)") ``` **Preview** ![Right image compare](/far-sdk/assets/images/original_wide-2f360425a0e0a779832de176b75c4354.jpg)![Left image compare](/far-sdk/assets/images/BackgroundScale-07b26044692d8592b470084f1a12587f.jpg) Drag ## Background blur[​](#background-blur "Direct link to Background blur") Sets the background blur radius. * `Background.blur(radius)` - set the background blur radius in range from 0 to 1. - config.js - Java - Swift - JavaScript ``` /* Feel free to add your custom code below */ Background.blur(0.6) ``` ``` // Effect mCurrentEffect = ... mCurrentEffect.evalJs("Background.blur(0.6)", null); ``` ``` // var currentEffect: BNBEffect = ... currentEffect?.evalJs("Background.blur(0.6)", resultCallback: nil) ``` ``` await effect.evalJs("Background.blur(0.6)") ``` **Preview** ![Right image compare](/far-sdk/assets/images/BackgroundTexture-fe6299595a550a13eec111677e4e2539.jpg)![Left image compare](/far-sdk/assets/images/BackgroundBlur-72d90fc5c1e1f874b4f53f8ddcf212eb.jpg) Drag ## Background transparency[​](#background-transparency "Direct link to Background transparency") Sets background transparency value. * `Background.transparency(value)` - set background transparency value in range from 0 to 1. 0 - transparent background disabled , 1 - fully transparent background enabled - config.js - Java - Swift - JavaScript ``` /* Feel free to add your custom code below */ Background.transparency(1) ``` ``` // Effect mCurrentEffect = ... mCurrentEffect.evalJs("Background.transparency(1)", null); ``` ``` // var currentEffect: BNBEffect = ... currentEffect?.evalJs("Background.transparency(1)", resultCallback: nil) ``` ``` await effect.evalJs("Background.transparency(1)") ``` **Preview** ![Right image compare](/far-sdk/assets/images/original_wide-2f360425a0e0a779832de176b75c4354.jpg)![Left image compare](/far-sdk/assets/images/BackgroundTransparent-1648360dee4dd0da0f4aa673866738e8.png) Drag --- # Support [View as Markdown](https://docs.banuba.com/far-sdk/support_page.md)[![](/far-sdk/img/ai-guide/chatgpt.svg)Open in ChatGPT](https://chatgpt.com/?q=Read%20https%3A%2F%2Fdocs.banuba.com%2Ffar-sdk%2Fsupport_page.md%20\(Support\)%20and%20help%20me%20with%20it%20in%20my%20Banuba%20Face%20AR%20SDK%20project.)[![](/far-sdk/img/ai-guide/claude.svg)Open in Claude](https://claude.ai/new?q=Read%20https%3A%2F%2Fdocs.banuba.com%2Ffar-sdk%2Fsupport_page.md%20\(Support\)%20and%20help%20me%20with%20it%20in%20my%20Banuba%20Face%20AR%20SDK%20project.)Install tools * [Dev Portal](https://community.banuba.com/) * [FAQ page](https://www.banuba.com/faq/). * [Contact our support](/far-sdk/support/.md). --- # Third parties library list [View as Markdown](https://docs.banuba.com/far-sdk/tutorials/capabilities/3rd_licenses.md)[![](/far-sdk/img/ai-guide/chatgpt.svg)Open in ChatGPT](https://chatgpt.com/?q=Read%20https%3A%2F%2Fdocs.banuba.com%2Ffar-sdk%2Ftutorials%2Fcapabilities%2F3rd_licenses.md%20\(Third%20parties%20library%20list\)%20and%20help%20me%20with%20it%20in%20my%20Banuba%20Face%20AR%20SDK%20project.)[![](/far-sdk/img/ai-guide/claude.svg)Open in Claude](https://claude.ai/new?q=Read%20https%3A%2F%2Fdocs.banuba.com%2Ffar-sdk%2Ftutorials%2Fcapabilities%2F3rd_licenses.md%20\(Third%20parties%20library%20list\)%20and%20help%20me%20with%20it%20in%20my%20Banuba%20Face%20AR%20SDK%20project.)Install tools ## 3-clause BSD License[​](#3-clause-bsd-license "Direct link to 3-clause BSD License") > Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met: > > 1. Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer. > 2. Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution. > 3. Neither the name of the copyright holder nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission. THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS ""AS IS"" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. " | Name | Version | Platform | Link | Copyright info | | ------------------------ | ------- | -------- | ----------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------- | | CLI11 | 1.9.1 | Desktop | | CLI11 1.8 Copyright (c) 2017-2019 University of Cincinnati, developed by Henry Schreiner under NSF AWARD 1414736. All rights reserved. | | ios-cmake | 3.0.2 | iOS | | Copyright (c) 2011-2014, Andrew Fischer (c) 2017, Alexander Widerberg All rights reserved. | | tinyexr | 0.9.5 | Desktop | | Syoyo Fujita() | | pybind11 | 2.6.2 | Python | | Copyright (c) 2016 Wenzel Jakob , All rights reserved. | | Win camera from chromium | N/A | Windows | | Copyright 2015 The Chromium Authors. All rights reserved. | | glslang | 16.2.0 | iOS, OSX | | Copyright (C) 2020-2025 The Khronos Group Inc. All rights reserved. | ## Apache License 2.0[​](#apache-license-20 "Direct link to Apache License 2.0") > "Licensed under the Apache License, Version 2.0 (the ""License""); you may not use this file except in compliance with the License. You may obtain a copy of the License at: Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an ""AS IS"" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License." | Name | Version | Platform | Link | Copyright info | | --------------- | ------- | -------- | --------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------- | | djinni | 1.3.0 | All | | Copyright (с) 2014 - 2019 Kannan Goundan, Tony Grue, Derek He, Steven Kabbes, Jacob Potter, Iulia Tamas, Andrew Twyman | | tensorflow lite | 2.1.4 | All | | "Copyright (c) Google Inc. Yuan Tang " | | SPIRV-Cross | 1.4.341 | iOS, OSX | | "Copyright (c) 2014-2020 The Khronos Group Inc." | | draco | 1.5.6 | All | | "Copyright (c) Google Inc. and other contributors" | | opencv | 4.13.0 | All | | "Copyright (c) 2025, OpenCV team" | ## Boost Software License 1.0[​](#boost-software-license-10 "Direct link to Boost Software License 1.0") > "Permission is hereby granted, free of charge, to any person or organization obtaining a copy of the software and accompanying documentation covered by this license (the ""Software"") to use, reproduce, display, distribute, execute, and transmit the Software, and to prepare derivative works of the Software, and to permit third-parties to whom the Software is furnished to do so, all subject to the following: The copyright notices in the Software and this entire statement, including the above license grant, this restriction and the following disclaimer, must be included in all copies of the Software, in whole or in part, and all derivative works of the Software, unless such copies or derivative works are solely in the form of machine-executable object code generated by a source language processor. THE SOFTWARE IS PROVIDED ""AS IS"", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, TITLE AND NON-INFRINGEMENT. IN NO EVENT SHALL THE COPYRIGHT HOLDERS OR ANYONE DISTRIBUTING THE SOFTWARE BE LIABLE FOR ANY DAMAGES OR OTHER LIABILITY, WHETHER IN CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE." | Name | Version | Platform | Link | Copyright info | | ------ | ------- | -------- | ------------------------------------ | -------------------------- | | catch2 | 2.5.0 | All | | Copyright Β© catch2 Authors | ## BSD 2-Clause Simplified[​](#bsd-2-clause-simplified "Direct link to BSD 2-Clause Simplified") > "Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met: > > * Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer. > * * Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution. HIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS ""AS IS"" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE." | Name | Version | Platform | Link | Copyright info | | ------- | ------- | -------- | ------------------------------------ | ----------------------------------------------------------------------------------------------------------------------------------------------------------- | | cpuinfo | N/A | All | | "Copyright (c) 2017-2018 Facebook Inc. Copyright (C) 2012-2017 Georgia Institute of Technology. Copyright (C) 2010-2012 Marat Dukhan. All rights reserved." | ## GNU Lesser General Public Licence version 2.1 or later[​](#gnu-lesser-general-public-licence-version-21-or-later "Direct link to GNU Lesser General Public Licence version 2.1 or later") > This library is free software and is governed by GNU Lesser General Public License, version 2.1, available at . | Name | Version | Platform | Link | Copyright info | | ----------- | ------- | -------- | ------------------------------------------- | ------------------------------------------------- | | ffmpeg | 7.1.3 | Windows | | Copyright (c) ffmpeg Authors | | openal soft | 1.18 | Windows | | Copyright (C) 1991 Free Software Foundation, Inc. | ## ISC License[​](#isc-license "Direct link to ISC License") > "Permission to use, copy, modify, and/or distribute this software for any purpose with or without fee is hereby granted, provided that the above copyright notice and this permission notice appear in all copies. THE SOFTWARE IS PROVIDED ""AS IS"" AND THE AUTHOR DISCLAIMS ALL WARRANTIES WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE." | Name | Version | Platform | Link | Copyright info | | ------ | ----------- | -------- | --------------------------------------- | ----------------------------------------------------------------------------------------- | | detex | 0.1.2alpha2 | All | | Copyright (c) 2015 Harm Hanemaaijer | | Sodium | 1.0.18 | All | | Copyright (c) Sodium authors, | ## MIT License[​](#mit-license "Direct link to MIT License") > "Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the ""Software""), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions: The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software. THE SOFTWARE IS PROVIDED ""AS IS"", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE." | Name | Version | Platform | Link | Copyright info | | ------------- | ---------- | --------------------- | ------------------------------------------- | ------------------------------------------------------------------------- | | asyncplusplus | 1.2 | All | | Copyright (c) 2015 Amanieu d'Antras | | imgui | 1.91.9 | Desktop | | Copyright (c) 2014-2025 Omar Cornut | | jnipp | N/A | Android | | Copyright (c) 2016 Mitchell Dowd | | json | 3.10.4 | All | | Copyright (c) 2013-2019 Niels Lohmann | | url-cpp | N/A | All | | Copyright (c) 2016-2017 SEOmoz, Inc. | | tinygltf | 3.0.0 | All | | Copyright (c) 2017 Syoyo Fujita, AurΓ©lien Chatelain and many contributors | | NumCpp | 2.14.1 | All | | Copyright (C) 2018-2023 David Pilger | | quickjs | 2025-09-13 | Android, Windows, Web | | Copyright (c) 2017-2025 Fabrice Bellard and Charlie Gordon. | ## Public domain[​](#public-domain "Direct link to Public domain") > This is free and unencumbered software released into the public domain. Anyone is free to copy, modify, publish, use, compile, sell, or distribute this software, either in source code form or as a compiled binary, for any purpose, commercial or non-commercial, and by any means. The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software. | Name | Version | Platform | Link | Copyright info | | --------------- | ------- | -------- | ------------------------------------------------- | --------------------------------------------------- | | drwav | 0.8.5 | All | | Copyright @ the Dr-wav Authors | | stb | 2.33 | All | | Copyright (C) stb authors | | HdrHistogram\_c | 0.11.2 | All | | Copyright (c) Gil Tene, Michael Barker, Matt Warren | ## The Happy Bunny and MIT Licences[​](#the-happy-bunny-and-mit-licences "Direct link to The Happy Bunny and MIT Licences") > "The Happy Bunny Licence: Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the ""Software""), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions: The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software. Restrictions: By making use of the Software for military purposes, you choose to make a Bunny unhappy. THE SOFTWARE IS PROVIDED ""AS IS"", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. MIT Licence: Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the ""Software""), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions: The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software. THE SOFTWARE IS PROVIDED ""AS IS"", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE." | Name | Version | Platform | Link | Copyright info | | ---- | ------- | -------- | ------------------------------- | ----------------------------------------- | | glm | 0.9.9.8 | All | | Copyright (c) 2005 - 2014 G-Truc Creation | ## zlib License[​](#zlib-license "Direct link to zlib License") > "This software is provided 'as-is', without any express or implied warranty. In no event will the authors be held liable for any damages arising from the use of this software. Permission is granted to anyone to use this software for any purpose, including commercial applications, and to alter it and redistribute it freely, subject to the following restrictions: > > 1. The origin of this software must not be misrepresented; you must not claim that you wrote the original software. If you use this software in a product, an acknowledgment in the product documentation would be appreciated but is not required. > 2. Altered source versions must be plainly marked as such, and must not be misrepresented as being the original software. > 3. This notice may not be removed or altered from any source distribution." | Name | Version | Platform | Link | Copyright info | | ---------- | ------- | -------- | ----------------------------------------------------- | -------------------------------------------------------------------------------------------------- | | glfw | 3.4 | All | | Copyright (c) 2002-2006 Marcus Geelnard Copyright (c) 2006-2016 Camilla LΓΆwy | | zlib | 1.2.11 | All | | Copyright (C) 1995-2017 Jean-loup Gailly and Mark Adler | | mikktspace | 1.0 | All | | Copyright (C) 2011 by Morten S. Mikkelsen | ## Mozilla Public License, Version 2.0[​](#mozilla-public-license-version-20 "Direct link to Mozilla Public License, Version 2.0") > Permissions of this weak copyleft license are conditioned on making available source code of licensed files and modifications of those files under the same license (or in certain cases, one of the GNU licenses). Copyright and license notices must be preserved. Contributors provide an express grant of patent rights. However, a larger work using the licensed work may be distributed under different terms and without source code for files added in the larger work. This Source Code Form is subject to the terms of the Mozilla Public License, v. 2.0. If a copy of the MPL was not distributed with this file, You can obtain one at . | Name | Version | Platform | Link | Copyright info | | ----- | ------- | -------- | ----------------------------------- | ------------------------------------------------------------------------------------------------------------------------ | | Eigen | 3.4.0 | All | | Copyright (C) 2008 Gael Guennebaud Copyright (C) 2008 Benoit Jacob | ## The LibYuv Project Authors[​](#the-libyuv-project-authors "Direct link to The LibYuv Project Authors") > Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met: Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer. Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution. Neither the name of TransGaming Inc., Google Inc., 3DLabs Inc. Ltd., nor the names of their contributors may be used to endorse or promote products derived from this software without specific prior written permission. THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. | Name | Version | Platform | Link | Copyright info | | ------ | ------- | -------- | -------------------------------------------------- | --------------------------------------------------------------- | | LibYuv | 1789 | All | | Copyright 2011 The LibYuv Project Authors. All rights reserved. | **Note:** The following libraries are also used in FaceAR SDK: OpenAL on iOS (Apple) and OpenSL on Android (Google). --- # Demo Face Filters [View as Markdown](https://docs.banuba.com/far-sdk/tutorials/capabilities/demo_face_filters.md)[![](/far-sdk/img/ai-guide/chatgpt.svg)Open in ChatGPT](https://chatgpt.com/?q=Read%20https%3A%2F%2Fdocs.banuba.com%2Ffar-sdk%2Ftutorials%2Fcapabilities%2Fdemo_face_filters.md%20\(Demo%20Face%20Filters\)%20and%20help%20me%20with%20it%20in%20my%20Banuba%20Face%20AR%20SDK%20project.)[![](/far-sdk/img/ai-guide/claude.svg)Open in Claude](https://claude.ai/new?q=Read%20https%3A%2F%2Fdocs.banuba.com%2Ffar-sdk%2Ftutorials%2Fcapabilities%2Fdemo_face_filters.md%20\(Demo%20Face%20Filters\)%20and%20help%20me%20with%20it%20in%20my%20Banuba%20Face%20AR%20SDK%20project.)Install tools tip You may also create your own effects with [**Banuba Studio**](https://studio.banuba.com/) or buy some from the [**Banuba Asset Store**](https://assetstore.banuba.net/). List of Facer AR SDK **Demo Face Filters** and [technologies](/far-sdk/tutorials/capabilities/sdk_features.md) represented with them. Face AR SDK release archives are supplied with a minimum built-in number of face filter (effects), all other Demo effects can be downloaded from this page. | Filters | Technologies represented | Required packages | | --------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------- | | [![0003\_cu\_Spider1\_v3\_b1 icon](/far-sdk/img/effects/0003_cu_Spider1_v3_b1.png)download](/far-sdk/generated/effects/0003_cu_Spider1_v3_b1.zip) | Animation, Triggers | face\_tracker | | [![ActionunitsGrout icon](/far-sdk/img/effects/ActionunitsRabbit.png)download](/far-sdk/effects/ActionunitsRabbit.zip) | Action units with blendshapes in the AR 3D Mask | face\_tracker | | [![Afro icon](/far-sdk/img/effects/afro.png)download](/far-sdk/generated/effects/Afro.zip) | AR 3D Mask | face\_tracker | | [![BG\_Metro icon](/far-sdk/img/effects/BG_Metro.png)download](/far-sdk/effects/BG_Metro.zip) | Simple effect that allow to set an image as the Background
Face filter performance can be lower on mid-end and low-end devices due to neural network usage. | background | | [![blur\_bg icon](/far-sdk/img/effects/blur_bg.png)download](/far-sdk/generated/effects/blur_bg.zip) | Background segmentation neural network, Blur
Face filter performance can be lower on mid-end and low-end devices due to neural network usage. | background | | [![bokeh icon](/far-sdk/img/effects/bokeh.png)download](/far-sdk/generated/effects/test_bokeh.zip) | Background segmentation neural network, Bokeh effect | face\_tracker, background | | [![BulldogHarlamov icon](/far-sdk/img/effects/BulldogHarlamov.png)download](/far-sdk/generated/effects/BulldogHarlamov.zip) | Transfer of facial expressions to the 3D model, AR 3D Mask | face\_tracker | | [![BurningMan2018 icon](/far-sdk/img/effects/HawaiiHairFlower.png)download](/far-sdk/effects/HawaiiHairFlower.zip) | Retouch, AR 3D Mask | face\_tracker | | [![CartoonOctopus icon](/far-sdk/img/effects/CartoonOctopus.png)download](/far-sdk/generated/effects/CartoonOctopus.zip) | Animation, Triggers | face\_tracker | | [![CubemapEverest icon](/far-sdk/img/effects/CubemapMoon.png)download](/far-sdk/effects/CubemapMoon.zip) | Background segmentation neural network, 3D environment cubemap texture, AR 3D Mask
Face filter performance can be lower on mid-end and low-end devices due to neural network usage. | face\_tracker, background | | [![DebugFRX icon](/far-sdk/img/effects/DebugWireframe.png)download](/far-sdk/generated/effects/DebugFRX.zip) | Display face recognition wireframe with landmarks | face\_tracker | | [![Glasses icon](/far-sdk/img/effects/glasses_RayBan4165_Dark.png)download](/far-sdk/effects/glasses_RayBan4165_Dark.zip) | Virtual glasses try-on | face\_tracker | | [![Makeup icon](/far-sdk/img/effects/Makeup.png)download](/far-sdk/generated/effects/Makeup.zip) | Virtual Makeup API effect. See more in Effect API section.
Face filter performance can be lower on mid-end and low-end devices due to neural network usage. | See comment below the table\* | | [![MonsterFactory icon](/far-sdk/img/effects/1001Nights_bloom.png)download](/far-sdk/effects/1001Nights_bloom.zip) | AR 3D Mask, Physics | face\_tracker | | [![nn\_api icon](/far-sdk/img/effects/nn_api.png)download](/far-sdk/generated/effects/nn_api.zip) | Effect with API for testing multiple neural networks work at the same time: Lips, Hair, Background, Hair and Skin
Face filter performance can be lower on mid-end and low-end devices due to neural network usage. | face\_tracker, lips, skin, background, hair | | [![PineappleGlasses icon](/far-sdk/img/effects/Pineappleglasses.png)download](/far-sdk/generated/effects/PineappleGlasses.zip) | Background segmentation neural network, Animated texture, AR 3D Mask
Face filter performance can be lower on mid-end and low-end devices due to neural network usage. | face\_tracker, background | | [![PoliceMan icon](/far-sdk/img/effects/PoliceMan.png)download](/far-sdk/generated/effects/PoliceMan.zip) | Morphing, AR 3D Mask | face\_tracker | | [![Rorschach icon](/far-sdk/img/effects/Rorschach.png)download](/far-sdk/generated/effects/Rorschach.zip) | Background segmentation neural network, Animated texture, AR 3D Mask
Face filter performance can be lower on mid-end and low-end devices due to neural network usage. | face\_tracker, background | | [![scene\_4\_faces icon](/far-sdk/img/effects/scene_4_faces.png)download](/far-sdk/generated/effects/scene_4_faces.zip) | Multi-face morphing example | face\_tracker | | [![HairPinkGirl icon](/far-sdk/img/effects/HairPinkGirl.png)download](/far-sdk/effects/HairPinkGirl.zip) | Animated texture, AR 3D Mask | face\_tracker | | [![test\_BG icon](/far-sdk/img/effects/test_BG.png)download](/far-sdk/generated/effects/test_BG.zip) | Background segmentation neural network
Face filter performance can be lower on mid-end and low-end devices due to neural network usage. | background | | [![test\_Eye\_lenses icon](/far-sdk/img/effects/test_Eye_lenses.png)download](/far-sdk/generated/effects/test_Eye_lenses.zip) | Virtual eye lenses example
Face filter performance can be lower on mid-end and low-end devices due to neural network usage. | face\_tracker, eyes | | [![test\_Eyelashes icon](/far-sdk/img/effects/test_Eyelashes.png)download](/far-sdk/generated/effects/test_Eyelashes.zip) | Effect for eyelashes tracking debug | face\_tracker | | [![test\_Eyes icon](/far-sdk/img/effects/test_Eyes.png)download](/far-sdk/generated/effects/test_Eyes.zip) | Eyes segmentation neural network
Face filter performance can be lower on mid-end and low-end devices due to neural network usage. | face\_tracker, eyes | | [![test\_gestures icon](/far-sdk/img/effects/test_gestures.png)download](/far-sdk/generated/effects/test_gestures.zip) | Hand gestures detection example | hands | | [![test\_Glasses icon](/far-sdk/img/effects/test_Glasses.png)download](/far-sdk/generated/effects/test_Glasses.zip) | Glasses detection (neural network approach), AR 3D Mask | face\_tracker, background | | [![test\_Hair icon](/far-sdk/img/effects/test_Hair.png)download](/far-sdk/generated/effects/test_Hair.zip) | Hair segmentation neural network
Face filter performance can be lower on mid-end and low-end devices due to neural network usage. | hair | | [![test\_Hair\_bound icon](/far-sdk/img/effects/test_Hair_bound.png)download](/far-sdk/generated/effects/test_Hair_bound.zip) | Hair recoloring in multiple shades, Hair segmentation neural network
Face filter performance can be lower on mid-end and low-end devices due to neural network usage. | face\_tracker, hair | | [![test\_Hair\_strand icon](/far-sdk/img/effects/test_Hair_strand.png)download](/far-sdk/generated/effects/test_Hair_strand.zip) | Hair strands recoloring
Face filter performance can be lower on mid-end and low-end devices due to neural network usage. | face\_tracker, hair | | [![test\_HandSkelet icon](/far-sdk/img/effects/test_HandSkelet.png)download](/far-sdk/generated/effects/test_HandSkelet.zip) | Displays hand skeleton model | hands | | [![test\_heart\_rate icon](/far-sdk/img/effects/test_heart_rate.png)download](/far-sdk/generated/effects/test_heart_rate.zip) | Heart rate measurement (Pulse) technology example | face\_tracker | | [![test\_image\_process\_cartoon icon](/far-sdk/img/effects/test_image_process_cartoon.png)download](/far-sdk/generated/effects/test_image_process_cartoon.zip) | Shader-based image filter | face\_tracker | | [![test\_Lips icon](/far-sdk/img/effects/test_Lips.png)download](/far-sdk/generated/effects/test_Lips.zip) | Lips segmentation neural network
Face filter performance can be lower on mid-end and low-end devices due to neural network usage. | face\_tracker, lips | | [![test\_Lips\_glitter icon](/far-sdk/img/effects/test_Lips_glitter.png)download](/far-sdk/generated/effects/test_Lips_glitter.zip) | Glitter lipstick effect, lips segmentation neural network
Face filter performance can be lower on mid-end and low-end devices due to neural network usage. | face\_tracker, lips | | [![test\_Lips\_shine icon](/far-sdk/img/effects/test_Lips_shine.png)download](/far-sdk/generated/effects/test_Lips_shine.zip) | Shiny lipstick effect, lips segmentation neural network
Face filter performance can be lower on mid-end and low-end devices due to neural network usage. | face\_tracker, lips | | [![test\_Nails icon](/far-sdk/img/effects/test_Nails.png)download](/far-sdk/generated/effects/test_Nails.zip) | Virtual nails try on effect
Face filter performance can be lower on mid-end and low-end devices due to neural network usage. | hands | | [![test\_Ring icon](/far-sdk/img/effects/test_Ring.png)download](/far-sdk/generated/effects/test_Ring.zip) | AR Ring demo effect
Face filter performance can be lower on mid-end and low-end devices due to neural network usage. | hands | | [![test\_Ruler icon](/far-sdk/img/effects/test_Ruler.png)download](/far-sdk/generated/effects/test_Ruler.zip) | Face-to-phone distance measurement | face\_tracker | | [![test\_Skin icon](/far-sdk/img/effects/test_Skin.png)download](/far-sdk/generated/effects/test_Skin.zip) | Skin segmentation neural network
Face filter performance can be lower on mid-end and low-end devices due to neural network usage. | skin | #### Legend[​](#legend "Direct link to Legend") * `Online / Offline` - Online face filters can be applied both for realtime work and photo. Offline face filters are designed to work only with photos (offline mode). * Click on filter name to download the effect. * **Required packages** column lists [the packages](/far-sdk/tutorials/development/installation.md) you must include within the app. For iOS package names follow this simple rule: *face\_tracker -> BNBFaceTracker* etc. \* *Makeup* effect dependencies are configured during runtime according to features enabled. You will see in log which package is missing in case of error. --- # FaceAR Glossary [View as Markdown](https://docs.banuba.com/far-sdk/tutorials/capabilities/glossary.md)[![](/far-sdk/img/ai-guide/chatgpt.svg)Open in ChatGPT](https://chatgpt.com/?q=Read%20https%3A%2F%2Fdocs.banuba.com%2Ffar-sdk%2Ftutorials%2Fcapabilities%2Fglossary.md%20\(FaceAR%20Glossary\)%20and%20help%20me%20with%20it%20in%20my%20Banuba%20Face%20AR%20SDK%20project.)[![](/far-sdk/img/ai-guide/claude.svg)Open in Claude](https://claude.ai/new?q=Read%20https%3A%2F%2Fdocs.banuba.com%2Ffar-sdk%2Ftutorials%2Fcapabilities%2Fglossary.md%20\(FaceAR%20Glossary\)%20and%20help%20me%20with%20it%20in%20my%20Banuba%20Face%20AR%20SDK%20project.)Install tools ## AR Technologies[​](#ar-technologies "Direct link to AR Technologies") ### FRX (Face tracking)[​](#frx-face-tracking "Direct link to FRX (Face tracking)") The technology to detect and track the presence of a human face in a digital video frame to enable FaceAR camera experiences. ### Face match[​](#face-match "Direct link to Face match") Comparing a face in the image/video to one reference image. ### Face attributes[​](#face-attributes "Direct link to Face attributes") Detection of multiple facial parameters: hair color, eye color, skin tone, gender, hair & facial hair style, nose & lips shape. ### Face shape[​](#face-shape "Direct link to Face shape") Face shape detection that can be used for things like personalized recommendations ### Background separation[​](#background-separation "Direct link to Background separation") Neural network that separates a user on the foreground from the background in a sequence of video frames to remove the background or replace it with another image. As a standalone feature, the background segmentation is used in video conferencing, live streaming and video communication apps. It also can be used as part of face filter together with facial animation in entertainment apps. ### Bokeh effect[​](#bokeh-effect "Direct link to Bokeh effect") The algorithm that separates a person on the foreground and blurs the background as in photography. ### Skin segmentation[​](#skin-segmentation "Direct link to Skin segmentation") Neural network trained to recognize and segment the skin area for recoloring tasks, e.g. make the skin tone lighter or darker. ### Skin smoothing[​](#skin-smoothing "Direct link to Skin smoothing") Neural network trained to segment and blur the skin on the face to create a retouch effect similar to professional photography. ### Neck segmentation and smoothing[​](#neck-segmentation-and-smoothing "Direct link to Neck segmentation and smoothing") Segmentation of the neck for applying effects like skin smoothing or trying on appropriate items (e.g. scarves) ### Hair Segmentation[​](#hair-segmentation "Direct link to Hair Segmentation") Neural network to detect and segment the image into hair, face and background to allow for real-time hair modification such as change the hair color. ### Eye segmentation[​](#eye-segmentation "Direct link to Eye segmentation") Neural network to detect and segment the eye into iris, pupil and eyeball for eye recoloring and virtual contact lens try on. ### Eye lenses[​](#eye-lenses "Direct link to Eye lenses") Applying virtual contact lenses and changing lenses color on physical glasses. ### Pupillary distance[​](#pupillary-distance "Direct link to Pupillary distance") Measuring distance between the centers of the person's pupils, a useful parameter for eyewear try-on. ### Brows segmentation[​](#brows-segmentation "Direct link to Brows segmentation") Segmentation of eyebrows for applying effects like facial feature editing. ### Lips segmentation[​](#lips-segmentation "Direct link to Lips segmentation") Neural network to detect and segment the lips on the users face for virtual lipstic try on effect. ### Hair strands painting[​](#hair-strands-painting "Direct link to Hair strands painting") The algorithms to change the hair color with several colors applied simultaneously, e.g. for strands highlight or coloring. ### Teeth tone[​](#teeth-tone "Direct link to Teeth tone") Detection of the person's teeth shade. Can be used for dental bleaching simulation. ## Features[​](#features "Direct link to Features") ### 2+ faces detection (Multi-face)[​](#2-faces-detection-multi-face "Direct link to 2+ faces detection (Multi-face)") Algorithm allowing for AR 3D Masks application to several people simultaneously for more engaging group AR experiences. For the quality user experience, we generally don’t recommend supporting more than 3 faces on mobile devices due to limited computing capabilities. ### Pulse (Heart rate)[​](#pulse-heart-rate "Direct link to Pulse (Heart rate)") Algorithm analyses fine patterns of the facial areas and their color variations within time to detect pulse frequency in real-time. ### Ruler (Distance to phone)[​](#ruler-distance-to-phone "Direct link to Ruler (Distance to phone)") Algorithm analyses face area size to estimate distance from user's face to camera in real-time. ### Text Texture (on AR 3D Mask)[​](#text-texture-on-ar-3d-mask "Direct link to Text Texture (on AR 3D Mask)") Allows to write text as texture on any 2D or 3D model in the effect. ### AR 3D Mask on a picture from Camera Roll[​](#ar-3d-mask-on-a-picture-from-camera-roll "Direct link to AR 3D Mask on a picture from Camera Roll") AR 3D Mask application on pre-recorded images the user uploads from the Camera Roll. ### AR 3D Mask on video from Camera Roll[​](#ar-3d-mask-on-video-from-camera-roll "Direct link to AR 3D Mask on video from Camera Roll") AR 3D Mask application on pre-recorded videos the user uploads from the Camera Roll. ### Post-processing effects[​](#post-processing-effects "Direct link to Post-processing effects") Graphical camera effects and animations applied on pre-recorded videos. ### Continuous photo editing[​](#continuous-photo-editing "Direct link to Continuous photo editing") AR effect is processed in real-time on the image. E.g. beautification slider to control the face modification or "Before/After" slider. ### Touches[​](#touches "Direct link to Touches") Small AR scenarios enabled thought the user touches on the screen. AR objects or camera effects can change color and behaviour. Applied in FaceAR games or interactive face filters to increase engagement. ### Trigger[​](#trigger "Direct link to Trigger") Small AR scenarios enabled through user facial expressions. The user can interact with effects or call them opening mouth, smiling, raising eyebrows or frowning. Applied in FaceAR games or interactive face filters to increase engagement. ### SFX[​](#sfx "Direct link to SFX") Sound effects support in FaceAR experiences, e.g. add music to filters. ### Glasses detection[​](#glasses-detection "Direct link to Glasses detection") The algorithm detects if the user wears glasses and removes them in a virtual AR 3D Mask. Applied in glasses try-on for convenient frame choice and face filters for AR glasses would not overlay the real glasses. ### Glasses frame color[​](#glasses-frame-color "Direct link to Glasses frame color") Detection of the color of the glasses that the person is wearing. ## Graphical technologies[​](#graphical-technologies "Direct link to Graphical technologies") ### Face beautification[​](#face-beautification "Direct link to Face beautification") The AR filter based on face tracking which automatically retouches the appearance applying skin smooth, morphing, eye makeup, teeth whitening, eye flare and LUT effect. ### Morphing[​](#morphing "Direct link to Morphing") Changing the size and proportions of the face, e.g. slim down the cheeks, nose or modify them for fun AR 3D Masks. ### Skinned mesh animation[​](#skinned-mesh-animation "Direct link to Skinned mesh animation") AR models look not static but moving, animated and transforming. ### Physically-based rendering[​](#physically-based-rendering "Direct link to Physically-based rendering") AR models behave like the real objects in the flow of real-world light and physics, e.g. support gravity or mirror the light with the camera rotates and user tilts. ### LUT post-processing[​](#lut-post-processing "Direct link to LUT post-processing") Real-time or offline color correction of pre-recorded images, e.g. Instagram-like filters. ### Texture sequences[​](#texture-sequences "Direct link to Texture sequences") The digital representation of the surface of an AR object providing the sophisticated and life-like object representation. ### Video textures[​](#video-textures "Direct link to Video textures") Infuse a static image with dynamic qualities and explicit action to achieve an enhanced look and feel of the FaceAR video experience. ### Action units[​](#action-units "Direct link to Action units") The fundamental actions of individual muscles or groups of muscles of the face that enable the AR 3D Mask to support user facial expressions, e.g. in emojis, avatars or full-face AR 3D Masks. ### Lips shine, gloss, chameleon[​](#lips-shine-gloss-chameleon "Direct link to Lips shine, gloss, chameleon") Effects simulating lipstick/lip gloss in several texture types: shine, gloss, & chameleon respectively. ### Eyelids gloss, chameleon[​](#eyelids-gloss-chameleon "Direct link to Eyelids gloss, chameleon") Effects simulating eyeshadow in several texture types: shine, gloss, and chameleon. ### Light correction[​](#light-correction "Direct link to Light correction") Automaatic correction of lighting to make the image/video feed look more aesthetically pleasing --- # SDK Features [View as Markdown](https://docs.banuba.com/far-sdk/tutorials/capabilities/sdk_features.md)[![](/far-sdk/img/ai-guide/chatgpt.svg)Open in ChatGPT](https://chatgpt.com/?q=Read%20https%3A%2F%2Fdocs.banuba.com%2Ffar-sdk%2Ftutorials%2Fcapabilities%2Fsdk_features.md%20\(SDK%20Features\)%20and%20help%20me%20with%20it%20in%20my%20Banuba%20Face%20AR%20SDK%20project.)[![](/far-sdk/img/ai-guide/claude.svg)Open in Claude](https://claude.ai/new?q=Read%20https%3A%2F%2Fdocs.banuba.com%2Ffar-sdk%2Ftutorials%2Fcapabilities%2Fsdk_features.md%20\(SDK%20Features\)%20and%20help%20me%20with%20it%20in%20my%20Banuba%20Face%20AR%20SDK%20project.)Install tools ## Face AR SDK[​](#face-ar-sdk "Direct link to Face AR SDK") | | ![Icon](/far-sdk/img/icons/ios.svg "iOS") | ![Icon](/far-sdk/img/icons/android.svg "Android") | ![Icon](/far-sdk/img/icons/apple.svg "MacOS") | ![Icon](/far-sdk/img/icons/windows.svg "Windows") | ![Icon](/far-sdk/img/icons/html5.svg "Web") | | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | :------------------------------------------------------------------: | :------------------------------------------------------------------: | :------------------------------------------------------------------: | :------------------------------------------------------------------: | :------------------------------------------------------------------: | | [Single Face Tracking](/far-sdk/tutorials/capabilities/glossary.md#frx-face-tracking)CPUGPU[![video icon](/far-sdk/img/icons/video.svg)](https://www.youtube.com/watch?v=Of7-xNDYknY) | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | | [Multi-Face Tracking](/far-sdk/tutorials/capabilities/glossary.md#frx-face-tracking)CPUGPU[![video icon](/far-sdk/img/icons/video.svg)](https://www.youtube.com/watch?v=IE4fC4gSWnA)[![video icon](/far-sdk/img/icons/video.svg)](https://www.youtube.com/watch?v=dJ7NBMzlAt8) | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | | [Makeup](/far-sdk/tutorials/capabilities/glossary.md#face-beautification)CPUGPU[![video icon](/far-sdk/img/icons/video.svg)](https://www.youtube.com/watch?v=ZGA-_oq9E2Q) | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | | [Mask on picture from Camera Roll (pre-recorded picture)](/far-sdk/tutorials/capabilities/glossary.md#ar-3d-mask-on-a-picture-from-camera-roll)CPUGPU[![video icon](/far-sdk/img/icons/video.svg)](https://www.youtube.com/watch?v=8XBdbmp8nSo) | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | | [Mask on video from Camera Roll (pre-recorded video)](/far-sdk/tutorials/capabilities/glossary.md#ar-3d-mask-on-video-from-camera-roll)CPUGPU[![video icon](/far-sdk/img/icons/video.svg)](https://www.youtube.com/watch?v=steQQNeQsxU) | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | | [Face attributes](/far-sdk/tutorials/capabilities/glossary.md#face-attributes)CPU | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | | [Face match](/far-sdk/tutorials/capabilities/glossary.md#face-match)CPU | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | | [Face shape](/far-sdk/tutorials/capabilities/glossary.md#face-shape)CPU | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | | [Pupillary distance](/far-sdk/tutorials/capabilities/glossary.md#pupillary-distance)CPU | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | ## Face AR SDK Neural Network Features[​](#face-ar-sdk-neural-network-features "Direct link to Face AR SDK Neural Network Features") | | ![Icon](/far-sdk/img/icons/ios.svg "iOS") | ![Icon](/far-sdk/img/icons/android.svg "Android") | ![Icon](/far-sdk/img/icons/apple.svg "MacOS") | ![Icon](/far-sdk/img/icons/windows.svg "Windows") | ![Icon](/far-sdk/img/icons/html5.svg "Web") | | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | :------------------------------------------------------------------: | :------------------------------------------------------------------: | :------------------------------------------------------------------: | :------------------------------------------------------------------: | :------------------------------------------------------------------: | | [Background separation](/far-sdk/tutorials/capabilities/glossary.md#background-separation)GPU[![video icon](/far-sdk/img/icons/video.svg)](https://www.youtube.com/watch?v=aAlsELbPTX0) | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | | [Skin segmentation](/far-sdk/tutorials/capabilities/glossary.md#skin-segmentation)GPU[![video icon](/far-sdk/img/icons/video.svg)](https://www.youtube.com/watch?v=CeLGmY9w2Kg) | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | | [Hair segmentation](/far-sdk/tutorials/capabilities/glossary.md#hair-segmentation)GPU[![video icon](/far-sdk/img/icons/video.svg)](https://www.youtube.com/watch?v=WvGeyA3FYS4) | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | | [Eye segmentation (3 layers)](/far-sdk/tutorials/capabilities/glossary.md#eye-segmentation)GPU[![video icon](/far-sdk/img/icons/video.svg)](https://www.youtube.com/watch?v=UZVM-sHbWfY) | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | | [Lips segmentation](/far-sdk/tutorials/capabilities/glossary.md#lips-segmentation)GPU[![video icon](/far-sdk/img/icons/video.svg)](https://www.youtube.com/watch?v=K2BSZotVM7U) | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | | [Hair strands painting](/far-sdk/tutorials/capabilities/glossary.md#hair-strands-painting)CPUGPU[![video icon](/far-sdk/img/icons/video.svg)](https://www.youtube.com/watch?v=MmZdVQSqa58) | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | *+* | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | *+* | *+* | | [Brows segmentation](/far-sdk/tutorials/capabilities/glossary.md#brows-segmentation)GPU | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | | [Glasses detection](/far-sdk/tutorials/capabilities/glossary.md#glasses-detection) | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | | [Neck segmentation ans smoothing](/far-sdk/tutorials/capabilities/glossary.md#neck-segmentation-and-smoothing)CPUGPU | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | ## Rendering Engine[​](#rendering-engine "Direct link to Rendering Engine") | | ![Icon](/far-sdk/img/icons/ios.svg "iOS") | ![Icon](/far-sdk/img/icons/android.svg "Android") | ![Icon](/far-sdk/img/icons/apple.svg "MacOS") | ![Icon](/far-sdk/img/icons/windows.svg "Windows") | ![Icon](/far-sdk/img/icons/html5.svg "Web") | | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | :------------------------------------------------------------------: | :------------------------------------------------------------------: | :------------------------------------------------------------------: | :------------------------------------------------------------------: | :------------------------------------------------------------------: | | 3d modeling & animationGPU | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | | Image-based lightingGPU | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | | [Physically based rendering](/far-sdk/tutorials/capabilities/glossary.md#physically-based-rendering)GPU[![video icon](/far-sdk/img/icons/video.svg)](https://www.youtube.com/watch?v=J6XTFaJL7wc) | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | | [Skinned Mesh Animations](/far-sdk/tutorials/capabilities/glossary.md#skinned-mesh-animation)GPU[![video icon](/far-sdk/img/icons/video.svg)](https://www.youtube.com/watch?v=j5jnWkwHIVM) | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | | [Face Morphing](/far-sdk/tutorials/capabilities/glossary.md#morphing)GPU[![video icon](/far-sdk/img/icons/video.svg)](https://www.youtube.com/watch?v=sw8sU2zcD_8) | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | | High dynamic range imaging (HDRI)GPU | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | | [Video textures](/far-sdk/tutorials/capabilities/glossary.md#video-textures)GPU[![video icon](/far-sdk/img/icons/video.svg)](https://www.youtube.com/watch?v=20EVsXlCGss) | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | | Multisample anti-aliasingGPU | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | | Sprite animationGPU | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | | [Lookup Tables (LUT)](/far-sdk/tutorials/capabilities/glossary.md#lut-post-processing)GPU[![video icon](/far-sdk/img/icons/video.svg)](https://www.youtube.com/watch?v=vMznyB7eyCI) | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | | [Texture sequences](/far-sdk/tutorials/capabilities/glossary.md#texture-sequences)GPU[![video icon](/far-sdk/img/icons/video.svg)](https://www.youtube.com/watch?v=62GXnNyLypg) | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | | [Lips shine, gloss, chameleon](/far-sdk/tutorials/capabilities/glossary.md#lips-shine-gloss-chameleon)GPU | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | | [Eyelids gloss, chameleon](/far-sdk/tutorials/capabilities/glossary.md#eyelids-gloss-chameleon)GPU | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | | [Light correction](/far-sdk/tutorials/capabilities/glossary.md#light-correction)CPUGPU | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | ## Other Features[​](#other-features "Direct link to Other Features") | | ![Icon](/far-sdk/img/icons/ios.svg "iOS") | ![Icon](/far-sdk/img/icons/android.svg "Android") | ![Icon](/far-sdk/img/icons/apple.svg "MacOS") | ![Icon](/far-sdk/img/icons/windows.svg "Windows") | ![Icon](/far-sdk/img/icons/html5.svg "Web") | | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | :------------------------------------------------------------------: | :------------------------------------------------------------------: | :------------------------------------------------------------------: | :------------------------------------------------------------------: | :------------------------------------------------------------------: | | [Face Filter Text Overlays (Text Texture)](/far-sdk/tutorials/capabilities/glossary.md#text-texture-on-ar-3d-mask)CPU[![video icon](/far-sdk/img/icons/video.svg)](https://www.youtube.com/watch?v=SGIRxCcrvMo) | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | | [Interactive Triggers](/far-sdk/tutorials/capabilities/glossary.md#trigger)CPU[![video icon](/far-sdk/img/icons/video.svg)](https://www.youtube.com/watch?v=7Y9nMgDzpbI) | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | | [Interactive Touch](/far-sdk/tutorials/capabilities/glossary.md#touches)CPU[![video icon](/far-sdk/img/icons/video.svg)](https://www.youtube.com/watch?v=6vRAuWvmlFw) | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | | [Face-To-Phone Distance (Ruler)](/far-sdk/tutorials/capabilities/glossary.md#ruler-distance-to-phone)CPU[![video icon](/far-sdk/img/icons/video.svg)](https://www.youtube.com/watch?v=4HO38U4C6HI) | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | | [Heart Rate detector](/far-sdk/tutorials/capabilities/glossary.md#pulse-heart-rate)CPU[![video icon](/far-sdk/img/icons/video.svg)](https://www.youtube.com/watch?v=FpFU8YhIXHI) | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | | AvatarsCPUGPU[![video icon](/far-sdk/img/icons/video.svg)](https://www.youtube.com/watch?v=oY69-kG4jZ0) | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | | [Eye lenses](/far-sdk/tutorials/capabilities/glossary.md#eye-lenses)CPU | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | | [Teeth tone](/far-sdk/tutorials/capabilities/glossary.md#teeth-tone)CPU | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | | [Glasses frame color](/far-sdk/tutorials/capabilities/glossary.md#glasses-frame-color)CPUGPU | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | ## Hand AR SDK[​](#hand-ar-sdk "Direct link to Hand AR SDK") Comes as part of Face AR bundle. | | ![Icon](/far-sdk/img/icons/ios.svg "iOS") | ![Icon](/far-sdk/img/icons/android.svg "Android") | ![Icon](/far-sdk/img/icons/apple.svg "MacOS") | ![Icon](/far-sdk/img/icons/windows.svg "Windows") | ![Icon](/far-sdk/img/icons/html5.svg "Web") | | ------------------------------------------------------------------------------------------------------------------------------------------------- | :------------------------------------------------------------------: | :------------------------------------------------------------------: | :------------------------------------------------------------------: | :------------------------------------------------------------------: | :------------------------------------------------------------------: | | Hand AR: NailsCPUGPU[![video icon](/far-sdk/img/icons/video.svg)](https://drive.google.com/file/d/1ch-dXgcN-arxZVYzCkbiIsctme1tsCde/preview) | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | | Hand AR: Hand GesturesCPU[![video icon](/far-sdk/img/icons/video.svg)](https://drive.google.com/file/d/1ch-dXgcN-arxZVYzCkbiIsctme1tsCde/preview) | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | | Hand AR: Hand SkeletonCPU[![video icon](/far-sdk/img/icons/video.svg)](https://drive.google.com/file/d/1ch-dXgcN-arxZVYzCkbiIsctme1tsCde/preview) | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | | Hand AR: RingsCPU[![video icon](/far-sdk/img/icons/video.svg)](https://drive.google.com/file/d/1ch-dXgcN-arxZVYzCkbiIsctme1tsCde/preview) | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | | Hand AR: WatchesCPU[![video icon](/far-sdk/img/icons/video.svg)](https://drive.google.com/file/d/1ch-dXgcN-arxZVYzCkbiIsctme1tsCde/preview) | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") | #### Legend[​](#legend "Direct link to Legend") * ![\`${props.title} icon\`](/far-sdk/img/icons/check.svg "Supported") - this technology fully supports the platform and has proper test coverage. * *+* - this technology doesn’t support the platform. * CPU, GPU - the badge shows the processing unit used by this technology. * [![video icon](/far-sdk/img/icons/video.svg)](#legend) - a link to the demonstration video is available for this technology. * [Info](#legend) - a link to an article describing work of this technology. --- # System Requirements [View as Markdown](https://docs.banuba.com/far-sdk/tutorials/capabilities/system_requirements.md)[![](/far-sdk/img/ai-guide/chatgpt.svg)Open in ChatGPT](https://chatgpt.com/?q=Read%20https%3A%2F%2Fdocs.banuba.com%2Ffar-sdk%2Ftutorials%2Fcapabilities%2Fsystem_requirements.md%20\(System%20Requirements\)%20and%20help%20me%20with%20it%20in%20my%20Banuba%20Face%20AR%20SDK%20project.)[![](/far-sdk/img/ai-guide/claude.svg)Open in Claude](https://claude.ai/new?q=Read%20https%3A%2F%2Fdocs.banuba.com%2Ffar-sdk%2Ftutorials%2Fcapabilities%2Fsystem_requirements.md%20\(System%20Requirements\)%20and%20help%20me%20with%20it%20in%20my%20Banuba%20Face%20AR%20SDK%20project.)Install tools ## Supported Platforms[​](#supported-platforms "Direct link to Supported Platforms") ### Mobile ![Icon](/far-sdk/img/icons/ios.svg "iOS") ![Icon](/far-sdk/img/icons/android.svg "Android") ![Icon](/far-sdk/img/icons/html5.svg "Web") ### Requirements * **Any platform: **OpenGL ES3.0 and higher * **Android: **supported from Android 8.0. API level 26 * **iOS: **supported from iOS 13.0. ### Desktop ![Icon](/far-sdk/img/icons/apple.svg "MacOS") ![Icon](/far-sdk/img/icons/windows.svg "Windows") ![Icon](/far-sdk/img/icons/html5.svg "Web") ### Requirements PC * Support OpenGL 4.3 and up (4.1 for MacOS) * Windows 8.1 and up * MacOS 10.13 and up *** ## Supported Browsers[​](#supported-browsers "Direct link to Supported Browsers") ### Mobile ![Icon](/far-sdk/img/icons/chrome.svg "Chrome") ![Icon](/far-sdk/img/icons/firefox.svg "Firefox") ![Icon](/far-sdk/img/icons/safari.svg "Safari") ### Desktop ![Icon](/far-sdk/img/icons/chrome.svg "Chrome") ![Icon](/far-sdk/img/icons/firefox.svg "Firefox") ![Icon](/far-sdk/img/icons/safari.svg "Safari") ### Requirements[​](#requirements "Direct link to Requirements") * Banuba Web SDK supports any browser with **WebGL 2.0 and higher**. All supported browsers are listed [here](https://caniuse.com/webgl2). *** ## Supported languages[​](#supported-languages "Direct link to Supported languages") | Languages | Platforms | | ------------------------------------------------------------------------------------------------- | --------- | | ![Icon](/far-sdk/img/icons/objc.svg) Objective-C
![Icon](/far-sdk/img/icons/swift.svg) Swift | MacOS | | ![Icon](/far-sdk/img/icons/kotlin.svg) Kotlin
![Icon](/far-sdk/img/icons/java.svg) Java | Android | | ![Icon](/far-sdk/img/icons/cpp.svg) C++ | Desktop | | ![Icon](/far-sdk/img/icons/csharp.svg) C# | Unity | | ![Icon](/far-sdk/img/icons/javascript.svg) JavaScript\* | Web | \* Types for TypeScript are also provided --- # Technical Specification [View as Markdown](https://docs.banuba.com/far-sdk/tutorials/capabilities/technical_specification.md)[![](/far-sdk/img/ai-guide/chatgpt.svg)Open in ChatGPT](https://chatgpt.com/?q=Read%20https%3A%2F%2Fdocs.banuba.com%2Ffar-sdk%2Ftutorials%2Fcapabilities%2Ftechnical_specification.md%20\(Technical%20Specification\)%20and%20help%20me%20with%20it%20in%20my%20Banuba%20Face%20AR%20SDK%20project.)[![](/far-sdk/img/ai-guide/claude.svg)Open in Claude](https://claude.ai/new?q=Read%20https%3A%2F%2Fdocs.banuba.com%2Ffar-sdk%2Ftutorials%2Fcapabilities%2Ftechnical_specification.md%20\(Technical%20Specification\)%20and%20help%20me%20with%20it%20in%20my%20Banuba%20Face%20AR%20SDK%20project.)Install tools This page provides technical metrics of the Face AR SDK feature performance. The values below are for your reference, as they were achieved under fixed lab conditions. Many factors can influence performance, including the state of the specific device, other apps running in the background, Wi-Fi being enabled, etc. We encourage you to test each feature within your environment. Please visit the [SDK Features](/far-sdk/tutorials/capabilities/sdk_features.md) page for more information on feature availability on different platforms. note * **FPS** β€” Frames per second of the face detection algorithm on a given device. * **Angles** β€” The maximum angle at which the technology was able to work during the measurement. * **Distance** β€” Maximum distance at which the technology was able to work during the measurement. * **Real-time (online)** β€” Technology performance in real-time. * **Photo (offline)** β€” The processing time needed to take a photo or process it from the gallery. ## SDK Features[​](#sdk-features "Direct link to SDK Features") ### Single-face Tracking[​](#single-face-tracking "Direct link to Single-face Tracking") **Android** | | Android Low | Android High | | -------------- | ----------- | ------------ | | FPS | 25 | 30 | | Angles, degree | 80 | 80 | | Distance, cm | 170 | 180 | **iOS** | | iOS Mid | iOS High | | -------------- | ------- | -------- | | FPS | 30 | 30 | | Angles, degree | 80 | 80 | | Distance, cm | 230 | 230 | ### Multi-face Tracking[​](#multi-face-tracking "Direct link to Multi-face Tracking") **Android** | | Android Low | Android High | | ------------ | ----------- | ------------ | | Max Faces | 5 | 5 | | 4 Faces, FPS | 23 | 28 | | 5 Faces, FPS | 22 | 27 | **iOS** | | iOS Mid | iOS High | | ------------ | ------- | -------- | | Max Faces | 5 | 5 | | 4 Faces, FPS | 30 | 30 | | 5 Faces, FPS | 30 | 30 | Max Faces\* β€” the maximum number of faces that the SDK can track with acceptable quality and performance on most mobile devices. The actual peak number is limited only by the physical capabilities of the device and its screen proportions. ## Effect performance[​](#effect-performance "Direct link to Effect performance") Banuba SDK allows for a variety of Face AR effects. Some of them only require face tracking and can be represented as a single AR 3D Mask with textures and materials. Other effects are implemented with separately trained neural networks. Below, you may find information on the real-time performance of Face AR effects which only require face tracking, i.e. face filters, avatars with action units, beautification, and makeup filter (without lipstick). **Android** | | Android Low | Android High | | --- | ----------- | ------------ | | FPS | 25 | 29 | **iOS** | | iOS Mid | iOS High | | --- | ------- | -------- | | FPS | 30 | 30 | ## Beautification[​](#beautification "Direct link to Beautification") ### Beautification filter[​](#beautification-filter "Direct link to Beautification filter") Basic face beautification filter includes skin smoothing, morphing, teeth whitening, eyes flare and LUT. It only requires face tracking, so please, refer to the [Effects performance](#effect-performance) section. ## Makeup[​](#makeup "Direct link to Makeup") The Makeup filter allows for a realistic try on of foundation, eyeshadow, eyeliner, highlighter, contour, and blusher. It only requires face tracking, so please, refer to the [Effects performance](#effect-performance) section. The lipstick try on requires lips segmentation neural network with a separate algorithm for Lips Shine effect. ### Lips coloring[​](#lips-coloring "Direct link to Lips coloring") **Android** | | Android Low | Android High | | -------------- | ----------- | ------------ | | Real-time, FPS | 25 | 30 | | Photo, sec | 1 | < 1 | **iOS** | | iOS Mid | iOS High | | -------------- | ------- | -------- | | Real-time, FPS | 30 | 30 | | Photo, sec | < 1 | < 1 | **Web** | | Real-time, FPS | | ------ | -------------- | | Chrome | 30 | | Safari | 26 | ### Lips Shine (Glossy lipstick)[​](#lips-shine-glossy-lipstick "Direct link to Lips Shine (Glossy lipstick)") **Android** | | Android Low | Android High | | -------------- | ----------- | ------------ | | Real-time, FPS | 25 | 30 | | Photo, sec | 1 | < 1 | **iOS** | | iOS Mid | iOS High | | -------------- | ------- | -------- | | Real-time, FPS | 30 | 30 | | Photo, sec | < 1 | < 1 | **Web** | | Real-time, FPS | | ------ | -------------- | | Chrome | 30 | | Safari | 25 | ## Background separation[​](#background-separation "Direct link to Background separation") **Android** | | Android Low | Android High | | ------------- | ----------- | ------------ | | Real-time,FPS | 25 | 30 | | Photo, sec | 1 | < 1 | **iOS** | | iOS Mid | iOS High | | -------------- | ------- | -------- | | Real-time, FPS | 30 | 30 | | Photo, sec | < 1 | < 1 | ### Distance[​](#distance "Direct link to Distance") | Device | Distance, cm | | ------------ | -------------------------------------- | | iOS High | Portrait 280 cm,
Landscape 360 cm | | iOS Low | Portrait 280 cm,
Landscape 360 cm | | Android High | Portrait 310 cm,
Landscape 370 cm | | Mac Mid | 330 cm | ### Image formats support[​](#image-formats-support "Direct link to Image formats support") Currently, the following images formats are supported as a background texture: `.jpeg`, `.jpg`, `.png`, `.ktx`, `.gif`. ### Video formats support[​](#video-formats-support "Direct link to Video formats support") Used as a part of an animated background. | Video format | MacOS\* | iOS | Android\*\* | Windows | | ------------ | ------- | --- | ----------- | ------- | | .mp4 | βœ… | βœ… | βœ… | βœ… | | .avi | βœ… | ❌ | ❌ | βœ… | | .flv | βœ… | ❌ | ❌ | βœ… | | .mkv | βœ… | ❌ | βœ… | βœ… | | .mov | βœ… | βœ… | ❌ | βœ… | | .mts | βœ… | ❌ | ❌ | βœ… | | .webm | βœ… | ❌ | βœ… | βœ… | | .wmv | βœ… | ❌ | ❌ | βœ… | \* MacOS is rather sensitive not only to containers (i.e. file extensions) but to video codecs itself. In case of problems observe application log to find corresponding error messages and test carefully before release. \*\* See more information about supported video formats on Android in the official Android developers guide β€” . ## Hair segmentation[​](#hair-segmentation "Direct link to Hair segmentation") ### Hair Recoloring[​](#hair-recoloring "Direct link to Hair Recoloring") **Android** | | Android Low | Android High | | ------------- | ----------- | ------------ | | Real-time,FPS | 25 | 30 | | Photo, sec | 1 | < 1 | **iOS** | | iOS Mid | iOS High | | ------------- | ------- | -------- | | Real-time,FPS | 30 | 30 | | Photo, sec | < 1 | < 1 | ## Skin segmentation[​](#skin-segmentation "Direct link to Skin segmentation") **Android** | | Android Low | Android High | | ------------- | ----------- | ------------ | | Real-time,FPS | 25 | 30 | | Photo, sec | < 1 | < 1 | **iOS** | | iOS Mid | iOS High | | ------------- | ------- | -------- | | Real-time,FPS | 30 | 30 | | Photo, sec | < 1 | < 1 | ## Eyes recoloring[​](#eyes-recoloring "Direct link to Eyes recoloring") **Android** | | Android Low | Android High | | ------------- | ----------- | ------------ | | Real-time,FPS | 25 | 30 | | Photo, sec | < 1 | < 1 | **iOS** | | iOS Mid | iOS High | | ------------- | ------- | -------- | | Real-time,FPS | 30 | 30 | | Photo, sec | < 1 | < 1 | ## Hand gestures[​](#hand-gestures "Direct link to Hand gestures") **Basic information** * Supported gestures: * Palm βœ‹ * Victory✌️ * Rock 🀘 * Like πŸ‘ * Ok πŸ‘Œ * Maximum distance β€” 2.5m **iOS** | Device | Realtime FPS | | ------ | ------------ | | Mid | 30 | | High | 30 | **Android** | Device | Realtime FPS | | ------ | ------------ | | Low | 30 | | High | 30 | --- # Token Management [View as Markdown](https://docs.banuba.com/far-sdk/tutorials/capabilities/token_management.md)[![](/far-sdk/img/ai-guide/chatgpt.svg)Open in ChatGPT](https://chatgpt.com/?q=Read%20https%3A%2F%2Fdocs.banuba.com%2Ffar-sdk%2Ftutorials%2Fcapabilities%2Ftoken_management.md%20\(Token%20Management\)%20and%20help%20me%20with%20it%20in%20my%20Banuba%20Face%20AR%20SDK%20project.)[![](/far-sdk/img/ai-guide/claude.svg)Open in Claude](https://claude.ai/new?q=Read%20https%3A%2F%2Fdocs.banuba.com%2Ffar-sdk%2Ftutorials%2Fcapabilities%2Ftoken_management.md%20\(Token%20Management\)%20and%20help%20me%20with%20it%20in%20my%20Banuba%20Face%20AR%20SDK%20project.)Install tools To use our SDK in your project, you need to have an SDK token. The FAQ below explains our token management process and guides you on how to store and update tokens after their expiration. Please, read all the information carefully. ## What is an SDK token?[​](#what-is-an-sdk-token "Direct link to What is an SDK token?") An SDK token is an automatically generated set of characters in .txt format unique to each client. It activates the licenced SDK functionality in the client app. There are two types of tokens: * A demo token is provided to start the SDK trial. It's *valid for 14 days*, our standard trial period. The token activates all SDK features for you to assess the SDK performance in your project. * A commercial token is provided after you make payment. It’s *valid throughout the prepaid period*. The token activates the SDK features defined by your software licence. note Once the token expires, the Banuba watermark and screen blur will appear automatically in your app. Therefore, please: * Don’t use demo tokens in live apps. * Observe payments of your SDK license and renew the commercial token on time. ## Why do we use tokens?[​](#why-do-we-use-tokens "Direct link to Why do we use tokens?") The token system helps us to manage billing, as well as protects our software from frauds, inappropriate and unconditioned usage that violates the SDK licensing terms. ## How do I get one?[​](#how-do-i-get-one "Direct link to How do I get one?") * To get the demo token and start the SDK trial, please contact your sales manager or [send us a request](https://www.banuba.com/facear-sdk/face-filters#form) via the website form. * The commercial token is generated and sent out by your account manager who guides your project within our company. ## How does it work?[​](#how-does-it-work "Direct link to How does it work?") **Token storage** We recommend storing your token on the server as it drastically speeds up the process of token renewal. caution If you store the token in the app, you will have to to upload its newest version to the App Store or Play Market after you renew the token. **Expiry and renewal** Tokens are valid only throughout the predefined period of time. For one month after the expiration date, you can still use the SDK in your app, but it will display Banuba watermark. After that, the SDK will stop working entirely, though the app will otherwise work normally. Extending the existing token’s life will be impossible. You will have to receive a new one. See the explanation of token expiration below: | Token expired? | What happens with Face AR SDK | | ------------------------------- | -------------------------------- | | No | Works as expected | | Yes, <= 1 month | Works but watermark is displayed | | Yes, > 1 month | Functionality won't work | To restore the access, you need to request a new token and renew it in your app. * **Demo tokens** may be renewed per client’s request in case the client hasn’t had enough time to evaluate the SDK. * **Commercial tokens** are renewed by an account manager only after the client’s pre-payment for an agreed period. **Other questions left?** Please, read all the information carefully and feel free to [contact us](/far-sdk/support/.md) if you have more questions. --- # Changelog [View as Markdown](https://docs.banuba.com/far-sdk/tutorials/changelog.md)[![](/far-sdk/img/ai-guide/chatgpt.svg)Open in ChatGPT](https://chatgpt.com/?q=Read%20https%3A%2F%2Fdocs.banuba.com%2Ffar-sdk%2Ftutorials%2Fchangelog.md%20\(Changelog\)%20and%20help%20me%20with%20it%20in%20my%20Banuba%20Face%20AR%20SDK%20project.)[![](/far-sdk/img/ai-guide/claude.svg)Open in Claude](https://claude.ai/new?q=Read%20https%3A%2F%2Fdocs.banuba.com%2Ffar-sdk%2Ftutorials%2Fchangelog.md%20\(Changelog\)%20and%20help%20me%20with%20it%20in%20my%20Banuba%20Face%20AR%20SDK%20project.)Install tools ## \[1.18.5] - 2026-08-10[​](#1185---2026-08-10 "Direct link to \[1.18.5] - 2026-08-10") **Changed** * General improvements and performance enhancements ## \[1.18.4] - 2026-07-17[​](#1184---2026-07-17 "Direct link to \[1.18.4] - 2026-07-17") **Added** * Acne severity detection algorithm **Changed** * Improved eyeshadow colors rendering * Improved makeup highlighter products application ## \[1.18.3] - 2026-07-02[​](#1183---2026-07-02 "Direct link to \[1.18.3] - 2026-07-02") **Added** * Facial skin smoothing algorithm **Changed** * Improved lipstick colors rendering **Fixed** * Other improvements and performance enhancements ## \[1.18.2] - 2026-06-03[​](#1182---2026-06-03 "Direct link to \[1.18.2] - 2026-06-03") **Added** * Acne detection algorithm **Fixed** * Other improvements and performance enhancements ## \[1.18.1] - 2026-04-30[​](#1181---2026-04-30 "Direct link to \[1.18.1] - 2026-04-30") **Changed** * Improved makeup effects performance **Fixed** * Other improvements and performance enhancements ## \[1.18.0] - 2026-03-09[​](#1180---2026-03-09 "Direct link to \[1.18.0] - 2026-03-09") **Changed** * Upgraded face tracking algorithm * Improved glasses lens segmentation * Improved glasses frame color detection * Improved skin tone detection * Improved 3d objects positioning on the face **Fixed** * Other improvements and performance enhancements ## \[1.17.7] - 2025-12-19[​](#1177---2025-12-19 "Direct link to \[1.17.7] - 2025-12-19") **Added** * Glitter effect for makeup * Glasses detection algorithm * Glasses lens segmentation algorithm * Glasses frame color detection algorithm **Changed** * Virtual Background segmentation improvements * Face shape detection algorithm improvements ## \[1.17.6] - 2025-09-29[​](#1176---2025-09-29 "Direct link to \[1.17.6] - 2025-09-29") **Added** * Face shape detection **Changed** * Enhancements for various platforms and effects ## \[1.17.5] - 2025-08-29[​](#1175---2025-08-29 "Direct link to \[1.17.5] - 2025-08-29") **Added** * Gloss and chameleon color effects for eyeshadows * Numerous enhancements for various platforms and effects **Changed** * Improved virtual background segmentation ## \[1.17.4] - 2025-07-18[​](#1174---2025-07-18 "Direct link to \[1.17.4] - 2025-07-18") **Added** * Gloss and chameleon color effects for makeup * Numerous enhancements for various platforms and effects **Changed** * Improved Gender Recognition model ## \[1.17.3] - 2025-06-06[​](#1173---2025-06-06 "Direct link to \[1.17.3] - 2025-06-06") **Added** * Lips Glare effect for Windows, OSX and Emscripten **Fixed** * Bug fixes and improvements for various platforms ## \[1.17.2] - 2025-05-23[​](#1172---2025-05-23 "Direct link to \[1.17.2] - 2025-05-23") **Added** * "Weatherman" mode * Support for video textures in effects **Changed** * Improved background segmentation * Improved hair segmentation **Fixed** * Bug fixes and improvements for various platforms ## \[1.17.1] - 2025-04-16[​](#1171---2025-04-16 "Direct link to \[1.17.1] - 2025-04-16") **Added** * Experimental feature "Weatherman" **Changed** * Resource linking ## \[1.17.0] - 2025-03-20[​](#1170---2025-03-20 "Direct link to \[1.17.0] - 2025-03-20") **Added** * Light source detection & correction **Changed** * Major performance optimization * Optimized performance of teeth whitening algorithm * Updated face tracking * Improved stability of face description model **Fixed** * Bug fixes and improvements for various platforms ## \[1.16.4] - 2025-01-28[​](#1164---2025-01-28 "Direct link to \[1.16.4] - 2025-01-28") **Added** * Hair segmentation mask stabilization **Fixed** * Bug fixes and improvements for various platforms ## \[1.16.3] - 2024-12-17[​](#1163---2024-12-17 "Direct link to \[1.16.3] - 2024-12-17") **Fixed** * Performance meter crash ## \[1.16.2] - 2024-12-12[​](#1162---2024-12-12 "Direct link to \[1.16.2] - 2024-12-12") **Added** * Light conversion effect **Changed** * Numerous documentation improvements * Switch to FFMPEG 4.10.2 for Windows **Fixed** * Functioning of Biometric Match feature * Scripting memory leak for iOS and macOS ## \[1.16.1] - 2024-11-29[​](#1161---2024-11-29 "Direct link to \[1.16.1] - 2024-11-29") **Changed** * Improved VITA Shade tone detection and teeth whitening **Fixed** * Bug fixes for various platforms ## \[1.16.0] - 2024-10-17[​](#1160---2024-10-17 "Direct link to \[1.16.0] - 2024-10-17") **Added** * Light Source Detection **Changed** * Improved face morphings * Improved hair segmentation **Fixed** * Acne removal * Music playback on certain effects ## \[1.15.1] - 2024-08-07[​](#1151---2024-08-07 "Direct link to \[1.15.1] - 2024-08-07") **Added** * Load Banuba Native Code using Relinker Library (Android) * Zoom and torch to player API * Teeth tone to generator **Fixed** * Permissions on start * Fixed long loading time for makeup * Fixed effects freezing in the new version of Chrome * Crash when frame size is not a multiple of 16 * Face Morphing fix ## \[1.14.0] - 2024-06-07[​](#1140---2024-06-07 "Direct link to \[1.14.0] - 2024-06-07") **Added** * New teeth whitening algorithm * Gender detection * ASTC format support * Frown Action Units **Changed** * Face attributes (Personalized 3D avatar) **Fixed** * Some fixes for prefabs * Texture formats on Metal ## \[1.13.2] - 2024-05-27[​](#1132---2024-05-27 "Direct link to \[1.13.2] - 2024-05-27") **Fixed** * KTX dynamic upload ## \[1.13.1] - 2024-05-30[​](#1131---2024-05-30 "Direct link to \[1.13.1] - 2024-05-30") **Fixed** * Android strip symbols ## \[1.13.0] - 2024-04-30[​](#1130---2024-04-30 "Direct link to \[1.13.0] - 2024-04-30") **Added** * Biometric match * Pupillary distance * Nails and eyelenses prefabs * GLTF 2.0 Support **Changed** * Eyelashes texture ## \[1.12.1] - 2024-04-10[​](#1121---2024-04-10 "Direct link to \[1.12.1] - 2024-04-10") **Fixed** * Web: * Frame leak, when source changed * Analytics crash * Video textures in effects (Win) * Unity: * Background & hair segmentation * Lips shine scale ## \[1.12.0] - 2024-04-02[​](#1120---2024-04-02 "Direct link to \[1.12.0] - 2024-04-02") **Added** * Personalized 3D avatar (iOS) - beta version * Nails segmentation (Android) * Improved creating effects process * FP16 flag XNNPACK for TFLite * Privacy manifest and signing (iOS) * Asynchronous pixels reading * Studio lightning effect * useFutureFilter option (Web) **Fixed** * Actions Units improvements * Face tracking performance improvements (Android + Web Safari) * Video track loading (Web) ## \[1.11.1] - 2024-02-23[​](#1111---2024-02-23 "Direct link to \[1.11.1] - 2024-02-23") **Added** * Fit mode for Unity **Fixed** * Win: GLTF models for AMD GPUs * Unity: * Segmentation clamp issues * Materials in effects * Hand skeleton transformation and rendering ## \[1.11.0] - 2024-02-09[​](#1110---2024-02-09 "Direct link to \[1.11.0] - 2024-02-09") **Added** * Improved background segmentation in landscape * New skin segmentation * Android: * VideoInput to Player API * TextureOutput to Player API * IRenderStatusCallback to Player API * IFrameRotationProviderCallback to Player API * WebAR: mediastream processor * iOS: * Player.onRender callback **Changed** * UV scale **Fixed** * FRX scattering * iOS: * breaking segmentation on the lower parts of the frame, camera orientation track * Android: * Calculation of strides for FrameOutput in the Player API ## \[1.10.1] - 2024-01-10[​](#1101---2024-01-10 "Direct link to \[1.10.1] - 2024-01-10") **Added** * Face-skin segmentation for Unity **Fixed** * Flashing morphing unity * Support for simulators ## \[1.10.0] - 2023-12-20[​](#1100---2023-12-20 "Direct link to \[1.10.0] - 2023-12-20") **Added** * Eye’s dark circles removal * New face morphs in the Makeup API * Teeth segmentation * ActionUnits antijitter * Ability to download iOS packages via SPM * `SeverityLevel.NONE` to disable logs **Changed** * Lip segmentation * Removed GLTF autoscale **Fixed** * iOS: Photo input now tracks device orientation (Player API) * Android: correctly resume audio from a paused or stopped state * Webar: * OpenCV SIMD instructions for Safari 16.4 * Empty files unzip * useFutureInterpolate option * Crash during module loading in an unsupported browser * Webcamera performance improved * Red AR 3D Mask bad colours ## \[1.9.3] - 2023-12-06][​](#193---2023-12-06 "Direct link to \[1.9.3] - 2023-12-06]") **Changed** * Remove auto-scale from GLTF loading ## \[1.9.2] - 2023-11-29[​](#192---2023-11-29 "Direct link to \[1.9.2] - 2023-11-29") **Added** * Face skin segmentation * Android & iOS: apply a watermark to recorded video (Player API) * Full HD video recording iOS ## \[1.9.1] - 2023-11-8[​](#191---2023-11-8 "Direct link to \[1.9.1] - 2023-11-8") **Fixed** * Action Units eyes blinking * Android: support for relative paths for video textures * WebAR: * Player FPS restriction * Perf measure * Rendering delay (Firefox) * NPM package missing modules * Agora Filter Extension v2.0.0 * Unity: * Prefabs ordering (export layers) * Makeup prefab * Camera UV after resize * Morphing prefab * UI rework ## \[1.9.0] - 2023-10-19[​](#190---2023-10-19 "Direct link to \[1.9.0] - 2023-10-19") **Added** * Rendering quality improvements for head wearable and arm wearable products * Earlobes detection * Skin smoothing and morphing improvements in Unity * Transmissive materials in GLTF files * New Effect Player API for Web, iOS & Android. Information about migration can be [here](/far-sdk/tutorials/development/guides/migration.md) **Changed** * Turned off future filter on eyes * Render improvements for caps & rings * Makeup softlight texture **Fixed** * iOS: Online mode for acne removal, some effects, call callback on the main thread, Video recording * WebAR: Process crash, broken textures in effects WebAR ## \[1.8.1] - 2023-09-01[​](#181---2023-09-01 "Direct link to \[1.8.1] - 2023-09-01") **Fixed** * Bug with special symbols in paths * Background transparency * WebGL context lost * Some fixes for OpenGL * Incorrect display of textures in Firefox ## \[1.8.0] - 2023-08-17[​](#180---2023-08-17 "Direct link to \[1.8.0] - 2023-08-17") **Added** * Acne removal for Android * Adjustable acne correction size through effect settings **Changed** * New lip segmentation * Support for Visual Studio 2022 instead of Visual Studio 2019 * WebAR: localhost logic * OEP callback status * GLTF serialization * Default enabled morphing * Skin smoothing **Fixed** * Android Demo App Crash * Added vector binding * TFlite model cache * Lips dithering * View scale * Background segmentation for M1 * Flipped MV in script * Makeup for Unity * Nose morphing * Accelerated physics on processed photos and videos * Makeup eyes colouring * Unity packages ## \[1.7.1] - 2023-06-15[​](#171---2023-06-15 "Direct link to \[1.7.1] - 2023-06-15") **Added** * Web telemetry * Portrait background segmentation for desktop and web * Manual acne removal for iOS **Changed** * Faster loading for segmentation neural networks * Enable OpenCL on Android 12 **Fixed** * Unity packages * Video track * Camera texture * Effect reset * Eyes morphings * Memory leak * JNI initialization * Camera switching * Bad JS cast ## \[1.7.0] - 2023-05-12[​](#170---2023-05-12 "Direct link to \[1.7.0] - 2023-05-12") **Added** * Turbo FRX (default is ON in web) * Motion detector for all segmentation neural networks * Lips ditering * New makeup morphings * Reduced detectors * Background neural networks's with Neural Engine support (iOS only) * Parallel FRX for multiface **Changed** * Sorting of effects for iOS * New EffectPlayer API for iOS * Desktop delivery * FRX update (FRX8) * Render Optimization * Reduced CPU consumption for the hand detector * Resize for metal * Eyebrow correctors **Fixed** * UI for rings * Segmentation postprocessing * TFlite initialization * Action Units eyebrows * Hair AR 3D Mask size * Shaders instancing and names for the new Makeup API * iOS: sound in recordered video, gif playback, video from gallery rotation, * Android: Crash, Assets load, Mips generation * WebAR: Memory leak and some bug fixes for Safari 16.4 * Win: Background issues * Unity: Makeup * OEP: Enable audio for Android, Video as a background ## \[1.6.1] - 2023-03-20[​](#161---2023-03-20 "Direct link to \[1.6.1] - 2023-03-20") **Added** * UI for ring fitting **Changed** * Replace int8 with fp16 in neural networks for Android **Fixed** * Background video play * Neuro\_beauty effect * Photo processing for Android & iOS * Sound in the recorded video * ANR during initialization * Android: Methods for Video Editor; Missed video textures in effects; * iOS: Effect events; * MacOS: Incorrect processing output; * WebAR: missing cleanup; memory leak; playback of the effect's video textures; * OEP: Crash after choosing .gif; background video textures; * Unity: Incorrect camera output. ## \[1.6.0] - 2023-01-23[​](#160---2023-01-23 "Direct link to \[1.6.0] - 2023-01-23") **Added** * Faster face tracking for Safari * Add an API to check compatibility between the browser and SDK * New rings for each finger * Background mode for effects (iOS) * Two types of denoising (Web only) * Play control functions (pause/play) for Android * Python build for M1 **Changed** * SSD Update * Webcam video enhancer opt-in * Agora example update (for Android) * Reduced size of skin\_segm\_tflite * Eyes neural network update **Fixed** * bnb\_UVMORPH texture * Broken face texture when using brow correctors * Missed background in gltf\_avatar effect iOS: MSAA crashed iOS, Crash on Makeup Android: Flipped-up effect test\_touch\_gestures MacOS: Video processing on MacOS WebAR: Token-caused crash of WebAR Win: Missed UI in viewer, spamming error messages during processing Unity: Unity Android crash OEP: Background video mode in OEP ## \[1.5.3] - 2022-11-17[​](#153---2022-11-17 "Direct link to \[1.5.3] - 2022-11-17") **Added** * Unity * Hand tracking * Hand skeleton * Eyebrow segmentation * Makeup **Fixed** * Accurate video seek on Android * Action Units update * Crash on multiface * Videodecoding on the Web ## \[1.5.1] - 2022-10-10[​](#151---2022-10-10 "Direct link to \[1.5.1] - 2022-10-10") **Added** * Variable frame video support (15, 30, 60 FPS etc.) **Changed** * A new blur background algorithm with more accurate borders * Ugly action unit MOUTH\_STRETCH has been excluded from the pipeline * New background segmentation models in landscape for Desktop platforms **Fixed** * Blur background radius settings * Flyout when no lips are found * Android * Crash during JavaScript execution * Video textures for MediaTek+PowerVR devices ## \[1.5.0] - 2022-08-24[​](#150---2022-08-24 "Direct link to \[1.5.0] - 2022-08-24") **Added** * OEP: Added support for BT601 and BT709, full and video ranges * Enabled neural network cache on Android * AR Avatars: technology updates, support for GLTF models added * Makeup API: Added return values for lips, hair, and teeth * Android: Ability to record original video without effects in the Demo app * Unity: More segmentation neural networks * Upgrade TFlite to version 2.9 * New face detector * C API: eval\_js method added * Face tracking for medical AR 3D Masks (does not ship with the regular Face AR SDK, contact your sales manager for more details) * Virtual Background: \*`Background.getBackgroundVideo()` method added * Ability to pause background video right after it's loaded **Changed** * WebAR is now delivered as an NPM package. * iOS Demo: Now video capture stops after pushing videoButton * Upgraded lips segmentation neural network for all platforms except Apple * Effects resources are now loaded asynchronously * iOS, MacOS: New background landscape model * Android Demo App: Improved UI Layout in landscape mode * Eyebrow technology: improved performance and stability * Smile and mouth opening trigger improvements * Improved stability of face tracking and detection * Unity * Plugin and Demo scene refactoring * Morphing refactoring **Fixed** * C API built on Windows * Crash on Viewer closing * Various serialization issues with effects (broken physics, etc.) * Crash on some devices with error: Resource deadlock would occur * Background video unexpected line * Deserialization of empty textures * Crash or image freeze when a face goes out of the screen border * OEP: * Memory leak in FPS Draw * Memory leak when loading effect synchronously * Released external surface before creating new (leak fix) * * WebAR: * Upside down screenshot on Safari 14 * Added better error messages for Outputs * Runtime error when using iframe.srcdoc * Inability to reuse MediaStream * Player.destroy() memory leak * Makeup API: * Makeup.lashes affects Eyelashes.color * Unity: * Various effects issues * Various startup errors * Demo scene UI fixes * Broken face in landscape * Beautification scene * Issue with multiple faces ## \[1.4.4] - 2022-07-26[​](#144---2022-07-26 "Direct link to \[1.4.4] - 2022-07-26") **Added** * Option to disable future frame filtration in the recognizer **Changed** * Disable the future frame feature for OEP ## \[1.4.3] - 2022-07-11[​](#143---2022-07-11 "Direct link to \[1.4.3] - 2022-07-11") **Added** * `Background.getBackgroundVideo()` method added to the virtual background feature * Android OEP Demo: Add virtual background samples **Changed** * Add play and loop parameters to the background texture **Fixed** * OEP: Frame is flashing when switching from blur to video * The app does not start on lower-end devices ## \[1.4.2] - 2022-07-04[​](#142---2022-07-04 "Direct link to \[1.4.2] - 2022-07-04") **Added** * The path to the application cache (internal storage) is available via `resource_manager` * Enable neural network model cache on Android * Enable the software MSAA on the Adreno 6xx series * iOS: support of the OpenGL backend for the OEP Demo app * Android, iOS OEP Demo functionality improvements **Changed** * Upgrade Tflite to version 2.8 (iOS, Android) * Virtual background: scaling and rotation are disabled when background is not set * Virtual background: rotation changed from ccw to cw * Virtual background: [video formats](/far-sdk/tutorials/capabilities/technical_specification.md#video-formats-support) support updates * Android: use xnnpack runner while gpu runner is is loading **Fixed** * Unexpected line on background video when using OEP * Background video frame is flashing when switching the background * SDK Version 1.3.X Crash on iOS system 12.X * Build made with modules crashed when the licenseΒ  expired (missing watermark file) * OEP: iOS 1.3.1 blur and transparency don't work * OEP: BG video orientation (recorded front, back cameras) is flipped, stretched, or squeezed * OEP: Background flipped upside down If portrait orientation lock is off * MacOS: hair and BG recognition * Xiaomi Mi8 Pro crash * Exception: 'transformation matrix is singular' * Windows: camera crashes when creating a new camera * Windows: deadlock ## \[1.4.1] - 2022-05-16[​](#141---2022-05-16 "Direct link to \[1.4.1] - 2022-05-16") **Fixed** * The makeup API didn't work on iOS versions below 14 * WebAR failed to start on Chrome 100 and up (Windows platform only) * Video processing was taking longer * Unity: Incorrect face texture display in some cases ## \[1.4.0] - 2022-04-20[​](#140---2022-04-20 "Direct link to \[1.4.0] - 2022-04-20") **Added** * Face AR SDK for [iOS](/far-sdk/tutorials/development/installation.md) and [Android](/far-sdk/tutorials/development/installation.md) distribution as pods or maven packages * iOS, Android: All Face AR SDK [examples on Github](/far-sdk/tutorials/development/samples.md) are switched to pods, and Maven packages * Component-based face tracking for all supported platforms (only in online mode) * Unity: brand-new demo scene with effects carousel * Android: restored external texture support to the `effect_player` * Hand AR: Textured nails * M1 simulator support has been restored * Makeup API: Eyebrows makeup * WebAR: ability to load Effect via Request * WebAR: ability to play gif as background texture * WebAR: Effect.preload progress listener * WebAR: Check out our new [tutorials section](/far-sdk/tutorials/development/basic_integration.md) with extensive WebAR insides and best-practices * Rendering Engine: GLTF support **Changed** * iOS: Upgraded lips segmentation neural network * B\&W lip colouring support without additional parameters * Makeup API: reduced number of uniforms * WebAR: Improved WebAR archive UX * Rendering Engine: Reduce LUT memory size * OEP: extend supported image formats **Fixed** * Black screen flash on effects loading * Android: memory leak in the OEP * Android: Pixel 3a crashes after applying certain face filters * Android: deadlock during video\_frame draw * Android: Xiaomi Mi 8 Pro crashes * iOS: iPhone 13 is recognised as middle-end hardware * Fixed invisible entities for some face filters * Effects API: `Api.drawingAreaWidth()` & `Api.drawingAreaHeight()` return 0 * Windows: Impossible to apply effect with path containing extended Unicode * Windows, MacOS: Banuba Viewer saves video without sound * MSAA issues in face filters * WebAR: Fixed processing of RGB images * WebAR: Fixed MediaStreamCapture crash on Safari 14 * Makeup API: Fixed serialization of empty textures * Hand AR: hand skeleton false detections * OEP: crash if input buffer format changes * OEP: green frame if texture is not ready * OEP: fixed rapid camera switch ## \[1.3.1] - 2022-03-02[​](#131---2022-03-02 "Direct link to \[1.3.1] - 2022-03-02") **Changed** * An OCV-based camera on Windows is able to select a camera by index **Fixed** * OEP crash or black background on first choice of background blur * OEP Metal crash * `noFaceActions` & `faceActions` functions are incorrect call in config.js * Viewer v1.3.0 crashes on Macbook * WebAR: MediaStreamCapture produces black frames * WebAR: Makeup effect crashes web page on iPhone 8, iOS 15 * WebAR: Fixed result of `ImagCapture.takePhoto` * WebAR: Effect crashes on iOS Safari * WebAR: Effect animations lag on iOS Safari 14 * Android alooper crash * Photo loading black screen: buffer is not large enough for dimensions * Incorrect face texture display when using lip correctors * Hand skeleton can be recognized on different objects ## \[0.38.6] - 2022-02-16[​](#0386---2022-02-16 "Direct link to \[0.38.6] - 2022-02-16") **Added** * Offscreen app performance improvement **Changed** * Drop any frame\_data fields on effect loading **Fixed** * Fix SDK work on Android 6 * Recording audio from effect\_player to file * Crash when making a photo or video from the Demo App * Avoid the copy frame in the case of the Banuba SDK Demo * OEP Android FATAL EXCEPTION: CameraThread * Distorted image after turning on blur for row stride not equal width ## \[1.3.0] - 2022-01-27[​](#130---2022-01-27 "Direct link to \[1.3.0] - 2022-01-27") Version 1.3.0 also includes all changes from SDK v0.x releases up to v0.38.5 version. Please, refer to the changelog below. **Added** * Face tracking antijitter based on optical flow algorithm * Ability to draw face mesh landmarks (and test effect) * Face mesh lip correctors (optional) * Face mesh eyebrow correctors (optional) * Native Metal API support (iOS, MacOS) * Metal multiinstance * Arm64 support with a Metal backend for simulators and MacOS * Makeup API: Lips morphing * WebAR: Tflite libs for emscripten (3.0.1) * WebAR: Throw an error when rendering an unexpected DOM element * unloadEffect method * OEP: Metal YUV converter * [SDK Manager](/far-sdk/generated/javadoc/banuba_sdk/com/banuba/sdk/manager/package-summary.html) added to the API documentation * AR Rings technology **Changed** * WebAR: TFLite delegate creation failed error now shows as warning * WebAR: Optimized CPU and GPU usage * WebAR: Reduced RAM usage and camera pixels retrieving time * WebAR: Added warning if effect.evalJs is called before the effect application * Rebuild OpenCV for Android only with the required set of features * Switch Hand Gestures and recognition to Tflite for iOS and Mac * The Android Beauty example switched to the Makeup API usage * Face mesh correctors are controlled directly from needed effects (including Makeup API) * An Android Demo app can be built now with Java 11+ and Gradle 7+ versions * Correct work of alpha channel blending (used in background transparency) * OEP: Improved performance of the YUV converter for Android * Makeup API: Improved the effect activation time * Removed copying of U and V textures for i420 **Fixed** * Makeup API: Correctly resolve loading resources from different modules * Makeup API: Hair blending incorrect brightness * Makeup API: Incorrect blur behaviour when comparing to 0.x version * WebAR: Electron: License error 0xff00f * WebAR: User-friendly error messages for misconfigured `locateFile` * WebAR: Delay in loading animation on Safari * WebAR: Makeup effect crashes iOS Safari * WebAR: Fixed GPU memory leak * Various crashes in the Offscreen Effect Player (OEP) * SDK v1.1.0 crashed on the unload effect in the Android quickstart app * Incorrect work of Api.playVideoRange * Windows: Effects cannot be enabled when extended Unicode is used in the app's location name * Windows: OEP-desktop-c-api example build has failed to launch * Android Demo app has crashed on switching effects and closing activity * iOS: `sdkManager.output?.takeSnapshot` method not working in SDK 1.x * iOS: Distorted videos/snapshots when using non-standard RenderSize ## \[0.38.5] - 2021-12-14[​](#0385---2021-12-14 "Direct link to \[0.38.5] - 2021-12-14") **Added** * EffectPlayer sound playback recording **Changed** * Android: Updated lips segmentation neural network **Fixed** * Incorrect shiny lip application * OEP: fix YUV aliasing ## \[1.2.1] - 2021-12-01[​](#121---2021-12-01 "Direct link to \[1.2.1] - 2021-12-01") **Fixed** * iOS: x86\_64 simulator support * Makeup API: Incorrect display of blurred background * Makeup API: Incorrect virtual background ratio in landscape mode * Virtual background: incorrect alpha blending when using transparent images ## \[1.2.0] - 2021-11-24[​](#120---2021-11-24 "Direct link to \[1.2.0] - 2021-11-24") Version 1.2.0 also includes all changes from SDK v0.x releases up to v0.38.4 version. Please, refer to the changelog below. **Added** * Hand gestures and Hand skeleton model for all platforms * Light Streaks effect support in Scene (1.x versions) * Add a warning if the versions of neural nework and FaceAR SDK are different * Face skin segmentation neural network for Tflite (all platforms except iOS and MacOS) * Makeup API: Eyelashes 3D support * Android: Automatically select RGB or YUV camera mode (better performance on some low and mid-end devices) * WebAR: Ability to enable heavy hair neural networks * Error message when trying to load an old effect (from 0.x versions) * Ruler (distance to face) effect * Introduce `evalJs` for calling effect methods from the application * iOS: Support for hair segmentation in landscape mode * Ability to [combine Face AR effects](/far-sdk/effects/virtual_background.md) with a virtual background in runtime * Eval js support for OEP **Changed** * New Tflite Lips neural network (all platforms except iOS and MacOS) * WebAR: Optimized Image processing * Remove unnecessary libraries and dependencies * iOS, MacOS: Switch face tracking neural networks to Tflite * Android: Updated hair segmentation neural network * The new body segmentation (v2) neural network is enabled by default **Fixed** * WebAR: Fixed Emscripten auto GC * WebAR: Added ability to release WASM memory * WebAR: Fixed Photo editing * WebAR: Hand segmentation and gesture support * WebAR: Fixed Next.js compatibility * Remove unneeded iteration for hair recolor * The second call of BNBUtilityManager.initialize or BanubaSdkManager.inialize causes a crash in the release * Eye segmentation nn performance incorrect display * Makeup API: Fixed Hair coloring algorithm * Various OEP fixes * Imgui display on Windows * Effects fix for Safari 15 ## \[1.1.1] - 2021-10-19[​](#111---2021-10-19 "Direct link to \[1.1.1] - 2021-10-19") **Added** * Makeup API: beauty morphings **Changed** * Makeup API: Blur algorithm **Fixed** * callJsMethod fails when pass parameters ## \[0.38.4] - 2021-11-02[​](#0384---2021-11-02 "Direct link to \[0.38.4] - 2021-11-02") **Fixed** * `full_image_from_yuv_i420_img` is too slow ## \[0.38.3] - 2021-10-07[​](#0383---2021-10-07 "Direct link to \[0.38.3] - 2021-10-07") **Added** * `face_search_mode` in the EffectPlayer API * Windows: Sign and add description to EffectPlayer dlls * i420 yuv pixel format support **Changed** * Improved face tracking performance * New tflite hair segmentation neural network for Android and Windows **Fixed** * Morphing behaviour at the screen edges ## \[0.38.2] - 2021-09-16[​](#0382---2021-09-16 "Direct link to \[0.38.2] - 2021-09-16") **Added** * iOS 15 support ## \[1.1.0] - 2021-09-30[​](#110---2021-09-30 "Direct link to \[1.1.0] - 2021-09-30") Version 1.1.0 also includes all changes from SDK v0.x releases up to v0.38 version. **Added** * Makeup API support * Action Units: Multi-face support * Full Body Segmentation v2 neural network (for all platforms) * Windows: Added description to dlls * Windows: Sign Banuba SDK dlls * Face triggers support (mouth open, smile, etc.) * iOS: Effect info UI in SDK Demo App * iOS 15 support * Api.isMirroring() * WebAR: Distance to phone support **Changed** * Android: Updated lips segmentation neural network * Skin segmentation neural network support on all platforms (including the Web) **Fixed** * nn\_api Lips aliasing * WebAR: iOS 13 does not show a video stream ## \[0.38.1] - 2021-08-17[​](#0381---2021-08-17 "Direct link to \[0.38.1] - 2021-08-17") **Added** * C API documentation + deadlock fixes (Windows) ## \[0.38.0] - 2021-08-16[​](#0380---2021-08-16 "Direct link to \[0.38.0] - 2021-08-16") **Added** * Windows: MSMF camera usage * iOS, Android: Hand gesture tracking **Changed** * Android: Use tflite GPU info library to range device classes * iOS: Use the same background segmentation neural networks for all devices * iOS: Remove unneeded UI from the demo app * Renamed license utils symbols **Fixed** * tflite\_runner assorted fixes * iOS: White eyelashes when making photos * iOS: Time range issues in video player * MacOS: Crash on M1 * CubemapEverest test effect autorotation * WebAR: broken SIMD support * WebAR: Fixed creation of MediaStreamCapture * Android: Multitouch crash in demo app * Unity: minimum version of Android SDK * Unity: Android build on Windows * Win32: Background NN work * Incorrect lips shine work ## \[0.37.1] - 2021-07-27[​](#0371---2021-07-27 "Direct link to \[0.37.1] - 2021-07-27") **Added** * Android x86\_64 preliminary support **Changed** * Android: Common gradle for SDK Demo projects * Updated background segmentation neural network models for Web and Desktop platforms **Fixed** * iOS: missing image from camera when using ARKit on iPhone 12 * Android: Crash on C API * Android: Missing photos in the gallery when using the Demo app on some Android devices ## \[0.37.0] - 2021-07-09[​](#0370---2021-07-09 "Direct link to \[0.37.0] - 2021-07-09") **Added** * New Eyes segmentation neural network with separate detection of eye parts: pupil, sclera, and iris (all platforms) * Token updates for Eye bags and Acne features (**new token required**) * iOS: Lips corrector * MacOS: reworked implementation of the MacOS framework * WebAR: API to set the number of faces to track * Unity: Action Units interface * Ability to show camera frames during effect initialization * M1 support (including simulators) * Accepting YUV i420 * Lip morphing effect * Ability to set Neck smoothing from JS (in effect) * Effect Player C API **Changed** * Update Win tflite x64 and x86 from 2.3 to 2.4.1 * Eyes corrector is included in release archives by default * Eyes corrector enabled for Win and Web * Preload all Android NN classes instead of creating them on request * Improved performance of the Lips Shine effect * Makeup API updates: * The SetInitialRotation method is added to the bg-image and bg-video classes * Transparent BG fix * Consistent methods of naming * Fixed usage of the skin segmentation with background features * Android: Remove unneeded rotateBg calls and all related code * iOS: Include bitcode into minimal builds * iOS: Updated background segmentation NNs for high-end and low-end devices * Updated Face tracking neural network (all platforms) **Fixed** * Makeup Transfer exception * WebAR: Fixed inactive tab video throttling * Standalone: remove legacy resource copy * Effects of video texture crashes on some devices with MediaTek and PowerVR * OEP long loading during app initialization ## \[1.0.0] - 2021-06-22[​](#100---2021-06-22 "Direct link to \[1.0.0] - 2021-06-22") **Added** * WebAR: Video texture support * WebAR: API to set the number of faces to track * WebAR: Lips effects support on iOS (Safari) * Xcode 12.5 supports **Changed** * [Examples apps](/far-sdk/tutorials/development/samples.md) optimised for SDK v1.x **Fixed** * Android: Screen is flashing when switching effects * Android: GPU-specific deadlock issues * iOS: Demo app crash on iOS < 13.5 * WebAR: Fixed texture alpha-blending * Windows: Effect with the video has failed to load * Windows: standalone build failure on the x86 platform * Do not crash the app when assert has failed * JS engine fixes * Lips shine effect * Various effects fixes ## \[0.36.1] - 2021-05-24[​](#0361---2021-05-24 "Direct link to \[0.36.1] - 2021-05-24") **Changed** * WebAR: FrameData is available in the WebAR SDK * Distance to phone improvements **Fixed** * Unity: Triggers incorrect work * Unity: iOS camera initialization ## \[0.36.0] - 2021-05-03[​](#0360---2021-05-03 "Direct link to \[0.36.0] - 2021-05-03") **Added** * WebAR: Human-readable exception messages * WebAR: Updated background segmentation * WebAR: Optional SIMD * WebAR: Made WebAR SDK SSR compatible * WebAR: Crop, resize, horizontalFlip support * Desktop: Updated background segmentation * Android: Offscreen Effect Player (OEP) example * iOS: Offscreen Effect Player (OEP) example * Unity: Face morphing support * GIF textures support * Lip segmentation support for Web and Desktop * Makeup API: Exposed extra APIs * Hand AR API: Nails segmentation * Windows: SDK dlls come signed **Changed** * Invalidate texture cache in case of file change * WebAR: Speed up frames obtaining * WebAR: Improved memory usage * WebAR: Throw if the effect has zero length * Mac: build SDK as a macOS framework * iOS: Remove ARKit dependency if ARKit face search is disabled **Fixed** * Error logs when loading empty effect * Android: Region cropping when applying zoom * WebAR: Prevent playback stops during unsuccessful effect application * WebAR: Inactive tab throttling * Makeup API: "black square" on lips with alpha channel * Unity: UI scaling for the Beautification scene * Crash on effect switching * Standalone demo app signing ## \[1.0.0-beta] - 2021-04-23[​](#100-beta---2021-04-23 "Direct link to \[1.0.0-beta] - 2021-04-23") **Added** * New render engine aka *Scene* * WebGL 1.0 support (for Safari) * Metal support ## \[0.35.0] - 2021-02-26[​](#0350---2021-02-26 "Direct link to \[0.35.0] - 2021-02-26") **Added** * Support non-ASCII symbols in paths * New Background segmentation neural networks (Desktop) * New Background segmentation neural networks (Web) * Hair segmentation support on the Windows platform * WebAR: `Effect.preload` and `Player.applyEffect` will now throw an exception if the effect's underlying source is not a .zip archive * Initial support for the Apple M1 **Changed** * ARKit disabled by default (iOS) * Strip unnecessary symbols on macOS * Use only the minimum required subset of OpenCV on macOS * Use TFLite 2.4.1 without Metal delegate on macOS * API to set animated background in effects dynamically **Fixed** * WebAR: Firefox video processing issue * Android: Orientation fixes * Android: Sound issues and minor improvements * Unity: iOS plugin size ## \[0.34.1] - 2021-01-26[​](#0341---2021-01-26 "Direct link to \[0.34.1] - 2021-01-26") **Added** * Face Ruler for Android platform * Unity: New Action Units effect with background segmentation **Changed** * Distribute EffectPlayer for iOS as xcframework **Fixed** * Build with disabled face tracking ## \[0.34.0] - 2021-01-19[​](#0340---2021-01-19 "Direct link to \[0.34.0] - 2021-01-19") **Added** * WebAR: Background support in landscape * The BG support field in effect\_info * Android: Java 8+ API desugaring support * Viewer extra options for processing **Changed** * tflite\_runner: different delegates support each feature * Android: Add static TensorFlow lite version * Enable RGB cameras on devices with Snapdragon 625 * Processed image location in Banuba Viewer * Skin smoothing NN update (iOS) **Fixed** * WebAR: ES6 to ES5 transpilation issue * WebAR: loading of non existing effects * WebAR: several performance issues * Hair segmentation: TFLite input copy error * Prior fixes to work with frx\_meta logic * Incorrect effects display on Android 10 * Android: Effect size after rotation ## \[0.33.1] - 2020-12-10[​](#0331---2020-12-10 "Direct link to \[0.33.1] - 2020-12-10") **Added** * Add listener as soon as test\_Ruler or FaceRuler effect is activated **Changed** * Enable a face recognition neural network for mid-end Android devices **Fixed** * `setEffectSize` fix for Android * Lips shine AR 3D Mask incorrect work with back camera ## \[0.33.0] - 2020-11-30[​](#0330---2020-11-30 "Direct link to \[0.33.0] - 2020-11-30") **Added** * Display FPS stats in the Desktop Viewer App * Beauty scene for Unity plugin * Android: Ability to Override Detected Resolution * Lips recoloring with a glitter effect (also supported in Banuba Viewer) * Distance to face (ruler feature) **Changed** * Updated tflite for Windows to 2.3 * Both tflite runners were created on first request * Text texture is enabled by default **Fixed** * iOS: incorrect BG work on photo in landscape * Unity: fix aspect on mobile devices * Delayed camera start * Repacking errors * Front camera flip * 'Face not found' message after loading photo from gallery * Added handling of IllegalArgumentException to prevent crashes dependent on surface configuration ## \[0.32.1] - 2020-11-05[​](#0321---2020-11-05 "Direct link to \[0.32.1] - 2020-11-05") **Added** * Effect activation listener * Unity: Separate render target for the beauty scene for the LUTs **Changed** * Decreases CPU load on MacOS **Fixed** * Crash during effect preload * Mesh trembling with fast face tracking * Unity: Aspect of Background segmentation * Crash during fast effect switching ## \[0.32.0] - 2020-10-20[​](#0320---2020-10-20 "Direct link to \[0.32.0] - 2020-10-20") **Added** * New background model * New WebAR API * WebAR Quickstart Demo App * WebAR beauty demo app * Native OSX camera implementation * Web and Desktop getting started added * Possibility to customise the capture session preset (iOS) * Desktop app examples for Windows and Mac * Demo effects without face recognition * Xcode 12 supports **Changed** * Eye corrector v2.0: improved stability and performance * Makeup API improvements * Use setBackgroundTexture with an absolute path * Face tracking stability and performance optimization * Update offline face tracking (Android) **Fixed** * Crash with bitcode in the JavaScript core * Separate AR 3D Mask for neck smoothing feature * Crash with effect reset (Android) * Missing logs in Viewer Standalone * Android video player loop * Memory leak on desktop when using animated textures * Quickstart example app fixes * Unity background fix * ARKit face detection failure ## \[0.31.0] - 2020-08-27[​](#0310---2020-08-27 "Direct link to \[0.31.0] - 2020-08-27") **Added** * SDK features control and repacking with client token and client configuration * Minimal SDK archive * SDK build for MacOS * Makeup transfer feature * Photo online processing in Banuba Viewer * Ability to enable effects and neural networks without a face recognizer * Eye brow segmentation NN * Neck smoothing neural network * Set camera FPS mode on Android (fixed/adaptive) * Represent SDK frames as OpenGL textures (WebRTC for Android) * New beautification API **Changed** * Updated Background Segmentation neural network for standalone builds * Face recognizer works on full frame * Landmarks smooth filter * Updated eye corrector * Updated face recognition neural network * Unity scene works in full screen mode * OpenCV updated to 4.3.0 * Range of android devices hardware class and max resolution for it **Fixed** * Unity plane does not update rect * Video player fix * Heart rate measurement with neural network face search * Segmentation neural networks work with arkit * Unity WebGL build * Fix Unity failure on the Windows platform * Crashes on effect unload * Correctly handle MRT rendering into background camera texture * Black screen on devices with ARKit * Background segmentation on Windows x86 * WebAR SDK blocks the backspace key * Banuba SDK works on devices with iOS 14 ## \[0.30.2] - 2020-07-15[​](#0302---2020-07-15 "Direct link to \[0.30.2] - 2020-07-15") **Fixed** * Second AR 3D Mask freezes on the screen in scene effects ## \[0.30.1] - 2020-07-14[​](#0301---2020-07-14 "Direct link to \[0.30.1] - 2020-07-14") **Added** * Eyes correction feature **Changed** * Hide Boost symbols **Fixed** * Asynchrony of sound and video after file import * iOS: The app freezes after background in Editing mode * Android Beauty: Screen is flashing after launch * Crash on Editing Image * App crashed in editing mod on iPhone XS Max * EffectPlayer is not launched for the first time * Second face is missed if to use the front Camera (with ARKit) ## \[0.30.0] - 2020-06-11[​](#0300---2020-06-11 "Direct link to \[0.30.0] - 2020-06-11") **Added** * WebAR support for Unity platform * Background segmentation for Unity platform * Max Faces support in client token * Videocall example for iOS * *Minimal* configuration of Banuba SDK * EffectPlayer EffectManager **Changed** * The EP version has changed to 5.6 * Enable bitcode by default (iOS) **Fixed** * Compilation error on Ubuntu * Body segmentation neural network rotation * Viewer Standalone build * MSVC x64 Eigen crash * The app won't throw an exception when neural network resources are missing * Optimized face beautification * Fix audio session (iOS) * Bakground segmentation failures (iOS) * Creepy smile fixes * Portrait match fixes * Skin smoothing fixes ## \[0.29.1] - 2020-05-19[​](#0291---2020-05-19 "Direct link to \[0.29.1] - 2020-05-19") **Fixed** * Crash on Android with neural face recognition ## \[0.29.0] - 2020-04-30[​](#0290---2020-04-30 "Direct link to \[0.29.0] - 2020-04-30") **Added** * Creepy smile neural network (iOS) * Manual audio session in BNBEffectPlayer * Skin smoothing neural network (iOS) * New face recognition and tracking algorithm for offline (Android) **Changed** * Banuba Viewer colour picker reacts to background and lip neural networks * Make WebAR SDK ES6 module * Updated llvm backend for WebAR * WebAR improvements **Fixed** * Lip segmentation on the Android HQ photo * Memset buffer overflow when using Action Units * Jaw mesh stretching fix in the face tracking algorithm ## \[0.28.3] - 2020-04-29[​](#0283---2020-04-29 "Direct link to \[0.28.3] - 2020-04-29") **Added** * Enabled bitcode in iOS release **Fixed** * Portrait match technology ## \[0.28.2] - 2020-04-22[​](#0282---2020-04-22 "Direct link to \[0.28.2] - 2020-04-22") **Added** * Portrait match technology ## \[0.28.1] - 2020-04-09[​](#0281---2020-04-09 "Direct link to \[0.28.1] - 2020-04-09") **Added** * Update Effect Player for video calls, support callkit audio session specifics **Changed** * Switch to a fast face recognition algorithm for weak iOS devices **Fixed** * Celebrity match technology fixes * Crash on Banuba Viewer close ## \[0.28.0] - 2020-03-23[​](#0280---2020-03-23 "Direct link to \[0.28.0] - 2020-03-23") **Added** * New face recognition and tracking algorithm for realtime (iOS) **Changed** * Full Body segmentation can be applied again (iOS) * Banuba Viewer UI changes * Adapt Action Units to use the new face recognition algorithm (iOS) * Adapt triggers to use Action Units (iOS) **Fixed** * Physics behaviour for effects on devices with ARKit (iOS) * Effects render on low-level Android devices ## \[0.27.2] - 2020-03-12[​](#0272---2020-03-12 "Direct link to \[0.27.2] - 2020-03-12") **Fixed** * Unity openCV error * Small recognizer fixes for the iOS platform ## \[0.27.1] - 2020-02-27[​](#0271---2020-02-27 "Direct link to \[0.27.1] - 2020-02-27") **Added** * ARKit multiface support **Changed** * Android strong device list updated **Fixed** * Multiface effects render * Multiface issues * Crash when processing a photo with two faces * Effects render with ARKit on iPhone X ## \[0.27.0] - 2020-02-19[​](#0270---2020-02-19 "Direct link to \[0.27.0] - 2020-02-19") **Added** * Greatly improved face detection in offline mode (iOS, for photos) * Objective-C full support * More examples for iOS and Android * Improved beauty effect * Lip shine effect improved * Ability to choose a camera from the command line on Desktops **Changed** * Persistent OpenGL context on Android (don't recreate it after the app goes in background) * Safely ignore GL errors on Android * The beauty effect is enabled by default **Fixed** * Lips shine effect * Neural network behaviour after the face was lost ## \[0.26.0] - 2020-01-17[​](#0260---2020-01-17 "Direct link to \[0.26.0] - 2020-01-17") **Added** * Advanced lip recoloring * Action Units from ARKit * x86 support for Windows * Lazy textures load **Changed** * Improve Unity sample effects * The Bokeh effect improved * Enable beauty by default in sample effects * Improve acne and bag removal performance * Hair stand blending performance improvement **Fixed** * Threads leak on Android * WebGL FPS stabilization * Memory issue on Android * Memory issue on iPhone6+ * Crash during rendering on Adreno 610 ## \[0.25.2] - 2020-01-09[​](#0252---2020-01-09 "Direct link to \[0.25.2] - 2020-01-09") **Fixed** * Camera open error on Android ## \[0.25.1] - 2019-12-31[​](#0251---2019-12-31 "Direct link to \[0.25.1] - 2019-12-31") **Fixed** * Bundle version in the xCode project ## \[0.25.0] - 2019-12-23[​](#0250---2019-12-23 "Direct link to \[0.25.0] - 2019-12-23") **Added** * Eye bug removal * Neural network based acne removal * Use `ARKit` for face tracking when available * `dvcam` post-process effect * Eyes state trigger and ruler features in recognizer API * API to change sound volume from Java Script * Option to add effects from an external folder in the sample application (Android) * Improve API (SDK for browsers) **Changed** * Don't reload effect if there was an error in JavaScript. **Fixed** * Decrease memory pressure while creating multiple `BanubaSdkManager` instances (Android) * Crash on effects with 3 or more faces * Improved camera FPS on selected low-end Android devices ## \[0.24.1] - 2019-11-06[​](#0241---2019-11-06 "Direct link to \[0.24.1] - 2019-11-06") **Changed** * Update documentation with examples of new UI **Fixed** * Video recording on Android * Crash after exiting from the application * Memory leak on Android * Crashes when interacting with the Android Demo app ## \[0.24.0] - 2019-11-01[​](#0240---2019-11-01 "Direct link to \[0.24.0] - 2019-11-01") **Added** * Glasses detection * Improved stability (aka jittersing) of face tracking * Extended `Recognizer` API * Recognition results in Python bindings **Changed** * Migration to AndroidX * New redesigned UI for Banuba SDK Demo AP (Android and iOS) **Fixed** * Video texture decoding on Android 10 * Crash while going to background on iOS * Audio recording speed on Android * Lag during neural network initialization * Various camera fixes for Android ## \[0.23.0] - 2019-10-02[​](#0230---2019-10-02 "Direct link to \[0.23.0] - 2019-10-02") **Added** * A neural network based approach to detecting faces. Quality, detection angles, and speed of the face detection was improved * Neural networks support for Windows and Web * Unity plugin **Changed** * Sync audio and video during recording on Android * Fast background on iPhone 6 and lower. * Correct neural network behaviour during device rotations **Fixed** * Video texture support (Android 10) * Crashes on Adreno chipsets * Stability fixes ## \[0.22.0] - 2019-08-28[​](#0220---2019-08-28 "Direct link to \[0.22.0] - 2019-08-28") **Added** * Lip colouring API in `Beauty` effect * Option to switch off face recognition in `config.json` * Option to set preferred frame-rate on iOS **Changed** * Lips segmentation neural network updated (Android) **Fixed** * Rendering on Andreno GPUs * Video texture decoding issue on Android * Android crash on app coming from background * Crash on iPhone 5 after video capture ## \[0.21.0] - 2019-08-05[​](#0210---2019-08-05 "Direct link to \[0.21.0] - 2019-08-05") **Added** * Hair and lips recoloring in the "Beauty" effect * `EffectPlayer` threading model documentation * SDK feature documentation * API to check if device is compatible with Neural Networks player **Changed** * `BanubaSdkManager` can be instantiated more than once (see "Migration Guides"). * Use the background AR 3D Mask transform for the background separation layer from `config.json` **Fixed** * Sample app signing (iOS) * Rendering bugs after the effect switch * Correct screenshot size on Android * "End touch" event (iOS) ## \[0.20.2] - 2019-07-24[​](#0202---2019-07-24 "Direct link to \[0.20.2] - 2019-07-24") **Fixed** * background separation layer from config.json (Android) ## \[0.20.1] - 2019-07-17[​](#0201---2019-07-17 "Direct link to \[0.20.1] - 2019-07-17") **Fixed** * Fix photo processing with MSAA enabled on Android ## \[0.20.0] - 2019-07-12[​](#0200---2019-07-12 "Direct link to \[0.20.0] - 2019-07-12") **Added** * Sample ASMR effects * Render passes * Rendered frame forwarding as byte arrays (Android) * Ability to debug JS * Reset effect cache API call **Changed** * Watermark gravity (Android) * Sample background separation effect blending improved * Background separation feature respects gyroscope data * Ignore the gyroscope during photo and video processing * Beauty effect improvements ## \[0.19.1] - 2019-06-24[​](#0191---2019-06-24 "Direct link to \[0.19.1] - 2019-06-24") **Added** * Colour post-processing effect * Face rect API from face recognition result **Changed** * Beauty effect improvements **Fixed** * Post-processing effect (when applied to framebuffer) * Image glitches and crashes in photo editing mode (Android) ## \[0.19.0] - 2019-06-17[​](#0190---2019-06-17 "Direct link to \[0.19.0] - 2019-06-17") **Added** * Bokeh effect example * Icons and cons for sample effects * New documentation, programming guides have been added * Rendering view transformation API * Post-process library * Beauty app example **Changed** * Removed `Beauty` effect API parameters: * eyes\_sharping\_str * blur\_bg\_enable * blur\_lod * remove\_bag\_intensity * eyes\_luts * Renamed `Beauty` effect API parameters: * makeup\_tex -> eyebrows\_tex * makeup\_alpha -> eyebrows\_alpha * eyebrows\_tex -> lashes\_tex * eyebrows\_alpha -> lashes\_alpha * Gravity in effect now respects device orientation * Remove life-cycle methods from `BanubaSDKManader` on Android * External texture is disabled by default (Android) * Action Units sample effect updated * Use only one camera session for all tasks (Android) * Improved photo processing speed * Sound Changer is supplied as a separate plugin * Swift 5 support **Fixed** * Gracefully handle exceptions on OS X * Missing and frozen video textures on iOS and Android * Open GL crashes on Android * Neural network overload on Android * Rendering bugs for the Web version * Stretched camera preview (iOS) ## \[0.18.1] - 2019-05-29[​](#0181---2019-05-29 "Direct link to \[0.18.1] - 2019-05-29") **Added** * SVG watermark support (Android) * Proguard rules for the banuba\_sdk module (Android) **Changed** * 'banuba\_sdk' is now supplied in compiled form * `BNBFullImageData` can be created from RGB `CVPixelBuffer` with padding * Suspend frame processing while taking low res photo (Android) * Beauty effect: return default parameters after animation * Restart camera preview session on HR photo (Android) **Fixed** * Video texture freeze (Android) * Crash during render size change ## \[0.18.0] - 2019-05-24[​](#0180---2019-05-24 "Direct link to \[0.18.0] - 2019-05-24") **Added** * Processing a bitmap and applying the selected effect to it (Android) * Image editing mode in platform modules (Android) * Acne removing technology in photo processing * Watermarks on video (Android) * Considering device orientation in photo taking * Ability to setup several listeners in EffectPlayer * Support for Bitmap in the FullImageData constructor (Android) * Universal framework for devices and simulators (iOS) **Changed** * Added assertions in the EffetPlayer life cycle (for video processing) * The hair segmentation neural network is updated **Fixed** * Losing face orientation after an Android activity restart is fixed * Physics on multiface effects is working correctly * The app has crashed on 32 bit Androids with enabled neural networks * ActionUnits improvements ## \[0.17.1] - 2019-04-24[​](#0171---2019-04-24 "Direct link to \[0.17.1] - 2019-04-24") **Fixed** * Exposure settings (iOS) * Continuous photo rendering with updated parameters ## \[0.17.0] - 2019-04-18[​](#0170---2019-04-18 "Direct link to \[0.17.0] - 2019-04-18") **Added** * Continuous photo rendering with updated parameters * Conversion-free RGB input support * Image file processing example (iOS) * Support for landscape frame input * API to check Android hardware performance **Changed** * Documentation improved * Swift 4.2 support in the example app * Updated lip segmentation neural network * Improved rendering quality on high-end Android devices **Fixed** * Removed duplicate functionality in Android samples * Correct video orientation (iOS) ## \[0.16.0] - 2019-04-03[​](#0160---2019-04-03 "Direct link to \[0.16.0] - 2019-04-03") **Added** * Neural networks for Android: lips, skin, hair, eyes, iris segmentation; background separation * Neural network rendering for Android * Full body segmentation neural network for iOS * Release binaries for Windows * New post process effects: acid whip, cathode and rave * x86\_64 build variant for iOS simulators * Face detection in any orientation (Android) **Changed** * Camera FPS increased on Huawei devices * Video now paused when app is in background * Process screenshot or HQ camera option for Android * Performance on low-end Android devices **Fixed** * Video textures playback * Audio resume after background (Android) * Launch time on first run (Android) ## \[0.15.0] - 2019-03-14[​](#0150---2019-03-14 "Direct link to \[0.15.0] - 2019-03-14") **Added** * Action Units and Blend Shapes * Take high resolution photos with effects, camera switches, and video recording (Android) * Post processing stage with simple effects * A new neural network for eye segmentation (iOS) * Multi-touch **Changed** * Method to process photos from a file readded to the API * Hide eyes if there is no face in the test\_Eyes effect * Binary size reduced for iOS **Fixed** * Photo in Landscape mode on iOS * Animation position on photos * Acne removal performance on photos * Sounds after background (Android) ## \[0.14.3] - 2019-02-21[​](#0143---2019-02-21 "Direct link to \[0.14.3] - 2019-02-21") **Fixed** * Android crash with external texture * Rendering area size for iOS * Java documentation ## \[0.14.2] - 2019-02-09[​](#0142---2019-02-09 "Direct link to \[0.14.2] - 2019-02-09") **Fixed** * Version number in the iOS framework ## \[0.14.1] - 2019-02-01[​](#0141---2019-02-01 "Direct link to \[0.14.1] - 2019-02-01") **Added** * iPad support for the Demo app * Analytics serialization * New lifecycle: effect is paused before background * Videoprocessing (desktop only) **Changed** * Photo processing optimization * Android Demo Activity GC optimization ## \[0.14.0] - 2019-01-15[​](#0140---2019-01-15 "Direct link to \[0.14.0] - 2019-01-15") **Added** * Haptic feedback. **Changed** * Client APIs are automatically generated for both Java and Obj-C. * Documentation reflecting Java and Obj-C classes. **Fixed** * Draw state after VAO modification made by external code (Android). * Sound session restoration on iOS. * FPS degradation on video textures (Android). ## \[0.13.3] - 2019-01-15[​](#0133---2019-01-15 "Direct link to \[0.13.3] - 2019-01-15") **Fixed** * Audio session configuration (iOS) ## \[0.13.2] - 2019-01-14[​](#0132---2019-01-14 "Direct link to \[0.13.2] - 2019-01-14") **Changed** * Restore the old RFX classifier ## \[0.13.1] - 2019-01-11[​](#0131---2019-01-11 "Direct link to \[0.13.1] - 2019-01-11") **Added** * Watermarks on video (iOS only) **Changed** * Beauty soft light texture without eye shadows **Fixed** * Stretched picture during video preview on iOS * Fix JS calls with arguments * Crashfixes ## \[0.13.0] - 2018-12-27[​](#0130---2018-12-27 "Direct link to \[0.13.0] - 2018-12-27") **Added** * Ability to modify the user's voice (voice changer); iOS only * A smaller face recognition classifier * Eye segmentation textures * Neural network for face detection (optional, disabled by default) * Acne removal * Lip segmentation **Changed** * Improved hair segmentation on Android * BanubaSdkManager improvements on Android **Fixed** * FPS calculation, * Overdraw on Android * Black screen on Mali devices ## \[0.12.6] - 2018-12-20[​](#0126---2018-12-20 "Direct link to \[0.12.6] - 2018-12-20") **Fixed** * Reverted unnecessary cropping of video pixel buffer ## \[0.12.5] - 2018-12-19[​](#0125---2018-12-19 "Direct link to \[0.12.5] - 2018-12-19") **Fixed** * Fix video recording for a custom size of input frame ## \[0.12.4] - 2018-12-18[​](#0124---2018-12-18 "Direct link to \[0.12.4] - 2018-12-18") **Added** * Functions for retrieving effects and screen sizes from JS **Fixed** * Rendering artefacts near the eyelid in a beauty effect (z-fighting) * Camera initial mode fix, custom aspect ratio support * Adjust the configuring exposure settings method ## \[0.12.3] - 2018-12-14[​](#0123---2018-12-14 "Direct link to \[0.12.3] - 2018-12-14") **Fixed** * Fix beautification issues at high resolution * Fix coordinate conversion in touch events ## \[0.12.2] - 2018-12-12[​](#0122---2018-12-12 "Direct link to \[0.12.2] - 2018-12-12") **Fixed** * Video recording (copy + flip on BanubaSDK side), memory management improvements ## \[0.12.1] - 2018-12-11[​](#0121---2018-12-11 "Direct link to \[0.12.1] - 2018-12-11") **Changed** * Turn off the frame\_brightness feature by default. * Enable gyroscope on demand. * Process image improved **Fixed** * Exposure point settings (iOS) **Added** * Effect events for Android * Touch events for Android ## \[0.12.0] - 2018-12-04[​](#0120---2018-12-04 "Direct link to \[0.12.0] - 2018-12-04") **Changed** * EffectPlayer life cycle methods updated * Strict checks of the surface lifecycle * Face detection algorithm has been reverted to a more stable implementation * Resource finding path changed to subfolder (bnb) * Naming banuba.core -> banuba.sdk (iOS) * VideoRecording via TextureCache (iOS) **Added** * Search locations in ResourceManager error message * Ability to setup log level and subscribe to SDK's log callback from client code * Methods for getting CPU Info on Android * Experimental neural network support on Android (background separation, hair segmentation) (special build is required) * Bin record interface in BanubaCore * Improved error reporting while effect loading * Beautification effect added to resources (special build is required) * Process a single image method with custom input and output formats (ability to take high-quality photos) * Experimental skin segmentation NN added to iOS * Experimental eye segmentation NN added to iOS * Ability to flip a rendered image along the Y axis * Touch events on iOS **Fixed** * Fix pushFrameYVU420 method * Fix crash in effect\_context::update (race condition) * Return draw error when effect loading failed * The Bokeh effect works on Android * Fix slow wireframe in DebugRenderer * Fix iOS crashes in shader compilation * Fix unpack alignment for textures with `width * components` not multiples of 4 * Fix process image when external camera texture * Fix BG copy MRT on ANGLE WebGL (Web) * Fix depth test\&write state after morph with hair compacting * Fix the minimum and maximum possible coordinates (in face recognition) * Fix the exposure point ## \[0.11.2] - 2018-11-20[​](#0112---2018-11-20 "Direct link to \[0.11.2] - 2018-11-20") **Fixed** * Drawing artefacts with some effects ## \[0.11.1] - 2018-11-15[​](#0111---2018-11-15 "Direct link to \[0.11.1] - 2018-11-15") **Changed** * Updated face recognition algorithm. **Added** * Ability to link with a simulator on iOS. **Fixed** * Release number for frameworks fixed * Fix runtime crashes with aligned new on iOS 10 * Correctly stop the Effect Player in onDestroy and initialization in onCreate ## \[0.11.0] - 2018-11-08[​](#0110---2018-11-08 "Direct link to \[0.11.0] - 2018-11-08") **Added** * Sound volume control is in effect. * Callback to receive events from effects (mainly for analytics). * Support for the Bokeh effect. **Changed** * A new face model classifier. * Rendering performance optimization. **Fixed** * Various crashes ## \[0.10.2] - 2018-10-08[​](#0102---2018-10-08 "Direct link to \[0.10.2] - 2018-10-08") **Fixed** * Issues with effects display (black background instead of camera texture). * Dynamic shadow lags by one frame. ## \[0.10.1] - 2018-10-05[​](#0101---2018-10-05 "Direct link to \[0.10.1] - 2018-10-05") **Fixed** * Face recognition black AR 3D Mask effects have been fixed on some Android devices. ## \[0.10.0] - 2018-10-05[​](#0100---2018-10-05 "Direct link to \[0.10.0] - 2018-10-05") **Added** * Support of new pixel formats in effect\_player::process\_frame: RGBA, BGRA, ARGB, RGB, and BGR. Not supported in BanubaCore yet. * Binding EffectPlayer and Recognizer for Python. **Fixed** * Launch on iOS 10. * Issue with audio engine lifecycle. * Render: The issue with the effect's shadows has been fixed. * Render: The issue with the depth buffer on the Xiaomi Redmi 4a has been fixed. **Changed** * Render optimization: * Excess loading of 1x1 textures for background and hair AR 3D Masks was removed when these features were not used. * Colour correction for easysnap (lut-textures background loading - speeds up beauty effect loading and small speed up of lut layer rendering). * Broken effects fixes. ## \[0.9.1] - 2018-10-02[​](#091---2018-10-02 "Direct link to \[0.9.1] - 2018-10-02") **Fixed** * The dynamic shadows drawing issue has been fixed (for Banuba 3.0). **Changed** * EffectPlayer for Backend major version was raised to 5.0. * Render performance optimization. * The Beauty Effect for both platforms should be taken from 2b959fa12a966956c6f158ded762b634eac988de or later (update Android effect). ## \[0.9.0] - 2018-09-28[​](#090---2018-09-28 "Direct link to \[0.9.0] - 2018-09-28") **Fixed** * A few crashes in the face recognition engine were fixed. * The issue with the AR 3D MaskΒ  not respecting head volume after switching the effects has been fixed. **Changed** * Render performance optimization. ## \[0.8.6] - 2018-09-24[​](#086---2018-09-24 "Direct link to \[0.8.6] - 2018-09-24") **Added** * Android version assembled with NDK 18. **Changed** * Face recognition improved performance, improved anti-tremble, smoothing and so on. ## \[0.8.5] - 2018-09-21[​](#085---2018-09-21 "Direct link to \[0.8.5] - 2018-09-21") **Fixed** * Fixed initialization crash. ## \[0.8.4] - 2018-09-21[​](#084---2018-09-21 "Direct link to \[0.8.4] - 2018-09-21") **Added** * Debug render antialiasing. **Changed** * Beautification effect performance has increased. * Face recognition performance has increased. ## \[0.8.3] - 2018-09-21[​](#083---2018-09-21 "Direct link to \[0.8.3] - 2018-09-21") **Changed** * The binary file size was reduced for iOS (17.7 MB against 19.7 MB). ## \[0.8.2] - 2018-09-19[​](#082---2018-09-19 "Direct link to \[0.8.2] - 2018-09-19") **Fixed** * Issues with single frame processing were fixed. **Changed** * Performance has improved. ## \[0.8.1] - 2018-09-14[​](#081---2018-09-14 "Direct link to \[0.8.1] - 2018-09-14") **Added** * Minor render optimizations (excess glGetInteger were removed). **Changed** * Low-light feature has been reverted because it has issues. ## \[0.8.0] - 2018-09-12[​](#080---2018-09-12 "Direct link to \[0.8.0] - 2018-09-12") **Added** * A new audio player. * Callback on low light detection. **Changed** * Improvements in face recognition library (performance in multiface mode has been increased, recognition angles have been increased, predictability of recognition work time has been improved - detection distribution by frames with self scheduler. * Improvements in render performance (number of passed parameters into shader interpolation were decreased - pixel shaders patch in glfx). * More accurate draw of the camera image (NEAREST filtration). ## \[0.7.2] - 2018-09-11[​](#072---2018-09-11 "Direct link to \[0.7.2] - 2018-09-11") **Fixed** * Issues with `effect_player_wrap-ios.framework` were fixed. ## \[0.7.1] - 2018-09-07[​](#071---2018-09-07 "Direct link to \[0.7.1] - 2018-09-07") **Fixed** * Fixed issue with crash in v0.7.0 release. ## \[0.7.0] - 2018-09-05[​](#070---2018-09-05 "Direct link to \[0.7.0] - 2018-09-05") **Added** * Photo mode frame processing (high resolution frame processing). * Consistency mode for camera external texture (Android). * Ability to get the version of EffectPlayer from the backend. * Ability to get the number of rendered frames and pass the number in push\_frame. * Face recognition finds faces at a large angle (up to 30 degrees). * Ability to set the texture parameters through suffixes in their names. * The framework version is transferred to Manifest after building, AAR assembling completely automated (Andorid). **Fixed** * Fixed issue with context loss on Android. * Effects with occlusion are fixed. * Small bug fixes. **Changed** * Strong reference on Delegate has been removed in iOS. ## \[0.6.2] - 2018-08-31[​](#062---2018-08-31 "Direct link to \[0.6.2] - 2018-08-31") **Fixed** * Fixed deadlock when drawing regular camera texture. ## \[0.6.1] - 2018-08-29[​](#061---2018-08-29 "Direct link to \[0.6.1] - 2018-08-29") **Added** * Consistent external texture for Android. * Zeroing face counter on onStop event. ## \[0.6.0] - 2018-08-21[​](#060---2018-08-21 "Direct link to \[0.6.0] - 2018-08-21") **Fixed** * Fixed and significantly improved inconsistency modes (which were given earlier in unversioned release). * Images strides from the camera were fixed for Android. * Fixed issue with long camera initialization. **Changed** * Default iOS mode was changed to inconsistency-without-face (was given earlier in unversioned release). * Updates in gender recognition (works fast and once in 3 seconds at the moment). ## \[0.5.2] - 2018-07-31[​](#052---2018-07-31 "Direct link to \[0.5.2] - 2018-07-31") **Added** * Possibility to enable inconsistency mode on iOS, it is possible to skip frame processing to render the image. * Possibility to receive device orientation in script (it is possible to disable background separation according to orientation). * Possibility to create a few recognizer instances (basically for DiffCat, not yet presented in EP API). * Possibility to transmit both camera matrix into the script (needed for morphing creation in accordance to distance from camera). * Recognizer coverage with performance tests has started. **Fixed** * Potential crash with keeping color\_plane was fixed. * Fixed wrong\_fb\_after\_morph. ## \[0.5.1] - 2018-07-26[​](#051---2018-07-26 "Direct link to \[0.5.1] - 2018-07-26") **Fixed** * Beauty settings doesn’t apply issue has been fixed. **Changed** * Unnecessary Android resources were removed. ## \[0.5.0] - 2018-07-24[​](#050---2018-07-24 "Direct link to \[0.5.0] - 2018-07-24") **Added** * Consistency/inconsistency modes switching (Android). * Blur background. * Performance collection using systrace (Android). * 32-bit support (but slow at the moment). * Possibility to transfer a frame number that has come from the camera. * Possibility to disable background separation and other recognizer features from scripts. * Switching between external textures and drawing from ImageReader (Android). **Fixed** * Huawei issues (Android). * Colours conversion bug fix (colour correction). **Changed** * Optimized morphing. --- # API Overview [View as Markdown](https://docs.banuba.com/far-sdk/tutorials/development/api_overview.md)[![](/far-sdk/img/ai-guide/chatgpt.svg)Open in ChatGPT](https://chatgpt.com/?q=Read%20https%3A%2F%2Fdocs.banuba.com%2Ffar-sdk%2Ftutorials%2Fdevelopment%2Fapi_overview.md%20\(API%20Overview\)%20and%20help%20me%20with%20it%20in%20my%20Banuba%20Face%20AR%20SDK%20project.)[![](/far-sdk/img/ai-guide/claude.svg)Open in Claude](https://claude.ai/new?q=Read%20https%3A%2F%2Fdocs.banuba.com%2Ffar-sdk%2Ftutorials%2Fdevelopment%2Fapi_overview.md%20\(API%20Overview\)%20and%20help%20me%20with%20it%20in%20my%20Banuba%20Face%20AR%20SDK%20project.)Install tools * iOS * Android * Web * Desktop ![image](/far-sdk/assets/images/ios_player_api_overview-329f7e9e59ebec11fef743ed4309d096.svg) **Banuba SDK** for **iOS** can be divided into three main entities: `Input`, `Player` and `Output`. The plethora of input options multiplied by the plethora of output options covers many use cases. ## Input[​](#input "Direct link to Input") Processes frames from one of the producers: * `Camera` - a real-time `CameraDevice` feed * `Photo` - an image from the gallery or a photo taken from `CameraDevice` * `Stream` - a frames sequence from the *WebRTC* stream or any other provider * Custom - your own implementation for the `Input` protocol ## Player[​](#player "Direct link to Player") Allows to `use` different data inputs like `Camera` feed or `Photo`, to apply an **effect** on top of it and to `use` several outputs like `View` and `Video` file simultaneously. The `Effect` component makes up an essential part of the SDK usage. The **effect** is represented as a folder with scripts and resources and can be loaded with the `load` method. Supports the following rendering modes: * `loop` *(default)* - render in the display-linked loop with the defined FPS * `manual` - render manually by calling the `render` method ## Output[​](#output "Direct link to Output") Presents a rendered frame onto one of the available surfaces: * `View` - on screen presentation * `Frame`, `PixelBuffer`, `PixelBufferYUV` - in memory presentation * `Video` - in video file presentation * Custom - your own implementation for the `Output` protocol ## CameraDevice[​](#cameradevice "Direct link to CameraDevice") Accesses the device's camera to generate a feed of frames in real time or takes high-quality photos.
By default, tracks UI orientation to properly manage frame rotation. ## RenderTarget[​](#rendertarget "Direct link to RenderTarget") Manages a `CALayer` object with a `Metal` context and provides offscreen rendering for the `Player` and presenting for the `Output`. ## Use cases[​](#use-cases "Direct link to Use cases") Common use cases and relevant samples are described in [this repository](https://github.com/Banuba/banuba-sdk-ios-samples).
tip See more use cases in [Samples](/far-sdk/tutorials/development/samples.md). ![image](/far-sdk/assets/images/android_player_api_overview-69f91128c9d7c1f1e3e7286f7f919f36.svg) **Player API** is a set of classes and methods that help facilitate and speed up the integration of the Banuba SDK into applications. The **Player API** concept distinguishes three main entities: **Input**, **Player**, and **Output**. Basic **Player API** packages: * `com.banuba.sdk.input` β€” all the classes responsible for the input data, the Input entity. * `com.banuba.sdk.player` β€” the rendering thread and the `Player`, the Player entity. * `com.banuba.sdk.output` β€” all the classes responsible for the output data, the Output entity. * `com.banuba.sdk.frame` β€” the pixel buffer used for both input and output. ## Input[​](#input "Direct link to Input") **Input** receives frames from a camera, image, or user input and provides them to the `Player`. The `Player` can only work with one Input at a time. * `CameraInput` β€” this class provides frames from the front or rear camera in real time. * `ProtoInput` β€” this class provides frames as photos taken by the camera or loaded from a file. * `StreamInput` β€” this class provides user frames from user data. * `VideoInput` β€” this class provides frames from a video file. ## Player[​](#player "Direct link to Player") The **Player** class requests frames from **Input**, then processes these frames and passes the results to one or more **Outputs**. By default, frame processing works automatically. Optionally, you can enable on-demand processing. ## Output[​](#output "Direct link to Output") **Output** receives the result of the work from the **Player** and renders it to the surface or a texture, writes it into a file, or provides the user with frames in a supported format. * `FrameOutput` β€” provides the user with data in the form of a buffer of pixels. * `SurfaceOutput` β€” this class renders frames to the `SurfaceView`. * `TextureOutput` β€” this class renders frames to the `TextureView`. * `VideoOutput` β€” this class writes frames to a video file. ## Input and Output of user data[​](#input-and-output-of-user-data "Direct link to Input and Output of user data") The **Input** and the **Output** can operate on a pixel buffer. * `FramePixelBuffer` β€” provides access to an array of pixels as a byte buffer. In the `StreamInput` class, it is used as the input data buffer, and in the `FrameOutput` class, it is used as the output data buffer. * `FramePixelBufferFormat` β€” pixel buffer format can be one of: `RGBA`, `I420_BT601_FULL`, `I420_BT601_VIDEO`, `I420_BT709_FULL` or `I420_BT709_VIDEO`. ## Use cases[​](#use-cases "Direct link to Use cases") ### CameraDevice[​](#cameradevice "Direct link to CameraDevice") The camera device is associated with the device's physical camera. All camera settings are made using this class. ``` // Variable declaration somewhere inside the activity private val cameraDevice by lazy(LazyThreadSafetyMode.NONE) { CameraDevice(requireNotNull(this.applicationContext), this@MainActivity) } ... // Somewhere in the initialization code /* You can change the camera settings at any time as follows. */ cameraDevice.configurator .setLens(CameraDeviceConfigurator.LensSelector.BACK) /* Set back camera as input */ .setVideoCaptureSize(CameraDeviceConfigurator.SD_CAPTURE_SIZE) /* Video capturing size 640, 480 */ .setImageCaptureSize(CameraDeviceConfigurator.HD_CAPTURE_SIZE) /* Image capture size 1280, 720 */ .commit() /* You must call this method to apply the new settings. */ /* But if you're happy with the camera's default settings, then you can safely skip manual settings. */ /* We start the camera and then player starts taking frames */ /* You must obtain permission before calling the cameraDevice.start() method. */ cameraDevice.start() ... // Somewhere in the interruption code /* After this method, the camera will stop capturing frames and transmitting them to player */ cameraDevice.stop() ``` ### CameraInput[​](#camerainput "Direct link to CameraInput") Allows to receive and process frames from the `CameraDevice`. The `Player` will only process the most recently received frame, all other frames will be discarded. Frames will be processed in online mode. ``` // Somewhere in the initialization code /* There is no need to create a variable for this class since this class is only used to transfer frames from the camera to the player. But it is necessary that cameraDevice is created */ player.use(CameraInput(cameraDevice), ...) ``` ### PhotoInput[​](#photoinput "Direct link to PhotoInput") Allows you to process photos from the `CameraDevice`, from the Android [`Bitmap`](https://developer.android.com/reference/android/graphics/Bitmap), from the Android [`Image`](https://developer.android.com/reference/android/media/Image), or from the [`FramePixelBuffer`](#framepixelbuffer) with or without a given orientation and mirroring. The photo will be processed in the offline mode. ``` // Somewhere in the image processing code val photoInput = PhotoInput() player.use(photoInput, ...) /* cameraDevice must be created and started before taking a photo */ photoInput.take(cameraDevice, object: CameraDevice.IErrorOccurred { override fun onError(exception: Exception) { /* Did an error occur? Now we just ignore it */ } }) ``` ### StreamInput[​](#streaminput "Direct link to StreamInput") Pushes the user data stream to the `Player`. User data can come from anywhere, for example, received over the network. Frames will be processed in online mode. ``` // Variable declaration private val streamInput by lazy(LazyThreadSafetyMode.NONE) { StreamInput() } ... // Somewhere in the initialization code player.use(streamInput, ...) ... // somewhere in the code when receiving the next frame /* You can see an example of creating a FramePixelBuffer below in section 1FramePixelBuffer */ val frame = FramePixelBuffer(...) /* myFrameTimestampInNanoseconds - it is not necessary to put the timestamp, you can always transmit 0. But if you use VideoOutput, then recording in the video file will be based on the time that you put here. Be careful with the timestamp; note that it must be transmitted in nanoseconds. */ streamInput.push(frame, myFrameTimestampInNanoseconds) ``` ### VideoInput[​](#videoinput "Direct link to VideoInput") Used when you need to process a video file, all the frames will be processed sequentially one by one. Must be used with `MANUAL` player rendering. Supported video formats depend on the specific device and installed **Android** codecs. Frames will be processed in offline mode. ``` // Variable declaration private val videoInput by lazy(LazyThreadSafetyMode.NONE) { VideoInput() } ... // Somewhere in the initialization code /* Asynchronous video file processing */ videoInput.processVideoFile(File("path_to/my_video.mp4"), object: VideoInput.IVideoFrameStatus { override fun onStart() { /* Start of video extraction */ /* We switch the player to the manual mode so that we can process the video file frame by frame */ player.setRenderMode(Player.RenderMode.MANUAL) } override fun onFrame() { /* The video frame was extracted and pushed to the player */ /* We call synchronous rendering so that the player has time to process the frame */ player.render() } override fun onError(throwable: Throwable) { /* Did an error occur? Now we just ignore it */ } override fun onFinish() { /* Processing of the video file has completed. If there were any errors and nothing was read from the video file, then this function is always called if function onStart() was called. We also return the player to the previous rendering mode */ player.setRenderMode(Player.RenderMode.LOOP) } }) ... // Somewhere in the interruption code /* If processing the current video file is no longer needed, then call this method */ videoInput.stopProcessing() ``` ### FramePixelBufferFormat[​](#framepixelbufferformat "Direct link to FramePixelBufferFormat") Pixel buffer format can be one of: * `BPC8_RGBA` - 4 bytes per pixel, analogue of android type [`Bitmap.Config.ARGB_8888`](https://developer.android.com/reference/android/graphics/Bitmap.Config). * `I420_BT601_FULL` - yuv i420 image encoded by standard bt601 full range. * `I420_BT601_VIDEO` - yuv i420 image encoded by standard bt601 video range. * `I420_BT709_FULL` - yuv i420 image encoded by standard bt709 full range. * `I420_BT709_VIDEO` - yuv i420 image encoded by standard bt709 video range. ### FramePixelBuffer[​](#framepixelbuffer "Direct link to FramePixelBuffer") A wrapper for pixel images that can store images of different formats and provide convenient access to all image parameters. example of creating RGBA FramePixelBuffer ``` /* The format BPC8_RGBA has a single plane of densely packed pixels */ val myWidth = 400 /* width of the image */ val myHeight = 300 /* height of the image */ val myPixelStride = 4 /* 4 because BPC8_RGBA format and the pixels are tightly packed */ val myRowStride = myWidth * myPixelStride /* Stride is bytes per row of pixels */ val myArrayOfPixels: ByteBuffer = ... /* array of pixels with RGBA data 400x300 pixels */ val frame = FramePixelBuffer(myArrayOfPixels, intArrayOf(0 /* 0 because the pixels start from a zero byte in the buffer */), intArrayOf(myRowStride), intArrayOf(myPixelStride), myWidth, myHeight, FramePixelBufferFormat.BPC8_RGBA ) ``` example of creating YUV FramePixelBuffer ``` /* You can read more about YUV format here: https://learn.microsoft.com/en-us/windows/win32/medfound/recommended-8-bit-yuv-formats-for-video-rendering */ /* Any format I420_*** has a single plane of densely packed pixels */ val myWidth = 400 /* width of the image */ val myHeight = 300 /* height of the image */ val myArrayOfPixels: ByteBuffer = ... /* array of pixels with i420 data 400x300 pixels */ val myYPlanePixelStride = 1 /* 1 because i420 format and the pixels are tightly packed */ val myUPlanePixelStride = 1 /* 1 because i420 format and the pixels are tightly packed */ val myVPlanePixelStride = 1 /* 1 because i420 format and the pixels are tightly packed */ val myYPlaneRowStride = myWidth /* in this case the stride is equal to the width */ val myUPlaneRowStride = myWidth /* in this case the stride is equal to the width */ val myVPlaneRowStride = myWidth /* in this case the stride is equal to the width */ val myOffsetToYPlane = 0 /* plane Y starts from the beginning of buffer myArrayOfPixels */ val myOffsetToUPlane = myOffsetToYPlane + myYPlanePixelStride * myHeight val myOffsetToVPlane = myOffsetToUPlane + myUPlanePixelStride * myHeight / 4 val frame = FramePixelBuffer(myArrayOfPixels, intArrayOf(myOffsetToYPlane, myOffsetToUPlane, myOffsetToVPlane), intArrayOf(myYPlaneRowStride, myUPlaneRowStride, myVPlaneRowStride), intArrayOf(myYPlanePixelStride, myUPlanePixelStride, myVPlanePixelStride), myWidth, myHeight, FramePixelBufferFormat.I420_BT601_FULL ) ``` ### Player[​](#player-1 "Direct link to Player") The main class with which you can manage **Banuba SDK**, **effects** and the entire rendering process. Rendering in this class is done in a separate thread, the **render thread**. ``` // Variable declaration private val player by lazy(LazyThreadSafetyMode.NONE) { Player() } ... // Somewhere in the initialization code /* Initialization of the Banuba SDK must occur before the player starts working, otherwise there will be a crash indicating an error */ BanubaSdkManager.initialize(this, <#MY BANUBA CLIENT TOKEN#>); /* cameraDevice and textureOutput must be created and declared before use */ /* In fact, you can specify more than one output as an output. The number of outputs is not limited. But every additional one affects performance. If this is an output to the surface, then you won’t notice the difference. But if this is rendering into a video file, then the performance directly depends on the capabilities of the phone. You can use multiple outputs as follows: player.use(CameraInput(cameraDevice), intArrayOf(myOutput1, myOutput2, ..., myOutputN)) */ player.use(CameraInput(cameraDevice), textureOutput) /* Loading any effect */ player.loadAsync("PineappleGlasses") /* And running the player */ player.play() ... // Somewhere in the destruction code /* After you are done using the player, you must free all resources by calling method close() */ player.close() ``` ### PlayerTouchListener[​](#playertouchlistener "Direct link to PlayerTouchListener") This class represents user clicks in **Banuba SDK**. This is required by some **effects**, and is one way to interact with them. ``` // Somewhere in the initialization code /* The player must be created and declared before use. The mySurfaceView is a UI element and must also exist in the layout */ mySurfaceView.setOnTouchListener(PlayerTouchListener(this.applicationContext, player)); ``` ### FrameOutput[​](#frameoutput "Direct link to FrameOutput") Allows you to receive the processing result frames in the form of an array of pixels in the desired format and in the desired orientation. ``` // Variable declaration private val frameOutput by lazy(LazyThreadSafetyMode.NONE) { FrameOutput(object : FrameOutput.IFramePixelBufferProvider { override fun onFrame(output: IOutput, framePixelBuffer: FramePixelBuffer?) { /* This is your code for working with the framePixelBuffer */ } }) } ... // Somewhere in the initialization code frameOutput.setFormat(FramePixelBufferFormat.I420_BT601_FULL) frameOutput.setOrientation(Orientation.UP, false) player.use(..., frameOutput) ... // Somewhere in the destruction code /* After finishing using the frameOutput, you must free all the resources by calling the close() method */ frameOutput.close() ``` ### SurfaceOutput[​](#surfaceoutput "Direct link to SurfaceOutput") Allows you to display the processing result on an [SurfaceView](https://developer.android.com/reference/android/view/SurfaceView). ``` // Variable declaration /* The mySurfaceView is a UI element and must exist in the layout */ private val surfaceOutput by lazy(LazyThreadSafetyMode.NONE) { SurfaceOutput(mySurfaceView.holder) } ... // Somewhere in the initialization code player.use(..., surfaceOutput) ... // Somewhere in the destruction code /* After finishing using the surfaceOutput, you must free all resources by calling the close() method */ surfaceOutput.close() ``` ### TextureOutput[​](#textureoutput "Direct link to TextureOutput") Allows you to display the processing result on an [TextureView](https://developer.android.com/reference/android/view/TextureView). ``` // Variable declaration /* The myTextureView is a UI element and must exist in the layout */ private val textureOutput by lazy(LazyThreadSafetyMode.NONE) { TextureOutput(myTextureView) } ... // Somewhere in the initialization code player.use(..., textureOutput) ... // Somewhere in the destruction code /* After finishing using the textureOutput, you must free all resources by calling the close() method */ textureOutput.close() ``` ### VideoOutput[​](#videooutput "Direct link to VideoOutput") The class allows you to record the processing result to a video file. ``` // Variable declaration private val videoOutput by lazy(LazyThreadSafetyMode.NONE) { VideoOutput() } ... // Somewhere in the initialization code player.use(..., videoOutput) /* Before you start recording a video, you must obtain permission to record to the storage. */ videoOutput.startRecording(File("path_to/my_output_video.mp4")) ... // Somewhere in the interruption code /* It is necessary to interrupt video recording when it is no longer needed */ videoOutput.stopRecording() ... // Somewhere in the destruction code /* After you are done using the videoOutput, you must free all the resources by calling the close() method */ videoOutput.close() ``` ![image](/far-sdk/assets/images/web_overview-747754fce3a1ba126bdff47046ba649d.svg) `BanubaSDK.js` exports different APIs for **Web AR** development like *Player*, *Effect*, several types of *Input* and *Output*. A generic workflow looks like: > *Input* -> *Player* + *Effect* -> *Output* ### Player[​](#player "Direct link to Player") The *Player* allows to consume different data inputs like webcam or image file, to apply an effect on top of it and to produce an output like rendering to DOM node or an image file. ### Effect[​](#effect "Direct link to Effect") The *Effect* allows to consume an effect or a face filter as remote or local archive. ### Input[​](#input "Direct link to Input") The *Input* can be one of the following: * Webcam * Image as Blob or URL * Video as Blob or URL * 3rd-party MediaStream like HTMLVideoElement stream or WebRTC stream ### Output[​](#output "Direct link to Output") The *Output* can be one of the following: * HTML Element * Image as Blob * Video as Blob * MediaStream that can be used by 3rd-parties like WebRTC peer connection The plenty of input options multiplied by the plenty of output options covers lots of use cases like: * Photo booth app with realtime webcam video processing and photo capturing * Photo and video files post-processing app * P2P video call app with face filter applied And many more. ## How it looks in JavaScript[​](#how-it-looks-in-javascript "Direct link to How it looks in JavaScript") Real-time webcam video processing and DOM rendering: ``` import { Webcam, Player, Module, Effect, Dom } from "https://cdn.jsdelivr.net/npm/@banuba/webar/dist/BanubaSDK.browser.esm.js" const player = await Player.create({ clientToken: "xxx-xxx-xxx" }) await player.addModule(new Module("https://cdn.jsdelivr.net/npm/@banuba/webar/dist/modules/face_tracker.zip")) await player.use(new Webcam()) player.applyEffect(new Effect("Glasses.zip")) Dom.render(player, "#webar-app") ``` And with screenshot capturing: ``` import { Webcam, Player, Effect, Module, Dom, ImageCapture } from "https://cdn.jsdelivr.net/npm/@banuba/webar/dist/BanubaSDK.browser.esm.js" const player = await Player.create({ clientToken: "xxx-xxx-xxx" }) await player.addModule(new Module("https://cdn.jsdelivr.net/npm/@banuba/webar/dist/modules/face_tracker.zip")) await player.use(new Webcam()) player.applyEffect(new Effect("Glasses.zip")) Dom.render(player, "#webar-app") const capture = new ImageCapture(player) const photo = await capture.takePhoto() ``` ![image](/far-sdk/assets/images/desktop_player_api_overview-45d2577b869d02247fa0b0dd15d29a70.svg) **Banuba SDK** for **desktop** can be divided into three main entities: `input`, `player` and `output`. The plethora of input options multiplied by the plethora of output options covers many use cases. ## Basic Player API interfaces:[​](#basic-player-api-interfaces "Direct link to Basic Player API interfaces:") * `bnb::player_api::interfaces::input` - receive frames and transfer them to the `player`. * `bnb::player_api::interfaces::output` - presents frames on the surface or read in memory. * `bnb::player_api::interfaces::player` - frames processing and rendering. * `bnb::player_api::interfaces::render_target` - rendering context. * `bnb::player_api::interfaces::render_delegate` - connection between the application rendering and `player` rendering. ## Input[​](#input "Direct link to Input") Processes frames from one of the producers: * `bnb::player_api::live_input` - live stream with the ability to skip frames. * `bnb::player_api::photo_input` - photo or image. * `bnb::player_api::stream_input` - stream without frames skipping. * Custom - your own implementation for the `bnb::player_api::interfaces::input` interface. ## Output[​](#output "Direct link to Output") Presents a rendered frame onto one of the available surfaces: * `bnb::player_api::opengl_frame_output` - array of pixels, should be used with `opengl_render_target`. * `bnb::player_api::metal_frame_output` - array of pixels, should be used with `metal_render_target`. * `bnb::player_api::texture_output` - **GPU** texture, texture type depends on the used `render_target`. * `bnb::player_api::window_output` - window should work with the same **GAPI** as the used `render_target`. * Custom - your own implementation for the `bnb::player_api::interfaces::output` interface. ## Player[​](#player "Direct link to Player") * `bnb::player_api::player` - processes frames and applies effects. ## Render target[​](#render-target "Direct link to Render target") * `bnb::player_api::opengl_render_target` - **OpenGL** implementation for the `render_target` * `bnb::player_api::metal_render_target` - **Metal** implementation for the `render_target`. ## Render delegate[​](#render-delegate "Direct link to Render delegate") This is always a custom implementation. To implement the interface, you need to inherit from interface `bnb::player_api::interfaces::render_delegate` and override three methods: * `activate()` - activating the rendering context, if necessary. * `started()` - frame rendering has started. After this, `finished(...)` will be called. * `finished(int64_t frame_number)` - frame rendering has ended. The frame number is any non-negative number. If `-1` is passed, then the frame rendering has failed. ## Input and output of user data[​](#input-and-output-of-user-data "Direct link to Input and output of user data") The **input** and the **output** can operate on a pixel buffer. * `bnb::full_image_t` - provides access to an array of pixels as a byte buffer. * `bnb::pixel_buffer_format` - pixel buffer format can be one of: `bpc8_rgb`, `bpc8_rgba`, `bpc8_bgr`, `bpc8_bgra`, `bpc8_argb`, `i420`, `nv12`. ## Use cases[​](#use-cases "Direct link to Use cases") ### live\_input[​](#live_input "Direct link to live_input") Used to receive and subsequently process frames in real time. If a new frame is received and the player has not yet processed the previous frame, the new frame will be skipped. Suitable for receiving frames from a camera. ``` #include ... // Creating an instance auto input = bnb::player_api::live_input::create(); ... // Adding an input to the player player->use(input); ... // Somewhere in the loop auto frame = bnb::full_image_t::create_bpc8(...); auto frame_time = get_my_current_timestamp_us(); // Pushing frame asynchronously into input input.push(frame, frame_time); ``` ### photo\_input[​](#photo_input "Direct link to photo_input") Used for obtaining and subsequent processing of photographs. ``` #include ... // Creating an instance auto input = bnb::player_api::photo_input::create(); ... // Adding an input to the player player->use(input); ... auto filepath = std::string("/path/to/the/photo.png"); // Somewhere where need to upload a photo input.push(filepath); ``` ### stream\_input[​](#stream_input "Direct link to stream_input") Used to receive and subsequently process a stream of frames. Does not skip frames if the previous frame is still not processed. Suitable for processing video streams. ``` #include ... // Creating an instance auto input = bnb::player_api::stream_input::create(); ... // Adding an input to the player player->use(input); ... // Somewhere in the loop auto frame = bnb::full_image_t::create_bpc8(...); auto frame_time = get_my_video_timestamp_us(); // Pushing frame synchronously into input input.push(frame, frame_time); ``` ### Custom input[​](#custom-input "Direct link to Custom input") A custom **input** is needed if the standard implementation does not contain the required logic. ``` #include #include class my_custom_input : public bnb::player_api::interfaces::input { public: my_custom_input() { auto config = bnb::interfaces::processor_configuration::create(); m_frame_processor = bnb::interfaces::frame_processor::create_video_processor(config); } void my_custom_push_method(...) { m_timestamp_us = get_timestamp(); auto fi = bnb::full_image_t::create_bpc8(...); // create an RGB image auto fd = bnb::interfaces::frame_data::create(); fd->add_full_img(fi); fd->add_timestamp_us(m_timestamp_us); m_frame_processor->push(fd); } frame_processor_sptr get_frame_processor() const noexcept override { return m_frame_processor; } uint64_t get_frame_time_us() const noexcept override { return m_timestamp_us; } private: bnb::player_api::frame_processor_sptr m_frame_processor; uint64_t m_timestamp_us; }; ``` ### opengl\_frame\_output[​](#opengl_frame_output "Direct link to opengl_frame_output") Allows you to receive the processing result frames as an array of pixels in the desired format and orientation. Important `opengl_frame_output` only works in conjunction with `opengl_render_target`. ``` #include ... // Creating an instance auto output = bnb::player_api::opengl_frame_output::create([](const bnb::full_image_t& image) { // We work here with the resulting `image` }, bnb::pixel_buffer_format::bpc8_rgba); output->set_orientation(bnb::orientation::left, true); ... // Adding an output to the player player->use(output); ``` ### metal\_frame\_output[​](#metal_frame_output "Direct link to metal_frame_output") Allows you to receive the processing result frames as an array of pixels in the desired format and orientation. Important `metal_frame_output` only works in conjunction with `metal_render_target` and it available only on `Apple` platform. ``` #include ... // Creating an instance auto output = bnb::player_api::metal_frame_output::create([](const bnb::full_image_t& image) { // We work here with the resulting `image` }, bnb::pixel_buffer_format::bpc8_rgba); output->set_orientation(bnb::orientation::left, true); ... // Adding an output to the player player->use(output); ``` ### texture\_output[​](#texture_output "Direct link to texture_output") Allows you to get texture. ``` #include ... // Creating an instance auto output = bnb::player_api::texture_output::create([](const bnb::player_api::texture_t texture) { // We work here with the resulting `texture` }); ... // Adding an output to the player player->use(output); ``` ### window\_output[​](#window_output "Direct link to window_output") Renders frames onto the window surface, at the specified location, and in the specified orientation. ``` #include ... // Creating an instance with opengl_render_target auto output = bnb::player_api::window_output::create(nullptr); ... // Or creating an instance with metal_render_target CAMetalLayer* metal_layer = get_my_metal_layer(); // When using metal rendering, you need to pass the render layer. auto output = bnb::player_api::window_output::create(static_cast(metal_layer)); ... // Setup orientaion and mirroring output->set_orientation(bnb::orientation::left, true); ... // Adding an output to the player player->use(output); ... // Set the rendering size and position. output->set_frame_layout(position_left, position_top, render_width, render_height); ``` ### Custom output[​](#custom-output "Direct link to Custom output") A custom **output** is needed if the standard implementation does not contain the required logic. ``` #include ... class my_custom_output : public bnb::player_api_interfaces { public: void attach() override { /* output is attached to the player */ } void detach() override { /* output is detached from the player */ } void present(const bnb::player_api::render_target_sptr& render_target) override { // custom code ... render_target->present(position_left, position_top, render_width, render_height, my_orientaion_matrix_4x4); } }; ``` ### full\_image\_t[​](#full_image_t "Direct link to full_image_t") Three functions are available to create images in different formats: * `bnb::full_image_t::create_bpc8` - for image formats `bpc8_rgb`, `bpc8_rgba`, `bpc8_bgr`, `bpc8_bgra`, `bpc8_argb`. * `bnb::full_image_t::create_nv12` - for nv12 yuv images with `bt601` or `bt709` color standard and with `full` or `video` color range. * `bnb::full_image_t::create_i420` - for i420 yuv images with `bt601` or `bt709` color standard and with `full` or `video` color range. ``` #include ... // creating RGB image auto rgb_image = bnb::full_image_t::create_bpc8( rgb_image_data, // uint8_t* raw pointer to image data width * 3, // rgb image stride. width, // width of the image height, // height of the image bnb::pixel_buffer_format::bpc8_rgb, // image format bnb::orientation::up, // image orientation false, // image mirroring [](uint8_t* data) { /* deleting the image data */ } // deleter of the image data ); ``` Retrieving image data: ``` #include ... // Getting RGB image data auto rgb_image = bnb::full_image_t::create_bpc8(...); uint8_t* rgb_image_bytes = rgb_image.get_base_ptr_of_plane(0); int32_t width = rgb_image.get_width_of_plane(0); int32_t height = rgb_image.get_height_of_plane(0); int32_t stride = rgb_image.get_bytes_per_row_of_plane(0); ``` ### player[​](#player-1 "Direct link to player") The **player** requests frames from the input, then processes those frames using the Banuba SDK and passes the results to one or more outputs. Default frame processing in Banuba SDK works automatically. If you wish, you can enable on-demand processing. ``` #include ... auto fps = 30; auto render_target = /* render target */; auto renderer = /* my custom renderer */ // Creating an instance auto player = bnb::player_api::player::create(fps, render_target, renderer); ... // Configuring the player and loading the effect player->use(/* some input */) .use(/* some output */) .use(/* some output */); player->load_async(/* path to the effect */); ``` ### opengl\_render\_target[​](#opengl_render_target "Direct link to opengl_render_target") The render target allows the player to render frames processed by the player onto outputs using OpenGL technology. Important `opengl_render_target` only works with OpenGL based outputs. ``` #include ... // Creating an instance auto render_target = bnb::player_api::opengl_render_target::create(); ... // Attach render target to the player auto player = bnb::player_api::player::create(/* fps value */, render_target, /* some renderer */); ``` ### metal\_render\_target[​](#metal_render_target "Direct link to metal_render_target") The render target allows the player to render frames processed by the player onto outputs using OpenGL technology. Important `metal_render_target` only works with METAL based outputs. ``` #include ... // Creating an instance auto render_target = bnb::player_api::metal_render_target::create(); ... // Attach render target to the player auto player = bnb::player_api::player::create(/* fps value */, render_target, /* some renderer */); ``` ### Custom render delegate[​](#custom-render-delegate "Direct link to Custom render delegate") Necessary for connecting the **player** and application in the context of rendering. ``` class my_custom_renderer : public bnb::player_api::interfaces::render_delegate { public: my_custom_renderer() { } void activate() override { /* make context current */ } void started() override { /* frame rendering started */ } void finished(int64_t frame_number) override { /* frame rendering finished */ } }; ``` Still have any questions about FaceAR SDK? Visit our [FAQ](https://www.banuba.com/faq/) or [contact our support](/far-sdk/support/.md). --- # android [View as Markdown](https://docs.banuba.com/far-sdk/tutorials/development/api_overview/android.md)[![](/far-sdk/img/ai-guide/chatgpt.svg)Open in ChatGPT](https://chatgpt.com/?q=Read%20https%3A%2F%2Fdocs.banuba.com%2Ffar-sdk%2Ftutorials%2Fdevelopment%2Fapi_overview%2Fandroid.md%20\(android\)%20and%20help%20me%20with%20it%20in%20my%20Banuba%20Face%20AR%20SDK%20project.)[![](/far-sdk/img/ai-guide/claude.svg)Open in Claude](https://claude.ai/new?q=Read%20https%3A%2F%2Fdocs.banuba.com%2Ffar-sdk%2Ftutorials%2Fdevelopment%2Fapi_overview%2Fandroid.md%20\(android\)%20and%20help%20me%20with%20it%20in%20my%20Banuba%20Face%20AR%20SDK%20project.)Install tools ![image](/far-sdk/assets/images/android_player_api_overview-69f91128c9d7c1f1e3e7286f7f919f36.svg) **Player API** is a set of classes and methods that help facilitate and speed up the integration of the Banuba SDK into applications. The **Player API** concept distinguishes three main entities: **Input**, **Player**, and **Output**. Basic **Player API** packages: * `com.banuba.sdk.input` β€” all the classes responsible for the input data, the Input entity. * `com.banuba.sdk.player` β€” the rendering thread and the `Player`, the Player entity. * `com.banuba.sdk.output` β€” all the classes responsible for the output data, the Output entity. * `com.banuba.sdk.frame` β€” the pixel buffer used for both input and output. ## Input[​](#input "Direct link to Input") **Input** receives frames from a camera, image, or user input and provides them to the `Player`. The `Player` can only work with one Input at a time. * `CameraInput` β€” this class provides frames from the front or rear camera in real time. * `ProtoInput` β€” this class provides frames as photos taken by the camera or loaded from a file. * `StreamInput` β€” this class provides user frames from user data. * `VideoInput` β€” this class provides frames from a video file. ## Player[​](#player "Direct link to Player") The **Player** class requests frames from **Input**, then processes these frames and passes the results to one or more **Outputs**. By default, frame processing works automatically. Optionally, you can enable on-demand processing. ## Output[​](#output "Direct link to Output") **Output** receives the result of the work from the **Player** and renders it to the surface or a texture, writes it into a file, or provides the user with frames in a supported format. * `FrameOutput` β€” provides the user with data in the form of a buffer of pixels. * `SurfaceOutput` β€” this class renders frames to the `SurfaceView`. * `TextureOutput` β€” this class renders frames to the `TextureView`. * `VideoOutput` β€” this class writes frames to a video file. ## Input and Output of user data[​](#input-and-output-of-user-data "Direct link to Input and Output of user data") The **Input** and the **Output** can operate on a pixel buffer. * `FramePixelBuffer` β€” provides access to an array of pixels as a byte buffer. In the `StreamInput` class, it is used as the input data buffer, and in the `FrameOutput` class, it is used as the output data buffer. * `FramePixelBufferFormat` β€” pixel buffer format can be one of: `RGBA`, `I420_BT601_FULL`, `I420_BT601_VIDEO`, `I420_BT709_FULL` or `I420_BT709_VIDEO`. ## Use cases[​](#use-cases "Direct link to Use cases") ### CameraDevice[​](#cameradevice "Direct link to CameraDevice") The camera device is associated with the device's physical camera. All camera settings are made using this class. ``` // Variable declaration somewhere inside the activity private val cameraDevice by lazy(LazyThreadSafetyMode.NONE) { CameraDevice(requireNotNull(this.applicationContext), this@MainActivity) } ... // Somewhere in the initialization code /* You can change the camera settings at any time as follows. */ cameraDevice.configurator .setLens(CameraDeviceConfigurator.LensSelector.BACK) /* Set back camera as input */ .setVideoCaptureSize(CameraDeviceConfigurator.SD_CAPTURE_SIZE) /* Video capturing size 640, 480 */ .setImageCaptureSize(CameraDeviceConfigurator.HD_CAPTURE_SIZE) /* Image capture size 1280, 720 */ .commit() /* You must call this method to apply the new settings. */ /* But if you're happy with the camera's default settings, then you can safely skip manual settings. */ /* We start the camera and then player starts taking frames */ /* You must obtain permission before calling the cameraDevice.start() method. */ cameraDevice.start() ... // Somewhere in the interruption code /* After this method, the camera will stop capturing frames and transmitting them to player */ cameraDevice.stop() ``` ### CameraInput[​](#camerainput "Direct link to CameraInput") Allows to receive and process frames from the `CameraDevice`. The `Player` will only process the most recently received frame, all other frames will be discarded. Frames will be processed in online mode. ``` // Somewhere in the initialization code /* There is no need to create a variable for this class since this class is only used to transfer frames from the camera to the player. But it is necessary that cameraDevice is created */ player.use(CameraInput(cameraDevice), ...) ``` ### PhotoInput[​](#photoinput "Direct link to PhotoInput") Allows you to process photos from the `CameraDevice`, from the Android [`Bitmap`](https://developer.android.com/reference/android/graphics/Bitmap), from the Android [`Image`](https://developer.android.com/reference/android/media/Image), or from the [`FramePixelBuffer`](#framepixelbuffer) with or without a given orientation and mirroring. The photo will be processed in the offline mode. ``` // Somewhere in the image processing code val photoInput = PhotoInput() player.use(photoInput, ...) /* cameraDevice must be created and started before taking a photo */ photoInput.take(cameraDevice, object: CameraDevice.IErrorOccurred { override fun onError(exception: Exception) { /* Did an error occur? Now we just ignore it */ } }) ``` ### StreamInput[​](#streaminput "Direct link to StreamInput") Pushes the user data stream to the `Player`. User data can come from anywhere, for example, received over the network. Frames will be processed in online mode. ``` // Variable declaration private val streamInput by lazy(LazyThreadSafetyMode.NONE) { StreamInput() } ... // Somewhere in the initialization code player.use(streamInput, ...) ... // somewhere in the code when receiving the next frame /* You can see an example of creating a FramePixelBuffer below in section 1FramePixelBuffer */ val frame = FramePixelBuffer(...) /* myFrameTimestampInNanoseconds - it is not necessary to put the timestamp, you can always transmit 0. But if you use VideoOutput, then recording in the video file will be based on the time that you put here. Be careful with the timestamp; note that it must be transmitted in nanoseconds. */ streamInput.push(frame, myFrameTimestampInNanoseconds) ``` ### VideoInput[​](#videoinput "Direct link to VideoInput") Used when you need to process a video file, all the frames will be processed sequentially one by one. Must be used with `MANUAL` player rendering. Supported video formats depend on the specific device and installed **Android** codecs. Frames will be processed in offline mode. ``` // Variable declaration private val videoInput by lazy(LazyThreadSafetyMode.NONE) { VideoInput() } ... // Somewhere in the initialization code /* Asynchronous video file processing */ videoInput.processVideoFile(File("path_to/my_video.mp4"), object: VideoInput.IVideoFrameStatus { override fun onStart() { /* Start of video extraction */ /* We switch the player to the manual mode so that we can process the video file frame by frame */ player.setRenderMode(Player.RenderMode.MANUAL) } override fun onFrame() { /* The video frame was extracted and pushed to the player */ /* We call synchronous rendering so that the player has time to process the frame */ player.render() } override fun onError(throwable: Throwable) { /* Did an error occur? Now we just ignore it */ } override fun onFinish() { /* Processing of the video file has completed. If there were any errors and nothing was read from the video file, then this function is always called if function onStart() was called. We also return the player to the previous rendering mode */ player.setRenderMode(Player.RenderMode.LOOP) } }) ... // Somewhere in the interruption code /* If processing the current video file is no longer needed, then call this method */ videoInput.stopProcessing() ``` ### FramePixelBufferFormat[​](#framepixelbufferformat "Direct link to FramePixelBufferFormat") Pixel buffer format can be one of: * `BPC8_RGBA` - 4 bytes per pixel, analogue of android type [`Bitmap.Config.ARGB_8888`](https://developer.android.com/reference/android/graphics/Bitmap.Config). * `I420_BT601_FULL` - yuv i420 image encoded by standard bt601 full range. * `I420_BT601_VIDEO` - yuv i420 image encoded by standard bt601 video range. * `I420_BT709_FULL` - yuv i420 image encoded by standard bt709 full range. * `I420_BT709_VIDEO` - yuv i420 image encoded by standard bt709 video range. ### FramePixelBuffer[​](#framepixelbuffer "Direct link to FramePixelBuffer") A wrapper for pixel images that can store images of different formats and provide convenient access to all image parameters. example of creating RGBA FramePixelBuffer ``` /* The format BPC8_RGBA has a single plane of densely packed pixels */ val myWidth = 400 /* width of the image */ val myHeight = 300 /* height of the image */ val myPixelStride = 4 /* 4 because BPC8_RGBA format and the pixels are tightly packed */ val myRowStride = myWidth * myPixelStride /* Stride is bytes per row of pixels */ val myArrayOfPixels: ByteBuffer = ... /* array of pixels with RGBA data 400x300 pixels */ val frame = FramePixelBuffer(myArrayOfPixels, intArrayOf(0 /* 0 because the pixels start from a zero byte in the buffer */), intArrayOf(myRowStride), intArrayOf(myPixelStride), myWidth, myHeight, FramePixelBufferFormat.BPC8_RGBA ) ``` example of creating YUV FramePixelBuffer ``` /* You can read more about YUV format here: https://learn.microsoft.com/en-us/windows/win32/medfound/recommended-8-bit-yuv-formats-for-video-rendering */ /* Any format I420_*** has a single plane of densely packed pixels */ val myWidth = 400 /* width of the image */ val myHeight = 300 /* height of the image */ val myArrayOfPixels: ByteBuffer = ... /* array of pixels with i420 data 400x300 pixels */ val myYPlanePixelStride = 1 /* 1 because i420 format and the pixels are tightly packed */ val myUPlanePixelStride = 1 /* 1 because i420 format and the pixels are tightly packed */ val myVPlanePixelStride = 1 /* 1 because i420 format and the pixels are tightly packed */ val myYPlaneRowStride = myWidth /* in this case the stride is equal to the width */ val myUPlaneRowStride = myWidth /* in this case the stride is equal to the width */ val myVPlaneRowStride = myWidth /* in this case the stride is equal to the width */ val myOffsetToYPlane = 0 /* plane Y starts from the beginning of buffer myArrayOfPixels */ val myOffsetToUPlane = myOffsetToYPlane + myYPlanePixelStride * myHeight val myOffsetToVPlane = myOffsetToUPlane + myUPlanePixelStride * myHeight / 4 val frame = FramePixelBuffer(myArrayOfPixels, intArrayOf(myOffsetToYPlane, myOffsetToUPlane, myOffsetToVPlane), intArrayOf(myYPlaneRowStride, myUPlaneRowStride, myVPlaneRowStride), intArrayOf(myYPlanePixelStride, myUPlanePixelStride, myVPlanePixelStride), myWidth, myHeight, FramePixelBufferFormat.I420_BT601_FULL ) ``` ### Player[​](#player-1 "Direct link to Player") The main class with which you can manage **Banuba SDK**, **effects** and the entire rendering process. Rendering in this class is done in a separate thread, the **render thread**. ``` // Variable declaration private val player by lazy(LazyThreadSafetyMode.NONE) { Player() } ... // Somewhere in the initialization code /* Initialization of the Banuba SDK must occur before the player starts working, otherwise there will be a crash indicating an error */ BanubaSdkManager.initialize(this, <#MY BANUBA CLIENT TOKEN#>); /* cameraDevice and textureOutput must be created and declared before use */ /* In fact, you can specify more than one output as an output. The number of outputs is not limited. But every additional one affects performance. If this is an output to the surface, then you won’t notice the difference. But if this is rendering into a video file, then the performance directly depends on the capabilities of the phone. You can use multiple outputs as follows: player.use(CameraInput(cameraDevice), intArrayOf(myOutput1, myOutput2, ..., myOutputN)) */ player.use(CameraInput(cameraDevice), textureOutput) /* Loading any effect */ player.loadAsync("PineappleGlasses") /* And running the player */ player.play() ... // Somewhere in the destruction code /* After you are done using the player, you must free all resources by calling method close() */ player.close() ``` ### PlayerTouchListener[​](#playertouchlistener "Direct link to PlayerTouchListener") This class represents user clicks in **Banuba SDK**. This is required by some **effects**, and is one way to interact with them. ``` // Somewhere in the initialization code /* The player must be created and declared before use. The mySurfaceView is a UI element and must also exist in the layout */ mySurfaceView.setOnTouchListener(PlayerTouchListener(this.applicationContext, player)); ``` ### FrameOutput[​](#frameoutput "Direct link to FrameOutput") Allows you to receive the processing result frames in the form of an array of pixels in the desired format and in the desired orientation. ``` // Variable declaration private val frameOutput by lazy(LazyThreadSafetyMode.NONE) { FrameOutput(object : FrameOutput.IFramePixelBufferProvider { override fun onFrame(output: IOutput, framePixelBuffer: FramePixelBuffer?) { /* This is your code for working with the framePixelBuffer */ } }) } ... // Somewhere in the initialization code frameOutput.setFormat(FramePixelBufferFormat.I420_BT601_FULL) frameOutput.setOrientation(Orientation.UP, false) player.use(..., frameOutput) ... // Somewhere in the destruction code /* After finishing using the frameOutput, you must free all the resources by calling the close() method */ frameOutput.close() ``` ### SurfaceOutput[​](#surfaceoutput "Direct link to SurfaceOutput") Allows you to display the processing result on an [SurfaceView](https://developer.android.com/reference/android/view/SurfaceView). ``` // Variable declaration /* The mySurfaceView is a UI element and must exist in the layout */ private val surfaceOutput by lazy(LazyThreadSafetyMode.NONE) { SurfaceOutput(mySurfaceView.holder) } ... // Somewhere in the initialization code player.use(..., surfaceOutput) ... // Somewhere in the destruction code /* After finishing using the surfaceOutput, you must free all resources by calling the close() method */ surfaceOutput.close() ``` ### TextureOutput[​](#textureoutput "Direct link to TextureOutput") Allows you to display the processing result on an [TextureView](https://developer.android.com/reference/android/view/TextureView). ``` // Variable declaration /* The myTextureView is a UI element and must exist in the layout */ private val textureOutput by lazy(LazyThreadSafetyMode.NONE) { TextureOutput(myTextureView) } ... // Somewhere in the initialization code player.use(..., textureOutput) ... // Somewhere in the destruction code /* After finishing using the textureOutput, you must free all resources by calling the close() method */ textureOutput.close() ``` ### VideoOutput[​](#videooutput "Direct link to VideoOutput") The class allows you to record the processing result to a video file. ``` // Variable declaration private val videoOutput by lazy(LazyThreadSafetyMode.NONE) { VideoOutput() } ... // Somewhere in the initialization code player.use(..., videoOutput) /* Before you start recording a video, you must obtain permission to record to the storage. */ videoOutput.startRecording(File("path_to/my_output_video.mp4")) ... // Somewhere in the interruption code /* It is necessary to interrupt video recording when it is no longer needed */ videoOutput.stopRecording() ... // Somewhere in the destruction code /* After you are done using the videoOutput, you must free all the resources by calling the close() method */ videoOutput.close() ``` --- # desktop [View as Markdown](https://docs.banuba.com/far-sdk/tutorials/development/api_overview/desktop.md)[![](/far-sdk/img/ai-guide/chatgpt.svg)Open in ChatGPT](https://chatgpt.com/?q=Read%20https%3A%2F%2Fdocs.banuba.com%2Ffar-sdk%2Ftutorials%2Fdevelopment%2Fapi_overview%2Fdesktop.md%20\(desktop\)%20and%20help%20me%20with%20it%20in%20my%20Banuba%20Face%20AR%20SDK%20project.)[![](/far-sdk/img/ai-guide/claude.svg)Open in Claude](https://claude.ai/new?q=Read%20https%3A%2F%2Fdocs.banuba.com%2Ffar-sdk%2Ftutorials%2Fdevelopment%2Fapi_overview%2Fdesktop.md%20\(desktop\)%20and%20help%20me%20with%20it%20in%20my%20Banuba%20Face%20AR%20SDK%20project.)Install tools ![image](/far-sdk/assets/images/desktop_player_api_overview-45d2577b869d02247fa0b0dd15d29a70.svg) **Banuba SDK** for **desktop** can be divided into three main entities: `input`, `player` and `output`. The plethora of input options multiplied by the plethora of output options covers many use cases. ## Basic Player API interfaces:[​](#basic-player-api-interfaces "Direct link to Basic Player API interfaces:") * `bnb::player_api::interfaces::input` - receive frames and transfer them to the `player`. * `bnb::player_api::interfaces::output` - presents frames on the surface or read in memory. * `bnb::player_api::interfaces::player` - frames processing and rendering. * `bnb::player_api::interfaces::render_target` - rendering context. * `bnb::player_api::interfaces::render_delegate` - connection between the application rendering and `player` rendering. ## Input[​](#input "Direct link to Input") Processes frames from one of the producers: * `bnb::player_api::live_input` - live stream with the ability to skip frames. * `bnb::player_api::photo_input` - photo or image. * `bnb::player_api::stream_input` - stream without frames skipping. * Custom - your own implementation for the `bnb::player_api::interfaces::input` interface. ## Output[​](#output "Direct link to Output") Presents a rendered frame onto one of the available surfaces: * `bnb::player_api::opengl_frame_output` - array of pixels, should be used with `opengl_render_target`. * `bnb::player_api::metal_frame_output` - array of pixels, should be used with `metal_render_target`. * `bnb::player_api::texture_output` - **GPU** texture, texture type depends on the used `render_target`. * `bnb::player_api::window_output` - window should work with the same **GAPI** as the used `render_target`. * Custom - your own implementation for the `bnb::player_api::interfaces::output` interface. ## Player[​](#player "Direct link to Player") * `bnb::player_api::player` - processes frames and applies effects. ## Render target[​](#render-target "Direct link to Render target") * `bnb::player_api::opengl_render_target` - **OpenGL** implementation for the `render_target` * `bnb::player_api::metal_render_target` - **Metal** implementation for the `render_target`. ## Render delegate[​](#render-delegate "Direct link to Render delegate") This is always a custom implementation. To implement the interface, you need to inherit from interface `bnb::player_api::interfaces::render_delegate` and override three methods: * `activate()` - activating the rendering context, if necessary. * `started()` - frame rendering has started. After this, `finished(...)` will be called. * `finished(int64_t frame_number)` - frame rendering has ended. The frame number is any non-negative number. If `-1` is passed, then the frame rendering has failed. ## Input and output of user data[​](#input-and-output-of-user-data "Direct link to Input and output of user data") The **input** and the **output** can operate on a pixel buffer. * `bnb::full_image_t` - provides access to an array of pixels as a byte buffer. * `bnb::pixel_buffer_format` - pixel buffer format can be one of: `bpc8_rgb`, `bpc8_rgba`, `bpc8_bgr`, `bpc8_bgra`, `bpc8_argb`, `i420`, `nv12`. ## Use cases[​](#use-cases "Direct link to Use cases") ### live\_input[​](#live_input "Direct link to live_input") Used to receive and subsequently process frames in real time. If a new frame is received and the player has not yet processed the previous frame, the new frame will be skipped. Suitable for receiving frames from a camera. ``` #include ... // Creating an instance auto input = bnb::player_api::live_input::create(); ... // Adding an input to the player player->use(input); ... // Somewhere in the loop auto frame = bnb::full_image_t::create_bpc8(...); auto frame_time = get_my_current_timestamp_us(); // Pushing frame asynchronously into input input.push(frame, frame_time); ``` ### photo\_input[​](#photo_input "Direct link to photo_input") Used for obtaining and subsequent processing of photographs. ``` #include ... // Creating an instance auto input = bnb::player_api::photo_input::create(); ... // Adding an input to the player player->use(input); ... auto filepath = std::string("/path/to/the/photo.png"); // Somewhere where need to upload a photo input.push(filepath); ``` ### stream\_input[​](#stream_input "Direct link to stream_input") Used to receive and subsequently process a stream of frames. Does not skip frames if the previous frame is still not processed. Suitable for processing video streams. ``` #include ... // Creating an instance auto input = bnb::player_api::stream_input::create(); ... // Adding an input to the player player->use(input); ... // Somewhere in the loop auto frame = bnb::full_image_t::create_bpc8(...); auto frame_time = get_my_video_timestamp_us(); // Pushing frame synchronously into input input.push(frame, frame_time); ``` ### Custom input[​](#custom-input "Direct link to Custom input") A custom **input** is needed if the standard implementation does not contain the required logic. ``` #include #include class my_custom_input : public bnb::player_api::interfaces::input { public: my_custom_input() { auto config = bnb::interfaces::processor_configuration::create(); m_frame_processor = bnb::interfaces::frame_processor::create_video_processor(config); } void my_custom_push_method(...) { m_timestamp_us = get_timestamp(); auto fi = bnb::full_image_t::create_bpc8(...); // create an RGB image auto fd = bnb::interfaces::frame_data::create(); fd->add_full_img(fi); fd->add_timestamp_us(m_timestamp_us); m_frame_processor->push(fd); } frame_processor_sptr get_frame_processor() const noexcept override { return m_frame_processor; } uint64_t get_frame_time_us() const noexcept override { return m_timestamp_us; } private: bnb::player_api::frame_processor_sptr m_frame_processor; uint64_t m_timestamp_us; }; ``` ### opengl\_frame\_output[​](#opengl_frame_output "Direct link to opengl_frame_output") Allows you to receive the processing result frames as an array of pixels in the desired format and orientation. Important `opengl_frame_output` only works in conjunction with `opengl_render_target`. ``` #include ... // Creating an instance auto output = bnb::player_api::opengl_frame_output::create([](const bnb::full_image_t& image) { // We work here with the resulting `image` }, bnb::pixel_buffer_format::bpc8_rgba); output->set_orientation(bnb::orientation::left, true); ... // Adding an output to the player player->use(output); ``` ### metal\_frame\_output[​](#metal_frame_output "Direct link to metal_frame_output") Allows you to receive the processing result frames as an array of pixels in the desired format and orientation. Important `metal_frame_output` only works in conjunction with `metal_render_target` and it available only on `Apple` platform. ``` #include ... // Creating an instance auto output = bnb::player_api::metal_frame_output::create([](const bnb::full_image_t& image) { // We work here with the resulting `image` }, bnb::pixel_buffer_format::bpc8_rgba); output->set_orientation(bnb::orientation::left, true); ... // Adding an output to the player player->use(output); ``` ### texture\_output[​](#texture_output "Direct link to texture_output") Allows you to get texture. ``` #include ... // Creating an instance auto output = bnb::player_api::texture_output::create([](const bnb::player_api::texture_t texture) { // We work here with the resulting `texture` }); ... // Adding an output to the player player->use(output); ``` ### window\_output[​](#window_output "Direct link to window_output") Renders frames onto the window surface, at the specified location, and in the specified orientation. ``` #include ... // Creating an instance with opengl_render_target auto output = bnb::player_api::window_output::create(nullptr); ... // Or creating an instance with metal_render_target CAMetalLayer* metal_layer = get_my_metal_layer(); // When using metal rendering, you need to pass the render layer. auto output = bnb::player_api::window_output::create(static_cast(metal_layer)); ... // Setup orientaion and mirroring output->set_orientation(bnb::orientation::left, true); ... // Adding an output to the player player->use(output); ... // Set the rendering size and position. output->set_frame_layout(position_left, position_top, render_width, render_height); ``` ### Custom output[​](#custom-output "Direct link to Custom output") A custom **output** is needed if the standard implementation does not contain the required logic. ``` #include ... class my_custom_output : public bnb::player_api_interfaces { public: void attach() override { /* output is attached to the player */ } void detach() override { /* output is detached from the player */ } void present(const bnb::player_api::render_target_sptr& render_target) override { // custom code ... render_target->present(position_left, position_top, render_width, render_height, my_orientaion_matrix_4x4); } }; ``` ### full\_image\_t[​](#full_image_t "Direct link to full_image_t") Three functions are available to create images in different formats: * `bnb::full_image_t::create_bpc8` - for image formats `bpc8_rgb`, `bpc8_rgba`, `bpc8_bgr`, `bpc8_bgra`, `bpc8_argb`. * `bnb::full_image_t::create_nv12` - for nv12 yuv images with `bt601` or `bt709` color standard and with `full` or `video` color range. * `bnb::full_image_t::create_i420` - for i420 yuv images with `bt601` or `bt709` color standard and with `full` or `video` color range. ``` #include ... // creating RGB image auto rgb_image = bnb::full_image_t::create_bpc8( rgb_image_data, // uint8_t* raw pointer to image data width * 3, // rgb image stride. width, // width of the image height, // height of the image bnb::pixel_buffer_format::bpc8_rgb, // image format bnb::orientation::up, // image orientation false, // image mirroring [](uint8_t* data) { /* deleting the image data */ } // deleter of the image data ); ``` Retrieving image data: ``` #include ... // Getting RGB image data auto rgb_image = bnb::full_image_t::create_bpc8(...); uint8_t* rgb_image_bytes = rgb_image.get_base_ptr_of_plane(0); int32_t width = rgb_image.get_width_of_plane(0); int32_t height = rgb_image.get_height_of_plane(0); int32_t stride = rgb_image.get_bytes_per_row_of_plane(0); ``` ### player[​](#player-1 "Direct link to player") The **player** requests frames from the input, then processes those frames using the Banuba SDK and passes the results to one or more outputs. Default frame processing in Banuba SDK works automatically. If you wish, you can enable on-demand processing. ``` #include ... auto fps = 30; auto render_target = /* render target */; auto renderer = /* my custom renderer */ // Creating an instance auto player = bnb::player_api::player::create(fps, render_target, renderer); ... // Configuring the player and loading the effect player->use(/* some input */) .use(/* some output */) .use(/* some output */); player->load_async(/* path to the effect */); ``` ### opengl\_render\_target[​](#opengl_render_target "Direct link to opengl_render_target") The render target allows the player to render frames processed by the player onto outputs using OpenGL technology. Important `opengl_render_target` only works with OpenGL based outputs. ``` #include ... // Creating an instance auto render_target = bnb::player_api::opengl_render_target::create(); ... // Attach render target to the player auto player = bnb::player_api::player::create(/* fps value */, render_target, /* some renderer */); ``` ### metal\_render\_target[​](#metal_render_target "Direct link to metal_render_target") The render target allows the player to render frames processed by the player onto outputs using OpenGL technology. Important `metal_render_target` only works with METAL based outputs. ``` #include ... // Creating an instance auto render_target = bnb::player_api::metal_render_target::create(); ... // Attach render target to the player auto player = bnb::player_api::player::create(/* fps value */, render_target, /* some renderer */); ``` ### Custom render delegate[​](#custom-render-delegate "Direct link to Custom render delegate") Necessary for connecting the **player** and application in the context of rendering. ``` class my_custom_renderer : public bnb::player_api::interfaces::render_delegate { public: my_custom_renderer() { } void activate() override { /* make context current */ } void started() override { /* frame rendering started */ } void finished(int64_t frame_number) override { /* frame rendering finished */ } }; ``` --- # ios [View as Markdown](https://docs.banuba.com/far-sdk/tutorials/development/api_overview/ios.md)[![](/far-sdk/img/ai-guide/chatgpt.svg)Open in ChatGPT](https://chatgpt.com/?q=Read%20https%3A%2F%2Fdocs.banuba.com%2Ffar-sdk%2Ftutorials%2Fdevelopment%2Fapi_overview%2Fios.md%20\(ios\)%20and%20help%20me%20with%20it%20in%20my%20Banuba%20Face%20AR%20SDK%20project.)[![](/far-sdk/img/ai-guide/claude.svg)Open in Claude](https://claude.ai/new?q=Read%20https%3A%2F%2Fdocs.banuba.com%2Ffar-sdk%2Ftutorials%2Fdevelopment%2Fapi_overview%2Fios.md%20\(ios\)%20and%20help%20me%20with%20it%20in%20my%20Banuba%20Face%20AR%20SDK%20project.)Install tools ![image](/far-sdk/assets/images/ios_player_api_overview-329f7e9e59ebec11fef743ed4309d096.svg) **Banuba SDK** for **iOS** can be divided into three main entities: `Input`, `Player` and `Output`. The plethora of input options multiplied by the plethora of output options covers many use cases. ## Input[​](#input "Direct link to Input") Processes frames from one of the producers: * `Camera` - a real-time `CameraDevice` feed * `Photo` - an image from the gallery or a photo taken from `CameraDevice` * `Stream` - a frames sequence from the *WebRTC* stream or any other provider * Custom - your own implementation for the `Input` protocol ## Player[​](#player "Direct link to Player") Allows to `use` different data inputs like `Camera` feed or `Photo`, to apply an **effect** on top of it and to `use` several outputs like `View` and `Video` file simultaneously. The `Effect` component makes up an essential part of the SDK usage. The **effect** is represented as a folder with scripts and resources and can be loaded with the `load` method. Supports the following rendering modes: * `loop` *(default)* - render in the display-linked loop with the defined FPS * `manual` - render manually by calling the `render` method ## Output[​](#output "Direct link to Output") Presents a rendered frame onto one of the available surfaces: * `View` - on screen presentation * `Frame`, `PixelBuffer`, `PixelBufferYUV` - in memory presentation * `Video` - in video file presentation * Custom - your own implementation for the `Output` protocol ## CameraDevice[​](#cameradevice "Direct link to CameraDevice") Accesses the device's camera to generate a feed of frames in real time or takes high-quality photos.
By default, tracks UI orientation to properly manage frame rotation. ## RenderTarget[​](#rendertarget "Direct link to RenderTarget") Manages a `CALayer` object with a `Metal` context and provides offscreen rendering for the `Player` and presenting for the `Output`. ## Use cases[​](#use-cases "Direct link to Use cases") Common use cases and relevant samples are described in [this repository](https://github.com/Banuba/banuba-sdk-ios-samples).
tip See more use cases in [Samples](/far-sdk/tutorials/development/samples.md). --- # web [View as Markdown](https://docs.banuba.com/far-sdk/tutorials/development/api_overview/web.md)[![](/far-sdk/img/ai-guide/chatgpt.svg)Open in ChatGPT](https://chatgpt.com/?q=Read%20https%3A%2F%2Fdocs.banuba.com%2Ffar-sdk%2Ftutorials%2Fdevelopment%2Fapi_overview%2Fweb.md%20\(web\)%20and%20help%20me%20with%20it%20in%20my%20Banuba%20Face%20AR%20SDK%20project.)[![](/far-sdk/img/ai-guide/claude.svg)Open in Claude](https://claude.ai/new?q=Read%20https%3A%2F%2Fdocs.banuba.com%2Ffar-sdk%2Ftutorials%2Fdevelopment%2Fapi_overview%2Fweb.md%20\(web\)%20and%20help%20me%20with%20it%20in%20my%20Banuba%20Face%20AR%20SDK%20project.)Install tools ![image](/far-sdk/assets/images/web_overview-747754fce3a1ba126bdff47046ba649d.svg) `BanubaSDK.js` exports different APIs for **Web AR** development like *Player*, *Effect*, several types of *Input* and *Output*. A generic workflow looks like: > *Input* -> *Player* + *Effect* -> *Output* ### Player[​](#player "Direct link to Player") The *Player* allows to consume different data inputs like webcam or image file, to apply an effect on top of it and to produce an output like rendering to DOM node or an image file. ### Effect[​](#effect "Direct link to Effect") The *Effect* allows to consume an effect or a face filter as remote or local archive. ### Input[​](#input "Direct link to Input") The *Input* can be one of the following: * Webcam * Image as Blob or URL * Video as Blob or URL * 3rd-party MediaStream like HTMLVideoElement stream or WebRTC stream ### Output[​](#output "Direct link to Output") The *Output* can be one of the following: * HTML Element * Image as Blob * Video as Blob * MediaStream that can be used by 3rd-parties like WebRTC peer connection The plenty of input options multiplied by the plenty of output options covers lots of use cases like: * Photo booth app with realtime webcam video processing and photo capturing * Photo and video files post-processing app * P2P video call app with face filter applied And many more. ## How it looks in JavaScript[​](#how-it-looks-in-javascript "Direct link to How it looks in JavaScript") Real-time webcam video processing and DOM rendering: ``` import { Webcam, Player, Module, Effect, Dom } from "https://cdn.jsdelivr.net/npm/@banuba/webar/dist/BanubaSDK.browser.esm.js" const player = await Player.create({ clientToken: "xxx-xxx-xxx" }) await player.addModule(new Module("https://cdn.jsdelivr.net/npm/@banuba/webar/dist/modules/face_tracker.zip")) await player.use(new Webcam()) player.applyEffect(new Effect("Glasses.zip")) Dom.render(player, "#webar-app") ``` And with screenshot capturing: ``` import { Webcam, Player, Effect, Module, Dom, ImageCapture } from "https://cdn.jsdelivr.net/npm/@banuba/webar/dist/BanubaSDK.browser.esm.js" const player = await Player.create({ clientToken: "xxx-xxx-xxx" }) await player.addModule(new Module("https://cdn.jsdelivr.net/npm/@banuba/webar/dist/modules/face_tracker.zip")) await player.use(new Webcam()) player.applyEffect(new Effect("Glasses.zip")) Dom.render(player, "#webar-app") const capture = new ImageCapture(player) const photo = await capture.takePhoto() ``` --- # Getting Started [View as Markdown](https://docs.banuba.com/far-sdk/tutorials/development/basic_integration.md)[![](/far-sdk/img/ai-guide/chatgpt.svg)Open in ChatGPT](https://chatgpt.com/?q=Read%20https%3A%2F%2Fdocs.banuba.com%2Ffar-sdk%2Ftutorials%2Fdevelopment%2Fbasic_integration.md%20\(Getting%20Started\)%20and%20help%20me%20with%20it%20in%20my%20Banuba%20Face%20AR%20SDK%20project.)[![](/far-sdk/img/ai-guide/claude.svg)Open in Claude](https://claude.ai/new?q=Read%20https%3A%2F%2Fdocs.banuba.com%2Ffar-sdk%2Ftutorials%2Fdevelopment%2Fbasic_integration.md%20\(Getting%20Started\)%20and%20help%20me%20with%20it%20in%20my%20Banuba%20Face%20AR%20SDK%20project.)Install tools ## Get the client token[​](#get-the-client-token "Direct link to Get the client token") To start working with **Banuba SDK** in your project, you need to have a client token. To receive it, please fill in the [form on banuba.com](https://www.banuba.com/face-filters-sdk), or contact us via . * iOS * Android * Web * Flutter * ReactNative * Desktop ## Installation[​](#installation "Direct link to Installation") 1. Add [BanubaSdk SPM packages](/far-sdk/tutorials/development/installation.md?ios-packages=spm#spm-packages) into your project info Details about the **SPM** and **CocoaPods** packages see in [Installation](/far-sdk/tutorials/development/installation.md). ## Integration[​](#integration "Direct link to Integration") 1. Setup `banubaClientToken` common/common/AppDelegate.swift ``` loading... ``` ![Icon](/far-sdk/img/icons/github.svg "GitHub") 2. Initialize `BanubaSdkManager` common/common/AppDelegate.swift ``` loading... ``` ![Icon](/far-sdk/img/icons/github.svg "GitHub") 3. Create `Player` and `load` the effect camera/camera/ViewController.swift ``` loading... ``` ![Icon](/far-sdk/img/icons/github.svg "GitHub") 4. Run the application! πŸŽ‰ πŸš€ πŸ’… tip See more use cases and code samples in the [GitHub repo](https://github.com/Banuba/banuba-sdk-ios-samples). ## Installation[​](#installation "Direct link to Installation") To get started, add the **Banuba SDK** packages to your project. Add the custom maven repo to your `build.gradle.kts`: camera/settings.gradle.kts ``` loading... ``` ![Icon](/far-sdk/img/icons/github.svg "GitHub") And add the dependency on **Banuba SDK** package to your `build.gradle.kts`: common/build.gradle.kts ``` loading... ``` ![Icon](/far-sdk/img/icons/github.svg "GitHub") info Details about the packages see in [Installation](/far-sdk/tutorials/development/installation.md). ## Integration[​](#integration "Direct link to Integration") camera/src/main/java/com/banuba/sdk/example/camera/MainActivity.kt ``` loading... ``` ![Icon](/far-sdk/img/icons/github.svg "GitHub") ## Requirements[​](#requirements "Direct link to Requirements") * [Nodejs](https://nodejs.org/en/) installed * Browser with [WebGL 2.0](https://caniuse.com/#feat=webgl2) and higher ## Integration[​](#integration "Direct link to Integration") 1. Setup the client token BanubaClientToken.js ``` loading... ``` ![Icon](/far-sdk/img/icons/github.svg "GitHub") 2. Import the required types from the [@banuba/webar](https://www.npmjs.com/package/@banuba/webar) NPM package BanubaPlayer.js ``` loading... ``` ![Icon](/far-sdk/img/icons/github.svg "GitHub") info See the details about the NPM package in [Installation](/far-sdk/tutorials/development/installation.md). 3. Initialize `Player` and apply the `Effect` BanubaPlayer.js ``` loading... ``` ![Icon](/far-sdk/img/icons/github.svg "GitHub") BanubaPlayer.js ``` loading... ``` ![Icon](/far-sdk/img/icons/github.svg "GitHub") BanubaPlayer.js ``` loading... ``` ![Icon](/far-sdk/img/icons/github.svg "GitHub") 4. Run a local web server from a terminal inside the folder ``` npx live-server ``` 5. Open [localhost:8080](http://localhost:8080) and start clicking πŸŽ‰ πŸš€ πŸ’… tip Follow the instructions of the demo app [README.md](https://github.com/Banuba/quickstart-web/blob/master/README.md) to get more info. note The demo app leverages [jsDelivr](https://jsdelivr.com/) CDN for ease of getting started, for a real life application please use the [@banuba/webar](https://www.npmjs.com/package/@banuba/webar) npm package. [Banuba SDK](https://pub.dev/packages/banuba_sdk) for [Flutter](https://docs.flutter.dev/) is available for iOS and Android and provides the following functionality: * Load and interact with any Effect (including the `Makeup` effect). * Still image processing. * Interaction with Camera (open/close, take photo, flashlight control, facing). * Screen recording. * [Videocall](/far-sdk/tutorials/development/videocall.md) powered by [Agora](https://docs.agora.io/). If this is not enough, you should go with native integration. ## Integration guide[​](#integration-guide "Direct link to Integration guide") * Android * iOS 1. Add `banuba_sdk` plugin: ``` flutter pub add banuba_sdk ``` 2. Add code from [the basic sample](https://github.com/Banuba/banuba-sdk-flutter/blob/master/example/lib/main.dart) into your app. Don't forget to `initialize` `BanubaSdkManager` with the Client Token: example/lib/main.dart ``` loading... ``` ![Icon](/far-sdk/img/icons/github.svg "GitHub") 3. Add `effects` folder into your project. Link it with your app: add the following code into app `build.gradle`. example/android/app/build.gradle ``` loading... ``` ![Icon](/far-sdk/img/icons/github.svg "GitHub") 1. Add `banuba_sdk` plugin: ``` flutter pub add banuba_sdk ``` 2. Link to **Banuba SDK** podspecs in `ios/Podfile`: ``` source 'https://github.com/sdk-banuba/banuba-sdk-podspecs.git' ``` 3. Add code from [the basic sample](https://github.com/Banuba/banuba-sdk-flutter/blob/master/example/lib/main.dart) into your app. Don't forget to `initialize` `BanubaSdkManager` with the Client Token: example/lib/main.dart ``` loading... ``` ![Icon](/far-sdk/img/icons/github.svg "GitHub") 4. Add `effects` folder into your project. Link it with your app: add the
folder into `Runner` **Xcode** project (`File` -> `Add Files to 'Runner'...`). [Banuba SDK](https://www.npmjs.com/package/@banuba/react-native) for [React Native](https://reactnative.dev/) available for iOS and Android and provides the following functionality: * Load and interact with any Effect (including `Makeup` effect). * Interaction with camera (open/close). * Screen recording (screenshots and video). * [Videocall](/far-sdk/tutorials/development/videocall.md) powered by [Agora](https://docs.agora.io/). If this is not enough, you should go with native integration. ## Integration guide[​](#integration-guide "Direct link to Integration guide") * Android * iOS 1. Add `@banuba/react-native` dependency ``` yarn add @banuba/react-native ``` 2. Add our **Maven repository** example/android/build.gradle ``` loading... ``` ![Icon](/far-sdk/img/icons/github.svg "GitHub") 3. Add [`effects` folder](https://github.com/Banuba/banuba-sdk-react-native/tree/master/example/effects) and add a task to copy them into app example/android/app/build.gradle ``` loading... ``` ![Icon](/far-sdk/img/icons/github.svg "GitHub") 4. Copy code from [this file](https://github.com/Banuba/banuba-sdk-react-native/blob/master/example/src/App.tsx) into your app. Don't forget to intialize the SDK with the Client Token example/src/App.tsx ``` loading... ``` ![Icon](/far-sdk/img/icons/github.svg "GitHub") 1. Add `@banuba/react-native` dependency ``` yarn add @banuba/react-native ``` 2. Add our podspecs repo to your `Podfile` example/ios/Podfile ``` loading... ``` ![Icon](/far-sdk/img/icons/github.svg "GitHub") 3. Add [`effects` folder](https://github.com/Banuba/banuba-sdk-react-native/tree/master/example/effects) and link it to Xcode project: `(File -> Add Files to ...)` 4. Copy code from [this file](https://github.com/Banuba/banuba-sdk-react-native/blob/master/example/src/App.tsx) into your app. Don't forget to intialize the SDK with the Client Token example/src/App.tsx ``` loading... ``` ![Icon](/far-sdk/img/icons/github.svg "GitHub") The steps below apply to desktop integration (**Windows** and/or **macOS**) with **C++**. 1. Download **Banuba SDK** binaries [from GitHub](https://github.com/Banuba/FaceAR-SDK-desktop-releases). 2. Integrate libraries downloaded on the previous step into your build system. If you use **CMake**, consider our [quickstart-desktop-cpp](https://github.com/Banuba/quickstart-desktop-cpp) sample. for Windows Besides the **Banuba SDK** itself, you will require third party libraries from the `bin` folder. 3. Create rendering context or copy and paste into your project [ready-to-use helpers](https://github.com/Banuba/quickstart-desktop-cpp/tree/master/helpers/src) sources based on [GLFW](https://www.glfw.org/) 4. Setup `BNB_CLIENT_TOKEN` helpers/src/BanubaClientToken.hpp ``` loading... ``` ![Icon](/far-sdk/img/icons/github.svg "GitHub") 5. Initialize **Banuba SDK** with the **Client Token** and path to resources from the archive with the binaries. Create `Player`, `Camera`, `Input` and `Output` and load the **effect**. realtime-camera-preview/main.cpp ``` loading... ``` ![Icon](/far-sdk/img/icons/github.svg "GitHub") 6. Run the application! πŸŽ‰ πŸš€ πŸ’… info The **effects** are also resources, you may initialize **Banuba SDK** with several resource paths (one for effects and one for SDK assets). for macOS Resources for **MacOS** are inside `BanubaEffectPlayer.xcframework`. Still have questions about FaceAR SDK? Visit our [FAQ](https://www.banuba.com/faq/) or [contact our support](/far-sdk/support/.md). --- # android [View as Markdown](https://docs.banuba.com/far-sdk/tutorials/development/basic_integration/android.md)[![](/far-sdk/img/ai-guide/chatgpt.svg)Open in ChatGPT](https://chatgpt.com/?q=Read%20https%3A%2F%2Fdocs.banuba.com%2Ffar-sdk%2Ftutorials%2Fdevelopment%2Fbasic_integration%2Fandroid.md%20\(android\)%20and%20help%20me%20with%20it%20in%20my%20Banuba%20Face%20AR%20SDK%20project.)[![](/far-sdk/img/ai-guide/claude.svg)Open in Claude](https://claude.ai/new?q=Read%20https%3A%2F%2Fdocs.banuba.com%2Ffar-sdk%2Ftutorials%2Fdevelopment%2Fbasic_integration%2Fandroid.md%20\(android\)%20and%20help%20me%20with%20it%20in%20my%20Banuba%20Face%20AR%20SDK%20project.)Install tools ## Installation[​](#installation "Direct link to Installation") To get started, add the **Banuba SDK** packages to your project. Add the custom maven repo to your `build.gradle.kts`: camera/settings.gradle.kts ``` loading... ``` ![Icon](/far-sdk/img/icons/github.svg "GitHub") And add the dependency on **Banuba SDK** package to your `build.gradle.kts`: common/build.gradle.kts ``` loading... ``` ![Icon](/far-sdk/img/icons/github.svg "GitHub") info Details about the packages see in [Installation](/far-sdk/tutorials/development/installation.md). ## Integration[​](#integration "Direct link to Integration") camera/src/main/java/com/banuba/sdk/example/camera/MainActivity.kt ``` loading... ``` ![Icon](/far-sdk/img/icons/github.svg "GitHub") --- # desktop [View as Markdown](https://docs.banuba.com/far-sdk/tutorials/development/basic_integration/desktop.md)[![](/far-sdk/img/ai-guide/chatgpt.svg)Open in ChatGPT](https://chatgpt.com/?q=Read%20https%3A%2F%2Fdocs.banuba.com%2Ffar-sdk%2Ftutorials%2Fdevelopment%2Fbasic_integration%2Fdesktop.md%20\(desktop\)%20and%20help%20me%20with%20it%20in%20my%20Banuba%20Face%20AR%20SDK%20project.)[![](/far-sdk/img/ai-guide/claude.svg)Open in Claude](https://claude.ai/new?q=Read%20https%3A%2F%2Fdocs.banuba.com%2Ffar-sdk%2Ftutorials%2Fdevelopment%2Fbasic_integration%2Fdesktop.md%20\(desktop\)%20and%20help%20me%20with%20it%20in%20my%20Banuba%20Face%20AR%20SDK%20project.)Install tools The steps below apply to desktop integration (**Windows** and/or **macOS**) with **C++**. 1. Download **Banuba SDK** binaries [from GitHub](https://github.com/Banuba/FaceAR-SDK-desktop-releases). 2. Integrate libraries downloaded on the previous step into your build system. If you use **CMake**, consider our [quickstart-desktop-cpp](https://github.com/Banuba/quickstart-desktop-cpp) sample. for Windows Besides the **Banuba SDK** itself, you will require third party libraries from the `bin` folder. 3. Create rendering context or copy and paste into your project [ready-to-use helpers](https://github.com/Banuba/quickstart-desktop-cpp/tree/master/helpers/src) sources based on [GLFW](https://www.glfw.org/) 4. Setup `BNB_CLIENT_TOKEN` helpers/src/BanubaClientToken.hpp ``` loading... ``` ![Icon](/far-sdk/img/icons/github.svg "GitHub") 5. Initialize **Banuba SDK** with the **Client Token** and path to resources from the archive with the binaries. Create `Player`, `Camera`, `Input` and `Output` and load the **effect**. realtime-camera-preview/main.cpp ``` loading... ``` ![Icon](/far-sdk/img/icons/github.svg "GitHub") 6. Run the application! πŸŽ‰ πŸš€ πŸ’… info The **effects** are also resources, you may initialize **Banuba SDK** with several resource paths (one for effects and one for SDK assets). for macOS Resources for **MacOS** are inside `BanubaEffectPlayer.xcframework`. --- # flutter [View as Markdown](https://docs.banuba.com/far-sdk/tutorials/development/basic_integration/flutter.md)[![](/far-sdk/img/ai-guide/chatgpt.svg)Open in ChatGPT](https://chatgpt.com/?q=Read%20https%3A%2F%2Fdocs.banuba.com%2Ffar-sdk%2Ftutorials%2Fdevelopment%2Fbasic_integration%2Fflutter.md%20\(flutter\)%20and%20help%20me%20with%20it%20in%20my%20Banuba%20Face%20AR%20SDK%20project.)[![](/far-sdk/img/ai-guide/claude.svg)Open in Claude](https://claude.ai/new?q=Read%20https%3A%2F%2Fdocs.banuba.com%2Ffar-sdk%2Ftutorials%2Fdevelopment%2Fbasic_integration%2Fflutter.md%20\(flutter\)%20and%20help%20me%20with%20it%20in%20my%20Banuba%20Face%20AR%20SDK%20project.)Install tools [Banuba SDK](https://pub.dev/packages/banuba_sdk) for [Flutter](https://docs.flutter.dev/) is available for iOS and Android and provides the following functionality: * Load and interact with any Effect (including the `Makeup` effect). * Still image processing. * Interaction with Camera (open/close, take photo, flashlight control, facing). * Screen recording. * [Videocall](/far-sdk/tutorials/development/videocall.md) powered by [Agora](https://docs.agora.io/). If this is not enough, you should go with native integration. ## Integration guide[​](#integration-guide "Direct link to Integration guide") * Android * iOS 1. Add `banuba_sdk` plugin: ``` flutter pub add banuba_sdk ``` 2. Add code from [the basic sample](https://github.com/Banuba/banuba-sdk-flutter/blob/master/example/lib/main.dart) into your app. Don't forget to `initialize` `BanubaSdkManager` with the Client Token: example/lib/main.dart ``` loading... ``` ![Icon](/far-sdk/img/icons/github.svg "GitHub") 3. Add `effects` folder into your project. Link it with your app: add the following code into app `build.gradle`. example/android/app/build.gradle ``` loading... ``` ![Icon](/far-sdk/img/icons/github.svg "GitHub") 1. Add `banuba_sdk` plugin: ``` flutter pub add banuba_sdk ``` 2. Link to **Banuba SDK** podspecs in `ios/Podfile`: ``` source 'https://github.com/sdk-banuba/banuba-sdk-podspecs.git' ``` 3. Add code from [the basic sample](https://github.com/Banuba/banuba-sdk-flutter/blob/master/example/lib/main.dart) into your app. Don't forget to `initialize` `BanubaSdkManager` with the Client Token: example/lib/main.dart ``` loading... ``` ![Icon](/far-sdk/img/icons/github.svg "GitHub") 4. Add `effects` folder into your project. Link it with your app: add the
folder into `Runner` **Xcode** project (`File` -> `Add Files to 'Runner'...`). --- # ios [View as Markdown](https://docs.banuba.com/far-sdk/tutorials/development/basic_integration/ios.md)[![](/far-sdk/img/ai-guide/chatgpt.svg)Open in ChatGPT](https://chatgpt.com/?q=Read%20https%3A%2F%2Fdocs.banuba.com%2Ffar-sdk%2Ftutorials%2Fdevelopment%2Fbasic_integration%2Fios.md%20\(ios\)%20and%20help%20me%20with%20it%20in%20my%20Banuba%20Face%20AR%20SDK%20project.)[![](/far-sdk/img/ai-guide/claude.svg)Open in Claude](https://claude.ai/new?q=Read%20https%3A%2F%2Fdocs.banuba.com%2Ffar-sdk%2Ftutorials%2Fdevelopment%2Fbasic_integration%2Fios.md%20\(ios\)%20and%20help%20me%20with%20it%20in%20my%20Banuba%20Face%20AR%20SDK%20project.)Install tools ## Installation[​](#installation "Direct link to Installation") 1. Add [BanubaSdk SPM packages](/far-sdk/tutorials/development/installation.md?ios-packages=spm#spm-packages) into your project info Details about the **SPM** and **CocoaPods** packages see in [Installation](/far-sdk/tutorials/development/installation.md). ## Integration[​](#integration "Direct link to Integration") 1. Setup `banubaClientToken` common/common/AppDelegate.swift ``` loading... ``` ![Icon](/far-sdk/img/icons/github.svg "GitHub") 2. Initialize `BanubaSdkManager` common/common/AppDelegate.swift ``` loading... ``` ![Icon](/far-sdk/img/icons/github.svg "GitHub") 3. Create `Player` and `load` the effect camera/camera/ViewController.swift ``` loading... ``` ![Icon](/far-sdk/img/icons/github.svg "GitHub") 4. Run the application! πŸŽ‰ πŸš€ πŸ’… tip See more use cases and code samples in the [GitHub repo](https://github.com/Banuba/banuba-sdk-ios-samples). --- # react\_native [View as Markdown](https://docs.banuba.com/far-sdk/tutorials/development/basic_integration/react_native.md)[![](/far-sdk/img/ai-guide/chatgpt.svg)Open in ChatGPT](https://chatgpt.com/?q=Read%20https%3A%2F%2Fdocs.banuba.com%2Ffar-sdk%2Ftutorials%2Fdevelopment%2Fbasic_integration%2Freact_native.md%20\(react_native\)%20and%20help%20me%20with%20it%20in%20my%20Banuba%20Face%20AR%20SDK%20project.)[![](/far-sdk/img/ai-guide/claude.svg)Open in Claude](https://claude.ai/new?q=Read%20https%3A%2F%2Fdocs.banuba.com%2Ffar-sdk%2Ftutorials%2Fdevelopment%2Fbasic_integration%2Freact_native.md%20\(react_native\)%20and%20help%20me%20with%20it%20in%20my%20Banuba%20Face%20AR%20SDK%20project.)Install tools [Banuba SDK](https://www.npmjs.com/package/@banuba/react-native) for [React Native](https://reactnative.dev/) available for iOS and Android and provides the following functionality: * Load and interact with any Effect (including `Makeup` effect). * Interaction with camera (open/close). * Screen recording (screenshots and video). * [Videocall](/far-sdk/tutorials/development/videocall.md) powered by [Agora](https://docs.agora.io/). If this is not enough, you should go with native integration. ## Integration guide[​](#integration-guide "Direct link to Integration guide") * Android * iOS 1. Add `@banuba/react-native` dependency ``` yarn add @banuba/react-native ``` 2. Add our **Maven repository** example/android/build.gradle ``` loading... ``` ![Icon](/far-sdk/img/icons/github.svg "GitHub") 3. Add [`effects` folder](https://github.com/Banuba/banuba-sdk-react-native/tree/master/example/effects) and add a task to copy them into app example/android/app/build.gradle ``` loading... ``` ![Icon](/far-sdk/img/icons/github.svg "GitHub") 4. Copy code from [this file](https://github.com/Banuba/banuba-sdk-react-native/blob/master/example/src/App.tsx) into your app. Don't forget to intialize the SDK with the Client Token example/src/App.tsx ``` loading... ``` ![Icon](/far-sdk/img/icons/github.svg "GitHub") 1. Add `@banuba/react-native` dependency ``` yarn add @banuba/react-native ``` 2. Add our podspecs repo to your `Podfile` example/ios/Podfile ``` loading... ``` ![Icon](/far-sdk/img/icons/github.svg "GitHub") 3. Add [`effects` folder](https://github.com/Banuba/banuba-sdk-react-native/tree/master/example/effects) and link it to Xcode project: `(File -> Add Files to ...)` 4. Copy code from [this file](https://github.com/Banuba/banuba-sdk-react-native/blob/master/example/src/App.tsx) into your app. Don't forget to intialize the SDK with the Client Token example/src/App.tsx ``` loading... ``` ![Icon](/far-sdk/img/icons/github.svg "GitHub") --- # web [View as Markdown](https://docs.banuba.com/far-sdk/tutorials/development/basic_integration/web.md)[![](/far-sdk/img/ai-guide/chatgpt.svg)Open in ChatGPT](https://chatgpt.com/?q=Read%20https%3A%2F%2Fdocs.banuba.com%2Ffar-sdk%2Ftutorials%2Fdevelopment%2Fbasic_integration%2Fweb.md%20\(web\)%20and%20help%20me%20with%20it%20in%20my%20Banuba%20Face%20AR%20SDK%20project.)[![](/far-sdk/img/ai-guide/claude.svg)Open in Claude](https://claude.ai/new?q=Read%20https%3A%2F%2Fdocs.banuba.com%2Ffar-sdk%2Ftutorials%2Fdevelopment%2Fbasic_integration%2Fweb.md%20\(web\)%20and%20help%20me%20with%20it%20in%20my%20Banuba%20Face%20AR%20SDK%20project.)Install tools ## Requirements[​](#requirements "Direct link to Requirements") * [Nodejs](https://nodejs.org/en/) installed * Browser with [WebGL 2.0](https://caniuse.com/#feat=webgl2) and higher ## Integration[​](#integration "Direct link to Integration") 1. Setup the client token BanubaClientToken.js ``` loading... ``` ![Icon](/far-sdk/img/icons/github.svg "GitHub") 2. Import the required types from the [@banuba/webar](https://www.npmjs.com/package/@banuba/webar) NPM package BanubaPlayer.js ``` loading... ``` ![Icon](/far-sdk/img/icons/github.svg "GitHub") info See the details about the NPM package in [Installation](/far-sdk/tutorials/development/installation.md). 3. Initialize `Player` and apply the `Effect` BanubaPlayer.js ``` loading... ``` ![Icon](/far-sdk/img/icons/github.svg "GitHub") BanubaPlayer.js ``` loading... ``` ![Icon](/far-sdk/img/icons/github.svg "GitHub") BanubaPlayer.js ``` loading... ``` ![Icon](/far-sdk/img/icons/github.svg "GitHub") 4. Run a local web server from a terminal inside the folder ``` npx live-server ``` 5. Open [localhost:8080](http://localhost:8080) and start clicking πŸŽ‰ πŸš€ πŸ’… tip Follow the instructions of the demo app [README.md](https://github.com/Banuba/quickstart-web/blob/master/README.md) to get more info. note The demo app leverages [jsDelivr](https://jsdelivr.com/) CDN for ease of getting started, for a real life application please use the [@banuba/webar](https://www.npmjs.com/package/@banuba/webar) npm package. --- # AR Cloud Guide [View as Markdown](https://docs.banuba.com/far-sdk/tutorials/development/guides/ar_cloud.md)[![](/far-sdk/img/ai-guide/chatgpt.svg)Open in ChatGPT](https://chatgpt.com/?q=Read%20https%3A%2F%2Fdocs.banuba.com%2Ffar-sdk%2Ftutorials%2Fdevelopment%2Fguides%2Far_cloud.md%20\(AR%20Cloud%20Guide\)%20and%20help%20me%20with%20it%20in%20my%20Banuba%20Face%20AR%20SDK%20project.)[![](/far-sdk/img/ai-guide/claude.svg)Open in Claude](https://claude.ai/new?q=Read%20https%3A%2F%2Fdocs.banuba.com%2Ffar-sdk%2Ftutorials%2Fdevelopment%2Fguides%2Far_cloud.md%20\(AR%20Cloud%20Guide\)%20and%20help%20me%20with%20it%20in%20my%20Banuba%20Face%20AR%20SDK%20project.)Install tools AR Cloud is a client-server solution that helps to save space in your application. This is a product used to store AR filters on a server instead of in the SDK code. After being selected by the user for the first time, the filter is going to be downloaded from the server and then saved on the phone's memory. * iOS * Android * Flutter [Example of using AR cloud](https://github.com/Banuba/arcloud-ios-swift)

**Banuba AR Cloud SDK** delivery solution includes the `BanubaARCloudSDK.xcframework` with `BanubaUtilities.xcframework` libraries that should be placed into [Frameworks folder](https://github.com/Banuba/arcloud-ios-swift/tree/master/Frameworks) directory or added as an [SPM](https://www.swift.org/package-manager/) dependecies to your project. You can find these libraries here: [BanubaARCloudSDK.xcframework](https://github.com/Banuba/BanubaARCloudSDK-IOS), [BanubaUtilities.xcframework](https://github.com/Banuba/BanubaUtilities-iOS) ## Follow these steps to integrate AR Cloud:[​](#follow-these-steps-to-integrate-ar-cloud "Direct link to Follow these steps to integrate AR Cloud:") 1. Set up `banubaArCloudURL` arcloud-ios-swift/BanubaClientToken.swift ``` loading... ``` ![Icon](/far-sdk/img/icons/github.svg "GitHub") 2. Initialize **Banuba AR Cloud SDK** arcloud-ios-swift/ARCloud/ARCloudManager.swift ``` loading... ``` ![Icon](/far-sdk/img/icons/github.svg "GitHub") 3. Fetch the available effects list using the `getAREffects` method arcloud-ios-swift/ARCloud/ARCloudManager.swift ``` loading... ``` ![Icon](/far-sdk/img/icons/github.svg "GitHub") 4. Download the effect, using the `downloadArEffect` method arcloud-ios-swift/ARCloud/ARCloudManager.swift ``` loading... ``` ![Icon](/far-sdk/img/icons/github.svg "GitHub") [Example of using AR cloud](https://github.com/Banuba/arcloud-android-kotlin/)

## Follow these steps to configure AR Cloud:[​](#follow-these-steps-to-configure-ar-cloud "Direct link to Follow these steps to configure AR Cloud:") ## Installation of the ArCloud library[​](#installation-of-the-arcloud-library "Direct link to Installation of the ArCloud library") To start using **Banuba SDK** with **ArCloud** from GitHub Packages, add a custom maven repo to your `build.gradle`: build.gradle ``` loading... ``` ![Icon](/far-sdk/img/icons/github.svg "GitHub") Add the `ar-cloud` dependency to your build.gradle: effect\_player\_arcloud\_example/build.gradle ``` loading... ``` ![Icon](/far-sdk/img/icons/github.svg "GitHub") ### Initialization of the ArCloudKoinModule module in Koin[​](#initialization-of-the-arcloudkoinmodule-module-in-koin "Direct link to Initialization of the ArCloudKoinModule module in Koin") **Koin** - this is a framework to help you build any kind of Kotlin & Kotlin Multiplatform application, from Android mobile and Multiplatform apps to backend Ktor server applications. You can read more about **Koin** [here](https://insert-koin.io/). In this example, we use **Koin** for dependency injection. effect\_player\_arcloud\_example/src/main/java/com/banuba/sdk/example/effect\_player\_arcloud\_example/Application.kt ``` loading... ``` ![Icon](/far-sdk/img/icons/github.svg "GitHub") note `ArCloudKoinModule` is the AR Cloud module which should be initialized and placed before `MainKoinModule`. `MainKoinModule` is the **Koin** module which should be implemented in your application. It is required for configuring AR Cloud dependencies. ### Configuring of AR Cloud dependencies in DI layer[​](#configuring-of-ar-cloud-dependencies-in-di-layer "Direct link to Configuring of AR Cloud dependencies in DI layer") [MainKoinModule.kt](https://github.com/Banuba/arcloud-android-kotlin/blob/master/effect_player_arcloud_example/src/main/java/com/banuba/sdk/example/effect_player_arcloud_example/arcloud/MainKoinModule.kt) effect\_player\_arcloud\_example/src/main/java/com/banuba/sdk/example/effect\_player\_arcloud\_example/Application.kt ``` loading... ``` ![Icon](/far-sdk/img/icons/github.svg "GitHub") These are additional important classes: * [ArCloudMasksActivity](https://github.com/Banuba/arcloud-android-kotlin/blob/master/effect_player_arcloud_example/src/main/java/com/banuba/sdk/example/effect_player_arcloud_example/arcloud/ArCloudMasksActivity.kt) - this is the main UI module that configures `Player` with dependent UI components. * [EffectWrapper](https://github.com/Banuba/arcloud-android-kotlin/blob/master/effect_player_arcloud_example/src/main/java/com/banuba/sdk/example/effect_player_arcloud_example/arcloud/EffectWrapper.kt) - this is a data class that wraps an effect taken from an **AR cloud**. * [EffectsAdapter](https://github.com/Banuba/arcloud-android-kotlin/blob/master/effect_player_arcloud_example/src/main/java/com/banuba/sdk/example/effect_player_arcloud_example/arcloud/EffectsAdapter.kt) - this is an adapter for the RecyclerView component used to populate effects data. * [EffectsViewModel](https://github.com/Banuba/arcloud-android-kotlin/blob/master/effect_player_arcloud_example/src/main/java/com/banuba/sdk/example/effect_player_arcloud_example/arcloud/EffectsViewModel.kt) - the [ViewModel](https://developer.android.com/topic/libraries/architecture/viewmodel) component which is responsible for loading and providing the effect data. You can use all classes mentioned above as examples or implement your own solution. Usage of this feature is described on the [ARCloud plugin page on Flutter Pub](https://pub.dev/packages/banuba_arcloud). Real **examples** can be found in [quickstart-flutter-plugin](https://github.com/Banuba/quickstart-flutter-plugin/blob/master/lib/page_arcloud.dart) and the source code of **Banuba** [arcloud-flutter](https://github.com/Banuba/arcloud-flutter/tree/master/example). Still have questions about FaceAR SDK? Visit our [FAQ](https://www.banuba.com/faq/) or [contact our support](/far-sdk/support/.md). --- # Face Landmarks Guide [View as Markdown](https://docs.banuba.com/far-sdk/tutorials/development/guides/landmarks.md)[![](/far-sdk/img/ai-guide/chatgpt.svg)Open in ChatGPT](https://chatgpt.com/?q=Read%20https%3A%2F%2Fdocs.banuba.com%2Ffar-sdk%2Ftutorials%2Fdevelopment%2Fguides%2Flandmarks.md%20\(Face%20Landmarks%20Guide\)%20and%20help%20me%20with%20it%20in%20my%20Banuba%20Face%20AR%20SDK%20project.)[![](/far-sdk/img/ai-guide/claude.svg)Open in Claude](https://claude.ai/new?q=Read%20https%3A%2F%2Fdocs.banuba.com%2Ffar-sdk%2Ftutorials%2Fdevelopment%2Fguides%2Flandmarks.md%20\(Face%20Landmarks%20Guide\)%20and%20help%20me%20with%20it%20in%20my%20Banuba%20Face%20AR%20SDK%20project.)Install tools Landmarks are anchor points that show the relative position and shape of the main elements of the face. **Banuba SDK** provides the coordinates of the landmarks. To get them, follow these steps. * iOS * Android * Web 1. Import the `BanubaEffectPlayer` framework into your project. ``` import BanubaEffectPlayer ``` 2. Add `BNBFrameDataListener` to your ViewController ``` player.effectPlayer?.add(self as BNBFrameDataListener) ``` note Remove `BNBFrameDataListener` when your ViewController is deinited ``` player.effectPlayer?.remove(self as BNBFrameDataListener) ``` 3. Inherit the `BNBFrameDataListener` interface and add protocol stubs ``` extension ViewController: BNBFrameDataListener { func onFrameDataProcessed(_ frameData: BNBFrameData?) { } } ``` 4. You can get information about the coordinates of the landmarks from the `BNBFrameData`. The `getLandmarks`function returns an array of `NSNumber` with size of `2 * (landmarks number)`. The first value responds to the *X* coord of the first landmark and the second value responds to the *Y* coord of the first landmark and so on. At first, these coordinates will be in the FRX (Camera) system, but you can transform them to the Screen system. You must set the variables `screenWidth` and `screenHeight` to the width and height of your screen beforehead. After the transformation, you can get an array of points in the Screen coordinates in correspondence with the landmarks. In this example it is the `landmarksPoints` array. ``` extension ViewController: BNBFrameDataListener { func onFrameDataProcessed(_ frameData: BNBFrameData?) { guard let fD = frameData else { return } let recognitionResult = fD.getFrxRecognitionResult() let faces = recognitionResult?.getFaces() let landmarksCoordinates = faces?[0].getLandmarks() guard let landmarks = landmarksCoordinates else { return } if landmarks.count != 0 { // get transformation from FRX (Camera) coordinates to Screen coordinates let screenRect = BNBPixelRect(x:0, y:0, w:screenWidth, h:screenHeight) guard let frxResultTransformation = recognitionResult?.getTransform(), let commonToFrxResult = BNBTransformation.makeData(frxResultTransformation.basisTransform), let commonRect = commonToFrxResult.inverseJ()?.transform(frxResultTransformation.fullRoi), let commonToScreen = BNBTransformation.makeRects(commonRect, targetRect: screenRect, rot: BNBRotation.deg0, flipX: false, flipY: false), let FrxResultToCommon = commonToFrxResult.inverseJ(), let frxResultToScreen = FrxResultToCommon.chainRight(commonToScreen) else { return } //create points from transformed coordinates var landmarksPoints: [CGPoint] = [] for i in 0 ..< (landmarks.count / 2) { let xCoord = Float(truncating: landmarks[i * 2]) let yCoord = Float(truncating: landmarks[i * 2 + 1]) let pointBeforeTransformation = BNBPoint2d(x: xCoord, y: yCoord) let pointAfterTransformation = frxResultToScreen.transformPoint(pointBeforeTransformation) landmarksPoints.append(CGPoint(x: CGFloat(pointAfterTransformation.x), y: CGFloat(pointAfterTransformation.y))) } } } } ``` 1. Import the following files into your project: ``` import com.banuba.sdk.effect_player.FrameDataListener import com.banuba.sdk.player.Player import com.banuba.sdk.types.FrxRecognitionResult import com.banuba.sdk.types.TransformableEvent import com.banuba.sdk.types.Transformation import com.banuba.sdk.types.FrameData import com.banuba.sdk.types.FaceData import com.banuba.sdk.types.PixelRect import com.banuba.sdk.types.Point2d import com.banuba.sdk.types.Rotation ``` 2. Add `FrameDataListener` to your `ViewController`: ``` player.effectPlayer.addFrameDataListener(this) ``` And don't forget to remove the `FrameDataListener` when your `ViewController` is destroyed: ``` player.effectPlayer.removeFrameDataListener(this) ``` 3. Inherit the `FrameDataListener` interface and override the `onFrameDataProcessed(...)` function: ``` class ViewController : FrameDataListener { override fun onFrameDataProcessed(frameData: FrameData?) { ... } } ``` 4. From FrameData you can get the information about the coordinates of the landmarks. The function `getLandmarks()` returns an array of `ArrayList` with a size of 2 \* (landmarks number). The first value corresponds to the X coord of the first landmark, the second value corresponds to the Y coord of the first landmark, and so on. At first, these coordinates are in the FRX camera space, but you can transform them into the screen space. You must set the variables `mScreenWidth` and `mScreenHeight` to the width and height of your screen beforehand. After the transformation, you can get an array of points in the Screen coordinates in correspondence with the landmarks. In this example, it is the `mLandmarksPoints` array. ``` class ViewController : FrameDataListener { val screenWidth = 720 /* input screen width */ val screenHeight = 1280 /* input screen height */ /* note: mLandmarksPoints - this variable is updated each frame in asynchronous mode. * To access it from another thread, adding synchronization is required. */ val landmarksPoints: ArrayList = ArrayList() /* output landmarks points */ override fun onFrameDataProcessed(frameData: FrameData?) { frameData ?: return val recognitionResult = frameData.frxRecognitionResult ?: return val faces = if (!recognitionResult.faces.isEmpty()) recognitionResult.faces else return /* note: The example only uses the first face data, but the SDK can recognize * and receive data from several faces. */ val landmarks = if (!faces[0].landmarks.isEmpty()) faces[0].landmarks else return /* Create transformation for landmarks */ val screenRect = PixelRect(0, 0, screenWidth, screenHeight) val frxResultTransformation = recognitionResult.transform val commonToFrxResult = Transformation.makeData(frxResultTransformation.basisTransform)!! val commonRect = commonToFrxResult.inverseJ()!!.transformRect(frxResultTransformation.fullRoi) val commonToScreen = Transformation.makeRects(commonRect, screenRect, Rotation.DEG_0, false, false) val frxResultToCommon = commonToFrxResult.inverseJ()!! val frxResultToScreen = frxResultToCommon.chainRight(commonToScreen)!! /* Create points from transformed coordinates */ val countPoints = landmarks.size / 2 for (i in 0..countPoints) { val xCoord = landmarks[i * 2] val yCoord = landmarks[i * 2 + 1] val pointBeforeTransformation = Point2d(xCoord, yCoord) val pointAfterTransformation = frxResultToScreen.transformPoint(pointBeforeTransformation) landmarksPoints.add(pointBeforeTransformation) } } } ``` It is possible to retrieve face landmarks from the face recognition performed by SDK: ``` player.addEventListener(Player.FRAME_DATA_EVENT, ({ detail: frameData }) => { const hasFace = frameData.get("frxRecognitionResult.faces.0.hasFace") if (!hasFace) return console.log("Face not found") const landmarks = frameData.get("frxRecognitionResult.faces.0.landmarks") console.log("Landmarks:", landmarks) }) ``` The landmarks array stores flattened pairs of 68 points in form of `[x1, y1, x2, y2, ... , x68, y68]`. warning Pay attention that an effect with [Face Recognition](/far-sdk/tutorials/capabilities/glossary.md#frx-face-tracking) (e.g. [DebugWireframe](/far-sdk/generated/effects/DebugWireframe.zip)) must be applied to the player. See the [FRAME\_DATA\_EVENT](/far-sdk/generated/typedoc/classes/Player.html#FRAME_DATA_EVENT) and [FrameData](/far-sdk/generated/typedoc/classes/FrameData.html#get) docs for more details and examples. Now you can use the face landmarks in the your app. For example, you can display them like in the picture below. ![image](/far-sdk/assets/images/landmarks_68-938993fb47df6b72e725c2acf71386eb.png) Still have questions about FaceAR SDK? Visit our [FAQ](https://www.banuba.com/faq/) or [contact our support](/far-sdk/support/.md). --- # Migration Guides [View as Markdown](https://docs.banuba.com/far-sdk/tutorials/development/guides/migration.md)[![](/far-sdk/img/ai-guide/chatgpt.svg)Open in ChatGPT](https://chatgpt.com/?q=Read%20https%3A%2F%2Fdocs.banuba.com%2Ffar-sdk%2Ftutorials%2Fdevelopment%2Fguides%2Fmigration.md%20\(Migration%20Guides\)%20and%20help%20me%20with%20it%20in%20my%20Banuba%20Face%20AR%20SDK%20project.)[![](/far-sdk/img/ai-guide/claude.svg)Open in Claude](https://claude.ai/new?q=Read%20https%3A%2F%2Fdocs.banuba.com%2Ffar-sdk%2Ftutorials%2Fdevelopment%2Fguides%2Fmigration.md%20\(Migration%20Guides\)%20and%20help%20me%20with%20it%20in%20my%20Banuba%20Face%20AR%20SDK%20project.)Install tools ## To version 1.17.0[​](#to-version-1170 "Direct link to To version 1.17.0") `RenderBackendType`(`render_backend_type`) was moved to `types` package. So, now for: ### Android[​](#android "Direct link to Android") Change `com.banuba.sdk.scene.RenderBackendType` to `com.banuba.sdk.types.RenderBackendType`. ### C++[​](#c "Direct link to C++") Change `#include ` to `#include ` ## To version 1.9.0[​](#to-version-190 "Direct link to To version 1.9.0") BanubaSDK introduces the `Player` API for iOS and Android, which implements the most popular use cases and is highly customizable. We continue to support the old API, but starting from this version it is marked as deprecated. The main changes are described below. ### iOS & Android[​](#ios--android "Direct link to iOS & Android") * The class `BanubaSdkManager` deprecated now. Static methods for initialization/deinitialization of the Banuba SDK with the client token and resources path still work. But we are suggest to switch to `BNBUtilityManager` for the SDK initialization instead. * The class `Player` introduced as a replacement for the `BanubaSdkManager`. Now it is the only way to process frames from the `Input`, manage effect playback and present them to the `Output`. * The protocol `Input` with basic use cases implementations: `Camera`, `Photo`, `Stream`; provides frames to the `Player` for further processing and rendering. * The protocol `Output` implements endpoint of the presentation surface, which may be `View`, `PixelBuffer`, `Video`, or any other surface, implemented by own. You can have several outputs in use at a time! * All the three main protocols can be connected between each other through the `player.use(input, outputs)` method call. More details about new `Player` API you can find in our [github examples](/far-sdk/tutorials/development/samples.md). --- # Optimization Guides [View as Markdown](https://docs.banuba.com/far-sdk/tutorials/development/guides/optimization.md)[![](/far-sdk/img/ai-guide/chatgpt.svg)Open in ChatGPT](https://chatgpt.com/?q=Read%20https%3A%2F%2Fdocs.banuba.com%2Ffar-sdk%2Ftutorials%2Fdevelopment%2Fguides%2Foptimization.md%20\(Optimization%20Guides\)%20and%20help%20me%20with%20it%20in%20my%20Banuba%20Face%20AR%20SDK%20project.)[![](/far-sdk/img/ai-guide/claude.svg)Open in Claude](https://claude.ai/new?q=Read%20https%3A%2F%2Fdocs.banuba.com%2Ffar-sdk%2Ftutorials%2Fdevelopment%2Fguides%2Foptimization.md%20\(Optimization%20Guides\)%20and%20help%20me%20with%20it%20in%20my%20Banuba%20Face%20AR%20SDK%20project.)Install tools * Web ## Optimizing WebAR SDK bundle size[​](#optimizing-webar-sdk-bundle-size "Direct link to Optimizing WebAR SDK bundle size") **Banuba WebAR SDK** is [tree-shakable](https://developer.mozilla.org/en-US/docs/Glossary/Tree_shaking), so import only the modules your application relies on: ``` // the named import saves extra KBs import { Webcam, Player, Effect, Dom } from "@banuba/webar" // ... ``` ## Optimizing WebAR SDK assets size[​](#optimizing-webar-sdk-assets-size "Direct link to Optimizing WebAR SDK assets size") `BanubaSDK.wasm` and `BanubaSDK.simd.wasm` are the heavy ones. But they have a good compressability due to the internal format of the files. | Asset | Original | Gzip | Brotli | | ------------------- | -------- | ----- | ------ | | BanubaSDK.wasm | 12Mb | 3.5Mb | 2.5Mb | | BanubaSDK.simd.wasm | 13Mb | 3.8Mb | 2.7Mb | Most hosting environments like [Netlify](https://www.netlify.com/) automatically precompress the files, but sometimes you may have to compress them yourself for better downloading times. You can run the command in the assets folder to get them compressed: ``` npx gzipper compress --brotli . ``` See [gzipper docs](https://www.npmjs.com/package/gzipper#compressc-1) for details. ## Speed up WebAR SDK on modern browsers[​](#speed-up-webar-sdk-on-modern-browsers "Direct link to Speed up WebAR SDK on modern browsers") Banuba WebAR SDK ships with the `BanubaSDK.simd.wasm` file - the SIMD version of the `BanubaSDK.wasm`. Without digging into the details of what SIMD is, the SIMD-enabled file can make processing performance up to several times faster. Taking into consideration that SIMD has [a good support across modern browsers](https://webassembly.org/roadmap/), you should definitely give it a try. SIMD support detection is built into the WebAR SDK. It means the SDK will try to load `BanubaSDK.simd.wasm` if the current browser supports SIMD and will load `BanubaSDK.wasm` otherwise. Don't forget to point BanubaSDK to the SIMD file location if you are using `locateFile`: ``` const player = await Player.create({ clientToken: "xxx-xxx-xxx", // point BanubaSDK where to find these vital files locateFile: { "BanubaSDK.data": "/path/to/BanubaSDK.data", "BanubaSDK.wasm": "/path/to/BanubaSDK.wasm", "BanubaSDK.simd.wasm": "/path/to/BanubaSDK.simd.wasm", }, }) ``` See [Player.create()](/far-sdk/generated/typedoc/classes/Player.html#create) and [locateFile](/far-sdk/generated/typedoc/types/SDKOptions.html) for details. ## Reducing CPU/GPU usage on HiDPI devices[​](#reducing-cpugpu-usage-on-hidpi-devices "Direct link to Reducing CPU/GPU usage on HiDPI devices") On HiDPI devices Banuba WebAR SDK scales the output frames by the [device pixel ratio](https://developer.mozilla.org/en-US/docs/Web/API/Window/devicePixelRatio). This approach allows the SDK to render face AR 3D Masks in a high quality and keep all the AR 3D Mask details. Despite the better rendering quality, this approach utilizes more CPU and GPU resources since the frame size to be processed scales geometrically. One can simply opt out of the default behavior and reduce CPU/GPU utilization by overriding the `devicePixelRatio` used by the SDK: ``` const player = await Player.create({ clientToken: "xxx-xxx-xxx", devicePixelRatio: 1, }) ``` See the [Player.create()](/far-sdk/generated/typedoc/classes/Player.html#create) method docs for more details. The CPU and GPU usage can be reduced even more by processing frames of smaller size, e.g the 640x480 webcam frame size can be used instead of the default 1280x720 frame size: ``` await player.use(new Webcam({ width: 640, height: 480 })) ``` Check out the [Video cropping](/far-sdk/tutorials/development/samples.md#video-cropping) sample for more details. ## Preloading Effects[​](#preloading-effects "Direct link to Preloading Effects") Sometimes you may experience a time lag between [player.applyEffect()](/far-sdk/generated/typedoc/classes/Player.html#applyEffect) call and a visual change due to the long time of the effect archive download. To speed up things, you can preload the Effect and apply it later on demand: ``` const preloaded = await Effect.preload("SomeBigEffect.zip")) // ... player.applyEffect(preloaded) ``` You can also scale the approach and add a local cache of the preloaded effects, or preload an effect on some user interaction like mouse hover a button. Still have questions about FaceAR SDK? Visit our [FAQ](https://www.banuba.com/faq/) or [contact our support](/far-sdk/support/.md). --- # Watermark Guide [View as Markdown](https://docs.banuba.com/far-sdk/tutorials/development/guides/watermark.md)[![](/far-sdk/img/ai-guide/chatgpt.svg)Open in ChatGPT](https://chatgpt.com/?q=Read%20https%3A%2F%2Fdocs.banuba.com%2Ffar-sdk%2Ftutorials%2Fdevelopment%2Fguides%2Fwatermark.md%20\(Watermark%20Guide\)%20and%20help%20me%20with%20it%20in%20my%20Banuba%20Face%20AR%20SDK%20project.)[![](/far-sdk/img/ai-guide/claude.svg)Open in Claude](https://claude.ai/new?q=Read%20https%3A%2F%2Fdocs.banuba.com%2Ffar-sdk%2Ftutorials%2Fdevelopment%2Fguides%2Fwatermark.md%20\(Watermark%20Guide\)%20and%20help%20me%20with%20it%20in%20my%20Banuba%20Face%20AR%20SDK%20project.)Install tools A watermark is a small image that is superimposed on top of the entire video. * iOS * Android If you want to apply a watermark to a recorded video, you should use the `video.watermark` property. ``` guard let watermark = UIImage(named: "YOUR_WATERMARK_IMAGE") else { return } let offset = CGPoint(x: 20.0, y: 20.0) let watermarkInfo = WatermarkInfo(image: watermark, corner: .bottomLeft, offset: offset, targetNormalizedWidth: 0.2) let video = Video(cameraDevice: cameraDevice) player.use(input: camera, outputs: [playerView.uiView, video]) video.watermark = watermarkInfo video.record(...) ``` If you want to apply a watermark to a video, you should create a `WatermarkInfo` structure: ``` val watermark: Drawable = ContextCompat.getDrawable(this, R.drawable.my_watermark_res)!! val width = 101 val height = 24 val aspectRatio = width.toFloat() / height.toFloat() val sizeProvider = { viewportSize: Size -> val targetWidth = (viewportSize.width * 0.5f).toInt() val targetHeight = (targetWidth / aspectRatio).toInt() Size(targetWidth, targetHeight) } val watermarkGravity = Gravity.BOTTOM // or Gravity.RIGHT val positionProvider = GravityPositionProviderAdapter(sizeProvider, watermarkGravity) val myWatermarkInfo = WatermarkInfo(watermark, sizeProvider, positionProvider, width, height, true) ``` Apply the watermark in the `VideoOutput.start` method: ``` videoOutput.start(..., myWatermarkInfo) ``` Still have questions about FaceAR SDK? Visit our [FAQ](https://www.banuba.com/faq/) or [contact our support](/far-sdk/support/.md). --- # Adding Banuba SDK to your project [View as Markdown](https://docs.banuba.com/far-sdk/tutorials/development/installation.md)[![](/far-sdk/img/ai-guide/chatgpt.svg)Open in ChatGPT](https://chatgpt.com/?q=Read%20https%3A%2F%2Fdocs.banuba.com%2Ffar-sdk%2Ftutorials%2Fdevelopment%2Finstallation.md%20\(Adding%20Banuba%20SDK%20to%20your%20project\)%20and%20help%20me%20with%20it%20in%20my%20Banuba%20Face%20AR%20SDK%20project.)[![](/far-sdk/img/ai-guide/claude.svg)Open in Claude](https://claude.ai/new?q=Read%20https%3A%2F%2Fdocs.banuba.com%2Ffar-sdk%2Ftutorials%2Fdevelopment%2Finstallation.md%20\(Adding%20Banuba%20SDK%20to%20your%20project\)%20and%20help%20me%20with%20it%20in%20my%20Banuba%20Face%20AR%20SDK%20project.)Install tools * iOS * Android * Web * Desktop - CocoaPods - Swift Package Manager ## CocoaPods packages[​](#cocoapods-packages "Direct link to CocoaPods packages") To start using **Banuba SDK** with [**CocoaPods**](https://guides.cocoapods.org/using/using-cocoapods.html), add a **custom repository** and desired **packages** to your `Podfile`: Podfile ``` loading... ``` ![Icon](/far-sdk/img/icons/github.svg "GitHub") Then install pods: ``` pod install --repo-update ``` See [banuba-sdk-podspecs](https://github.com/sdk-banuba/banuba-sdk-podspecs) repo for the list of all available packages and versions. The complete example, configured for **CocoaPods** usage, can be found in our [Objective-C](https://github.com/Banuba/quickstart-ios-objc) quickstart example. ## SPM packages[​](#spm-packages "Direct link to SPM packages") **Banuba SDK** provides [Swift Package Manager](https://www.swift.org/package-manager/) packages in the custom repositories. Add **Banuba SDK** packages, to your Xcode project: 1. **File** > **Add Package Dependencies...** 2. Search for the package, for example 3. Press the **Add Package** button. After verifying the package, press **Add Package** again. See the [sdk-banuba](https://github.com/sdk-banuba?tab=repositories\&q=swift+package\&type=\&language=\&sort=) repositories for the list of all available packages and versions. The complete example, configured for **SPM** usage, can be found in our [Beauty-iOS](https://github.com/Banuba/banuba-sdk-ios-samples) quickstart example. ## How to choose required packages[​](#how-to-choose-required-packages "Direct link to How to choose required packages") warning Only use the packages with the same version! Packages with different versions (even minor) may conflict or work incorrectly with each other. tip If **feature** or **effect** works incorrect, see application **logs**, to figure out which package is missed. Add packages depends on the specific **features** or **effects**, which your app will use. It is your responsibility to include everything required for the desired behaviour.
See detailed [packages description](#list-of-all-available-packages) in the table below. Example of the packages set for [Face Tracking](/far-sdk/tutorials/capabilities/glossary.md#frx-face-tracking) and [Background Separation](/far-sdk/tutorials/capabilities/glossary.md#background-separation): * BNBSdkApi * BNBFaceTracker * BNBBackground See detailed [packages description](#list-of-all-available-packages) in the table below. ## List of all available packages[​](#list-of-all-available-packages "Direct link to List of all available packages") | Package name | Description | | ----------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | BNBSdkApi | platform-specific API, like `Player`, `Input`, `Output`, etc | | BNBSdkCore | provides the functionality of the native `EffectPlayer`. | | BNBEffectPlayer | contains the necessary shaders, used by `sdk_core` package and provides the following features: math utilities, texture utilities, morphing, beautification, etc. | | BNBScripting | includes the basic functionality used by the effect api, see [Effects](/far-sdk/effects/overview.md). | | BNBFaceTracker | package consists of neural network models used to track face and its features: lips, eyes, etc. Include it whenever you deal with tracking. See more about [Face Tracking](/far-sdk/tutorials/capabilities/glossary.md#frx-face-tracking). | | BNBFaceAttributes | package consists of neural network models used to extract face attributes: skin color, gender, face shape etc. | | BNBFaceMatch | package provides facilities to measure how faces are similar on two photos | | BNBMakeup | provides prefabs for [Makeup](/far-sdk/effects/prefabs/makeup.md). | | BNBLips | provides neural network models for lips segmentation. See more about [Lips Segmentation](/far-sdk/tutorials/capabilities/glossary.md#lips-segmentation). | | BNBHair | provides neural network models for hair segmentation. See more about [Hair Segmentation](/far-sdk/tutorials/capabilities/glossary.md#hair-segmentation). | | BNBHands | provides neural network models for hand, nail, and finger segmentation. | | BNBEyes | provides neural network models for eyes segmentation. See more about [Eye Segmentation](/far-sdk/tutorials/capabilities/glossary.md#eye-segmentation). | | BNBSkin | provides neural network models for skin segmentation. See more about [Skin Segmentation](/far-sdk/tutorials/capabilities/glossary.md#skin-segmentation). | | BNBBackground | provides neural network models for background separation. See more about [Background Separation](/far-sdk/tutorials/capabilities/glossary.md#background-separation). | | BNBAcne | provides neural network models for acne removal. | | BNBNeck | provides neural network models for neck segmentation. | | BNBResources | includes all the resources of the all packages. **Use it when you don't care about the size or you need all the features!** | | BNBPoseEstimation | private | | BanubaSdk | depends on the all the packages for the operation of all available features. **Use it when you don't care about the size or you need all the features!** | ## Packages[​](#packages "Direct link to Packages") **Maven** To start using **Banuba SDK** , add a custom maven repo to your `build.gradle.kts`: build.gradle.kts ``` loading... ``` ![Icon](/far-sdk/img/icons/github.svg "GitHub") Refer to our [Maven](https://nexus.banuba.net/#browse/browse:maven-releases) to find the list of all available packages and their versions. ## How to choose required packages[​](#how-to-choose-required-packages "Direct link to How to choose required packages") warning Use only the packages with the same version! Packages with different versions (even minor) may conflict or work incorrectly with each other. tip If **feature** or **effect** works incorrect, see application **logs**, to figure out which package is missed. Add packages depends on the specific **features** or **effects**, which your app will use. It is your responsibility to include everything required for the desired behaviour.
See detailed [packages description](#list-of-all-available-packages) in the table below. build.gradle.kts ``` loading... ``` ![Icon](/far-sdk/img/icons/github.svg "GitHub") ## List of all available packages[​](#list-of-all-available-packages "Direct link to List of all available packages") | Package name | Description | | ------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | com.banuba.sdk.sdk\_api | platform-specific APIs, like `Player`, `Input`, `Output`, etc. | | com.banuba.sdk.sdk\_core | provides the functionality of the native `EffectPlayer`. | | com.banuba.sdk.effect\_player | contains the necessary shaders, used by `sdk_core` package and provides the following features: math utilities, texture utilities, morphing, beautification, etc. | | com.banuba.sdk.scripting | includes the basic functionality used by the effect api, see [Effects](/far-sdk/effects/overview.md). | | com.banuba.sdk.face\_tracker | consists of neural network models used to track a face and its features: lips, eyes, etc. Include it whenever you deal with tracking. See more about [FRX](/far-sdk/tutorials/capabilities/glossary.md#frx-face-tracking). | | com.banuba.sdk.face\_attributes | package consists of neural network models used to extract face attributes: skin color, gender, face shape etc. | | com.banuba.sdk.face\_match | package provides facilities to measure how faces are similar on two photos | | com.banuba.sdk.makeup | provides prefabs for [Makeup](/far-sdk/effects/prefabs/makeup.md). | | com.banuba.sdk.lips | provides neural network models for lips segmentation. See more about [Lips Segmentation](/far-sdk/tutorials/capabilities/glossary.md#lips-segmentation). | | com.banuba.sdk.hair | provides neural network models for hair segmentation. See more about [Hair Segmentation](/far-sdk/tutorials/capabilities/glossary.md#hair-segmentation). | | com.banuba.sdk.hands | provides neural network models for hand, nail, and finger segmentation. | | com.banuba.sdk.eyes | provides neural network models for eyes segmentation. See more about [Eye Segmentation](/far-sdk/tutorials/capabilities/glossary.md#eye-segmentation). | | com.banuba.sdk.skin | provides neural network models for skin segmentation. See more about [Skin Segmentation](/far-sdk/tutorials/capabilities/glossary.md#skin-segmentation). | | com.banuba.sdk.background | provides neural network models for background separation. See more about [Background Separation](/far-sdk/tutorials/capabilities/glossary.md#background-separation). | | com.banuba.sdk.acne | provides neural network models for acne removal. | | com.banuba.sdk.neck | provides neural network models for neck segmentation. | | com.banuba.sdk.banuba\_sdk\_resources | includes all the resources of the all packages. **Use it when you don't care about the size or you need all the features!** | | com.banuba.sdk.pose\_estimation | private | | com.banuba.sdk.banuba\_sdk | depends on the all the packages for the operation of all available features. **Use it when you don't care about the size or you need all the features!** | ## NPM Package[​](#npm-package "Direct link to NPM Package") **[Banuba WebAR](https://www.npmjs.com/package/@banuba/webar)** is delivered as an NPM package, which includes executables (`.js`, `.wasm`, `.simd.wasm`) and resources modules (`modules/*.zip`). ``` npm i @banuba/webar ``` ## Resources Modules[​](#resources-modules "Direct link to Resources Modules") | Module name | Description | | -------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | face\_tracker.zip | consists of neural network models used to track a face and its features: lips, eyes, etc. Include it whenever you deal with tracking. See more about [FRX](/far-sdk/tutorials/capabilities/glossary.md#frx-face-tracking). | | face\_attributes.zip | package consists of neural network models used to extract face attributes: skin color, gender, face shape etc. | | face\_match.zip | package provides facilities to measure how faces are similar on two photos | | makeup.zip | provides prefabs for [Makeup](/far-sdk/effects/prefabs/makeup.md). | | lips.zip | provides neural network models for lips segmentation. See more about [Lips Segmentation](/far-sdk/tutorials/capabilities/glossary.md#lips-segmentation). | | hair.zip | provides neural network models for hair segmentation. See more about [Hair Segmentation](/far-sdk/tutorials/capabilities/glossary.md#hair-segmentation). | | hands.zip | provides neural network models for hand, nail, and finger segmentation. | | eyes.zip | provides neural network models for eyes segmentation. See more about [Eye Segmentation](/far-sdk/tutorials/capabilities/glossary.md#eye-segmentation). | | skin.zip | provides neural network models for skin segmentation. See more about [Skin Segmentation](/far-sdk/tutorials/capabilities/glossary.md#skin-segmentation). | | background.zip | provides neural network models for background separation. See more about [Background Separation](/far-sdk/tutorials/capabilities/glossary.md#background-separation). | | acne.zip | provides neural network models for acne removal. | | neck.zip | provides neural network models for neck segmentation. | | pose\_estimation.zip | private | ## Bundlers[​](#bundlers "Direct link to Bundlers") **[Banuba WebAR](https://www.npmjs.com/package/@banuba/webar)** depends on `BanubaSDK.data` and `BanubaSDK.wasm` (or `BanubaSDK.simd.wasm` if you are targeting [SIMD](/far-sdk/tutorials/development/guides/optimization.md#speed-up-webar-sdk-on-modern-browsers)) files. By default the SDK expects these files to be accessible from the application root i.e. by the `/BanubaSDK.data`, `/BanubaSDK.wasm` `/BanubaSDK.simd.wasm` links. It must be taken into consideration when working with application bundlers like [Vite](https://vitejs.dev/), [Rollup](https://rollupjs.org/guide/en/) or [Webpack](https://webpack.js.org/). Generally speaking one should be able to put `BanubaSDK.data`, `BanubaSDK.wasm` and `BanubaSDK.simd.wasm` files into the application assets folder (usually `public/`) and get the SDK loading these files properly. But you may want to place the files somewhere else, that case the [locateFile](/far-sdk/generated/typedoc/types/SDKOptions.html) property of the [Player.create()](/far-sdk/generated/typedoc/classes/Player.html#create) method should help you to set-up SDK properly. * Vite * Rollup * Webpack ### Vite[​](#vite "Direct link to Vite") ``` import { Player, Module /* ... */ } from "@banuba/webar" // vite uses special ?url syntax to import files as URLs import data from "@banuba/webar/BanubaSDK.data?url" import wasm from "@banuba/webar/BanubaSDK.wasm?url" import simd from "@banuba/webar/BanubaSDK.simd.wasm?url" import FaceTracker from "@banuba/webar/face_tracker.zip?url" import Background from "@banuba/webar/background.zip?url" // ... const player = await Player.create({ clientToken: "xxx-xxx-xxx", // point BanubaSDK where to find these vital files locateFile: { "BanubaSDK.data": data, "BanubaSDK.wasm": wasm, "BanubaSDK.simd.wasm": simd, }, }) await player.addModule(new Module(FaceTracker), new Module(Background)) // ... ``` tip See Vite [Explicit URL imports](https://vitejs.dev/guide/assets.html#explicit-url-imports) docs for details. ### Rollup[​](#rollup "Direct link to Rollup") ``` import { Player, Module /* ... */ } from "@banuba/webar" // you need to set-up @rollup/plugin-url for the import syntax to work import data from "@banuba/webar/BanubaSDK.data" import wasm from "@banuba/webar/BanubaSDK.wasm" import simd from "@banuba/webar/BanubaSDK.simd.wasm" import FaceTracker from "@banuba/webar/face_tracker.zip" import Background from "@banuba/webar/background.zip" // ... const player = await Player.create({ clientToken: "xxx-xxx-xxx", // point BanubaSDK where to find these vital files locateFile: { "BanubaSDK.data": data, "BanubaSDK.wasm": wasm, "BanubaSDK.simd.wasm": simd, }, }) await player.addModule(new Module(FaceTracker), new Module(Background)) // ... ``` tip See [@rollup/plugin-url](https://www.npmjs.com/package/@rollup/plugin-url#include) docs for details. ### Webpack[​](#webpack "Direct link to Webpack") Depending on the version of **Webpack** used, you may have to add following rule to the `module.rules` section of the `webpack.config.js`: ``` module.exports = { module: { rules: [ // ... { test: /\.wasm$/, type: 'javascript/auto', loader: 'file-loader', }, // ... ], }, }, } ``` Now import of `.wasm` files as URLs should work properly: ``` import { Player, Module /* ... */ } from "@banuba/webar" import data from "@banuba/webar/BanubaSDK.data" import wasm from "@banuba/webar/BanubaSDK.wasm" import simd from "@banuba/webar/BanubaSDK.simd.wasm" import FaceTracker from "@banuba/webar/face_tracker.zip" import Background from "@banuba/webar/background.zip" // ... const player = await Player.create({ clientToken: "xxx-xxx-xxx", // point BanubaSDK where to find these vital files locateFile: { "BanubaSDK.data": data, "BanubaSDK.wasm": wasm, "BanubaSDK.simd.wasm": simd, }, }) await player.addModule(new Module(FaceTracker), new Module(Background)) // ... ``` info See the related [Webpack issue](https://github.com/webpack/webpack/issues/7352) for details. **Banuba SDK** for desktop platforms (i.e. **Windows** and **MacOS**) is distributed via [GitHub Releases](https://github.com/Banuba/FaceAR-SDK-desktop-releases/releases). Release archives for **Windows** are packed in `.zip` (`bnd_sdk.zip`), for **MacOS** in `.tar.gz` (`bnb_sdk.tar.gz`). Archives for **Windows** and **MacOS** contains identical C++ API, **MacOS** archive also contains **Objective-C** API identical to **iOS**. As usual, **Objecive-C** API is designed to be callable from **Swift**. Still have questions about FaceAR SDK? Visit our [FAQ](https://www.banuba.com/faq/) or [contact our support](/far-sdk/support/.md). --- # android [View as Markdown](https://docs.banuba.com/far-sdk/tutorials/development/installation/android.md)[![](/far-sdk/img/ai-guide/chatgpt.svg)Open in ChatGPT](https://chatgpt.com/?q=Read%20https%3A%2F%2Fdocs.banuba.com%2Ffar-sdk%2Ftutorials%2Fdevelopment%2Finstallation%2Fandroid.md%20\(android\)%20and%20help%20me%20with%20it%20in%20my%20Banuba%20Face%20AR%20SDK%20project.)[![](/far-sdk/img/ai-guide/claude.svg)Open in Claude](https://claude.ai/new?q=Read%20https%3A%2F%2Fdocs.banuba.com%2Ffar-sdk%2Ftutorials%2Fdevelopment%2Finstallation%2Fandroid.md%20\(android\)%20and%20help%20me%20with%20it%20in%20my%20Banuba%20Face%20AR%20SDK%20project.)Install tools ## Packages[​](#packages "Direct link to Packages") **Maven** To start using **Banuba SDK** , add a custom maven repo to your `build.gradle.kts`: build.gradle.kts ``` loading... ``` ![Icon](/far-sdk/img/icons/github.svg "GitHub") Refer to our [Maven](https://nexus.banuba.net/#browse/browse:maven-releases) to find the list of all available packages and their versions. ## How to choose required packages[​](#how-to-choose-required-packages "Direct link to How to choose required packages") warning Use only the packages with the same version! Packages with different versions (even minor) may conflict or work incorrectly with each other. tip If **feature** or **effect** works incorrect, see application **logs**, to figure out which package is missed. Add packages depends on the specific **features** or **effects**, which your app will use. It is your responsibility to include everything required for the desired behaviour.
See detailed [packages description](#list-of-all-available-packages) in the table below. build.gradle.kts ``` loading... ``` ![Icon](/far-sdk/img/icons/github.svg "GitHub") ## List of all available packages[​](#list-of-all-available-packages "Direct link to List of all available packages") | Package name | Description | | ------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | com.banuba.sdk.sdk\_api | platform-specific APIs, like `Player`, `Input`, `Output`, etc. | | com.banuba.sdk.sdk\_core | provides the functionality of the native `EffectPlayer`. | | com.banuba.sdk.effect\_player | contains the necessary shaders, used by `sdk_core` package and provides the following features: math utilities, texture utilities, morphing, beautification, etc. | | com.banuba.sdk.scripting | includes the basic functionality used by the effect api, see [Effects](/far-sdk/effects/overview.md). | | com.banuba.sdk.face\_tracker | consists of neural network models used to track a face and its features: lips, eyes, etc. Include it whenever you deal with tracking. See more about [FRX](/far-sdk/tutorials/capabilities/glossary.md#frx-face-tracking). | | com.banuba.sdk.face\_attributes | package consists of neural network models used to extract face attributes: skin color, gender, face shape etc. | | com.banuba.sdk.face\_match | package provides facilities to measure how faces are similar on two photos | | com.banuba.sdk.makeup | provides prefabs for [Makeup](/far-sdk/effects/prefabs/makeup.md). | | com.banuba.sdk.lips | provides neural network models for lips segmentation. See more about [Lips Segmentation](/far-sdk/tutorials/capabilities/glossary.md#lips-segmentation). | | com.banuba.sdk.hair | provides neural network models for hair segmentation. See more about [Hair Segmentation](/far-sdk/tutorials/capabilities/glossary.md#hair-segmentation). | | com.banuba.sdk.hands | provides neural network models for hand, nail, and finger segmentation. | | com.banuba.sdk.eyes | provides neural network models for eyes segmentation. See more about [Eye Segmentation](/far-sdk/tutorials/capabilities/glossary.md#eye-segmentation). | | com.banuba.sdk.skin | provides neural network models for skin segmentation. See more about [Skin Segmentation](/far-sdk/tutorials/capabilities/glossary.md#skin-segmentation). | | com.banuba.sdk.background | provides neural network models for background separation. See more about [Background Separation](/far-sdk/tutorials/capabilities/glossary.md#background-separation). | | com.banuba.sdk.acne | provides neural network models for acne removal. | | com.banuba.sdk.neck | provides neural network models for neck segmentation. | | com.banuba.sdk.banuba\_sdk\_resources | includes all the resources of the all packages. **Use it when you don't care about the size or you need all the features!** | | com.banuba.sdk.pose\_estimation | private | | com.banuba.sdk.banuba\_sdk | depends on the all the packages for the operation of all available features. **Use it when you don't care about the size or you need all the features!** | --- # desktop [View as Markdown](https://docs.banuba.com/far-sdk/tutorials/development/installation/desktop.md)[![](/far-sdk/img/ai-guide/chatgpt.svg)Open in ChatGPT](https://chatgpt.com/?q=Read%20https%3A%2F%2Fdocs.banuba.com%2Ffar-sdk%2Ftutorials%2Fdevelopment%2Finstallation%2Fdesktop.md%20\(desktop\)%20and%20help%20me%20with%20it%20in%20my%20Banuba%20Face%20AR%20SDK%20project.)[![](/far-sdk/img/ai-guide/claude.svg)Open in Claude](https://claude.ai/new?q=Read%20https%3A%2F%2Fdocs.banuba.com%2Ffar-sdk%2Ftutorials%2Fdevelopment%2Finstallation%2Fdesktop.md%20\(desktop\)%20and%20help%20me%20with%20it%20in%20my%20Banuba%20Face%20AR%20SDK%20project.)Install tools **Banuba SDK** for desktop platforms (i.e. **Windows** and **MacOS**) is distributed via [GitHub Releases](https://github.com/Banuba/FaceAR-SDK-desktop-releases/releases). Release archives for **Windows** are packed in `.zip` (`bnd_sdk.zip`), for **MacOS** in `.tar.gz` (`bnb_sdk.tar.gz`). Archives for **Windows** and **MacOS** contains identical C++ API, **MacOS** archive also contains **Objective-C** API identical to **iOS**. As usual, **Objecive-C** API is designed to be callable from **Swift**. --- # ios [View as Markdown](https://docs.banuba.com/far-sdk/tutorials/development/installation/ios.md)[![](/far-sdk/img/ai-guide/chatgpt.svg)Open in ChatGPT](https://chatgpt.com/?q=Read%20https%3A%2F%2Fdocs.banuba.com%2Ffar-sdk%2Ftutorials%2Fdevelopment%2Finstallation%2Fios.md%20\(ios\)%20and%20help%20me%20with%20it%20in%20my%20Banuba%20Face%20AR%20SDK%20project.)[![](/far-sdk/img/ai-guide/claude.svg)Open in Claude](https://claude.ai/new?q=Read%20https%3A%2F%2Fdocs.banuba.com%2Ffar-sdk%2Ftutorials%2Fdevelopment%2Finstallation%2Fios.md%20\(ios\)%20and%20help%20me%20with%20it%20in%20my%20Banuba%20Face%20AR%20SDK%20project.)Install tools * CocoaPods * Swift Package Manager ## CocoaPods packages[​](#cocoapods-packages "Direct link to CocoaPods packages") To start using **Banuba SDK** with [**CocoaPods**](https://guides.cocoapods.org/using/using-cocoapods.html), add a **custom repository** and desired **packages** to your `Podfile`: Podfile ``` loading... ``` ![Icon](/far-sdk/img/icons/github.svg "GitHub") Then install pods: ``` pod install --repo-update ``` See [banuba-sdk-podspecs](https://github.com/sdk-banuba/banuba-sdk-podspecs) repo for the list of all available packages and versions. The complete example, configured for **CocoaPods** usage, can be found in our [Objective-C](https://github.com/Banuba/quickstart-ios-objc) quickstart example. ## SPM packages[​](#spm-packages "Direct link to SPM packages") **Banuba SDK** provides [Swift Package Manager](https://www.swift.org/package-manager/) packages in the custom repositories. Add **Banuba SDK** packages, to your Xcode project: 1. **File** > **Add Package Dependencies...** 2. Search for the package, for example 3. Press the **Add Package** button. After verifying the package, press **Add Package** again. See the [sdk-banuba](https://github.com/sdk-banuba?tab=repositories\&q=swift+package\&type=\&language=\&sort=) repositories for the list of all available packages and versions. The complete example, configured for **SPM** usage, can be found in our [Beauty-iOS](https://github.com/Banuba/banuba-sdk-ios-samples) quickstart example. ## How to choose required packages[​](#how-to-choose-required-packages "Direct link to How to choose required packages") warning Only use the packages with the same version! Packages with different versions (even minor) may conflict or work incorrectly with each other. tip If **feature** or **effect** works incorrect, see application **logs**, to figure out which package is missed. Add packages depends on the specific **features** or **effects**, which your app will use. It is your responsibility to include everything required for the desired behaviour.
See detailed [packages description](#list-of-all-available-packages) in the table below. Example of the packages set for [Face Tracking](/far-sdk/tutorials/capabilities/glossary.md#frx-face-tracking) and [Background Separation](/far-sdk/tutorials/capabilities/glossary.md#background-separation): * BNBSdkApi * BNBFaceTracker * BNBBackground See detailed [packages description](#list-of-all-available-packages) in the table below. ## List of all available packages[​](#list-of-all-available-packages "Direct link to List of all available packages") | Package name | Description | | ----------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | BNBSdkApi | platform-specific API, like `Player`, `Input`, `Output`, etc | | BNBSdkCore | provides the functionality of the native `EffectPlayer`. | | BNBEffectPlayer | contains the necessary shaders, used by `sdk_core` package and provides the following features: math utilities, texture utilities, morphing, beautification, etc. | | BNBScripting | includes the basic functionality used by the effect api, see [Effects](/far-sdk/effects/overview.md). | | BNBFaceTracker | package consists of neural network models used to track face and its features: lips, eyes, etc. Include it whenever you deal with tracking. See more about [Face Tracking](/far-sdk/tutorials/capabilities/glossary.md#frx-face-tracking). | | BNBFaceAttributes | package consists of neural network models used to extract face attributes: skin color, gender, face shape etc. | | BNBFaceMatch | package provides facilities to measure how faces are similar on two photos | | BNBMakeup | provides prefabs for [Makeup](/far-sdk/effects/prefabs/makeup.md). | | BNBLips | provides neural network models for lips segmentation. See more about [Lips Segmentation](/far-sdk/tutorials/capabilities/glossary.md#lips-segmentation). | | BNBHair | provides neural network models for hair segmentation. See more about [Hair Segmentation](/far-sdk/tutorials/capabilities/glossary.md#hair-segmentation). | | BNBHands | provides neural network models for hand, nail, and finger segmentation. | | BNBEyes | provides neural network models for eyes segmentation. See more about [Eye Segmentation](/far-sdk/tutorials/capabilities/glossary.md#eye-segmentation). | | BNBSkin | provides neural network models for skin segmentation. See more about [Skin Segmentation](/far-sdk/tutorials/capabilities/glossary.md#skin-segmentation). | | BNBBackground | provides neural network models for background separation. See more about [Background Separation](/far-sdk/tutorials/capabilities/glossary.md#background-separation). | | BNBAcne | provides neural network models for acne removal. | | BNBNeck | provides neural network models for neck segmentation. | | BNBResources | includes all the resources of the all packages. **Use it when you don't care about the size or you need all the features!** | | BNBPoseEstimation | private | | BanubaSdk | depends on the all the packages for the operation of all available features. **Use it when you don't care about the size or you need all the features!** | --- # web [View as Markdown](https://docs.banuba.com/far-sdk/tutorials/development/installation/web.md)[![](/far-sdk/img/ai-guide/chatgpt.svg)Open in ChatGPT](https://chatgpt.com/?q=Read%20https%3A%2F%2Fdocs.banuba.com%2Ffar-sdk%2Ftutorials%2Fdevelopment%2Finstallation%2Fweb.md%20\(web\)%20and%20help%20me%20with%20it%20in%20my%20Banuba%20Face%20AR%20SDK%20project.)[![](/far-sdk/img/ai-guide/claude.svg)Open in Claude](https://claude.ai/new?q=Read%20https%3A%2F%2Fdocs.banuba.com%2Ffar-sdk%2Ftutorials%2Fdevelopment%2Finstallation%2Fweb.md%20\(web\)%20and%20help%20me%20with%20it%20in%20my%20Banuba%20Face%20AR%20SDK%20project.)Install tools ## NPM Package[​](#npm-package "Direct link to NPM Package") **[Banuba WebAR](https://www.npmjs.com/package/@banuba/webar)** is delivered as an NPM package, which includes executables (`.js`, `.wasm`, `.simd.wasm`) and resources modules (`modules/*.zip`). ``` npm i @banuba/webar ``` ## Resources Modules[​](#resources-modules "Direct link to Resources Modules") | Module name | Description | | -------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | face\_tracker.zip | consists of neural network models used to track a face and its features: lips, eyes, etc. Include it whenever you deal with tracking. See more about [FRX](/far-sdk/tutorials/capabilities/glossary.md#frx-face-tracking). | | face\_attributes.zip | package consists of neural network models used to extract face attributes: skin color, gender, face shape etc. | | face\_match.zip | package provides facilities to measure how faces are similar on two photos | | makeup.zip | provides prefabs for [Makeup](/far-sdk/effects/prefabs/makeup.md). | | lips.zip | provides neural network models for lips segmentation. See more about [Lips Segmentation](/far-sdk/tutorials/capabilities/glossary.md#lips-segmentation). | | hair.zip | provides neural network models for hair segmentation. See more about [Hair Segmentation](/far-sdk/tutorials/capabilities/glossary.md#hair-segmentation). | | hands.zip | provides neural network models for hand, nail, and finger segmentation. | | eyes.zip | provides neural network models for eyes segmentation. See more about [Eye Segmentation](/far-sdk/tutorials/capabilities/glossary.md#eye-segmentation). | | skin.zip | provides neural network models for skin segmentation. See more about [Skin Segmentation](/far-sdk/tutorials/capabilities/glossary.md#skin-segmentation). | | background.zip | provides neural network models for background separation. See more about [Background Separation](/far-sdk/tutorials/capabilities/glossary.md#background-separation). | | acne.zip | provides neural network models for acne removal. | | neck.zip | provides neural network models for neck segmentation. | | pose\_estimation.zip | private | ## Bundlers[​](#bundlers "Direct link to Bundlers") **[Banuba WebAR](https://www.npmjs.com/package/@banuba/webar)** depends on `BanubaSDK.data` and `BanubaSDK.wasm` (or `BanubaSDK.simd.wasm` if you are targeting [SIMD](/far-sdk/tutorials/development/guides/optimization.md#speed-up-webar-sdk-on-modern-browsers)) files. By default the SDK expects these files to be accessible from the application root i.e. by the `/BanubaSDK.data`, `/BanubaSDK.wasm` `/BanubaSDK.simd.wasm` links. It must be taken into consideration when working with application bundlers like [Vite](https://vitejs.dev/), [Rollup](https://rollupjs.org/guide/en/) or [Webpack](https://webpack.js.org/). Generally speaking one should be able to put `BanubaSDK.data`, `BanubaSDK.wasm` and `BanubaSDK.simd.wasm` files into the application assets folder (usually `public/`) and get the SDK loading these files properly. But you may want to place the files somewhere else, that case the [locateFile](/far-sdk/generated/typedoc/types/SDKOptions.html) property of the [Player.create()](/far-sdk/generated/typedoc/classes/Player.html#create) method should help you to set-up SDK properly. * Vite * Rollup * Webpack ### Vite[​](#vite "Direct link to Vite") ``` import { Player, Module /* ... */ } from "@banuba/webar" // vite uses special ?url syntax to import files as URLs import data from "@banuba/webar/BanubaSDK.data?url" import wasm from "@banuba/webar/BanubaSDK.wasm?url" import simd from "@banuba/webar/BanubaSDK.simd.wasm?url" import FaceTracker from "@banuba/webar/face_tracker.zip?url" import Background from "@banuba/webar/background.zip?url" // ... const player = await Player.create({ clientToken: "xxx-xxx-xxx", // point BanubaSDK where to find these vital files locateFile: { "BanubaSDK.data": data, "BanubaSDK.wasm": wasm, "BanubaSDK.simd.wasm": simd, }, }) await player.addModule(new Module(FaceTracker), new Module(Background)) // ... ``` tip See Vite [Explicit URL imports](https://vitejs.dev/guide/assets.html#explicit-url-imports) docs for details. ### Rollup[​](#rollup "Direct link to Rollup") ``` import { Player, Module /* ... */ } from "@banuba/webar" // you need to set-up @rollup/plugin-url for the import syntax to work import data from "@banuba/webar/BanubaSDK.data" import wasm from "@banuba/webar/BanubaSDK.wasm" import simd from "@banuba/webar/BanubaSDK.simd.wasm" import FaceTracker from "@banuba/webar/face_tracker.zip" import Background from "@banuba/webar/background.zip" // ... const player = await Player.create({ clientToken: "xxx-xxx-xxx", // point BanubaSDK where to find these vital files locateFile: { "BanubaSDK.data": data, "BanubaSDK.wasm": wasm, "BanubaSDK.simd.wasm": simd, }, }) await player.addModule(new Module(FaceTracker), new Module(Background)) // ... ``` tip See [@rollup/plugin-url](https://www.npmjs.com/package/@rollup/plugin-url#include) docs for details. ### Webpack[​](#webpack "Direct link to Webpack") Depending on the version of **Webpack** used, you may have to add following rule to the `module.rules` section of the `webpack.config.js`: ``` module.exports = { module: { rules: [ // ... { test: /\.wasm$/, type: 'javascript/auto', loader: 'file-loader', }, // ... ], }, }, } ``` Now import of `.wasm` files as URLs should work properly: ``` import { Player, Module /* ... */ } from "@banuba/webar" import data from "@banuba/webar/BanubaSDK.data" import wasm from "@banuba/webar/BanubaSDK.wasm" import simd from "@banuba/webar/BanubaSDK.simd.wasm" import FaceTracker from "@banuba/webar/face_tracker.zip" import Background from "@banuba/webar/background.zip" // ... const player = await Player.create({ clientToken: "xxx-xxx-xxx", // point BanubaSDK where to find these vital files locateFile: { "BanubaSDK.data": data, "BanubaSDK.wasm": wasm, "BanubaSDK.simd.wasm": simd, }, }) await player.addModule(new Module(FaceTracker), new Module(Background)) // ... ``` info See the related [Webpack issue](https://github.com/webpack/webpack/issues/7352) for details. --- # Known Issues [View as Markdown](https://docs.banuba.com/far-sdk/tutorials/development/known_issues.md)[![](/far-sdk/img/ai-guide/chatgpt.svg)Open in ChatGPT](https://chatgpt.com/?q=Read%20https%3A%2F%2Fdocs.banuba.com%2Ffar-sdk%2Ftutorials%2Fdevelopment%2Fknown_issues.md%20\(Known%20Issues\)%20and%20help%20me%20with%20it%20in%20my%20Banuba%20Face%20AR%20SDK%20project.)[![](/far-sdk/img/ai-guide/claude.svg)Open in Claude](https://claude.ai/new?q=Read%20https%3A%2F%2Fdocs.banuba.com%2Ffar-sdk%2Ftutorials%2Fdevelopment%2Fknown_issues.md%20\(Known%20Issues\)%20and%20help%20me%20with%20it%20in%20my%20Banuba%20Face%20AR%20SDK%20project.)Install tools * Web ## MediaStreamCapture stream freezes when a browser tab becomes inactive in Safari[​](#mediastreamcapture-stream-freezes-when-a-browser-tab-becomes-inactive-in-safari "Direct link to MediaStreamCapture stream freezes when a browser tab becomes inactive in Safari") Starting from the 15.3 release Safari began to pause [MediaStream](https://developer.mozilla.org/en-US/docs/Web/API/MediaStream)s obtained from [canvas.captureStream()](https://developer.mozilla.org/en-US/docs/Web/API/HTMLCanvasElement/captureStream), which used internally by WebAR's [MediaStreamCapture](/far-sdk/generated/typedoc/classes/MediaStreamCapture.html), when the browser's tab [is not visible](https://developer.mozilla.org/en-US/docs/Web/API/Page_Visibility_API#properties_added_to_the_document_interface). This leads to the video freeze of the WebRTC call app participant using WebAR SDK when the participant minimizes or hides the browser or the browser's tab running the app. Unfortunately, currently there is no known workaround for this issue. ## Page running WebAR SDK consumes too much memory[​](#page-running-webar-sdk-consumes-too-much-memory "Direct link to Page running WebAR SDK consumes too much memory") The most likely reason is a memory leak caused by a dangling [Player](/far-sdk/generated/typedoc/classes/Player.html#constructor) instance which can not be automatically collected by the browser's GC. Consider the code: ``` let webcam document.querySelector("#start").onclick = async () => { const player = await Player.create({ clientToken: "xxx-xxx-xxx" }) await player.use((webcam = new Webcam())) player.play() Dom.render(player, "#webar") } document.querySelector("#stop").onclick = () => { webcam.stop() Dom.unmount("#webar") } ``` Sequence of clicks on the `#start` followed by a click on the `#stop` leads to a memory leak, since the `player` object is still held in the browser memory. To fix it, you should call [Player.destroy()](/far-sdk/generated/typedoc/classes/Player.html#destroy) once the `player` object is not needed anymore. The following fixed code will not cause a memory leak: ``` let webcam, player document.querySelector("#start").onclick = async () => { player = await Player.create({ clientToken: "xxx-xxx-xxx" }) await player.use((webcam = new Webcam())) player.play() Dom.render(player, "#webar") } document.querySelector("#stop").onclick = () => { webcam.stop() Dom.unmount("#webar") // destroy the player object to prevent accidental memory leaks player.destroy() } ``` note If the app has such a "start - stop - repeat" logic, you may also consider to cache the `player` object instead of constantly re-creating it: ``` let player, webcam document.querySelector("#start").onclick = async () => { // reuse the player instance instead of re-creation if (!player) player = await Player.create({ clientToken: "xxx-xxx-xxx" }) await player.use((webcam = new Webcam())) player.play() Dom.render(player, "#webar") } document.querySelector("#stop").onclick = () => { webcam.stop() Dom.unmount("#webar") // no need to destroy the player since it will be reused on the next "#start" click } ``` ## Page running WebAR SDK crashes[​](#page-running-webar-sdk-crashes "Direct link to Page running WebAR SDK crashes") One of the most widespread reasons is a memory leak which drains all the device's RAM. Please check out the [Page running WebAR SDK consumes too much memory](#page-running-webar-sdk-consumes-too-much-memory) section. If that's not your case, please [contact support](/far-sdk/support/.md) and submit the issue. ## Effect animations are delayed in Safari[​](#effect-animations-are-delayed-in-safari "Direct link to Effect animations are delayed in Safari") This is [a known Safari bug](https://bugs.webkit.org/show_bug.cgi?id=232076), and for your convenience we provide you with [the ready-to-go fix](/far-sdk/js/range-requests.sw.js). Assume you have an html page from the [Basic Integration](/far-sdk/tutorials/development/basic_integration.md) section. Download the [range-requests.sw.js](/far-sdk/js/range-requests.sw.js) file and put it next to the WebAR running page. To fix the Safari playback issue prepend the `navigator.serviceWorker.register("./range-requests.sw.js")` to the page's script and point Player to proxy video requests to the service worker: ``` Banuba SDK Web AR
``` You can verify the fix with help of the [Rorschach](/far-sdk/generated/effects/Rorschach.zip) animated effect. note You may want to conditionally include the fix for Safari but not for the other browsers. Check the [quickstart-web](https://github.com/Banuba/quickstart-web) demo app for a possible implementation. note If your app already has a ServiceWorker in your app, simply import the [range-requests.sw.js](/far-sdk/js/range-requests.sw.js) into it: ``` importScripts("range-requests.sw.js") ``` Still have questions about FaceAR SDK? Visit our [FAQ](https://www.banuba.com/faq/) or [contact our support](/far-sdk/support/.md). --- # web [View as Markdown](https://docs.banuba.com/far-sdk/tutorials/development/known_issues/web.md)[![](/far-sdk/img/ai-guide/chatgpt.svg)Open in ChatGPT](https://chatgpt.com/?q=Read%20https%3A%2F%2Fdocs.banuba.com%2Ffar-sdk%2Ftutorials%2Fdevelopment%2Fknown_issues%2Fweb.md%20\(web\)%20and%20help%20me%20with%20it%20in%20my%20Banuba%20Face%20AR%20SDK%20project.)[![](/far-sdk/img/ai-guide/claude.svg)Open in Claude](https://claude.ai/new?q=Read%20https%3A%2F%2Fdocs.banuba.com%2Ffar-sdk%2Ftutorials%2Fdevelopment%2Fknown_issues%2Fweb.md%20\(web\)%20and%20help%20me%20with%20it%20in%20my%20Banuba%20Face%20AR%20SDK%20project.)Install tools ## MediaStreamCapture stream freezes when a browser tab becomes inactive in Safari[​](#mediastreamcapture-stream-freezes-when-a-browser-tab-becomes-inactive-in-safari "Direct link to MediaStreamCapture stream freezes when a browser tab becomes inactive in Safari") Starting from the 15.3 release Safari began to pause [MediaStream](https://developer.mozilla.org/en-US/docs/Web/API/MediaStream)s obtained from [canvas.captureStream()](https://developer.mozilla.org/en-US/docs/Web/API/HTMLCanvasElement/captureStream), which used internally by WebAR's [MediaStreamCapture](/far-sdk/generated/typedoc/classes/MediaStreamCapture.html), when the browser's tab [is not visible](https://developer.mozilla.org/en-US/docs/Web/API/Page_Visibility_API#properties_added_to_the_document_interface). This leads to the video freeze of the WebRTC call app participant using WebAR SDK when the participant minimizes or hides the browser or the browser's tab running the app. Unfortunately, currently there is no known workaround for this issue. ## Page running WebAR SDK consumes too much memory[​](#page-running-webar-sdk-consumes-too-much-memory "Direct link to Page running WebAR SDK consumes too much memory") The most likely reason is a memory leak caused by a dangling [Player](/far-sdk/generated/typedoc/classes/Player.html#constructor) instance which can not be automatically collected by the browser's GC. Consider the code: ``` let webcam document.querySelector("#start").onclick = async () => { const player = await Player.create({ clientToken: "xxx-xxx-xxx" }) await player.use((webcam = new Webcam())) player.play() Dom.render(player, "#webar") } document.querySelector("#stop").onclick = () => { webcam.stop() Dom.unmount("#webar") } ``` Sequence of clicks on the `#start` followed by a click on the `#stop` leads to a memory leak, since the `player` object is still held in the browser memory. To fix it, you should call [Player.destroy()](/far-sdk/generated/typedoc/classes/Player.html#destroy) once the `player` object is not needed anymore. The following fixed code will not cause a memory leak: ``` let webcam, player document.querySelector("#start").onclick = async () => { player = await Player.create({ clientToken: "xxx-xxx-xxx" }) await player.use((webcam = new Webcam())) player.play() Dom.render(player, "#webar") } document.querySelector("#stop").onclick = () => { webcam.stop() Dom.unmount("#webar") // destroy the player object to prevent accidental memory leaks player.destroy() } ``` note If the app has such a "start - stop - repeat" logic, you may also consider to cache the `player` object instead of constantly re-creating it: ``` let player, webcam document.querySelector("#start").onclick = async () => { // reuse the player instance instead of re-creation if (!player) player = await Player.create({ clientToken: "xxx-xxx-xxx" }) await player.use((webcam = new Webcam())) player.play() Dom.render(player, "#webar") } document.querySelector("#stop").onclick = () => { webcam.stop() Dom.unmount("#webar") // no need to destroy the player since it will be reused on the next "#start" click } ``` ## Page running WebAR SDK crashes[​](#page-running-webar-sdk-crashes "Direct link to Page running WebAR SDK crashes") One of the most widespread reasons is a memory leak which drains all the device's RAM. Please check out the [Page running WebAR SDK consumes too much memory](#page-running-webar-sdk-consumes-too-much-memory) section. If that's not your case, please [contact support](/far-sdk/support/.md) and submit the issue. ## Effect animations are delayed in Safari[​](#effect-animations-are-delayed-in-safari "Direct link to Effect animations are delayed in Safari") This is [a known Safari bug](https://bugs.webkit.org/show_bug.cgi?id=232076), and for your convenience we provide you with [the ready-to-go fix](/far-sdk/js/range-requests.sw.js). Assume you have an html page from the [Basic Integration](/far-sdk/tutorials/development/basic_integration.md) section. Download the [range-requests.sw.js](/far-sdk/js/range-requests.sw.js) file and put it next to the WebAR running page. To fix the Safari playback issue prepend the `navigator.serviceWorker.register("./range-requests.sw.js")` to the page's script and point Player to proxy video requests to the service worker: ``` Banuba SDK Web AR
``` You can verify the fix with help of the [Rorschach](/far-sdk/generated/effects/Rorschach.zip) animated effect. note You may want to conditionally include the fix for Safari but not for the other browsers. Check the [quickstart-web](https://github.com/Banuba/quickstart-web) demo app for a possible implementation. note If your app already has a ServiceWorker in your app, simply import the [range-requests.sw.js](/far-sdk/js/range-requests.sw.js) into it: ``` importScripts("range-requests.sw.js") ``` --- # Build a Face AR App in Minutes with AI [View as Markdown](https://docs.banuba.com/far-sdk/tutorials/development/llms.md)[![](/far-sdk/img/ai-guide/chatgpt.svg)Open in ChatGPT](https://chatgpt.com/?q=Read%20https%3A%2F%2Fdocs.banuba.com%2Ffar-sdk%2Fllms-full.txt%20and%20help%20me%20integrate%20the%20Banuba%20Face%20AR%20SDK%20into%20a%20web%20app.)[![](/far-sdk/img/ai-guide/claude.svg)Open in Claude](https://claude.ai/new?q=Read%20https%3A%2F%2Fdocs.banuba.com%2Ffar-sdk%2Fllms-full.txt%20and%20help%20me%20integrate%20the%20Banuba%20Face%20AR%20SDK%20into%20a%20web%20app.)Install tools ## ![](/far-sdk/img/ai-guide/icon-pro.svg) Quick Start \~3 min[​](#quick-start "Direct link to quick-start") For developers who already have Node.js and Claude Code installed. Copy each command and run it in your terminal. 1 Install Banuba AI Skills: `npx skills add Banuba/ai-skills -a claude-code --all --yes`Copy 2 Start Claude Code: `claude`Copy 3 Fill out the form to receive a free trial license key: [Get a Free Trial License Key](https://www.banuba.com/face-filters-sdk) 4 Build your Face AR demo. Copy this prompt and paste it into Claude Code: `/far-general Build a Face AR camera for Web with beauty filters, face effects, and a virtual background. Use sensible defaults and create a working demo. Banuba license key: [paste your license key]`Copy Claude Code can ask follow-up questions later, but you should not need to choose a use case before installing and testing the skill. 5 Run your app. Follow the command provided by Claude Code. It will usually look similar to: `npm install npm run dev`Copy Open the local browser link shown in the terminal. ![](/far-sdk/img/ai-guide/check-green.svg) πŸŽ‰ **Your Face AR camera is running.** Need to install Node.js or Claude Code first? [See Start from Scratch below β†’](#start-from-scratch) *** ## What Can I Build?[​](#what-can-i-build "Direct link to what-can-i-build") VideocallVirtual BackgroundFace EffectsFace TrackingBeautificationMakeupVirtual Try-OnCustom Effects Every prompt starts the same skill. Not sure which to pick? Start with **Videocall**. ![Video call with AR backgrounds and masks](/far-sdk/img/ai/1.jpg)Preview coming soon Add Face AR effects and virtual backgrounds to a Web video-call experience. prompt Β· videocallCopy /far-general Add Face AR effects, beauty filters, and virtual backgrounds to my Web video-call experience. Banuba license key: \[paste your token] Copy the prompt, paste it into Claude Code, replace \[paste your token] with your [Banuba license key](https://www.banuba.com/face-filters-sdk), and press Enter. *** ## Step-by-step setup \~15 min[​](#start-from-scratch "Direct link to start-from-scratch") Steps 1 .Get a Banuba license key2 .Install Node.js3 .Install Claude4 .Install AI Skills5 .Build and run Follow the steps in order. Each one ends with a check you can confirm before moving on. Fill out the form to receive a free trial license key. It arrives by email. [Get a free trial license key](https://www.banuba.com/face-filters-sdk) Need these features? Not included by default β€” tick them in the form to unlock: Hand gesturesAdvanced makeupVirtual background Save the key β€” you will paste it into the prompt in **step 5**. **Done when:** you receive the license key from Banuba PreviousNext *** ## Already Have a Web App?[​](#already-have-a-web-app "Direct link to Already Have a Web App?") Open your existing project folder in Claude Code and ask Banuba AI Skills to add Face AR SDK to your current experience. Examples: prompt Β· video call Copy `/far-general Add Banuba Face AR SDK to the video-call screen in my existing Web app. Include beauty filters and virtual background support. Banuba license key: [paste your license key]` prompt Β· ecommerce try-on Copy `/far-general Add a makeup try-on camera to my existing ecommerce Web app. Include lipstick, eyeshadow, foundation, and a shade picker. Banuba license key: [paste your license key]` prompt Β· profile photo Copy `/far-general Add custom Face AR effects to the profile-photo flow in my existing Web app. Banuba license key: [paste your license key]` *** ## How AI Skills Work[​](#how-ai-skills-work "Direct link to How AI Skills Work") Your Web Project+Claude Code+Banuba AI Skills+Banuba Face AR SDK↓Working Face AR Experience Banuba Face AR SDK provides the AR camera engine. Claude Code reads your project, answers questions, and helps create or modify code. Banuba AI Skills provide Claude Code with SDK-specific instructions, starter templates, and integration guidance. AI Skills do not automatically create a finished production product. They help the coding agent understand Banuba Face AR SDK and guide implementation using current SDK documentation. *** ## Why Use Banuba AI Skills?[​](#why-use-banuba-ai-skills "Direct link to Why Use Banuba AI Skills?") Banuba AI Skills cut integration time dramatically: instead of spending days reading documentation and assembling the SDK by hand, you get a working Face AR demo in minutes. Without AI Skills Search documentation↓Find relevant examples↓Understand SDK APIs↓Assemble the integration↓Debug compatibility issues With AI Skills Install the skill↓Describe what you want↓Let the agent build↓Review and iterate *** ## Troubleshooting[​](#troubleshooting "Direct link to Troubleshooting") `npx: command not found` Terminal doesn't recognize the npx command `cause`Node.js is not installed, or your terminal has not picked it up yet. `fix`Install the current LTS version of [Node.js](https://nodejs.org), close and reopen the terminal, then run `node --version` again. `claude: command not found` Terminal doesn't recognize the claude command `cause`Claude Code is not installed or is not available in your terminal PATH. `fix`Reinstall Claude Code, close and reopen the terminal, then run `claude --version` again. `/far-general not recognized` The agent doesn't know the Banuba slash command `cause`Banuba AI Skills were not installed, or Claude Code was started from a different folder. `fix`Open the folder where you installed Banuba AI Skills and start Claude Code again. You can also reinstall the skill: terminal Β· reinstall Copy `npx skills add Banuba/ai-skills --skill far-general -a claude-code` `app does not start` The Web app does not start `cause`Dependencies may be missing, or the run command failed. `fix`Ask Claude Code: prompt Copy `Check why the Web app does not start and fix the setup. Run the appropriate checks and explain any remaining issue.` `black camera` The camera is black `cause`Camera access is blocked, or another app is using the camera. `fix`Allow camera access in the browser, close other apps using the camera, and refresh the page. If needed, try another modern browser. `license error` License error at runtime `cause`The license key was copied incorrectly, has expired, does not support the requested platform, or does not include the required features. `fix`Confirm the key, then ask Claude Code: prompt Copy `Update the Banuba license key in the project and check that it is configured correctly.` `missing add-on` Advanced features do not work `cause`Hand gestures, advanced makeup, and background replacement may require additional neural networks or license add-ons. `fix`Request access to the required features, then update the project with the new license. `effects do not load` Effects do not load `cause`Effect files may be missing, in the wrong folder, or referenced with an incorrect path. `fix`Ask Claude Code: prompt Copy `Check how Banuba effects are loaded in this project and fix missing files, incorrect folders, or invalid paths.` `low fps` Performance is poor `cause`The browser, device, camera resolution, or enabled effects may be too heavy for the current setup. `fix`Ask Claude Code: prompt Copy `Optimize this Face AR Web project. Reduce camera resolution if appropriate, limit preloaded effects, and keep only the required neural networks and assets.` `bundle too large` The Web bundle is too large `cause`The project may include more neural networks or assets than your demo needs. `fix`Ask Claude Code: prompt Copy `Optimize the Web bundle. Keep only the neural networks, effects, and assets required for this experience.` ![](/far-sdk/img/ai-guide/bubble.svg) **Still stuck?** Open an issue or contact [Banuba support](https://www.banuba.com/support) with the error message, browser name, operating system, and the steps you followed. *** ## Supported AI Coding Agents[​](#supported-ai-coding-agents "Direct link to Supported AI Coding Agents") | Agent | Status | Install command | | ----------- | --------- | ------------------------------------------------------------ | | Claude Code | Supported | `npx skills add Banuba/ai-skills -a claude-code --all --yes` | | Codex | Supported | `npx skills add Banuba/ai-skills -a codex --all --yes` | | Qwen Code | Supported | `npx skills add Banuba/ai-skills -a qwen-code --all --yes` | *** ## No AI Coding Agent?[​](#no-ai-coding-agent "Direct link to No AI Coding Agent?") Use Banuba's LLM-ready documentation: * [llms.txt](https://docs.banuba.com/far-sdk/llms.txt) - quick context * [llms-full.txt](https://docs.banuba.com/far-sdk/llms-full.txt) - complete documentation You can provide these files to ChatGPT, Claude, Gemini, or another LLM for product scoping, documentation questions, or technical analysis. An LLM chat cannot directly create or modify files on your computer unless it is connected to a coding environment. *** ## Developer Handoff[​](#developer-handoff "Direct link to Developer Handoff") handoff template Β· paste to ticket or Slack Copy `Banuba Face AR SDK - Web Demo Status: A working Web demo was created with Banuba AI Skills. Features enabled: [face filters / beauty / makeup / virtual background / hand gestures / other] Source: [path or repository link] Engineering review: - Review the integration against the production Web architecture. - Replace the trial license with the production license. - Connect the Face AR camera to the target product flow. - Review performance across target browsers and devices. - Confirm that required neural networks and add-ons are covered. - Complete security, privacy, and accessibility reviews. For SDK-specific questions in Claude Code, use /far-general.` Copy *** ## Reference[​](#reference "Direct link to Reference") ### Available skill[​](#available-skill "Direct link to Available skill") | Skill | Purpose | | ------------- | -------------------------------------------------------------------------------------------------- | | `far-general` | Understand, plan, build, review, and troubleshoot Banuba Face AR SDK integrations for Web projects | ### Common capabilities[​](#common-capabilities "Direct link to Common capabilities") * Face tracking * Face filters and masks * AR glasses * Beauty effects * Virtual makeup * Background blur and replacement * Hand gesture triggers * Hand AR effects * Custom `.zip` effects * Video-call integrations ### Glossary[​](#glossary "Direct link to Glossary") | Term | Definition | | ------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | **AI coding agent** | A coding tool that can read a project, answer questions, and help create or modify code. | | **Agent skill** | Packaged instructions that teach an AI coding agent how to complete a specific task. | | **Effect** | A Face AR scene that can include a face filter, makeup look, virtual background, gesture trigger, or a combination of features. Effects are distributed as `.zip` files and loaded at runtime. | | **License key** | The Banuba credential required to use Face AR SDK, including trial integrations. | | **Neural network** | A model used by the SDK to recognize faces, hands, gestures, backgrounds, and other inputs. Some neural networks require additional license access. | | **npx** | A tool included with Node.js that runs installer commands without permanently installing the installer. | | **Slash command** | A command inside an AI coding agent that begins with `/`, such as `/far-general`. | | **Terminal** | A text-based application used to run commands. Use Terminal on macOS and Terminal or PowerShell on Windows. | --- # Examples of using Banuba SDK [View as Markdown](https://docs.banuba.com/far-sdk/tutorials/development/samples.md)[![](/far-sdk/img/ai-guide/chatgpt.svg)Open in ChatGPT](https://chatgpt.com/?q=Read%20https%3A%2F%2Fdocs.banuba.com%2Ffar-sdk%2Ftutorials%2Fdevelopment%2Fsamples.md%20\(Examples%20of%20using%20Banuba%20SDK\)%20and%20help%20me%20with%20it%20in%20my%20Banuba%20Face%20AR%20SDK%20project.)[![](/far-sdk/img/ai-guide/claude.svg)Open in Claude](https://claude.ai/new?q=Read%20https%3A%2F%2Fdocs.banuba.com%2Ffar-sdk%2Ftutorials%2Fdevelopment%2Fsamples.md%20\(Examples%20of%20using%20Banuba%20SDK\)%20and%20help%20me%20with%20it%20in%20my%20Banuba%20Face%20AR%20SDK%20project.)Install tools To start working with **Banuba SDK** examples, you need to have a client token. To receive it, please fill in our [form on banuba.com](https://www.banuba.com/face-filters-sdk), or contact us via . ## Our GitHub[​](#our-github "Direct link to Our GitHub") Visit our [GitHub page](https://github.com/Banuba) to see all available examples. * iOS * Android * Web * Flutter * ReactNative * macOS * Desktop ## iOS samples (Swift)[​](#ios-samples-swift "Direct link to iOS samples (Swift)") This repository contains basic samples how to use [Player API](/far-sdk/tutorials/development/api_overview.md). ## Agora plugin example (Swift)[​](#agora-plugin-example-swift "Direct link to Agora plugin example (Swift)") This example shows how to use **Banuba SDK** as an **Agora** plugin for a video call between two devices. ## Opentok example (Swift)[​](#opentok-example-swift "Direct link to Opentok example (Swift)") This example demonstrates the use of **Banuba SDK** in conjunction with **Opentok SDK**. ## WebRTC example (Objective-C)[​](#webrtc-example-objective-c "Direct link to WebRTC example (Objective-C)") This example demonstrates how to use **Banuba SDK** in conjunction with **WebRTC**. ## Beauty example (Swift)[​](#beauty-example-swift "Direct link to Beauty example (Swift)") This example demonstrates how to correctly use the **Makeup** effect. info This example uses **SPM** ## [ZEGOCLOUD](https://www.zegocloud.com) example (Swift)[​](#zegocloud-example-swift "Direct link to zegocloud-example-swift") This example demonstrates integration with ZEGOCLOUD videocalling platform. ## ARCloud example (Swift)[​](#arcloud-example-swift "Direct link to ARCloud example (Swift)") This example shows how to dynamically load effects from the network. ## Quickstart example (Objective-C)[​](#quickstart-example-objective-c "Direct link to Quickstart example (Objective-C)") This example shows how to use **Banuba SDK** in **Objective-C** application. ## Requirements[​](#requirements "Direct link to Requirements") * Latest stable Android Studio * Latest Gradle plugin * Latest NDK ## Banuba SDK examples (Kotlin)[​](#banuba-sdk-examples-kotlin "Direct link to Banuba SDK examples (Kotlin)") This example shows how to use **PlayerAPI** for various tasks. It uses the `Player`, `CameraInput`, `SurfaceOutput`, and `VideoOutput` classes. ## Agora plugin example (Kotlin)[​](#agora-plugin-example-kotlin "Direct link to Agora plugin example (Kotlin)") An example of using **Banuba SDK** as an **Agora** plugin for a video call between two devices. ## Opentok example (Java)[​](#opentok-example-java "Direct link to Opentok example (Java)") This example demonstrates the use of **Banuba SDK** in conjunction with **Opentok SDK**. ## WebRTC example (Kotlin)[​](#webrtc-example-kotlin "Direct link to WebRTC example (Kotlin)") This example demonstrates how to use **Banuba SDK** in conjunction with **WebRTC**. In it, a **WebRTC** camera is used, and after processing, the frame is drawn onto the surface using **WebRTC**. This example based on **PlayerAPI**. ## [ZEGOCLOUD](https://www.zegocloud.com) example (Java)[​](#zegocloud-example-java "Direct link to zegocloud-example-java") This example demonstrates integration with ZEGOCLOUD videocalling platform. ## Beauty example (Java)[​](#beauty-example-java "Direct link to Beauty example (Java)") This example demonstrates how to correctly use the **Makeup** effect and how to call **MakeupAPI** scripts. This example is based on **PlayerAPI**. ## Beauty example (Kotlin)[​](#beauty-example-kotlin "Direct link to Beauty example (Kotlin)") This example demonstrates how to correctly use the **Makeup** effect and how to call **MakeupAPI** scripts. ## ARCloud example (Kotlin)[​](#arcloud-example-kotlin "Direct link to ARCloud example (Kotlin)") This example shows how to dynamically load effects from the network. This example is based on **PlayerAPI**. ## Quickstart[​](#quickstart "Direct link to Quickstart") **[Quickstart Web](https://github.com/Banuba/quickstart-web)** ## Beauty[​](#beauty "Direct link to Beauty") **[Beauty demo app](https://github.com/Banuba/beauty-web)** ## Vue[​](#vue "Direct link to Vue") **[Vue demo app](https://github.com/Banuba/quickstart-web-vue)** ## Angular[​](#angular "Direct link to Angular") **[Angular demo app](https://github.com/Banuba/quickstart-web-angular)** ## React[​](#react "Direct link to React") **[React demo app](https://github.com/Banuba/quickstart-web-react)** ``` import React, { useEffect } from "react" import data from "@banuba/webar/BanubaSDK.data" import wasm from "@banuba/webar/BanubaSDK.wasm" import simd from "@banuba/webar/BanubaSDK.simd.wasm" import FaceTracker from "@banuba/webar/face_tracker.zip" import Glasses from "/path/to/Glasses.zip" import { Webcam, Player, Module, Effect, Dom } from "@banuba/webar" export default function WebARComponent() { // componentDidMount useEffect(() => { let webcam Player.create({ clientToken: "xxx-xxx-xxx", locateFile: { "BanubaSDK.data": data, "BanubaSDK.wasm": wasm, "BanubaSDK.simd.wasm": simd, }, }).then(async (player) => { await player.addModule(new Module(FaceTracker)) await player.use(webcam = new Webcam()) player.applyEffect(new Effect(Glasses)) Dom.render(player, "#webar") }) // componentWillUnmount return () => { webcam?.stop() Dom.unmount("#webar") } }) return
} ``` tip See [Bundlers](/far-sdk/tutorials/development/installation.md#bundlers) for notes about specific bundlers and `locateFile` usage. ## Agora[​](#agora "Direct link to Agora") **[Banuba Agora Extension](https://github.com/Banuba/agora-plugin-filters-web)** **[Video call demo app](https://github.com/Banuba/videocall-web)** ## ZEGOCLOUD[​](#zegocloud "Direct link to ZEGOCLOUD") **[ZEGOCLOUD sample](https://github.com/Banuba/banuba_sdk_zegocloud_sdk_web)** ## OpenTok (TokBox)[​](#opentok-tokbox "Direct link to OpenTok (TokBox)") **[TokBox demo app](https://github.com/Banuba/videocall-tokbox-web)** ## Customizing video source[​](#customizing-video-source "Direct link to Customizing video source") You can easily modify the built-in [Webcam](/far-sdk/generated/typedoc/classes/Webcam.html#constructor) module video by passing [parameters](https://developer.mozilla.org/en-US/docs/Web/API/MediaTrackConstraints#properties_of_video_tracks) to it. ### Switching from front to back camera[​](#switching-from-front-to-back-camera "Direct link to Switching from front to back camera") For example, you can use back camera of the device by passing [facingMode](https://developer.mozilla.org/en-US/docs/Web/API/MediaTrackConstraints/facingMode) parameter to it: ``` // ... // The default facingMode value is "user" which means front camera // The "environment" value here means back camera await player.use(new Webcam({ facingMode: "environment" })) // ... ``` ### Rendering WebAR video in full screen on mobile[​](#rendering-webar-video-in-full-screen-on-mobile "Direct link to Rendering WebAR video in full screen on mobile") Simply add the following CSS to force WebAR AR video to fill viewport: ``` ``` If you decide to specify the exact `width` and `height` for the [Webcam](/far-sdk/generated/typedoc/classes/Webcam.html#constructor), pay attention that on several mobile devices/operating systems webcam video width and height may be flipped. It's a known platform-specific [webcam bug](https://stackoverflow.com/questions/62538271/getusermedia-selfie-full-screen-on-mobile/62598616#62598616). To work around it, swap the `width` and `height` values: ``` const desiredWidth = 360 const desiredHeight = 540 await player.use(new Webcam({ width: desiredHeight, height: desiredWidth, })) ``` Also, you may want to check out the [Video cropping](#video-cropping) section for more advanced scenarios. ### External MediaStream[​](#external-mediastream "Direct link to External MediaStream") If the built-in [Webcam](/far-sdk/generated/typedoc/classes/Webcam.html#constructor) can not fit your needs you can use a custom [MediaStream](/far-sdk/generated/typedoc/classes/MediaStream.html) with [Player](/far-sdk/generated/typedoc/classes/Player.html#constructor): ``` import { MediaStream /* ... */ } from "@banuba/webar" // ... /* process video from the camera */ const camera = await navigator.mediaDevices.getUserMedia({ audio: true, video: true }) await player.use(new MediaStream(camera)) /* or even from another canvas */ const canvas = $("canvas").captureStream() await player.use(new MediaStream(canvas)) // ... ``` See [MediaStream](/far-sdk/generated/typedoc/classes/MediaStream.html) docs for more details. ## Capturing processed video[​](#capturing-processed-video "Direct link to Capturing processed video") You can easily capture the processed video, take screenshots, video recordings or pass the captured video to a WebRTC connection. ### Screenshot[​](#screenshot "Direct link to Screenshot") ``` import { ImageCapture /* ... */ } from "@banuba/webar" // ... const capture = new ImageCapture(player) const photo = await capture.takePhoto() // ... ``` See [ImageCapture.takePhoto()](/far-sdk/generated/typedoc/classes/ImageCapture.html#takePhoto) docs for more details. ### Video[​](#video "Direct link to Video") ``` import { VideoRecorder /* ... */ } from "@banuba/webar" // ... const recorder = new VideoRecorder(player) recorder.start() await new Promise((r) => setTimeout(r, 5000)) // wait for 5 sec const video = await recorder.stop() // ... ``` See [VideoRecorder](/far-sdk/generated/typedoc/classes/VideoRecorder.html) docs for more details. ### MediaStream[​](#mediastream "Direct link to MediaStream") ``` import { MediaStreamCapture /* ... */ } from "@banuba/webar" // ... // the capture is an instance of window.MediaStream const capture = new MediaStreamCapture(player) // so it can be used as a video source $("video").srcObject = capture // or can be added to a WebRTC peer connection const connection = new RTCPeerConnection() connection.addTrack(capture.getVideoTrack()) // ... ``` See [MediaStreamCapture](/far-sdk/generated/typedoc/classes/MediaStreamCapture.html) docs for more details. ## Video cropping[​](#video-cropping "Direct link to Video cropping") You can adjust video frame dimensions via [Webcam](/far-sdk/generated/typedoc/classes/Webcam.html#constructor) constructor parameters: ``` const wcam = new Webcam({ width: 320, height: 240 }) ``` But this approach is platform-dependent and varies between browsers, e.g. some browsers may be unable to produces frames of the requested dimensions and can yield frames of close but different dimensions instead (e.g. 352x288 instead of requested 320x240). To work around this platform-specific limitations, you can leverage the built-in SDK crop modificator: ``` const desiredWidth = 320 const desiredHeight = 240 function crop(renderWidth, renderHeight) { const dx = (renderWidth - desiredWidth) / 2 const dy = (renderHeight - desiredHeight) / 2 return [dx, dy, desiredWidth, desiredHeight] } await player.use(webcam, { crop }) ``` This way you can get the desired frame size regardless of the platform used. See [Player.use()](/far-sdk/generated/typedoc/classes/Player.html#use) and [InputOptions](//generated/typedoc/types/InputOptions.html) docs for more datails. ## Postprocessing[​](#postprocessing "Direct link to Postprocessing") It's possible to post-process the video processed by WebAR SDK. You can grab the idea from the code-snippet: ``` import { MediaStreamCapture /* ... */ } from "@banuba/webar" // ... const capture = document.createElement("video") capture.autoplay = true capture.srcObject = new MediaStreamCapture(player) const canvas = document.getElementById("postprocessed") const ctx = canvas.getContext("2d") const fontSize = 48 * window.devicePixelRatio function postprocess() { canvas.width = capture.videoWidth canvas.height = capture.videoHeight ctx.drawImage(capture, 0, 0) ctx.font = `${fontSize}px serif` ctx.fillStyle = "red" ctx.fillText("A Watermark", 0.5 * fontSize, 1.25 * fontSize) } ;(function loop() { postprocess() requestAnimationFrame(loop) })() ``` See [Capturing processed video](#capturing-processed-video) > [MediaStream](#mediastream) for details. ## Minimal sample[​](#minimal-sample "Direct link to Minimal sample") A one-file example, a good starting point. ## Quickstart Flutter sample[​](#quickstart-flutter-sample "Direct link to Quickstart Flutter sample") This project demonstrates how to apply effects, makeup; process photos. ## Video call sample[​](#video-call-sample "Direct link to Video call sample") This example demonstrates the use of **Banuba SDK** in conjunction with **AgoraRTC SDK** for a video call on Flutter. This example is forked from the [Flutter plugin of Agora](https://github.com/AgoraIO-Extensions/Agora-Flutter-SDK). Banuba SDK is integrated in `JoinChannelVideo` subsample. See [videocall](/far-sdk/tutorials/development/videocall.md). ## ARCloud sample[​](#arcloud-sample "Direct link to ARCloud sample") Read more about [ARCloud](/far-sdk/tutorials/development/guides/ar_cloud.md). ## Minimal sample[​](#minimal-sample "Direct link to Minimal sample") One-file example, good starting point. This a part of React Native over Banuba module. ## Videocall example[​](#videocall-example "Direct link to Videocall example") This example demonstrates the use of **Banuba SDK** in conjunction with **AgoraRTC SDK** for a video call on React Native. This example is forked from [Agora module for React Native](https://github.com/AgoraIO-Extensions/react-native-agora). Banuba SDK integrated in `JoinChannelVideo` subsample. See [videocall](/far-sdk/tutorials/development/videocall.md). ## macOS sample (Swift)[​](#macos-sample-swift "Direct link to macOS sample (Swift)") This repository contains basic samples how to use Banuba SDK on macOS. Examples bellow are written in C++ and will run both on Windows and macOS. ## Quickstart Desktop (C++)[​](#quickstart-desktop-c "Direct link to Quickstart Desktop (C++)") The starting point for desktop integration in C++. Demonstrates: 1. [on-screen rendering with realtime camera](https://github.com/Banuba/quickstart-desktop-cpp/blob/master/realtime-camera-preview/main.cpp) 2. [photo processing](https://github.com/Banuba/quickstart-desktop-cpp/blob/master/single-image-processing/main.cpp) 3. [video stream processing](https://github.com/Banuba/quickstart-desktop-cpp/blob/master/videostream-processing/main.cpp) --- # android [View as Markdown](https://docs.banuba.com/far-sdk/tutorials/development/samples/android.md)[![](/far-sdk/img/ai-guide/chatgpt.svg)Open in ChatGPT](https://chatgpt.com/?q=Read%20https%3A%2F%2Fdocs.banuba.com%2Ffar-sdk%2Ftutorials%2Fdevelopment%2Fsamples%2Fandroid.md%20\(android\)%20and%20help%20me%20with%20it%20in%20my%20Banuba%20Face%20AR%20SDK%20project.)[![](/far-sdk/img/ai-guide/claude.svg)Open in Claude](https://claude.ai/new?q=Read%20https%3A%2F%2Fdocs.banuba.com%2Ffar-sdk%2Ftutorials%2Fdevelopment%2Fsamples%2Fandroid.md%20\(android\)%20and%20help%20me%20with%20it%20in%20my%20Banuba%20Face%20AR%20SDK%20project.)Install tools ## Requirements[​](#requirements "Direct link to Requirements") * Latest stable Android Studio * Latest Gradle plugin * Latest NDK ## Banuba SDK examples (Kotlin)[​](#banuba-sdk-examples-kotlin "Direct link to Banuba SDK examples (Kotlin)") This example shows how to use **PlayerAPI** for various tasks. It uses the `Player`, `CameraInput`, `SurfaceOutput`, and `VideoOutput` classes. ## Agora plugin example (Kotlin)[​](#agora-plugin-example-kotlin "Direct link to Agora plugin example (Kotlin)") An example of using **Banuba SDK** as an **Agora** plugin for a video call between two devices. ## Opentok example (Java)[​](#opentok-example-java "Direct link to Opentok example (Java)") This example demonstrates the use of **Banuba SDK** in conjunction with **Opentok SDK**. ## WebRTC example (Kotlin)[​](#webrtc-example-kotlin "Direct link to WebRTC example (Kotlin)") This example demonstrates how to use **Banuba SDK** in conjunction with **WebRTC**. In it, a **WebRTC** camera is used, and after processing, the frame is drawn onto the surface using **WebRTC**. This example based on **PlayerAPI**. ## [ZEGOCLOUD](https://www.zegocloud.com) example (Java)[​](#zegocloud-example-java "Direct link to zegocloud-example-java") This example demonstrates integration with ZEGOCLOUD videocalling platform. ## Beauty example (Java)[​](#beauty-example-java "Direct link to Beauty example (Java)") This example demonstrates how to correctly use the **Makeup** effect and how to call **MakeupAPI** scripts. This example is based on **PlayerAPI**. ## Beauty example (Kotlin)[​](#beauty-example-kotlin "Direct link to Beauty example (Kotlin)") This example demonstrates how to correctly use the **Makeup** effect and how to call **MakeupAPI** scripts. ## ARCloud example (Kotlin)[​](#arcloud-example-kotlin "Direct link to ARCloud example (Kotlin)") This example shows how to dynamically load effects from the network. This example is based on **PlayerAPI**. --- # desktop [View as Markdown](https://docs.banuba.com/far-sdk/tutorials/development/samples/desktop.md)[![](/far-sdk/img/ai-guide/chatgpt.svg)Open in ChatGPT](https://chatgpt.com/?q=Read%20https%3A%2F%2Fdocs.banuba.com%2Ffar-sdk%2Ftutorials%2Fdevelopment%2Fsamples%2Fdesktop.md%20\(desktop\)%20and%20help%20me%20with%20it%20in%20my%20Banuba%20Face%20AR%20SDK%20project.)[![](/far-sdk/img/ai-guide/claude.svg)Open in Claude](https://claude.ai/new?q=Read%20https%3A%2F%2Fdocs.banuba.com%2Ffar-sdk%2Ftutorials%2Fdevelopment%2Fsamples%2Fdesktop.md%20\(desktop\)%20and%20help%20me%20with%20it%20in%20my%20Banuba%20Face%20AR%20SDK%20project.)Install tools Examples bellow are written in C++ and will run both on Windows and macOS. ## Quickstart Desktop (C++)[​](#quickstart-desktop-c "Direct link to Quickstart Desktop (C++)") The starting point for desktop integration in C++. Demonstrates: 1. [on-screen rendering with realtime camera](https://github.com/Banuba/quickstart-desktop-cpp/blob/master/realtime-camera-preview/main.cpp) 2. [photo processing](https://github.com/Banuba/quickstart-desktop-cpp/blob/master/single-image-processing/main.cpp) 3. [video stream processing](https://github.com/Banuba/quickstart-desktop-cpp/blob/master/videostream-processing/main.cpp) --- # flutter [View as Markdown](https://docs.banuba.com/far-sdk/tutorials/development/samples/flutter.md)[![](/far-sdk/img/ai-guide/chatgpt.svg)Open in ChatGPT](https://chatgpt.com/?q=Read%20https%3A%2F%2Fdocs.banuba.com%2Ffar-sdk%2Ftutorials%2Fdevelopment%2Fsamples%2Fflutter.md%20\(flutter\)%20and%20help%20me%20with%20it%20in%20my%20Banuba%20Face%20AR%20SDK%20project.)[![](/far-sdk/img/ai-guide/claude.svg)Open in Claude](https://claude.ai/new?q=Read%20https%3A%2F%2Fdocs.banuba.com%2Ffar-sdk%2Ftutorials%2Fdevelopment%2Fsamples%2Fflutter.md%20\(flutter\)%20and%20help%20me%20with%20it%20in%20my%20Banuba%20Face%20AR%20SDK%20project.)Install tools ## Minimal sample[​](#minimal-sample "Direct link to Minimal sample") A one-file example, a good starting point. ## Quickstart Flutter sample[​](#quickstart-flutter-sample "Direct link to Quickstart Flutter sample") This project demonstrates how to apply effects, makeup; process photos. ## Video call sample[​](#video-call-sample "Direct link to Video call sample") This example demonstrates the use of **Banuba SDK** in conjunction with **AgoraRTC SDK** for a video call on Flutter. This example is forked from the [Flutter plugin of Agora](https://github.com/AgoraIO-Extensions/Agora-Flutter-SDK). Banuba SDK is integrated in `JoinChannelVideo` subsample. See [videocall](/far-sdk/tutorials/development/videocall.md). ## ARCloud sample[​](#arcloud-sample "Direct link to ARCloud sample") Read more about [ARCloud](/far-sdk/tutorials/development/guides/ar_cloud.md). --- # ios [View as Markdown](https://docs.banuba.com/far-sdk/tutorials/development/samples/ios.md)[![](/far-sdk/img/ai-guide/chatgpt.svg)Open in ChatGPT](https://chatgpt.com/?q=Read%20https%3A%2F%2Fdocs.banuba.com%2Ffar-sdk%2Ftutorials%2Fdevelopment%2Fsamples%2Fios.md%20\(ios\)%20and%20help%20me%20with%20it%20in%20my%20Banuba%20Face%20AR%20SDK%20project.)[![](/far-sdk/img/ai-guide/claude.svg)Open in Claude](https://claude.ai/new?q=Read%20https%3A%2F%2Fdocs.banuba.com%2Ffar-sdk%2Ftutorials%2Fdevelopment%2Fsamples%2Fios.md%20\(ios\)%20and%20help%20me%20with%20it%20in%20my%20Banuba%20Face%20AR%20SDK%20project.)Install tools ## iOS samples (Swift)[​](#ios-samples-swift "Direct link to iOS samples (Swift)") This repository contains basic samples how to use [Player API](/far-sdk/tutorials/development/api_overview.md). ## Agora plugin example (Swift)[​](#agora-plugin-example-swift "Direct link to Agora plugin example (Swift)") This example shows how to use **Banuba SDK** as an **Agora** plugin for a video call between two devices. ## Opentok example (Swift)[​](#opentok-example-swift "Direct link to Opentok example (Swift)") This example demonstrates the use of **Banuba SDK** in conjunction with **Opentok SDK**. ## WebRTC example (Objective-C)[​](#webrtc-example-objective-c "Direct link to WebRTC example (Objective-C)") This example demonstrates how to use **Banuba SDK** in conjunction with **WebRTC**. ## Beauty example (Swift)[​](#beauty-example-swift "Direct link to Beauty example (Swift)") This example demonstrates how to correctly use the **Makeup** effect. info This example uses **SPM** ## [ZEGOCLOUD](https://www.zegocloud.com) example (Swift)[​](#zegocloud-example-swift "Direct link to zegocloud-example-swift") This example demonstrates integration with ZEGOCLOUD videocalling platform. ## ARCloud example (Swift)[​](#arcloud-example-swift "Direct link to ARCloud example (Swift)") This example shows how to dynamically load effects from the network. ## Quickstart example (Objective-C)[​](#quickstart-example-objective-c "Direct link to Quickstart example (Objective-C)") This example shows how to use **Banuba SDK** in **Objective-C** application. --- # macos [View as Markdown](https://docs.banuba.com/far-sdk/tutorials/development/samples/macos.md)[![](/far-sdk/img/ai-guide/chatgpt.svg)Open in ChatGPT](https://chatgpt.com/?q=Read%20https%3A%2F%2Fdocs.banuba.com%2Ffar-sdk%2Ftutorials%2Fdevelopment%2Fsamples%2Fmacos.md%20\(macos\)%20and%20help%20me%20with%20it%20in%20my%20Banuba%20Face%20AR%20SDK%20project.)[![](/far-sdk/img/ai-guide/claude.svg)Open in Claude](https://claude.ai/new?q=Read%20https%3A%2F%2Fdocs.banuba.com%2Ffar-sdk%2Ftutorials%2Fdevelopment%2Fsamples%2Fmacos.md%20\(macos\)%20and%20help%20me%20with%20it%20in%20my%20Banuba%20Face%20AR%20SDK%20project.)Install tools ## macOS sample (Swift)[​](#macos-sample-swift "Direct link to macOS sample (Swift)") This repository contains basic samples how to use Banuba SDK on macOS. --- # react\_native [View as Markdown](https://docs.banuba.com/far-sdk/tutorials/development/samples/react_native.md)[![](/far-sdk/img/ai-guide/chatgpt.svg)Open in ChatGPT](https://chatgpt.com/?q=Read%20https%3A%2F%2Fdocs.banuba.com%2Ffar-sdk%2Ftutorials%2Fdevelopment%2Fsamples%2Freact_native.md%20\(react_native\)%20and%20help%20me%20with%20it%20in%20my%20Banuba%20Face%20AR%20SDK%20project.)[![](/far-sdk/img/ai-guide/claude.svg)Open in Claude](https://claude.ai/new?q=Read%20https%3A%2F%2Fdocs.banuba.com%2Ffar-sdk%2Ftutorials%2Fdevelopment%2Fsamples%2Freact_native.md%20\(react_native\)%20and%20help%20me%20with%20it%20in%20my%20Banuba%20Face%20AR%20SDK%20project.)Install tools ## Minimal sample[​](#minimal-sample "Direct link to Minimal sample") One-file example, good starting point. This a part of React Native over Banuba module. ## Videocall example[​](#videocall-example "Direct link to Videocall example") This example demonstrates the use of **Banuba SDK** in conjunction with **AgoraRTC SDK** for a video call on React Native. This example is forked from [Agora module for React Native](https://github.com/AgoraIO-Extensions/react-native-agora). Banuba SDK integrated in `JoinChannelVideo` subsample. See [videocall](/far-sdk/tutorials/development/videocall.md). --- # web [View as Markdown](https://docs.banuba.com/far-sdk/tutorials/development/samples/web.md)[![](/far-sdk/img/ai-guide/chatgpt.svg)Open in ChatGPT](https://chatgpt.com/?q=Read%20https%3A%2F%2Fdocs.banuba.com%2Ffar-sdk%2Ftutorials%2Fdevelopment%2Fsamples%2Fweb.md%20\(web\)%20and%20help%20me%20with%20it%20in%20my%20Banuba%20Face%20AR%20SDK%20project.)[![](/far-sdk/img/ai-guide/claude.svg)Open in Claude](https://claude.ai/new?q=Read%20https%3A%2F%2Fdocs.banuba.com%2Ffar-sdk%2Ftutorials%2Fdevelopment%2Fsamples%2Fweb.md%20\(web\)%20and%20help%20me%20with%20it%20in%20my%20Banuba%20Face%20AR%20SDK%20project.)Install tools ## Quickstart[​](#quickstart "Direct link to Quickstart") **[Quickstart Web](https://github.com/Banuba/quickstart-web)** ## Beauty[​](#beauty "Direct link to Beauty") **[Beauty demo app](https://github.com/Banuba/beauty-web)** ## Vue[​](#vue "Direct link to Vue") **[Vue demo app](https://github.com/Banuba/quickstart-web-vue)** ## Angular[​](#angular "Direct link to Angular") **[Angular demo app](https://github.com/Banuba/quickstart-web-angular)** ## React[​](#react "Direct link to React") **[React demo app](https://github.com/Banuba/quickstart-web-react)** ``` import React, { useEffect } from "react" import data from "@banuba/webar/BanubaSDK.data" import wasm from "@banuba/webar/BanubaSDK.wasm" import simd from "@banuba/webar/BanubaSDK.simd.wasm" import FaceTracker from "@banuba/webar/face_tracker.zip" import Glasses from "/path/to/Glasses.zip" import { Webcam, Player, Module, Effect, Dom } from "@banuba/webar" export default function WebARComponent() { // componentDidMount useEffect(() => { let webcam Player.create({ clientToken: "xxx-xxx-xxx", locateFile: { "BanubaSDK.data": data, "BanubaSDK.wasm": wasm, "BanubaSDK.simd.wasm": simd, }, }).then(async (player) => { await player.addModule(new Module(FaceTracker)) await player.use(webcam = new Webcam()) player.applyEffect(new Effect(Glasses)) Dom.render(player, "#webar") }) // componentWillUnmount return () => { webcam?.stop() Dom.unmount("#webar") } }) return
} ``` tip See [Bundlers](/far-sdk/tutorials/development/installation.md#bundlers) for notes about specific bundlers and `locateFile` usage. ## Agora[​](#agora "Direct link to Agora") **[Banuba Agora Extension](https://github.com/Banuba/agora-plugin-filters-web)** **[Video call demo app](https://github.com/Banuba/videocall-web)** ## ZEGOCLOUD[​](#zegocloud "Direct link to ZEGOCLOUD") **[ZEGOCLOUD sample](https://github.com/Banuba/banuba_sdk_zegocloud_sdk_web)** ## OpenTok (TokBox)[​](#opentok-tokbox "Direct link to OpenTok (TokBox)") **[TokBox demo app](https://github.com/Banuba/videocall-tokbox-web)** ## Customizing video source[​](#customizing-video-source "Direct link to Customizing video source") You can easily modify the built-in [Webcam](/far-sdk/generated/typedoc/classes/Webcam.html#constructor) module video by passing [parameters](https://developer.mozilla.org/en-US/docs/Web/API/MediaTrackConstraints#properties_of_video_tracks) to it. ### Switching from front to back camera[​](#switching-from-front-to-back-camera "Direct link to Switching from front to back camera") For example, you can use back camera of the device by passing [facingMode](https://developer.mozilla.org/en-US/docs/Web/API/MediaTrackConstraints/facingMode) parameter to it: ``` // ... // The default facingMode value is "user" which means front camera // The "environment" value here means back camera await player.use(new Webcam({ facingMode: "environment" })) // ... ``` ### Rendering WebAR video in full screen on mobile[​](#rendering-webar-video-in-full-screen-on-mobile "Direct link to Rendering WebAR video in full screen on mobile") Simply add the following CSS to force WebAR AR video to fill viewport: ``` ``` If you decide to specify the exact `width` and `height` for the [Webcam](/far-sdk/generated/typedoc/classes/Webcam.html#constructor), pay attention that on several mobile devices/operating systems webcam video width and height may be flipped. It's a known platform-specific [webcam bug](https://stackoverflow.com/questions/62538271/getusermedia-selfie-full-screen-on-mobile/62598616#62598616). To work around it, swap the `width` and `height` values: ``` const desiredWidth = 360 const desiredHeight = 540 await player.use(new Webcam({ width: desiredHeight, height: desiredWidth, })) ``` Also, you may want to check out the [Video cropping](#video-cropping) section for more advanced scenarios. ### External MediaStream[​](#external-mediastream "Direct link to External MediaStream") If the built-in [Webcam](/far-sdk/generated/typedoc/classes/Webcam.html#constructor) can not fit your needs you can use a custom [MediaStream](/far-sdk/generated/typedoc/classes/MediaStream.html) with [Player](/far-sdk/generated/typedoc/classes/Player.html#constructor): ``` import { MediaStream /* ... */ } from "@banuba/webar" // ... /* process video from the camera */ const camera = await navigator.mediaDevices.getUserMedia({ audio: true, video: true }) await player.use(new MediaStream(camera)) /* or even from another canvas */ const canvas = $("canvas").captureStream() await player.use(new MediaStream(canvas)) // ... ``` See [MediaStream](/far-sdk/generated/typedoc/classes/MediaStream.html) docs for more details. ## Capturing processed video[​](#capturing-processed-video "Direct link to Capturing processed video") You can easily capture the processed video, take screenshots, video recordings or pass the captured video to a WebRTC connection. ### Screenshot[​](#screenshot "Direct link to Screenshot") ``` import { ImageCapture /* ... */ } from "@banuba/webar" // ... const capture = new ImageCapture(player) const photo = await capture.takePhoto() // ... ``` See [ImageCapture.takePhoto()](/far-sdk/generated/typedoc/classes/ImageCapture.html#takePhoto) docs for more details. ### Video[​](#video "Direct link to Video") ``` import { VideoRecorder /* ... */ } from "@banuba/webar" // ... const recorder = new VideoRecorder(player) recorder.start() await new Promise((r) => setTimeout(r, 5000)) // wait for 5 sec const video = await recorder.stop() // ... ``` See [VideoRecorder](/far-sdk/generated/typedoc/classes/VideoRecorder.html) docs for more details. ### MediaStream[​](#mediastream "Direct link to MediaStream") ``` import { MediaStreamCapture /* ... */ } from "@banuba/webar" // ... // the capture is an instance of window.MediaStream const capture = new MediaStreamCapture(player) // so it can be used as a video source $("video").srcObject = capture // or can be added to a WebRTC peer connection const connection = new RTCPeerConnection() connection.addTrack(capture.getVideoTrack()) // ... ``` See [MediaStreamCapture](/far-sdk/generated/typedoc/classes/MediaStreamCapture.html) docs for more details. ## Video cropping[​](#video-cropping "Direct link to Video cropping") You can adjust video frame dimensions via [Webcam](/far-sdk/generated/typedoc/classes/Webcam.html#constructor) constructor parameters: ``` const wcam = new Webcam({ width: 320, height: 240 }) ``` But this approach is platform-dependent and varies between browsers, e.g. some browsers may be unable to produces frames of the requested dimensions and can yield frames of close but different dimensions instead (e.g. 352x288 instead of requested 320x240). To work around this platform-specific limitations, you can leverage the built-in SDK crop modificator: ``` const desiredWidth = 320 const desiredHeight = 240 function crop(renderWidth, renderHeight) { const dx = (renderWidth - desiredWidth) / 2 const dy = (renderHeight - desiredHeight) / 2 return [dx, dy, desiredWidth, desiredHeight] } await player.use(webcam, { crop }) ``` This way you can get the desired frame size regardless of the platform used. See [Player.use()](/far-sdk/generated/typedoc/classes/Player.html#use) and [InputOptions](//generated/typedoc/types/InputOptions.html) docs for more datails. ## Postprocessing[​](#postprocessing "Direct link to Postprocessing") It's possible to post-process the video processed by WebAR SDK. You can grab the idea from the code-snippet: ``` import { MediaStreamCapture /* ... */ } from "@banuba/webar" // ... const capture = document.createElement("video") capture.autoplay = true capture.srcObject = new MediaStreamCapture(player) const canvas = document.getElementById("postprocessed") const ctx = canvas.getContext("2d") const fontSize = 48 * window.devicePixelRatio function postprocess() { canvas.width = capture.videoWidth canvas.height = capture.videoHeight ctx.drawImage(capture, 0, 0) ctx.font = `${fontSize}px serif` ctx.fillStyle = "red" ctx.fillText("A Watermark", 0.5 * fontSize, 1.25 * fontSize) } ;(function loop() { postprocess() requestAnimationFrame(loop) })() ``` See [Capturing processed video](#capturing-processed-video) > [MediaStream](#mediastream) for details. --- # Using video calls with the Banuba SDK [View as Markdown](https://docs.banuba.com/far-sdk/tutorials/development/videocall.md)[![](/far-sdk/img/ai-guide/chatgpt.svg)Open in ChatGPT](https://chatgpt.com/?q=Read%20https%3A%2F%2Fdocs.banuba.com%2Ffar-sdk%2Ftutorials%2Fdevelopment%2Fvideocall.md%20\(Using%20video%20calls%20with%20the%20Banuba%20SDK\)%20and%20help%20me%20with%20it%20in%20my%20Banuba%20Face%20AR%20SDK%20project.)[![](/far-sdk/img/ai-guide/claude.svg)Open in Claude](https://claude.ai/new?q=Read%20https%3A%2F%2Fdocs.banuba.com%2Ffar-sdk%2Ftutorials%2Fdevelopment%2Fvideocall.md%20\(Using%20video%20calls%20with%20the%20Banuba%20SDK\)%20and%20help%20me%20with%20it%20in%20my%20Banuba%20Face%20AR%20SDK%20project.)Install tools In our example, **AgoraRTC SDK** is used for video streaming. But the integration can be done based on any video streaming library. important You should have client tokens for both **AgoraRTC SDK** and **Banuba SDK**.
To receive **Banuba SDK** token, fill in the [form on banuba.com](https://www.banuba.com/facear-sdk/face-filters#form), or contact us via .
To generate an **AgoraRTC SDK** tokens, visit [Agora website](https://www.agora.io/). * iOS * Android * Web * Flutter * ReactNative [Example of using video calls with the Banuba SDK](https://github.com/Banuba/banuba-sdk-ios-samples/tree/master/videocall)

## Installation[​](#installation "Direct link to Installation") 1. Add `banuba-sdk-podspecs` repo along with `AgoraRtcEngine_iOS` and `BanubaSdk` packages into the your `Podfile`. Alternatively you may use our [SPM modules](/far-sdk/tutorials/development/installation.md#spm-packages) info See the details about the **Banuba SDK** packages in [Installation](/far-sdk/tutorials/development/installation.md). ## Integration[​](#integration "Direct link to Integration") 1. Setup client tokens videocall/videocall/ViewModel.swift ``` loading... ``` ![Icon](/far-sdk/img/icons/github.svg "GitHub") 2. Initialize `BanubaSdkManager` common/common/AppDelegate.swift ``` loading... ``` ![Icon](/far-sdk/img/icons/github.svg "GitHub") 3. Initialize `AgoraRtcEngineKit`, setup video/audio encoders and join the channel videocall/videocall/ViewModel.swift ``` loading... ``` ![Icon](/far-sdk/img/icons/github.svg "GitHub") 4. Setup `Player`, load the effect and start `Camera` frames forwarding videocall/videocall/ViewModel.swift ``` loading... ``` ![Icon](/far-sdk/img/icons/github.svg "GitHub") 5. Run the application! πŸŽ‰ πŸš€ πŸ’… [Example of using video calls with the Banuba SDK](https://github.com/Banuba/banuba-sdk-android-samples/tree/master/videocall)

For a video call, you need to receive frames as an array of pixels frame by frame in RGBA format. This can be done using `FrameOutput`. Just create a variable called `frameOutput` and add a callback that will receive an array of pixels `framePixelBuffer`: videocall/src/main/java/com/banuba/sdk/example/videocall/MainActivity.kt ``` loading... ``` ![Icon](/far-sdk/img/icons/github.svg "GitHub") Now, when initializing the player, the created variable is set to `player.use(myInput, frameOutput)`. ## Follow these steps to configure Videocall[​](#follow-these-steps-to-configure-videocall "Direct link to Follow these steps to configure Videocall") important To get started, add the **Banuba SDK** [integration code](/far-sdk/tutorials/development/basic_integration.md#integration) to your project. Add the **AgoraRTC SDK** dependency to your `build.gradle.kts`: videocall/build.gradle.kts ``` loading... ``` ![Icon](/far-sdk/img/icons/github.svg "GitHub") This example is based on the [**Player API**](/far-sdk/tutorials/development/basic_integration.md#integration). First, create the **Banuba SDK** core - `player`. Create a `surfaceOutput` that will draw the processed image from the **Banuba SDK**. Create a `frameOutput` that will produce the processed image and transfer it to **Agora** as an array of pixels. And create a camera `cameraDevice` and manage it yourself. **Agora** also has its own camera module, but in this example the **Agora** camera is not used, so `setExternalVideoSource(...)` is called to disable the **Agora** camera: videocall/src/main/java/com/banuba/sdk/example/videocall/MainActivity.kt ``` loading... ``` ![Icon](/far-sdk/img/icons/github.svg "GitHub") Create the **Agora** core with which the videocall will be made - `agoraRtc`, inside we indicate where the **Agora** will draw the received frames: videocall/src/main/java/com/banuba/sdk/example/videocall/MainActivity.kt ``` loading... ``` ![Icon](/far-sdk/img/icons/github.svg "GitHub") And then initialize everything and set up a video call in the` onCreate(...)` method. ## How it works[​](#how-it-works "Direct link to How it works") Frames from the **Banuba** camera are processed in the `player`, and then the result is passed to the `onFrame(...)` handler. In the handler, frames are passed to the **Agora** module via `agoraRtc.pushExternalVideoFrame(...)`. And then **Agora** transmits the launched frame to the server, and this is how the video call works. ## Fully working code[​](#fully-working-code "Direct link to Fully working code") MainActivity.kt ``` loading... ``` ![Icon](/far-sdk/img/icons/github.svg "GitHub") * Agora * OpenTok * WebRTC ### Agora[​](#agora "Direct link to Agora") tip Check the official **[Banuba Agora Extension](https://github.com/Banuba/agora-plugin-filters-web)** for Web [@banuba/agora-extension](https://www.npmjs.com/package/@banuba/agora-extension). Or if you need finer control, you may use the **Banuba WebAR** directly: 1. Import **AgoraRTC** and **Banuba WebAR** index.html ``` loading... ``` ![Icon](/far-sdk/img/icons/github.svg "GitHub") index.html ``` loading... ``` ![Icon](/far-sdk/img/icons/github.svg "GitHub") 2. Setup client tokens AgoraAppId.js ``` loading... ``` ![Icon](/far-sdk/img/icons/github.svg "GitHub") BanubaClientToken.js ``` loading... ``` ![Icon](/far-sdk/img/icons/github.svg "GitHub") 3. Initialize `Player` index.html ``` loading... ``` ![Icon](/far-sdk/img/icons/github.svg "GitHub") 4. Initialize `AgoraRTC` index.html ``` loading... ``` ![Icon](/far-sdk/img/icons/github.svg "GitHub") 5. Connect `Player` to `AgoraRTC` index.html ``` loading... ``` ![Icon](/far-sdk/img/icons/github.svg "GitHub") 6. Run the application! πŸŽ‰ πŸš€ πŸ’… tip See [AgoraWebSDK NG API docs](https://agoraio-community.github.io/AgoraWebSDK-NG/api/en/interfaces/iagorartc.html#createcustomvideotrack) for details. tip See [Banuba Video call demo app](https://github.com/Banuba/videocall-web) for more code examples. ### OpenTok (TokBox)[​](#opentok-tokbox "Direct link to OpenTok (TokBox)") ``` import "https://cdn.jsdelivr.net/npm/@opentok/client" import { MediaStream, Player, Module Effect, MediaStreamCapture } from "https://cdn.jsdelivr.net/npm/@banuba/webar/dist/BanubaSDK.browser.esm.js" // ... const camera = await navigator.mediaDevices.getUserMedia({ audio: true, video: true }) const player = await Player.create({ clientToken: "xxx-xxx-xxx" }) await player.addModule(new Module("https://cdn.jsdelivr.net/npm/@banuba/webar/dist/modules/background.zip")) const webar = new MediaStreamCapture(player) await player.use(new MediaStream(camera)) player.applyEffect(new Effect("BackgroundBlur.zip")) player.play() // original audio const audio = camera.getAudioTracks()[0] // webar processed video const video = webar.getVideoTracks()[0] const session = OT.initSession("OT API KEY", "OT SESSION ID") session.connect("OT SESSION TOKEN", async () => { const publisher = await OT.initPublisher( "publisher", { insertMode: "append", audioSource: audio, videoSource: video, width: "100%", height: "100%", }, () => {}, ) session.publish(publisher, () => {}) }) // ... ``` tip See [TokBox Video API docs](https://tokbox.com/developer/sdks/js/reference/OT.html#initPublisher) for details. tip See [Banuba Video call (TokBox) demo app](https://github.com/Banuba/videocall-tokbox-web) for more code examples. ### WebRTC[​](#webrtc "Direct link to WebRTC") Considering the [Fireship WebRTC demo](https://github.com/fireship-io/webrtc-firebase-demo/blob/main/main.js) ``` import { MediaStream as BanubaMediaStream, Player, Module, Effect, MediaStreamCapture, } from "https://cdn.jsdelivr.net/npm/@banuba/webar/dist/BanubaSDK.browser.esm.js" // ... webcamButton.onclick = async () => { localStream = await navigator.mediaDevices.getUserMedia({ video: true, audio: true }) remoteStream = new MediaStream() const player = await Player.create({ clientToken: "xxx-xxx-xxx" }) await player.addModule(new Module("https://cdn.jsdelivr.net/npm/@banuba/webar/dist/modules/background.zip")) const webar = new MediaStreamCapture(player) await player.use(new BanubaMediaStream(localStream)) player.applyEffect(new Effect("BackgroundBlur.zip")) player.play() // original audio const audio = localStream.getAudioTracks()[0] // webar processed video const video = webar.getVideoTracks()[0] localStream = new MediaStream([audio, video]) // Push tracks from local stream to peer connection localStream.getTracks().forEach((track) => { pc.addTrack(track, localStream) }) // ... } ``` Due to **Flutter** limitations, for every videocall solution you have to create a **native Flutter plugin**. We have developed one for integration with [Agora](https://docs.agora.io). It is expected that you will develop your own [Flutter plugin](https://docs.flutter.dev/packages-and-plugins/developing-packages) if **Agora** isn't suitable for you. We have created [Agora Extension](https://docs.agora.io/en/video-calling/develop/use-an-extension?platform=flutter) which is accessible from **Flutter**. [The sample](https://github.com/Banuba/banuba-agora-flutter-sdk) described below is a fork of [Flutter plugin of Agora](https://github.com/AgoraIO-Extensions/Agora-Flutter-SDK). Follow the instructions in `README.md` to run it. These are the general steps to integrate the sample code into your app: * Android * iOS 1. Add the **Client Token** and extension properties keys constants example/lib/examples/basic/join\_channel\_video/join\_channel\_video.dart ``` loading... ``` ![Icon](/far-sdk/img/icons/github.svg "GitHub") 2. Add common methods to interact with **Banuba extension** example/lib/examples/basic/join\_channel\_video/join\_channel\_video.dart ``` loading... ``` ![Icon](/far-sdk/img/icons/github.svg "GitHub") 3. Initialize **Banuba** and load an **effect** example/lib/examples/basic/join\_channel\_video/join\_channel\_video.dart ``` loading... ``` ![Icon](/far-sdk/img/icons/github.svg "GitHub") 4. Copy effects in [`assets/effects`](https://github.com/Banuba/banuba-agora-flutter-sdk/tree/main/example/assets/effects) folder 5. Add a reference to **Banuba Maven repo** example/android/build.gradle ``` loading... ``` ![Icon](/far-sdk/img/icons/github.svg "GitHub") 6. Add **Banuba dependencies** and prepare a task to copy effects into app example/android/app/build.gradle ``` loading... ``` ![Icon](/far-sdk/img/icons/github.svg "GitHub") 1. Add **Client Token** and extension properties keys constants example/lib/examples/basic/join\_channel\_video/join\_channel\_video.dart ``` loading... ``` ![Icon](/far-sdk/img/icons/github.svg "GitHub") 2. Add common methods to interact with **Banuba extension** example/lib/examples/basic/join\_channel\_video/join\_channel\_video.dart ``` loading... ``` ![Icon](/far-sdk/img/icons/github.svg "GitHub") 3. Initialize **Banuba** and load an **effect** example/lib/examples/basic/join\_channel\_video/join\_channel\_video.dart ``` loading... ``` ![Icon](/far-sdk/img/icons/github.svg "GitHub") 4. Copy effects in [`assets/effects`](https://github.com/Banuba/banuba-agora-flutter-sdk/tree/main/example/assets/effects) folder 5. Add Banuba dependencies to `Podfile` example/ios/Podfile ``` loading... ``` ![Icon](/far-sdk/img/icons/github.svg "GitHub") 6. Add the `effects` folder added earlier into your project. Link it with your app: add the folder into `Runner` **Xcode** project (`File` -> `Add Files to 'Runner'...`). Due to **React Native** limitations, for every videocall solution you have to create a **React native module**. We developed one for integration with [Agora](https://docs.agora.io). It is expected that you will develop your own [React native module](https://reactnative.dev/docs/native-modules-intro) if **Agora** isn't suitable for you. We have created [Agora Extension](https://docs.agora.io/en/video-calling/develop/use-an-extension?platform=react-native) which is accessible from **React Native**. [The sample](https://github.com/Banuba/banuba-react-native-agora) described below is a fork of [React Native around Agora](https://github.com/AgoraIO-Extensions/react-native-agora). Follow the instructions in `README.md` to run it. These are general steps to integrate the sample code into your app: * Android * iOS 1. Add the **Client Token** and extension properties keys constants example/src/examples/basic/JoinChannelVideo/JoinChannelVideo.tsx ``` loading... ``` ![Icon](/far-sdk/img/icons/github.svg "GitHub") 2. Add the common methods to interact with **Banuba extension** example/src/examples/basic/JoinChannelVideo/JoinChannelVideo.tsx ``` loading... ``` ![Icon](/far-sdk/img/icons/github.svg "GitHub") 3. Initialize **Banuba** and load an **effect** example/src/examples/basic/JoinChannelVideo/JoinChannelVideo.tsx ``` loading... ``` ![Icon](/far-sdk/img/icons/github.svg "GitHub") warning `intiBanuba()` must be called just after `engine.initialize(...)`, calling it later will cause an error during extention loading. 4. Copy effects in [`effects`](https://github.com/Banuba/banuba-react-native-agora/tree/main/example/effects) folder 5. Add a reference to the **Banuba Maven repo** example/android/build.gradle ``` loading... ``` ![Icon](/far-sdk/img/icons/github.svg "GitHub") 6. Add **Banuba dependencies** and prepare a task to copy effects into app example/android/app/build.gradle ``` loading... ``` ![Icon](/far-sdk/img/icons/github.svg "GitHub") 1. Add **Client Token** and extension properties keys constants example/src/examples/basic/JoinChannelVideo/JoinChannelVideo.tsx ``` loading... ``` ![Icon](/far-sdk/img/icons/github.svg "GitHub") 2. Add common methods to interact with the **Banuba extension** example/src/examples/basic/JoinChannelVideo/JoinChannelVideo.tsx ``` loading... ``` ![Icon](/far-sdk/img/icons/github.svg "GitHub") 3. Initialize **Banuba** and load an **effect** example/src/examples/basic/JoinChannelVideo/JoinChannelVideo.tsx ``` loading... ``` ![Icon](/far-sdk/img/icons/github.svg "GitHub") warning `intiBanuba()` must be called immediately after `engine.initialize(...)`, calling it later will cause an error during extention loading. 4. Copy effects in [`effects`](https://github.com/Banuba/banuba-react-native-agora/tree/main/example/effects) folder 5. Add **Banuba dependencies** to `Podfile` example/ios/Podfile ``` loading... ``` ![Icon](/far-sdk/img/icons/github.svg "GitHub") 6. Add the `effects` folder added earlier into your project. Link it with your app: add the folder into **Xcode** project (`File` -> `Add Files to ''...`). --- # android [View as Markdown](https://docs.banuba.com/far-sdk/tutorials/development/videocall/android.md)[![](/far-sdk/img/ai-guide/chatgpt.svg)Open in ChatGPT](https://chatgpt.com/?q=Read%20https%3A%2F%2Fdocs.banuba.com%2Ffar-sdk%2Ftutorials%2Fdevelopment%2Fvideocall%2Fandroid.md%20\(android\)%20and%20help%20me%20with%20it%20in%20my%20Banuba%20Face%20AR%20SDK%20project.)[![](/far-sdk/img/ai-guide/claude.svg)Open in Claude](https://claude.ai/new?q=Read%20https%3A%2F%2Fdocs.banuba.com%2Ffar-sdk%2Ftutorials%2Fdevelopment%2Fvideocall%2Fandroid.md%20\(android\)%20and%20help%20me%20with%20it%20in%20my%20Banuba%20Face%20AR%20SDK%20project.)Install tools [Example of using video calls with the Banuba SDK](https://github.com/Banuba/banuba-sdk-android-samples/tree/master/videocall)

For a video call, you need to receive frames as an array of pixels frame by frame in RGBA format. This can be done using `FrameOutput`. Just create a variable called `frameOutput` and add a callback that will receive an array of pixels `framePixelBuffer`: videocall/src/main/java/com/banuba/sdk/example/videocall/MainActivity.kt ``` loading... ``` ![Icon](/far-sdk/img/icons/github.svg "GitHub") Now, when initializing the player, the created variable is set to `player.use(myInput, frameOutput)`. ## Follow these steps to configure Videocall[​](#follow-these-steps-to-configure-videocall "Direct link to Follow these steps to configure Videocall") important To get started, add the **Banuba SDK** [integration code](/far-sdk/tutorials/development/basic_integration.md#integration) to your project. Add the **AgoraRTC SDK** dependency to your `build.gradle.kts`: videocall/build.gradle.kts ``` loading... ``` ![Icon](/far-sdk/img/icons/github.svg "GitHub") This example is based on the [**Player API**](/far-sdk/tutorials/development/basic_integration.md#integration). First, create the **Banuba SDK** core - `player`. Create a `surfaceOutput` that will draw the processed image from the **Banuba SDK**. Create a `frameOutput` that will produce the processed image and transfer it to **Agora** as an array of pixels. And create a camera `cameraDevice` and manage it yourself. **Agora** also has its own camera module, but in this example the **Agora** camera is not used, so `setExternalVideoSource(...)` is called to disable the **Agora** camera: videocall/src/main/java/com/banuba/sdk/example/videocall/MainActivity.kt ``` loading... ``` ![Icon](/far-sdk/img/icons/github.svg "GitHub") Create the **Agora** core with which the videocall will be made - `agoraRtc`, inside we indicate where the **Agora** will draw the received frames: videocall/src/main/java/com/banuba/sdk/example/videocall/MainActivity.kt ``` loading... ``` ![Icon](/far-sdk/img/icons/github.svg "GitHub") And then initialize everything and set up a video call in the` onCreate(...)` method. ## How it works[​](#how-it-works "Direct link to How it works") Frames from the **Banuba** camera are processed in the `player`, and then the result is passed to the `onFrame(...)` handler. In the handler, frames are passed to the **Agora** module via `agoraRtc.pushExternalVideoFrame(...)`. And then **Agora** transmits the launched frame to the server, and this is how the video call works. ## Fully working code[​](#fully-working-code "Direct link to Fully working code") MainActivity.kt ``` loading... ``` ![Icon](/far-sdk/img/icons/github.svg "GitHub") --- # flutter [View as Markdown](https://docs.banuba.com/far-sdk/tutorials/development/videocall/flutter.md)[![](/far-sdk/img/ai-guide/chatgpt.svg)Open in ChatGPT](https://chatgpt.com/?q=Read%20https%3A%2F%2Fdocs.banuba.com%2Ffar-sdk%2Ftutorials%2Fdevelopment%2Fvideocall%2Fflutter.md%20\(flutter\)%20and%20help%20me%20with%20it%20in%20my%20Banuba%20Face%20AR%20SDK%20project.)[![](/far-sdk/img/ai-guide/claude.svg)Open in Claude](https://claude.ai/new?q=Read%20https%3A%2F%2Fdocs.banuba.com%2Ffar-sdk%2Ftutorials%2Fdevelopment%2Fvideocall%2Fflutter.md%20\(flutter\)%20and%20help%20me%20with%20it%20in%20my%20Banuba%20Face%20AR%20SDK%20project.)Install tools Due to **Flutter** limitations, for every videocall solution you have to create a **native Flutter plugin**. We have developed one for integration with [Agora](https://docs.agora.io). It is expected that you will develop your own [Flutter plugin](https://docs.flutter.dev/packages-and-plugins/developing-packages) if **Agora** isn't suitable for you. We have created [Agora Extension](https://docs.agora.io/en/video-calling/develop/use-an-extension?platform=flutter) which is accessible from **Flutter**. [The sample](https://github.com/Banuba/banuba-agora-flutter-sdk) described below is a fork of [Flutter plugin of Agora](https://github.com/AgoraIO-Extensions/Agora-Flutter-SDK). Follow the instructions in `README.md` to run it. These are the general steps to integrate the sample code into your app: * Android * iOS 1. Add the **Client Token** and extension properties keys constants example/lib/examples/basic/join\_channel\_video/join\_channel\_video.dart ``` loading... ``` ![Icon](/far-sdk/img/icons/github.svg "GitHub") 2. Add common methods to interact with **Banuba extension** example/lib/examples/basic/join\_channel\_video/join\_channel\_video.dart ``` loading... ``` ![Icon](/far-sdk/img/icons/github.svg "GitHub") 3. Initialize **Banuba** and load an **effect** example/lib/examples/basic/join\_channel\_video/join\_channel\_video.dart ``` loading... ``` ![Icon](/far-sdk/img/icons/github.svg "GitHub") 4. Copy effects in [`assets/effects`](https://github.com/Banuba/banuba-agora-flutter-sdk/tree/main/example/assets/effects) folder 5. Add a reference to **Banuba Maven repo** example/android/build.gradle ``` loading... ``` ![Icon](/far-sdk/img/icons/github.svg "GitHub") 6. Add **Banuba dependencies** and prepare a task to copy effects into app example/android/app/build.gradle ``` loading... ``` ![Icon](/far-sdk/img/icons/github.svg "GitHub") 1. Add **Client Token** and extension properties keys constants example/lib/examples/basic/join\_channel\_video/join\_channel\_video.dart ``` loading... ``` ![Icon](/far-sdk/img/icons/github.svg "GitHub") 2. Add common methods to interact with **Banuba extension** example/lib/examples/basic/join\_channel\_video/join\_channel\_video.dart ``` loading... ``` ![Icon](/far-sdk/img/icons/github.svg "GitHub") 3. Initialize **Banuba** and load an **effect** example/lib/examples/basic/join\_channel\_video/join\_channel\_video.dart ``` loading... ``` ![Icon](/far-sdk/img/icons/github.svg "GitHub") 4. Copy effects in [`assets/effects`](https://github.com/Banuba/banuba-agora-flutter-sdk/tree/main/example/assets/effects) folder 5. Add Banuba dependencies to `Podfile` example/ios/Podfile ``` loading... ``` ![Icon](/far-sdk/img/icons/github.svg "GitHub") 6. Add the `effects` folder added earlier into your project. Link it with your app: add the folder into `Runner` **Xcode** project (`File` -> `Add Files to 'Runner'...`). --- # ios [View as Markdown](https://docs.banuba.com/far-sdk/tutorials/development/videocall/ios.md)[![](/far-sdk/img/ai-guide/chatgpt.svg)Open in ChatGPT](https://chatgpt.com/?q=Read%20https%3A%2F%2Fdocs.banuba.com%2Ffar-sdk%2Ftutorials%2Fdevelopment%2Fvideocall%2Fios.md%20\(ios\)%20and%20help%20me%20with%20it%20in%20my%20Banuba%20Face%20AR%20SDK%20project.)[![](/far-sdk/img/ai-guide/claude.svg)Open in Claude](https://claude.ai/new?q=Read%20https%3A%2F%2Fdocs.banuba.com%2Ffar-sdk%2Ftutorials%2Fdevelopment%2Fvideocall%2Fios.md%20\(ios\)%20and%20help%20me%20with%20it%20in%20my%20Banuba%20Face%20AR%20SDK%20project.)Install tools [Example of using video calls with the Banuba SDK](https://github.com/Banuba/banuba-sdk-ios-samples/tree/master/videocall)

## Installation[​](#installation "Direct link to Installation") 1. Add `banuba-sdk-podspecs` repo along with `AgoraRtcEngine_iOS` and `BanubaSdk` packages into the your `Podfile`. Alternatively you may use our [SPM modules](/far-sdk/tutorials/development/installation.md#spm-packages) info See the details about the **Banuba SDK** packages in [Installation](/far-sdk/tutorials/development/installation.md). ## Integration[​](#integration "Direct link to Integration") 1. Setup client tokens videocall/videocall/ViewModel.swift ``` loading... ``` ![Icon](/far-sdk/img/icons/github.svg "GitHub") 2. Initialize `BanubaSdkManager` common/common/AppDelegate.swift ``` loading... ``` ![Icon](/far-sdk/img/icons/github.svg "GitHub") 3. Initialize `AgoraRtcEngineKit`, setup video/audio encoders and join the channel videocall/videocall/ViewModel.swift ``` loading... ``` ![Icon](/far-sdk/img/icons/github.svg "GitHub") 4. Setup `Player`, load the effect and start `Camera` frames forwarding videocall/videocall/ViewModel.swift ``` loading... ``` ![Icon](/far-sdk/img/icons/github.svg "GitHub") 5. Run the application! πŸŽ‰ πŸš€ πŸ’… --- # react\_native [View as Markdown](https://docs.banuba.com/far-sdk/tutorials/development/videocall/react_native.md)[![](/far-sdk/img/ai-guide/chatgpt.svg)Open in ChatGPT](https://chatgpt.com/?q=Read%20https%3A%2F%2Fdocs.banuba.com%2Ffar-sdk%2Ftutorials%2Fdevelopment%2Fvideocall%2Freact_native.md%20\(react_native\)%20and%20help%20me%20with%20it%20in%20my%20Banuba%20Face%20AR%20SDK%20project.)[![](/far-sdk/img/ai-guide/claude.svg)Open in Claude](https://claude.ai/new?q=Read%20https%3A%2F%2Fdocs.banuba.com%2Ffar-sdk%2Ftutorials%2Fdevelopment%2Fvideocall%2Freact_native.md%20\(react_native\)%20and%20help%20me%20with%20it%20in%20my%20Banuba%20Face%20AR%20SDK%20project.)Install tools Due to **React Native** limitations, for every videocall solution you have to create a **React native module**. We developed one for integration with [Agora](https://docs.agora.io). It is expected that you will develop your own [React native module](https://reactnative.dev/docs/native-modules-intro) if **Agora** isn't suitable for you. We have created [Agora Extension](https://docs.agora.io/en/video-calling/develop/use-an-extension?platform=react-native) which is accessible from **React Native**. [The sample](https://github.com/Banuba/banuba-react-native-agora) described below is a fork of [React Native around Agora](https://github.com/AgoraIO-Extensions/react-native-agora). Follow the instructions in `README.md` to run it. These are general steps to integrate the sample code into your app: * Android * iOS 1. Add the **Client Token** and extension properties keys constants example/src/examples/basic/JoinChannelVideo/JoinChannelVideo.tsx ``` loading... ``` ![Icon](/far-sdk/img/icons/github.svg "GitHub") 2. Add the common methods to interact with **Banuba extension** example/src/examples/basic/JoinChannelVideo/JoinChannelVideo.tsx ``` loading... ``` ![Icon](/far-sdk/img/icons/github.svg "GitHub") 3. Initialize **Banuba** and load an **effect** example/src/examples/basic/JoinChannelVideo/JoinChannelVideo.tsx ``` loading... ``` ![Icon](/far-sdk/img/icons/github.svg "GitHub") warning `intiBanuba()` must be called just after `engine.initialize(...)`, calling it later will cause an error during extention loading. 4. Copy effects in [`effects`](https://github.com/Banuba/banuba-react-native-agora/tree/main/example/effects) folder 5. Add a reference to the **Banuba Maven repo** example/android/build.gradle ``` loading... ``` ![Icon](/far-sdk/img/icons/github.svg "GitHub") 6. Add **Banuba dependencies** and prepare a task to copy effects into app example/android/app/build.gradle ``` loading... ``` ![Icon](/far-sdk/img/icons/github.svg "GitHub") 1. Add **Client Token** and extension properties keys constants example/src/examples/basic/JoinChannelVideo/JoinChannelVideo.tsx ``` loading... ``` ![Icon](/far-sdk/img/icons/github.svg "GitHub") 2. Add common methods to interact with the **Banuba extension** example/src/examples/basic/JoinChannelVideo/JoinChannelVideo.tsx ``` loading... ``` ![Icon](/far-sdk/img/icons/github.svg "GitHub") 3. Initialize **Banuba** and load an **effect** example/src/examples/basic/JoinChannelVideo/JoinChannelVideo.tsx ``` loading... ``` ![Icon](/far-sdk/img/icons/github.svg "GitHub") warning `intiBanuba()` must be called immediately after `engine.initialize(...)`, calling it later will cause an error during extention loading. 4. Copy effects in [`effects`](https://github.com/Banuba/banuba-react-native-agora/tree/main/example/effects) folder 5. Add **Banuba dependencies** to `Podfile` example/ios/Podfile ``` loading... ``` ![Icon](/far-sdk/img/icons/github.svg "GitHub") 6. Add the `effects` folder added earlier into your project. Link it with your app: add the folder into **Xcode** project (`File` -> `Add Files to ''...`). --- # web [View as Markdown](https://docs.banuba.com/far-sdk/tutorials/development/videocall/web.md)[![](/far-sdk/img/ai-guide/chatgpt.svg)Open in ChatGPT](https://chatgpt.com/?q=Read%20https%3A%2F%2Fdocs.banuba.com%2Ffar-sdk%2Ftutorials%2Fdevelopment%2Fvideocall%2Fweb.md%20\(web\)%20and%20help%20me%20with%20it%20in%20my%20Banuba%20Face%20AR%20SDK%20project.)[![](/far-sdk/img/ai-guide/claude.svg)Open in Claude](https://claude.ai/new?q=Read%20https%3A%2F%2Fdocs.banuba.com%2Ffar-sdk%2Ftutorials%2Fdevelopment%2Fvideocall%2Fweb.md%20\(web\)%20and%20help%20me%20with%20it%20in%20my%20Banuba%20Face%20AR%20SDK%20project.)Install tools * Agora * OpenTok * WebRTC ### Agora[​](#agora "Direct link to Agora") tip Check the official **[Banuba Agora Extension](https://github.com/Banuba/agora-plugin-filters-web)** for Web [@banuba/agora-extension](https://www.npmjs.com/package/@banuba/agora-extension). Or if you need finer control, you may use the **Banuba WebAR** directly: 1. Import **AgoraRTC** and **Banuba WebAR** index.html ``` loading... ``` ![Icon](/far-sdk/img/icons/github.svg "GitHub") index.html ``` loading... ``` ![Icon](/far-sdk/img/icons/github.svg "GitHub") 2. Setup client tokens AgoraAppId.js ``` loading... ``` ![Icon](/far-sdk/img/icons/github.svg "GitHub") BanubaClientToken.js ``` loading... ``` ![Icon](/far-sdk/img/icons/github.svg "GitHub") 3. Initialize `Player` index.html ``` loading... ``` ![Icon](/far-sdk/img/icons/github.svg "GitHub") 4. Initialize `AgoraRTC` index.html ``` loading... ``` ![Icon](/far-sdk/img/icons/github.svg "GitHub") 5. Connect `Player` to `AgoraRTC` index.html ``` loading... ``` ![Icon](/far-sdk/img/icons/github.svg "GitHub") 6. Run the application! πŸŽ‰ πŸš€ πŸ’… tip See [AgoraWebSDK NG API docs](https://agoraio-community.github.io/AgoraWebSDK-NG/api/en/interfaces/iagorartc.html#createcustomvideotrack) for details. tip See [Banuba Video call demo app](https://github.com/Banuba/videocall-web) for more code examples. ### OpenTok (TokBox)[​](#opentok-tokbox "Direct link to OpenTok (TokBox)") ``` import "https://cdn.jsdelivr.net/npm/@opentok/client" import { MediaStream, Player, Module Effect, MediaStreamCapture } from "https://cdn.jsdelivr.net/npm/@banuba/webar/dist/BanubaSDK.browser.esm.js" // ... const camera = await navigator.mediaDevices.getUserMedia({ audio: true, video: true }) const player = await Player.create({ clientToken: "xxx-xxx-xxx" }) await player.addModule(new Module("https://cdn.jsdelivr.net/npm/@banuba/webar/dist/modules/background.zip")) const webar = new MediaStreamCapture(player) await player.use(new MediaStream(camera)) player.applyEffect(new Effect("BackgroundBlur.zip")) player.play() // original audio const audio = camera.getAudioTracks()[0] // webar processed video const video = webar.getVideoTracks()[0] const session = OT.initSession("OT API KEY", "OT SESSION ID") session.connect("OT SESSION TOKEN", async () => { const publisher = await OT.initPublisher( "publisher", { insertMode: "append", audioSource: audio, videoSource: video, width: "100%", height: "100%", }, () => {}, ) session.publish(publisher, () => {}) }) // ... ``` tip See [TokBox Video API docs](https://tokbox.com/developer/sdks/js/reference/OT.html#initPublisher) for details. tip See [Banuba Video call (TokBox) demo app](https://github.com/Banuba/videocall-tokbox-web) for more code examples. ### WebRTC[​](#webrtc "Direct link to WebRTC") Considering the [Fireship WebRTC demo](https://github.com/fireship-io/webrtc-firebase-demo/blob/main/main.js) ``` import { MediaStream as BanubaMediaStream, Player, Module, Effect, MediaStreamCapture, } from "https://cdn.jsdelivr.net/npm/@banuba/webar/dist/BanubaSDK.browser.esm.js" // ... webcamButton.onclick = async () => { localStream = await navigator.mediaDevices.getUserMedia({ video: true, audio: true }) remoteStream = new MediaStream() const player = await Player.create({ clientToken: "xxx-xxx-xxx" }) await player.addModule(new Module("https://cdn.jsdelivr.net/npm/@banuba/webar/dist/modules/background.zip")) const webar = new MediaStreamCapture(player) await player.use(new BanubaMediaStream(localStream)) player.applyEffect(new Effect("BackgroundBlur.zip")) player.play() // original audio const audio = localStream.getAudioTracks()[0] // webar processed video const video = webar.getVideoTracks()[0] localStream = new MediaStream([audio, video]) // Push tracks from local stream to peer connection localStream.getTracks().forEach((track) => { pc.addTrack(track, localStream) }) // ... } ``` --- [View as Markdown](https://docs.banuba.com/far-sdk/tutorials/unity/basic_integration.md)[![](/far-sdk/img/ai-guide/chatgpt.svg)Open in ChatGPT](https://chatgpt.com/?q=Read%20https%3A%2F%2Fdocs.banuba.com%2Ffar-sdk%2Ftutorials%2Funity%2Fbasic_integration.md%20\(Unity%20Face%20AR%20SDK%20Basic%20Integration%20guide\)%20and%20help%20me%20with%20it%20in%20my%20Banuba%20Face%20AR%20SDK%20project.)[![](/far-sdk/img/ai-guide/claude.svg)Open in Claude](https://claude.ai/new?q=Read%20https%3A%2F%2Fdocs.banuba.com%2Ffar-sdk%2Ftutorials%2Funity%2Fbasic_integration.md%20\(Unity%20Face%20AR%20SDK%20Basic%20Integration%20guide\)%20and%20help%20me%20with%20it%20in%20my%20Banuba%20Face%20AR%20SDK%20project.)Install tools # Banuba Face AR SDK for Unity Unity Face AR SDK allows developers to create cross-platform face tracking apps with custom AR effects in Unity3D. Banuba Face AR SDK for Unity is a native library compiled for the following platforms: * Windows * MacOS * iOS * Android ## Requirements[​](#requirements "Direct link to Requirements") * Latest Unity Hub with latest installed **Unity Editor LTS**. Minimum Unity Editor Version: **2019.3.13f1** * Installed Unity platform plugins for each required platform: Android, iOS or Desktop * **Windows**: Before launching Unity you should check presence of the following tools in Microsoft VS2019: ![Image](/far-sdk/assets/images/tools-52640adedca1f7d9dc839b2ab8ebeaeb.png) ## Usage[​](#usage "Direct link to Usage") ### Token[​](#token "Direct link to Token") Before you commit to a license, you are free to test all the features of the SDK for free. To start it, [send us a message](https://www.banuba.com/facear-sdk/face-filters#form).
We will get back to you with the trial token. You can store the token within the app. Feel free to [contact us](https://docs.banuba.com/far-sdk/support) if you have any questions. ### How To Import To Your Project[​](#how-to-import-to-your-project "Direct link to How To Import To Your Project") 1. Download the latest assets package [BanubaSDK-import.unitypackage ](https://github.com/Banuba/quickstart-unity/releases)and import it using Unity Package Manager. 2. Put your Client Token to the `Assets/Resources/BanubaClientToken.txt` 3. Find [Demo Scene](/far-sdk/tutorials/unity/demo_scene.md), open and click run! πŸŽ‰ πŸš€ πŸ’… --- # Unity Demo Scene [View as Markdown](https://docs.banuba.com/far-sdk/tutorials/unity/demo_scene.md)[![](/far-sdk/img/ai-guide/chatgpt.svg)Open in ChatGPT](https://chatgpt.com/?q=Read%20https%3A%2F%2Fdocs.banuba.com%2Ffar-sdk%2Ftutorials%2Funity%2Fdemo_scene.md%20\(Unity%20Demo%20Scene\)%20and%20help%20me%20with%20it%20in%20my%20Banuba%20Face%20AR%20SDK%20project.)[![](/far-sdk/img/ai-guide/claude.svg)Open in Claude](https://claude.ai/new?q=Read%20https%3A%2F%2Fdocs.banuba.com%2Ffar-sdk%2Ftutorials%2Funity%2Fdemo_scene.md%20\(Unity%20Demo%20Scene\)%20and%20help%20me%20with%20it%20in%20my%20Banuba%20Face%20AR%20SDK%20project.)Install tools Banuba SDK provides a **Demo Scene** for our Unity SDK. The Demo scene contains several AR effects. To launch the Demo scene on the device follow the steps bellow: 1. In the project files tree, find and open **BanubaSDKDemo.unity** which is located under *Assets* -> *BanubaFaceAR* -> *Demo* 2. To add effects, find the `EffectsManager` object in Hierarchy and populate `Effects` list in Inspector with desired effects prefabs that you can find inside folders under *Assets* -> *BanubaFaceAR* -> *Effects*. At launch, the list will already be filled with all available effects. ![image](/far-sdk/assets/images/demo_1-ca9fb65dd771d6a37a982d28d743b31b.png) 3. Select the **LoaderScene** and **BanubaSDKDemo** scenes in **File** -> **Build Settings** and make sure their indexes are 0 and 1 respectively. Then launch it on the needed platform as usual. ThatοΏ½s how a properly configured scene should look like. ![image](/far-sdk/assets/images/demo_2-5c138f26f171ec2ec2116b4bb145d819.png) --- [View as Markdown](https://docs.banuba.com/far-sdk/tutorials/unity/overview.md)[![](/far-sdk/img/ai-guide/chatgpt.svg)Open in ChatGPT](https://chatgpt.com/?q=Read%20https%3A%2F%2Fdocs.banuba.com%2Ffar-sdk%2Ftutorials%2Funity%2Foverview.md%20\(Unity%20Face%20AR%20SDK%20Overview\)%20and%20help%20me%20with%20it%20in%20my%20Banuba%20Face%20AR%20SDK%20project.)[![](/far-sdk/img/ai-guide/claude.svg)Open in Claude](https://claude.ai/new?q=Read%20https%3A%2F%2Fdocs.banuba.com%2Ffar-sdk%2Ftutorials%2Funity%2Foverview.md%20\(Unity%20Face%20AR%20SDK%20Overview\)%20and%20help%20me%20with%20it%20in%20my%20Banuba%20Face%20AR%20SDK%20project.)Install tools # Face AR SDK for Unity Overview ![image](/far-sdk/assets/images/unity_overview-90d94c3d663f22905d904de0c25da6dc.svg) Unity Face AR SDK provides ready-to-use assets (prefabs, scripts, materials, scenes and etc.) for fast integration based on BanubaSDKBridge API. ### BanubaSDKManager[​](#banubasdkmanager "Direct link to BanubaSDKManager") *script:* `Assets/BanubaFaceAR/BaseAssets/Scripts/BanubaSDKManager.cs` Public Properties: * Max Face Count - Set the maximum number of faces to search BanubaSDKManager.cs main functionality: * initialize BanubaSDK with [token](/far-sdk/tutorials/unity/basic_integration.md#usage) Assets/BanubaFaceAR/BaseAssets/Scripts/BanubaSDKManager.cs ``` private void Awake() { //... // Load Token var tokenResourceFile = Resources.Load("BanubaClientToken"); var tokenLine = tokenResourceFile.text.Trim(); // Banuba Face AR SDK static environment initialization var error = IntPtr.Zero; BanubaSDKBridge.bnb_recognizer_env_init(tokenLine, out error); Utils.CheckError(error); // The recognizer object init method needs a path to its resources. They are placed in // Assets/StreamingAssets folder, and unity does not compress resources placed there which is important. // Full path to Assets/StreamingAssets is platform dependent. Unity provides it as // Application.streamingAssetsPath property. // We recommend using only one instance of the recognizer object to decrease memory consumption. #if (UNITY_ANDROID || UNITY_WEBGL) && !UNITY_EDITOR var resourcesPath = Application.persistentDataPath; #else var resourcesPath = Application.streamingAssetsPath; #endif Recognizer = new Recognizer(resourcesPath + "/BanubaFaceAR/"); // set maximum faces to search BanubaSDKBridge.bnb_recognizer_set_max_faces(Recognizer, _maxFaceCount, out error); Utils.CheckError(error); //... } ``` * Process the input image and notify the subscribers if there is a recognition result with `public event Action onRecognitionResult;` Assets/BanubaFaceAR/BaseAssets/Scripts/BanubaSDKManager.cs ``` public static bool processCameraImage(BanubaSDKBridge.bnb_bpc8_image_t cameraImage) { if (instance == null) { return false; } var error = IntPtr.Zero; var frameData = BanubaSDKBridge.bnb_frame_data_init(out error); Utils.CheckError(error); BanubaSDKBridge.bnb_frame_data_set_bpc8_img(frameData, ref cameraImage, out error); Utils.CheckError(error); BanubaSDKBridge.bnb_recognizer_push_frame_data(instance.Recognizer, frameData, out error); Utils.CheckError(error); var outFrameData = new FrameData(); bool process = BanubaSDKBridge.bnb_recognizer_pop_frame_data(instance.Recognizer, outFrameData, out error); Utils.CheckError(error); if (process) { instance.onRecognitionResult?.Invoke(outFrameData); } return process; } ``` All other feature-based instances must subscribe to `onRecognitionResult` Feature Based Class Example ``` private void Awake() { BanubaSDKManager.instance.onRecognitionResult += OnRecognitionResult; } private void OnDestroy() { BanubaSDKManager.instance.onRecognitionResult -= OnRecognitionResult; } private void OnRecognitionResult(FrameData frameData) { // Do Something with frameData } ``` note Currently BanubaSDKManager.cs provides synchronous image processing only, but BanubaSDK API also privides async processing. ### Camera Device[​](#camera-device "Direct link to Camera Device") *script:* `Assets/BanubaFaceAR/BaseAssets/Scripts/CameraDevice.cs` Camera device class is based on [WebCamTexture](https://docs.unity3d.com/ScriptReference/WebCamTexture.html). It takes an image from the camera device and pushes it to the BanubaSDKManager Push Camera Frame Example ``` // Create Camera Image var cameraImage = new BanubaSDKBridge.bnb_bpc8_image_t { format = new BanubaSDKBridge.bnb_image_format_t() }; // Create Color32 pixel array var data = new Color32[texSize]; //... // Fill the data with WebCamTexture.GetPixels32(data) or any other method // depends on what you want to process //... // Marshaling GCHandle pinnedData = GCHandle.Alloc(data, GCHandleType.Pinned);; cameraImage.format.orientation = AngleToOrientation(0); // image orientation cameraImage.format.require_mirroring = 1; // selfie mode (horisontal flip) cameraImage.format.face_orientation = 0; cameraImage.format.width = (uint) width; cameraImage.format.height = (uint) height; cameraImage.data = pinnedData.AddrOfPinnedObject(); //retrieve the IntPtr cameraImage.pixel_format = BanubaSDKBridge.bnb_pixel_format_t.BNB_RGBA; //image format // Process image with BanubaSDKManager BanubaSDKManager.processCameraImage(cameraImage); // Free pinned data pinnedData.Free(); ``` It also provides `public event Action onCameraTexture;` that allows subscribers to retrieve some data from the original camera image like `Plane Controller` ### Camera[​](#camera "Direct link to Camera") Components: * *script:* `Assets/BanubaFaceAR/BaseAssets/Scripts/CameraController.cs` Retrieve the needed camera settings based on the recognition result. ### Camera Plane[​](#camera-plane "Direct link to Camera Plane") Components: * *script:* `Assets/BanubaFaceAR/BaseAssets/Scripts/PlaneController.cs` A Canvas UI element for rendering camera image with right transformation. A child of the `Surface Canvas` ## Face AR Effect[​](#face-ar-effect "Direct link to Face AR Effect") ### Faces Controller[​](#faces-controller "Direct link to Faces Controller") *script:* `Assets/BanubaFaceAR/BaseAssets/Scripts/FacesController.cs` A ready-to-use prefab for rendering Face Mesh Properties: * Enable UV Draw - Instanciate a UV Draw Component (Difference between static and dynamic face meshes) * Enable Static Pos - Instanciate a Static Pos Component (Face Mesh without mimics) Instanciate GameObjects with [FaceController](/far-sdk/tutorials/unity/overview.md#face-controller) Component depending on faces detected on the camera image. By default it contains Face0 Object as a child and if detected faces > 1, it copies Face0 and increments it's index. note Maximum searching face count setup placed in the properties of the [BanubaSDKManager](/far-sdk/tutorials/unity/overview.md#banubasdkmanager) ### Face Controller[​](#face-controller "Direct link to Face Controller") *script:* `Assets/BanubaFaceAR/BaseAssets/Scripts/FaceController.cs` Apply transformations retrieved from the Frame Data ### Face Mesh Controller[​](#face-mesh-controller "Direct link to Face Mesh Controller") *script:* `Assets/BanubaFaceAR/BaseAssets/Scripts/FaceMeshController.cs` A basic Component for rendering Face Mesh ### Face AR example[​](#face-ar-example "Direct link to Face AR example") ![Image](/far-sdk/assets/images/minimal_example-0b3c3601b09d49182318ac66d76b4554.png) ## Morphing[​](#morphing "Direct link to Morphing") ### Morphing Feature[​](#morphing-feature "Direct link to Morphing Feature") Components: * *script:* `Assets/BanubaFaceAR/FeatureMorphing/Scripts/MorphingFeature.cs` A ready-to-use prefab for applying face morphing filter. Public Properies: * Morph Shape - `IMorphDraw` component field. Defines the type of the morphing filter. Available shapes: * [UV Morphing](/far-sdk/tutorials/unity/overview.md#uv-morphing) * [Custom Morphing](/far-sdk/tutorials/unity/overview.md#custom-morphing) * [Faces Controller](/far-sdk/tutorials/unity/overview.md#faces-controller) * [Effect Reference](/far-sdk/tutorials/unity/overview.md#morphing-result-camera) ### Morphing Draw Camera[​](#morphing-draw-camera "Direct link to Morphing Draw Camera") Components: * *script:* `Assets/BanubaFaceAR/BaseAssets/Scripts/Blur.cs` * *script:* `Assets/BanubaFaceAR/BaseAssets/Scripts/CameraDevice.cs` * *script:* `Assets/BanubaFaceAR/BaseAssets/Scripts/RenderToTexture.cs` This camera renders a blurred difference between Face Mesh vertices and Morphing Shape vertices to the [RenderTexture](https://docs.unity3d.com/ScriptReference/RenderTexture.html) with `RenderToTexture.cs` Component #### UV Morphing[​](#uv-morphing "Direct link to UV Morphing") This shape morphs face mesh with the mesh provided by user. You can find a basic example of a uv morphing shape here `platform/unity/BanubaSdk/Assets/BanubaFaceAR/FeatureMorphing/effect/MorphTest.prefab` **Components:** * [MeshRenderer](https://docs.unity3d.com/ScriptReference/MeshRenderer.html) with the mesh of morph shape * *script:* `MorphDraw.cs`. Updates material properties. * *material:* `MorphDraw.mat`. important UV Morph draw shapes required `Enable UV Draw` and `Enable Static Pos` in [Faces Controller](/far-sdk/tutorials/unity/overview.md#faces-controller) ![morph](/far-sdk/assets/images/morph_draw-d234572f5a8cb8950f656becd28e1c08.png) #### Custom Morphing[​](#custom-morphing "Direct link to Custom Morphing") A ready-to-use Morph Shape with 28 parts of face that can be changed in the runtime. **Components:** * [MeshRenderer](https://docs.unity3d.com/ScriptReference/MeshRenderer.html) with the mesh of morph shape * *script:* `CustomMorphDraw.cs`. Updates material properties. * *material:* `CustomMorphDraw.mat`. ![custom morph](/far-sdk/assets/images/custom_morph_draw-32b063e5003825105e9319cc0bb85ffe.png) ### Morphing Result Camera[​](#morphing-result-camera "Direct link to Morphing Result Camera") **Components:** * *script:* `Assets/BanubaFaceAR/FeatureMorphing/Scripts/MorphingPostEffect.cs` * *script:* `Assets/BanubaFaceAR/BaseAssets/Scripts/CameraController.cs` It takes a Morphing result from the [Morphing Draw Camera](/far-sdk/tutorials/unity/overview.md#morphing-draw-camera) and applies morphing in the [OnRenderImage](https://docs.unity3d.com/ScriptReference/MonoBehaviour.OnRenderImage.html) event function ### Morphing Example[​](#morphing-example "Direct link to Morphing Example") You can find the basic example here: ![Image](/far-sdk/assets/images/morphing_example-ea67a7628f92f43a5314ac0115f3c882.png) ## Segmentation[​](#segmentation "Direct link to Segmentation") ### Segmentation Feature[​](#segmentation-feature "Direct link to Segmentation Feature") Components: * *script:* `Assets/BanubaFaceAR/FeatureSegmentation/Scripts/SegmentationFeature.cs` Properties: * [Type](/far-sdk/tutorials/unity/overview.md#segmentation-types) - type of the segmentation feature * Use Segmentation Shader - if enabled, uses the default segmentation shader located here `Assets/BanubaFaceAR/FeatureSegmentation/Shaders/Segmentation.shader` * Plane - [Camera Plain Reference](/far-sdk/tutorials/unity/overview.md#camera-plane) * *unityUI:* [Raw Image](https://docs.unity3d.com/2018.2/Documentation/ScriptReference/UI.RawImage.html) Location: `Assets/BanubaFaceAR/FeatureSegmentation/Prefabs/SegmentationFeature.prefab` A ready-to-use prefab with all segmentation features provided by **Banuba SDK**. ### Segmentation Types:[​](#segmentation-types "Direct link to Segmentation Types:") ``` public enum bnb_segm_type_t { BNB_BACKGROUND = 0, BNB_FACE, BNB_HAIR, BNB_NECK, BNB_SKIN, BNB_LIPS, BNB_BROW_LEFT, BNB_BROW_RIGHT, BNB_EYE_PUPIL_LEFT, BNB_EYE_PUPIL_RIGHT, BNB_EYE_SCLERA_LEFT, BNB_EYE_SCLERA_RIGHT, BNB_EYE_IRIS_LEFT, BNB_EYE_IRIS_RIGHT, BNB_FACE_SKIN } ``` ### Segmentation Example:[​](#segmentation-example "Direct link to Segmentation Example:") **Background:** ![Image](/far-sdk/assets/images/segmentation_example-ff253946249a7179eff32aa4acea9c65.png) **Face Skin:** ![Image](/far-sdk/assets/images/segmentation_face_example-7a3cfca57e4ac3ad05e62602b84fa0ca.png) ## MakeUp[​](#makeup "Direct link to MakeUp") ### How to add Makeup to your Effect[​](#how-to-add-makeup-to-your-effect "Direct link to How to add Makeup to your Effect") 1. Find `Makeup.prefab` in Project window, drag it and attach to your effect in Hierarchy window. ![image](/far-sdk/assets/images/unity_makeup_1_1-980e894d2e5407a7a691c64dafe87efc.png) 2. Then you need to assign a reference of your effect's PlaneController component to each segmentation AR 3D Mask (*SmoothCamera* is not segmentation) under **Canvas** object by dragging a **CameraPlane** object from Hierarchy to a **Plane** field of a Segmentation Feature component. You can do it one by one, or just by selecting all of them in the Hierarchy and assigning reference to all of them at once as shown below. ![image](/far-sdk/assets/images/unity_makeup_1_2-78be8d8cb28d821972e588d065a0fe2a.png) 3. Add the required references for FacesController. If you haven't got FacesController in your current effect, you could find it here Assets/BanubaFaceAR/BaseAssets/Prefabs/FacesController.prefab and drag it into the effect root. ![image](/far-sdk/assets/images/unity_makeup_1_4-f9867e2d5b30569edea54b1ae037d608.jpg) 4. Assign effect's **ResultCamera** to **RenderCamera** field on **Canvas** object. ![image](/far-sdk/assets/images/unity_makeup_1_3-097af1f60f71b3ea90aa6a0eb6d757d8.png) 5. Change the FaceMesh Material on Assets/BanubaFaceAR/Makeup/Materials/EyeFaceMakeup.mat: ![image](/far-sdk/assets/images/unity_makeup_1_5-7d368aca622eaf451dd2432877ca3994.jpg) 6. Makeup is ready! ### How to use[​](#how-to-use "Direct link to How to use") After Makeup is added to your object, it's time to tweak some options. Each separate makeup feature is represented as a component on the Makeup object that provides you with different options to adjust. You can do it both from code and Inspector. ![image](/far-sdk/assets/images/unity_makeup_2_1-ee9f09c41ccbf00391a76e320faac492.png) *Tweak options from Inspector to see makeup in Editor Playmode* note Lips makeup component has 3 option presets for aa particular lips look. To apply them, find and click the button in the upper-right corner of the Lips Makeup component window and choose the preset you want from the end of the list. ![image](/far-sdk/assets/images/unity_makeup_2_2-ad72af3b5336225ba6c8b1313bfa9c08.png) To access these options from the code at runtime, you can get a reference to MakeAPI component which stores references to all other makeup components. ``` MakeupAPI makeupAPI = makeupGameObject.GetComponent(); makeupAPI.Lips.color = new Color(0.8f, 0.1f, 0, 0.6f); makeupAPI.Lips.brightness = 0.9f; makeupAPI.Skin.softeningStrength = 1; ``` *Adjusting Makeup from code* If you want to completely disable or enable a particular makeup feature, just change the state of the MonoBehaviour's `enabled` property. ``` makeupAPI.Skin.enabled = false; // now Skin makeup is completely disabled ``` ### Makeup example scene[​](#makeup-example-scene "Direct link to Makeup example scene") Banuba Unity plugin provides an example scene with Makeup effect. Everything is already set up. Find and open `MakeupExample.unity` scene under *Assets* => *BanubaFaceAR* => *Makeup*. Click Play button and see how it works! ## Hand skeleton[​](#hand-skeleton "Direct link to Hand skeleton") Hand skeleton feature allows you to detect and render hand skeleton 2D. 1. Enable Hand skeleton feature with **BanubaSDKBridge.bnb\_recognizer\_insert\_feature** function. ``` // var recognizer = BanubaSDKManager.instance.Recognizer // ... var featuresId = BanubaSDKBridge.bnb_recognizer_get_features_id(); BanubaSDKBridge.bnb_recognizer_insert_feature(recognizer, featureId.hand_skeleton, out var error); ``` 2. Get the detected hand with **BanubaSDKBridge.bnb\_frame\_data\_get\_hand** function. ``` // frameData = BanubaSDKBridge.bnb_recognizer_process_frame_data(..., frameData, ...) /// ... var error = IntPtr.Zero; var hand = BanubaSDKBridge.bnb_frame_data_get_hand(frameData, Screen.height, Screen.height, BanubaSDKBri, bnb_rect_fit_mode_t, bnb_fit_height, out error); Utils.CheckError(error); ``` 3. bnb\_hand\_data\_t contains landmarks for hand skeleton, transformation for the landmarks and current detected gesture(For gesture detecting see Hand Gestures section). Apply these landmarks as vertices of you mesh. ``` [StructLayout(LayoutKind.Sequential)] public struct bnb_hand_data_t { public bnb_hand_gesture_t gesture; public int vertices_count; public IntPtr vertices; [MarshalAs(UnmanagedType.ByValArray, SizeConst = 16)] public float[] transform; }; ``` ### How to add and use the Hand gestures feature[​](#how-to-add-and-use-the-hand-gestures-feature "Direct link to How to add and use the Hand gestures feature") With the Unity Face AR Hand Gestures feature you can get and use hand gestures triggers in your app. At the the moment, our algorithms are able to recognize 5 hand gestures: * Like πŸ‘ * Ok πŸ‘Œ * Palm βœ‹ * Rock 🀘 * Victory/Peace ✌️ 1. Enable the Hand gestures feature with the **BanubaSDKBridge.bnb\_recognizer\_insert\_feature** function. ``` // var recognizer = BanubaSDKManager.instance.Recognizer // ... var featuresId = BanubaSDKBridge.bnb_recognizer_get_features_id(); BanubaSDKBridge.bnb_recognizer_insert_feature(recognizer, featureId.hand_gestures, out var error); ``` **NOTE:** **featureId.hand\_gestures** always enables **featureId.hand\_skeleton**. 2. Get this frame's detected gesture with **BanubaSDKBridge.bnb\_frame\_data\_get\_gesture** function. ``` // frameData = BanubaSDKBridge.bnb_recognizer_process_frame_data(..., frameData, ...) /// ... var gesture = BanubaSDKBridge.bnb_frame_data_get_gesture(frameData, out var error); ``` ### Example prefab and script[​](#example-prefab-and-script "Direct link to Example prefab and script") You can find **HandSkeleton.prefab** example prefab under *Assets* -> *BanubaFaceAR* -> *Hands* -> *Prefabs*. It has an attached HandSkeleton.cs component that contains all needed logic to enable and use hand skeleton and gestures. 1. Drop the prefab into scene. ![image](/far-sdk/assets/images/hand_skelet_unity1-29e93577d533286dc01b73c26e1fb9a1.jpg) 2. Add render camera reference for the **HandSkeleton.prefab**. ![image](/far-sdk/assets/images/hand_skelet_unity2-8bc208f3d0f8814d15e6c7933d266524.jpg) 3. Hit Play to see how it works! --- # Using video calls with the Banuba SDK [View as Markdown](https://docs.banuba.com/far-sdk/tutorials/unity/videocall.md)[![](/far-sdk/img/ai-guide/chatgpt.svg)Open in ChatGPT](https://chatgpt.com/?q=Read%20https%3A%2F%2Fdocs.banuba.com%2Ffar-sdk%2Ftutorials%2Funity%2Fvideocall.md%20\(Using%20video%20calls%20with%20the%20Banuba%20SDK\)%20and%20help%20me%20with%20it%20in%20my%20Banuba%20Face%20AR%20SDK%20project.)[![](/far-sdk/img/ai-guide/claude.svg)Open in Claude](https://claude.ai/new?q=Read%20https%3A%2F%2Fdocs.banuba.com%2Ffar-sdk%2Ftutorials%2Funity%2Fvideocall.md%20\(Using%20video%20calls%20with%20the%20Banuba%20SDK\)%20and%20help%20me%20with%20it%20in%20my%20Banuba%20Face%20AR%20SDK%20project.)Install tools In our example, **AgoraRTC SDK** is used for video streaming. But integration can be done based on any video streaming library. You can read more about Agora [here](https://www.agora.io/en/unity/). [Example of using video calling with the Banuba SDK](https://github.com/Banuba/videocall-unity)

## How To Run[​](#how-to-run "Direct link to How To Run") 1. Get the client token for Banuba SDK. Please fill in our form on banuba.com website, or contact us via . 2. Open the the project. 3. Download and import the [Agora SDK package from Unity Asset Store](https://assetstore.unity.com/packages/tools/video/agora-video-sdk-for-unity-134502) with Unity Package Manager. If it is not available, contact [Agora Support](https://www.agora.io/en/customer-support/) 4. Download and import the [BanubaSDK-import.unitypackage](https://github.com/Banuba/quickstart-unity/releases) 5. Visit [agora.io](https://www.agora.io) to sign up and get a token, as well as an app and channel ID. 6. Find the `Assets/Resources/BanubaClientToken.txt` and past your client token here. 7. Open the scene `VideoCallDemo/demo/MainScene.scene.` 8. Find the the VideoCanvas object in the scene and set AppID, Token, and your channel name in the properties of the DemoVideoCall script. ![image](/far-sdk/assets/images/videocall_example-048b1ac9667bd95430cb05f16581596d.png) 8. Run the project in the Editor. ## How It Works[​](#how-it-works "Direct link to How It Works") 1. Initialize `AgoraSDK` in `Start` method with methods below: Assets/VideoCallDemo/demo/DemoVideoCall.cs ``` loading... ``` ![Icon](/far-sdk/img/icons/github.svg "GitHub") 2. Initialize **BanubaSDK**. The MainScene.scene contains [BanubaSDKManager](/far-sdk/tutorials/unity/overview.md#banubasdkmanager) reference from `BanubaSDK-import.unitypackage` 3. Render Camera and any AR Effect with the `BNB.RenderToTexture.cs` to the RenderTexture ![image](/far-sdk/assets/images/videocall_example_2-4bb607e6eccf828c6fb7ad34571d1d05.png) 3. Send Video Frames in `Update` method Assets/VideoCallDemo/demo/DemoVideoCall.cs ``` loading... ``` ![Icon](/far-sdk/img/icons/github.svg "GitHub") --- # Banuba Face AR SDK [View as Markdown](https://docs.banuba.com/far-sdk/index.md)[![](/far-sdk/img/ai-guide/chatgpt.svg)Open in ChatGPT](https://chatgpt.com/?q=Read%20https%3A%2F%2Fdocs.banuba.com%2Ffar-sdk%2Findex.md%20\(Banuba%20Face%20AR%20SDK\)%20and%20help%20me%20with%20it%20in%20my%20Banuba%20Face%20AR%20SDK%20project.)[![](/far-sdk/img/ai-guide/claude.svg)Open in Claude](https://claude.ai/new?q=Read%20https%3A%2F%2Fdocs.banuba.com%2Ffar-sdk%2Findex.md%20\(Banuba%20Face%20AR%20SDK\)%20and%20help%20me%20with%20it%20in%20my%20Banuba%20Face%20AR%20SDK%20project.)Install tools ## Introduction[​](#introduction "Direct link to Introduction") Welcome to **Banuba Face AR SDK**. This document will help you to get started with our SDK and guide you on how to create your project with **Face AR** features. Banuba SDK allows developers to include AR features into their applications. Our documentation will give you a complite guide how to use the features described. How to read this documentation: 1. [**View the samples.**](/far-sdk/tutorials/development/samples.md) The SDK is delivered with examples of feature applications for each platform. They cover a variety of real-life use cases and give a comprehensive overview of how to run and use the SDK. 2. [**Setup your project**](/far-sdk/tutorials/development/basic_integration.md) using our examples. 3. [**This document**](/far-sdk/tutorials/capabilities/sdk_features.md) review the features across the supported platforms. 4. Try to create your own effect with [**Banuba Studio**](https://studio.banuba.com/) or buy some from the [**Banuba Asset Store**](https://assetstore.banuba.net/). ## Architecture[​](#architecture "Direct link to Architecture") The image below shows the components of the **Banuba Face AR SDK**. ![image](/far-sdk/assets/images/introduction_0-e6e520e3eafab1de9d6776fe0c97fc71.svg) ### EffectPlayer[​](#effectplayer "Direct link to EffectPlayer") EffectPlayer is a low-level library for effects' playing. Its code doesn’t depend on any platform-specific APIs or compilers. EffectPlayer is written in **C++**, but has bindings to all supported platform-specific languages and runtimes. You need to use **Java** or **Kotlin** on Android, and **Objective-C** or **Swift** on iOS and macOS for dealing with this API. Additionally, **C++** API is available on Windows and macOS. EffectPlayer features: * Consumes the camera frames and requests for the frame drawing. Serves it in an asynchronous manner using multithreading if available on a platform. * Runs all recognition operations on the input frames. * Runs and manages all platform-specific modules encapsulated in C++. For example audio playback, video playback, accelerometer, scripting engine, etc. * Implements all logic of loading and playing interactive effects (loading from disk, rendering, scripting of effect logic, etc). ### Platform modules[​](#platform-modules "Direct link to Platform modules") The functionality of the platform modules depends on the specific platform. Generally, it includes: * camera features (permission management, configuration, lifecycle implementation) * effect's rendering context setup * video recording * high-resolution photo taking * preparing of EffectPlayer's resources on the app's launch The source code of the platform modules is included in SDK's distribution archive. You can modify the code and adapt the SDK up to your use case if its default functionality is not enough. ## More info[​](#more-info "Direct link to More info") * Visit our [getting started](/far-sdk/tutorials/development/basic_integration.md) page for more information about SDK integration and examples of Demo apps. * Have questions about Face AR SDK? Visit the [FAQ page](https://www.banuba.com/faq/). * Can't find an answer? [Contact our support](/far-sdk/support/.md). --- # Banuba Video and Photo Editor SDKs Documentation - Version 7205f9c62354ceda95082d07dfef2073 > Docs provides full information about Banuba Video and Photo Editor SDKs, integration and customization guides. This file contains all documentation content in a single document following the llmstxt.org standard. ## Overview Video Editor SDK on Android Banuba Video Editor SDK has built in UI/UX experience and provides a number of customizations you can use to meet your requirements. **AVAILABLE** :white_check_mark: Use your branded icons, colors, and text styles. [See details](ve-faq.md) :white_check_mark: Localize and change text resources. Default locale is :us: :white_check_mark: Make content you want i.e. a number of video with different resolutions and durations, an audio file. :white_check_mark: Masks and filters order. [See details](ve-faq.md) **NOT AVAILABLE** :x: Change layout or order of screens after entry point You can, however, [ask](https://www.banuba.com/support) us to customize the mobile video editor UI as a separate contract. --- ## AI Clipping on Android The [AI Clipping](https://www.banuba.com/ai-sdk) feature automates the video creation process by leveraging the power of artificial intelligence. The neural network transforms raw input into ready-to-post videos with various transitions, effects, and a precise music match. :::important The ```AI Clipping``` is based on the [Banuba Face AR SDK](https://www.banuba.com/facear-sdk/face-filters) product. Since this feature uses additional services, it is disabled by default. Please contact Banuba representatives to know more about using this feature. ::: Here is how users can interact with it: 1. User selects clips and music. People can choose their desired clips and accompanying music within the app. 2. AI trims the videos. The AI algorithm intelligently trims the selected clips to match the tempo and rhythm of the chosen track. 3. AI adds various effects. AI Clipping enhances the content by adding a variety of effects to create visually stunning content. 4. User exports, edits, or regenerates the clip. Upon completion, users have the flexibility to export the video as is, make further edits, or regenerate the clip for a fresh perspective. Regardless of the user’s editing skills, with the help of AI Clipping, everyone can get a stunning video within seconds.   ## Supported Music Providers - [Banuba Music](guide_audio_content.md#connect-banuba-music) - [Soundstripe](guide_audio_content.md#connect-soundstripe) ## Integration ### Setup configuration Add ```Banuba Face AR SDK``` module by following the [Face AR instruction](guide_far_arcloud.md#integrate-face-ar). Set up ```AiClippingConfig```, ```AutoCutTrackLoader``` and ```ContentFeatureProvider``` dependencies in [VideoEditorModule](https://github.com/Banuba/ve-sdk-android-integration-sample/blob/main/app/src/main/java/com/banuba/example/integrationapp/VideoEditorModule.kt#L51) :::important Contact Banuba representative to get trial keys for ```audioDataUrl``` and ```audioTracksUrl``` ::: ### Config with Banuba Music provider ```kotlin factory { AiClippingConfig( audioDataUrl = "https://d27n29bgbvbeer.cloudfront.net/index-staging.zip", audioTracksUrl = "" ) } factory { AiClippingBanubaMusicTrackLoader(contentProvider = get()) } factory>( named("recommendedSoundsMusicTrackProvider") ) { AiClippingRecommendedSoundProvider() } ``` ### Config with Soundstripe provider ```kotlin factory { AiClippingConfig( audioDataUrl = "...", audioTracksUrl = "..." ) } factory { AiClippingSoundstripeTrackLoader(soundstripeApi = get()) } factory>( named("recommendedSoundsMusicTrackProvider") ) { AiClippingRecommendedSoundProvider() } ``` ### Launch AI Clipping :::info Video creation with AI Clipping is available on Gallery screen by default. ::: For better experience we added new entry point to `VideoCreationActivity` for opening AI Clipping as a separate mode. In this scenario the user starts from the gallery screen and is taken to AI Clipping screen after selecting media. Use the `Intent` to start new mode. ```kotlin val intent = VideoCreationActivity.startFromAiClipping(Context) ``` --- ## Closed Captions on Android Closed captions(CC) are a textual representation of the audio within a media file. :::important The Close Captions feature is disabled by default. Please contact Banuba representatives to know more about using this feature. ::: Over 80% of videos played on mobile devices don’t have sound turned on. This means many forms of content (e.g. skits, monologues, educational clips, etc.) will be skipped if there are no subtitles. But making captions by hand is tedious. AI-generated subtitles solve this issue, as they are created and placed automatically. The users can then edit the text as well as change its style and color.       [AWS Transcribe service](https://docs.aws.amazon.com/transcribe/) is used to generate captions. ## Supported languages - Arabic - English - Mandarin - Spanish - Portuguese ## Integration Create ```Bundle``` with Closed Captions configuration and pass ```extras``` to any Video Editor start method. ### Closed Captions V2 (Recommended) :::important Request API V2 key from Banuba representatives. ::: ```kotlin val extras = bundleOf( CaptionsApiService.ARG_API_KEY_V2 to "...", ) ``` ### Closed Captions V1 :::important Request keys from Banuba representatives. ::: ```kotlin val extras = bundleOf( CaptionsApiService.ARG_CAPTIONS_UPLOAD_URL to "...", CaptionsApiService.ARG_CAPTIONS_TRANSCRIBE_URL to "...", CaptionsApiService.ARG_API_KEY to "...", ) ``` Use ```extras``` in any start method of the Video Editor SDK ```kotlin VideoCreationActivity.startFromCamera( context = applicationContext, pictureInPictureConfig = PipConfig( video = pipVideo, openPipSettings = editorSettingsProvider.openPipSettings ), audioTrackData = initialTrackData, extras = extras ) ``` --- ## Dependencies and Licenses on Android Lists used dependencies and licenses in the SDK. ## Dependencies | Name | Version | | --------- |---------| | androidx.activity:activity-ktx | 1.10.1 | | androidx.annotation:annotation | 1.9.1 | | androidx.appcompat:appcompat | 1.7.0 | | androidx.constraintlayout:constraintlayout | 2.1.4 | | androidx.core:core-ktx | 1.15.0 | | androidx.exifinterface:exifinterface | 1.4.0 | | androidx.fragment:fragment-ktx | 1.8.6 | | androidx.lifecycle:lifecycle-livedata-ktx | 2.8.7 | | androidx.lifecycle:lifecycle-runtime-ktx | 2.8.7 | | androidx.lifecycle:lifecycle-service | 2.8.7 | | androidx.lifecycle:lifecycle-viewmodel-ktx | 2.8.7 | | androidx.localbroadcastmanager:localbroadcastmanager | 1.0.0 | | androidx.recyclerview:recyclerview | 1.4.0 | | com.airbnb.android:lottie | 3.5.0 | | com.github.bumptech.glide:glide | 4.15.0 | | androidx.media3:media3-exoplayer | 1.6.0 | | com.google.android.material:material | 1.12.0 | | com.squareup.moshi:moshi-kotlin | 1.15.2 | | com.squareup.moshi:moshi-kotlin-codegen | 1.15.2 | | com.squareup.okhttp3:logging-interceptor | 4.12.0 | | com.squareup.okhttp3:okhttp | 4.12.0 | | com.squareup.retrofit2:converter-moshi | 2.11.0 | | com.squareup.retrofit2:retrofit | 2.11.0 | | io.insert-koin:koin-android | 3.5.6 | | org.apache.commons:commons-math3 | 3.6.1 | | org.jetbrains.kotlin:kotlin-stdlib-jdk7 | 2.1.0 | | org.jetbrains.kotlin:kotlin-reflect | 2.1.0 | | org.jetbrains.kotlinx:kotlinx-coroutines-android | 1.10.1 | | org.jetbrains.kotlinx:kotlinx-coroutines-core | 1.10.1 | ## 3rd party licenses ### **GNU Lesser General Public Licence version 3.0 or later** This library is free software and is governed by GNU Lesser General Public License, version 3.0, available at https://www.gnu.org/licenses/lgpl-3.0.en.html. | Name | Link | Copyright info | | --- | --- | --- | | FFmpeg | [https://github.com/tanersener/mobile-ffmpeg](https://github.com/tanersener/mobile-ffmpeg) | Copyright (c) ffmpeg Authors | ## **MIT License** "Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the ""Software""), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions: The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software. THE SOFTWARE IS PROVIDED ""AS IS"", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE." | Name | Link | Copyright info | | --- | --- | --- | | json | [https://github.com/nlohmann/json](https://github.com/nlohmann/json) | Copyright (c) 2013-2019 Niels Lohmann | | Android-gif-drawable | [https://github.com/koral--/android-gif-drawable](https://github.com/koral--/android-gif-drawable) | Copyright (c) 2013 - present Karol WrΓ³tniak, Droids on Roids LLC | ## **Apache License 2.0** "Licensed under the Apache License, Version 2.0 (the ""License""); you may not use this file except in compliance with the License. You may obtain a copy of the License at: [http://www.apache.org/licenses/LICENSE-2.0](http://www.apache.org/licenses/LICENSE-2.0) Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an ""AS IS"" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License." | Name | Link | Copyright info | |-------------------| --- | --- | | Media 3 ExoPlayer | [https://developer.android.com/media/media3/exoplayer](https://developer.android.com/media/media3/exoplayer) | | | OkHttp | [https://github.com/square/okhttp](https://github.com/square/okhttp) | Copyright 2019 Square, Inc. | | Retrofit | [https://github.com/square/retrofit](https://github.com/square/retrofit) | Copyright 2013 Square, Inc. | | Okio | [https://github.com/square/okio](https://github.com/square/okio) | Copyright 2013 Square, Inc. | | Koin | [https://github.com/InsertKoinIO/koin](https://github.com/InsertKoinIO/koin) | Copyright 2017-2021 Arnaud GIULIANI, Laurent BARESSE | | Moshi | [https://github.com/square/moshi](https://github.com/square/moshi) | Copyright 2015 Square, Inc. | | Lottie | [https://github.com/airbnb/lottie-android](https://github.com/airbnb/lottie-android) | Copyright 2018 Airbnb, Inc. | ## **3-clause BSD License** Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met: 1. Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer. 2. Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution. 3. Neither the name of the copyright holder nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission. THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS ""AS IS"" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. " | Name | Link | Copyright info | | --- | --- | --- | | Libyuv | [https://chromium.googlesource.com/libyuv/libyuv](https://chromium.googlesource.com/libyuv/libyuv) | Copyright 2011 The LibYuv Project Authors | | Glide | [https://github.com/bumptech/glide](https://github.com/bumptech/glide) | Copyright 2014 Google, Inc. All rights reserved. | ## Face AR SDK [View](https://docs.banuba.com/face-ar-sdk/overview/3rd_licenses) third party libraries for Banuba Face AR SDK --- ## Use of FFmpeg Banuba Video Editor SDK uses FFmpeg version ```5.3.0 ``` ## Guidelines to use FFmpeg dependency in your app: Add Banuba FFmpeg dependency from gradle file. ```groovy implementation "com.banuba.sdk:ffmpeg:5.3.0" ``` Add the following code in your Activity to check whether everything is correct: ```kotlin com.banuba.sdk.ve.processing.FFmpeg(context = this).execute(emptyArray()).run { waitFor() Log.d("FFmpeg", errorStream.reader().readText()) } ``` The output should contain information about version of FFmpeg libraries. We recommend using our FFMpeg functionality, as it offers a very wide range of tools. --- ## Audio content on Android Guide to integrating and customizing audio providers in Video Editor SDK. ## Overview Audio content is a key part of making awesome video. Video Editor SDK can play, trim, merge and add audio content to a video. :::info 1. Banuba does not deliver audio content for Video Editor SDK. 2. Video Editor can apply audio file stored on the device. The SDK is not responsible for downloading audio content except [Soundstripe](https://www.soundstripe.com/) and [Banuba Music](#connect-banuba-music) 3. Video Editor SDK supports only one music provider per launch. ::: There are 2 approaches of using audio content: 1. ```AudioBrowser``` - specific module and a set of screens that includes built in support of browsing and applying audio content within video editor. The user does not leave the sdk while using audio. 2. ```External API``` - the client implements specific API for managing audio content. The user leaves the SDK and is taken to an app screen when audio is requested. ## Audio Browser Audio Browser is a specific Android module that allows to browse, play and apply audio content within video editor. It supports 3 sources for audio content: 1. ```Banuba Music``` - includes build in integration with Banuba Music, 2. ```Soundstripe``` - includes built in integration with [Soundstripe](https://www.soundstripe.com/) API, 3. ```My Library``` - includes audio content available on the user's device. Add the dependency ```kotlin implementation "com.banuba.sdk:ve-audio-browser-sdk:${version}" ``` to your [gradle](https://github.com/Banuba/ve-sdk-android-integration-sample/blob/main/app/build.gradle#L77) file and specify ```AudioBrowserKoinModule``` Koin module in [VideoEditorModule](https://github.com/Banuba/ve-sdk-android-integration-sample/blob/main/app/src/main/java/com/banuba/example/integrationapp/VideoEditorModule.kt#L57) ```diff startKoin { ... modules( // highlight-add-next-line + AudioBrowserKoinModule().module, VideoEditorKoinModule().module ) } ``` to integrate ```AudioBrowser```. ## Connect Banuba Music Over 35 GB of royalty-free tracks available from within the Video Editor SDK. Your users could check them out through an inbuilt music browser and legally include them in their content. :::info The feature is not activated by default. Please contact Banuba representatives to know more about using this feature. :::   Use ```BanubaMusicProvider``` implementation in [VideoEditorModule](https://github.com/Banuba/ve-sdk-android-integration-sample/blob/main/app/src/main/java/com/banuba/example/integrationapp/VideoEditorModule.kt#L64) ```kotlin single>(named("musicTrackProvider")){ BanubaMusicProvider() } ``` ## Connect Soundstripe [Soundstripe](https://www.soundstripe.com/) is a service for providing the best audio tracks for creating video content. Your users will be able to add audio tracks while recording or editing video content. :::info The feature is not activated by default. Please contact Banuba representatives to know more about using this feature. :::     Use ```SoundstripeProvider``` implementation in [VideoEditorModule](https://github.com/Banuba/ve-sdk-android-integration-sample/blob/main/app/src/main/java/com/banuba/example/integrationapp/VideoEditorModule.kt#L64) ```kotlin single>(named("musicTrackProvider")){ SoundstripeProvider() } ``` ## Connect My Library ```My Library``` is a default implementation in ```AudioBrowser``` . It allows the user to apply audio that is available on a device. Use ```AudioBrowserMusicProvider``` implementation in [VideoEditorModule](https://github.com/Banuba/ve-sdk-android-integration-sample/blob/main/app/src/main/java/com/banuba/example/integrationapp/VideoEditorModule.kt#L75) to enable ```My Library```. ```kotlin single>(named("musicTrackProvider")) { AudioBrowserMusicProvider() } ``` ## Integrate Client Audio provider on Android Video Editor includes special API for integrating your custom audio content provider and applying this content in video editor. The user will be taken to your app specific screen when audio is requested on video editor screen i.e. camera or editor. Next, once the user picks audio content on your app screen you need to follow API and return the user to video editor. Any audio file should stored on the device before applying. Below is a guide of using API to provide your audio to video editor. First, create new Activity ```CustomAudioContentActivity``` that will handle API and create new method to start it from video editor. This Activity you can use to implement any API for downloading audio content and showing your beautiful UI to your users. ```kotlin class CustomAudioContentActivity : AppCompatActivity() { ... companion object { fun buildPickMusicResourceIntent( context: Context, extras: Bundle ) = Intent(context, AwesomeAudioContentActivity::class.java).apply { putExtras(extras) } } } ``` where ```extras``` includes a data that can be used in the Activity. 1. ```ProvideTrackContract.EXTRA_LAST_PROVIDED_TRACK``` of TrackData. Can be null. ```null``` is used to dismiss audio. 2. ```ProvideTrackContract.EXTRA_TRACK_TYPE``` of TrackType. Next, create ```CustomActivityMusicProvider``` and implement ```ContentFeatureProvider```. ```kotlin class CustomActivityMusicProvider : ContentFeatureProvider { private var activityResultLauncher: ActivityResultLauncher? = null private val activityResultCallback: (TrackData?) -> Unit = { activityResultCallbackInternal(it) } private var activityResultCallbackInternal: (TrackData?) -> Unit = {} override fun init(hostFragment: WeakReference) { activityResultLauncher = hostFragment.get()?.registerForActivityResult( ProvideTrackContract(), activityResultCallback ) } override fun requestContent( context: Context, extras: Bundle ): ContentFeatureProvider.Result = ContentFeatureProvider.Result.RequestUi( intent = CustomAudioContentActivity.buildPickMusicResourceIntent( context, extras ) ) override fun handleResult( hostFragment: WeakReference, intent: Intent, block: (TrackData?) -> Unit ) { activityResultCallbackInternal = block activityResultLauncher?.launch(intent) } } ``` And set ```CustomActivityMusicProvider``` to [VideoEditorModule](https://github.com/Banuba/ve-sdk-android-integration-sample/blob/main/app/src/main/java/com/banuba/example/integrationapp/VideoEditorModule.kt#L75) ```kotlin single>(named("musicTrackProvider"), override = true) { CustomActivityMusicProvider() } ``` Please keep in mind that only one instance of ```musicTrackProvider``` can exist either your custom of```External API``` or ```Audio Browser```. Finally, pass audio content to apply audio in video editor. Instance of ```TrackData``` is used for passing in ```Intent``` to video editor. The audio should be stored on the device. ```kotlin val trackData = TrackData( UUID.randomUUID(), "My awesome track", audioTrackUri, // Uri of the audio track on local storage // file:///data/user/0//files//awesome.wav "Awesome Artist" ) ``` To pass ```TrackData``` to video editor your need to use ```setResult``` with ```Intent``` and finish current Activity. ```kotlin val trackToApply: TrackData = ... val resultIntent = Intent().apply { putExtra(ProvideTrackContract.EXTRA_RESULT_TRACK_DATA, trackToApply) } setResult(Activity.RESULT_OK, resultIntent) finish() ``` To dismiss previously selected audio track you can pass ```null``` for ```TrackData```. ## Resources The following string resources are used in ```AudioBrowser```. | ResourceId | Value | | ------------- | :----------- | | apply_track | Use | | remove_track| Stop\nusing | | track_loading_failed | Sorry, audio content is temporarily unavailable | | track_search_cancel | Cancel | | audio_browser_title_library | My library | | audio_browser_title_category | Music | | audio_browser_title_empty_category | Music | | audio_browser_load_more | Show more | | audio_browser_error_tracks_not_found | No tracks found | | audio_browser_error_categories_not_found | No categories found | | audio_browser_error_empty_library | No tracks yet | | audio_browser_error_license_not_active | The license is not active | | audio_browser_error_license_expired | The license expired | | audio_browser_error_license_access | Access denied, check license access type | | audio_browser_error_license_api_version | Access denied, check license API version | | audio_browser_error_license_wrong_key | Mubert key is missing. In order to get it contact Banuba rep. | | audio_browser_hint_search_categories | Search by categories | | audio_browser_hint_search_sub_categories | Search by groups | | audio_browser_hint_search_tracks | Search by tracks | | audio_browser_error_dialog_title | Oops, something went wrong… | | audio_browser_error_dialog_description | Please, try again later. | | audio_browser_error_dialog_retry | Retry | | audio_browser_error_dialog_close | Close | | permission_library_description_message | Allow to access to your storage to select an audio tracks from your device. | | audio_browser_connection_error_title | No internet connection | | audio_browser_connection_error_message | Please, check your connection and try again. | | audio_browser_connection_error_btn | Retry | | audio_browser_connection_error_toast | No internet connection | | action_add_music_track | Tracks | | action_add_voice_recording | Record | | action_effects | Effects | | action_edit | Edit | | action_delete | Delete | | edit_track_volume_title | Volume | | edit_track_volume_percent | %1$d%% | | edit_track_audio_duration | Audio duration | | edit_track_duration_error | Audio should be longer than %1$.1f sec | | error_voice_recording_start | Error on voice recording start | | error_invalid_duration_voice_recording | Min voice recording duration - %1$.1f sec | | error_invalid_duration_music_track | Min music track duration - %1$.1f sec | | error_voice_recording_delete_file | Internal error when try to delete voice recording file | | error_track_limit | Max available tracks - %1$d | | error_no_space | No space | --- ## Camera screen on Android Guide to modifying camera UI elements and screen configuration. ## Change icons You can use your app specific icons in Video Editor SDK. Add icon files to Android ```drawable``` folder with predefined names. :::info We strongly recommend using vector assets (.svg files). :::   Below is a list of icons. Each icon has a few states - on, off and modes. Specific file name should be assigned to each state. The filename on the right. Example, ```ic_camera_switch_on_new.xml``` file is used in the SDK to denote the state when a flipper is on. 1. Flip 1. **on** - ```ic_camera_switch_on_new``` 2. **off** - ```ic_camera_switch_off_new``` 2. Flash 1. **on** - ```ic_flashlight_on_new``` 2. **off** - ```ic_flashlight_off_new``` 3. Speed 1. **x0.5** - ```ic_recording_speed_x0_5_new``` 2. **x1** - ```ic_recording_speed_x1_new``` 3. **x2** - ```ic_recording_speed_x2_new``` 4. **x3** - ```ic_recording_speed_x3_new``` 5. **x0.5 selected** - ```ic_recording_speed_x0_5_selected_new``` 6. **x1 selected** - ```ic_recording_speed_x1_selected_new``` 7. **x2 selected** - ```ic_recording_speed_x2_selected_new``` 8. **x3 selected** - ```ic_recording_speed_x3_selected_new``` 4. Mic 1. **on** - ```ic_mute_mic_on_new``` 2. **off** - ```ic_mute_mic_off_new``` 5. Beauty 1. **on** - ```ic_beauty_on_new``` 2. **off** - ```ic_beauty_off_new``` 6. Filter 1. **on** - ```ic_color_on_new``` 2. **off** - ```ic_color_off_new``` 7. PIP 1. **on** - ```ic_pip_on_new``` 2. **off** - ```ic_pip_off_new``` 8. Mask 1. **on** - ```ic_mask_on_new``` 2. **off** - ```ic_mask_off_new``` 9. Music 1. **on** - ```ic_music_on_white``` 2. **off** - ```ic_music_on_white``` 10. Timer 1. ```ic_timer_new``` --- ## Cover image on Android Cover image screen allows users to pick any frame of video as image or choose an image from gallery. ```CoverProvider``` supports 2 modes - ```EXTENDED``` - enables the screen - ```NONE``` - disables the screen. The user is taken to export screen. ```EXTENDED``` is **default** mode in the SDK. You can change the mode in [VideoEditorModule](https://github.com/Banuba/ve-sdk-android-integration-sample/blob/main/app/src/main/java/com/banuba/example/integrationapp/VideoEditorModule.kt#L75) ``` kotlin single(override = true) { CoverProvider.EXTENDED } ``` ## Resources You can override the following string resources in your app. | ResourceId | Value | |--------------| :----------- | | cover_image_text | Choose cover | | cover_progress_text | Please, wait | | err_cover_image | Failed to create cover image | --- ## Drafts screen on Android Guide to integrating and customizing drafts screen on Video Editor SDK. ## Configuration Drafts are enabled by default, asks the user to save a draft before leave any VideoEditor screen. If you need to change drafts configuration you should add the code below in the [VideoEditorModule](https://github.com/Banuba/ve-sdk-android-integration-sample/blob/main/app/src/main/java/com/banuba/example/integrationapp/VideoEditorModule.kt#L60): ```kotlin override val draftConfig: BeanDefinition = factory(override = true) { DraftConfig.ENABLED_ASK_TO_SAVE } ``` You can choose one of these options: 1. `ENABLED_ASK_TO_SAVE` - drafts enabled, asks the user to save a draft 2. `ENABLED_ASK_IF_SAVE_NOT_EXPORT` - drafts enabled, asks the user to save a draft without export 3. `ENABLED_SAVE_BY_DEFAULT` - drafts enabled, saved by default without asking the user 4. `DISABLED` - disabled drafts ## Draft Helper ```DraftsHelper``` interface is used for managing drafts. ``` kotlin interface DraftsHelper { val allDrafts: StateFlow> fun delete(draft: Draft) fun deleteAll() fun openDraft(draft: Draft): Intent fun openLastDraft(): Intent } ``` To get the instance of ```DraftsHelper``` use the following in your Fragment or Activity. ``` kotlin val draftsHelper: DraftsHelper by inject() ``` ### Used string resources | ResourceId | Value | |:----------:|:-----:| |drafts_title|Drafts| |drafts_empty_description|No Drafts| |drafts_options_edit|Edit| |drafts_options_delete|Delete| |editor_trim_video|Adjust Clips| |editor_discard_changes|Discard changes| |editor_update_draft|Update draft| --- ## Editor screen on Android Guide to integrating and customizing Editor screen. ## Change icons You can use your app specific icons in Video Editor SDK. Add icon files to Android ```drawable``` folder with predefined names. :::info We strongly recommend using vector assets (.svg files). :::   Below is a list of icons. Each icon has a few states - on, off, disabled. Specific file name should be assigned to each state. The filename on the right. Example, ```ic_gif_on.xml``` file is used in the SDK to denote the state when a sticker is added to the video. 1. Stickers 1. **on** - ```ic_gif_on``` 2. **off** - ```ic_gif_off``` 2. Text 1. **on** - ```ic_text_effect_on``` 2. **off** - ```ic_text_effect_off``` 3. Captions 1. **on** - ```ic_captions_on``` 2. **off** - ```ic_captions_off``` 3. **disabled** - ```ic_captions_disabled``` 4. Effects 1. **on** - ```ic_visual_effect_on``` 2. **off** - ```ic_visual_effect_off``` 5. Masks 1. **on** - ```ic_mask_on``` 2. **off** - ```ic_mask_off``` 6. Music 1. **on** - ```ic_music_on``` 2. **off** - ```ic_music_off``` 7. Time 1. **on** - ```ic_time_effect_on``` 2. **off** - ```ic_time_effect_off``` 8. Filters 1. **on** - ```ic_lut_on``` 2. **off** - ```ic_lut_off``` 9. Blur 1. **on** - ```ic_blur_on``` 2. **off** - ```ic_blur_off``` 10. AI Clipping 1. **on** - ```ic_autocut_on``` 2. **off** - ```ic_autocut_off``` --- ## NEW Editor screen on Android Guide to integrating and customizing New Editor screen. ## Overview With the new interface, better controls, and additional quality of life improvements, making stunning videos is easier and more fun than ever. Design and user experience principles are constantly evolving. To keep up with the latest developments and best practices, our team has completely redesigned the Video Editor SDK to be as convenient and enjoyable as possible.       ## Integration Create ```Bundle``` with Editor UI V2 configuration and pass ```extras``` to any Video Editor start method. ```kotlin val extras = bundleOf( "EXTRA_USE_EDITOR_V2" to true ) ``` ```kotlin VideoCreationActivity.startFromCamera( context = applicationContext, pictureInPictureConfig = PipConfig( video = pipVideo, openPipSettings = false ), audioTrackData = initialTrackData, extras = extras ) ``` --- ## Export media on Android Video Editor SDK allows to export a number of media files i.e. video and audio with various resolutions and other configurations. Video is exported as ```.mp4``` file. :::info Export is a very heavy computational task that takes time and the user has to wait. Execution time depends on 1. Video duration - the longer video the longer execution time. 2. Number of video sources - the many sources the longer execution time. 3. Number of effects and their usage in video - the more effects and their usage the longer execution time. 4. Number of exported video - the more video and audio you want to export the longer execution time. 5. Device hardware - the most powerful devices can execute export much quicker and with higher resolution. ::: Export supports 2 modes: - ```Foreground``` - the user has to wait on progress screen until processing is done. **Default** - ```Background``` - the user can close the editor and open an app specific screen. A certain notification is sent when processing is done. ```ForegroundExportFlowManager``` and ```BackgroundExportFlowManager``` are corresponding implementations. Here is a screen that is shown in ```Foreground``` mode. ## Video quality Video Editor supports video codec options: 1. ```HEVC``` - H265 codec. **Default** 2. ```AVC_PROFILES``` - H264 codec with profiles 3. ```BASELINE``` - H264 codec without profiles The following table presents list of video resolution and bitrate values for codec ```H264(AVC_PROFILES)```. | 240p(240x426) | 360p(360x640) | 480p(480x854) | QHD540(540x960) | HD(720x1280) | FHD(1080x1920) | QHD(1440x2560) | UHD(2160x3840) | |---------------|------------|---------------|-----------------|--------------|----------------|----------------|----------------| | 1000 kb/s | 1200 kb/s | 2000 kb/s | 2400 kb/s | 3600 kb/s | 5800 kb/s | 10000 kb/s | 20000 kb/s | :::info Video Editor includes built-in feature for detecting device performance capabilities and finding ```auto``` video quality for exported video. ::: ## Export storage All exported media files are stored in ```export``` directory on the device storage. You can specify another directory by overriding ```exportDir``` dependency in [VideoEditorModule](https://github.com/Banuba/ve-sdk-android-integration-sample/blob/main/app/src/main/java/com/banuba/example/integrationapp/VideoEditorModule.kt#L51). Default implementation is ```kotlin single(named("exportDir")) { get().getExternalFilesDir("")?.toUri() ?.buildUpon() ?.appendPath("export") ?.build() ?: throw NullPointerException("exportDir cannot be null!") } ``` ## Implement export flow :::info Default implementation exports single video file with auto quality(based on device hardware capabilities). ::: You can create your own export flow to meet your requirements. First, specify list of media files to export. Create new class ```CustomExportParamsProvider``` and implement ```ExportParamsProvider```. Method ```provideExportParams``` returns ```List``` which is a list of media files to export. The following implementation exports single video with ```HD``` and ```auto``` quality ```kotlin class CustomExportParamsProvider( private val exportDir: Uri, private val watermarkBuilder: WatermarkBuilder ) : ExportParamsProvider { override fun provideExportParams( effects: Effects, videoRangeList: VideoRangeList, musicEffects: List, videoVolume: Float ): List { val exportSessionDir = exportDir.toFile().apply { deleteRecursively() mkdirs() } // Video is in HD resolution val exportVideoHD = ExportParams.Builder(VideoResolution.Exact.HD) .effects(effects) .fileName("export_video_hd") .videoRangeList(videoRangeList) .destDir(exportSessionDir) .musicEffects(musicEffects) .volumeVideo(videoVolume) .build() // Video is in auto resolution val exportVideoAuto = ExportParams.Builder() .effects(effects) .fileName("export_video_auto") .videoRangeList(videoRangeList) .destDir(exportSessionDir) .musicEffects(musicEffects) .volumeVideo(videoVolume) .build() return listOf(exportVideoHD, exportVideoAuto) } } ``` Use constructor ```ExportParams.Builder()``` or specific method ```videoResolution()```to set up custom video quality. ```diff val exportVideoHD = // highlight-add-next-line + ExportParams.Builder(VideoResolution.Exact.HD) .effects(effects) .fileName("export_video_hd") .videoRangeList(videoRangeList) .destDir(exportSessionDir) .musicEffects(musicEffects) .volumeVideo(videoVolume) .build() ``` :::warning Please keep in mind that low level devices might not be able to export video with high quality. Our recommendations 1. Use ```HD``` or ```VGA480``` resolutions with codec ```H264(AVC_PROFILES)``` 2. Do not specify certain video quality. In this case ```auto``` quality will be used. ::: Next, specify ```CustomExportParamsProvider``` implementation in [VideoEditorModule](https://github.com/Banuba/ve-sdk-android-integration-sample/blob/main/app/src/main/java/com/banuba/example/integrationapp/VideoEditorModule.kt#L51) ```kotlin factory { CustomExportParamsProvider( exportDir = get(named("exportDir")), watermarkBuilder = get() ) } ``` Finally, use the most suitable export mode for your application in [VideoEditorModule](https://github.com/Banuba/ve-sdk-android-integration-sample/blob/main/app/src/main/java/com/banuba/example/integrationapp/VideoEditorModule.kt#L51). ```ForegroundExportFlowManager``` is used by default. ## Use AVC codec The SDK prefers ```H265(HEVC)``` codec by default for exporting video file. You can specify ```H264(AVC_PROFILES)``` by setting ```false``` to ```useHevcIfPossible```. :::info ```H264(AVC_PROFILES)``` is recommended for low level devices ::: ```diff ExportParams.Builder(VideoResolution.Exact.HD) ... // highlight-add-next-line + useHevcIfPossible(false) .build(), ``` ## Add watermark Watermark is not added to exported video by default. Create new class ```CustomWatermarkProvider``` and implement ```WatermarkProvider``` to use your custom watermark image. ```kotlin private class CustomWatermarkProvider(private val context: Context) : WatermarkProvider { override fun getWatermarkBitmap(): Bitmap? { val watermarkDrawableRes = ... // R.drawable. val watermark = BitmapFactory.decodeResource( context.resources, watermarkDrawableRes ) return watermark } } ``` and specify it in [VideoEditorModule](https://github.com/Banuba/ve-sdk-android-integration-sample/blob/main/app/src/main/java/com/banuba/example/integrationapp/VideoEditorModule.kt#L51) ```kotlin factory { CustomWatermarkProvider() } ``` Use extension method ```Effects.withWatermark``` for adding a watermark to ```ExportParams.Builder```. ```diff ExportParams.Builder(VideoResolution.Exact.HD) // highlight-add-next-line + .effects(effects.withWatermark(watermarkBuilder, WatermarkAlignment.BottomRight(marginRightPx = 16.toPx))) ... .build(), ``` where ```WatermarkBuilder``` provides watermark drawable and ```alignment``` is used where to locate drawable ```kotlin WatermarkAlignment { TOP_LEFT, TOP_RIGHT, BOTTOM_LEFT, BOTTOM_RIGHT } ``` ## Export soundtrack track You can export video and audio soundtrack file separately. For example, ```Kotlin val extraSoundtrackUri = Uri.parse(exportSessionDir.toString()).buildUpon() .appendPath("exported_soundtrack.${MediaFileNameHelper.DEFAULT_SOUND_FORMAT}") .build() val exportVideoAndSoundtrack = ExportParams.Builder(VideoResolution.Exact.HD) .fileName("export_extra_soundtrack") .videoRangeList(videoRangeList) .destDir(exportSessionDir) .musicEffects(musicEffects) .extraAudioFile(extraSoundtrackUri) .volumeVideo(videoVolume) .build() ``` and add ```exportVideoAndSoundtrack``` to the list of exported video files in your custom ```ExportParamsProvider```. ## Handle export result The result is returned to your controller in [registerForActivityResult](https://github.com/Banuba/ve-sdk-android-integration-sample/blob/main/app/src/main/java/com/banuba/example/integrationapp/MainActivity.kt#L27) method as an instance of ```ExportResult```. ```kotlin private val createVideoRequest = registerForActivityResult(CustomExportResultVideoContract()) { exportResult -> exportResult?.let { if (exportResult is ExportResult.Success) { // Get uri of first exported video file val videoUri = exportResult.videoList.getOrNull(0) ... } } } ``` ```ExportResult``` class ```kotlin sealed class ExportResult { object Inactive : ExportResult() object Stopped : ExportResult() data class Progress( val preview: Uri ) : ExportResult() @Parcelize data class Success( val videoList: List, val preview: Uri, val metaUri: Uri, val additionalExportData: Parcelable? = null ) : ExportResult(), Parcelable @Parcelize data class Error(val type: ExportError) : ExportResult(), Parcelable } ``` Instance of ```ExportResult.Success``` is returned with all export data when export finishes successfully. ```ExportResult.Error``` is returned when an error is occurred while exporting media content. ## Export in background If you want to export media in the background and avoid showing progress screen to your user you can use ```BackgroundExportFlowManager``` implementation in [VideoEditorModule](https://github.com/Banuba/ve-sdk-android-integration-sample/blob/main/app/src/main/java/com/banuba/example/integrationapp/VideoEditorModule.kt#L51) and provide your custom implementations. ```kotlin single { BackgroundExportFlowManager( exportDataProvider = get(), exportSessionHelper = get(), exportNotificationManager = get(), exportDir = get(named("exportDir")), shouldClearSessionOnFinish = true, publishManager = get(), errorParser = get(), exportBundleProvider = get(), eventConverter = get() ) } ``` Next, specify Android ```CustomActivity``` that opens after export in ```AndroidManifest.xml``` file and set special ``` ``` . Please use ```applicationId``` as a part of intent action name to make it unique among other possible intent actions ```kotlin ``` Create new class ```CustomExportResultHandler``` and implement ```ExportResultHandler``` that will start your Activity mentioned above. ```kotlin class CustomExportResultHandler : ExportResultHandler { override fun doAction(activity: AppCompatActivity, result: ExportResult.Success?) { val intent = Intent("${activity.packageName}.ShowExportResult").apply { result?.let { putExtra(EXTRA_EXPORTED_SUCCESS, it) } addFlags(Intent.FLAG_ACTIVITY_CLEAR_TOP) } activity.startActivity(intent) } } ``` **Note**: action that is passed into ```Intent``` constructor **must be the same** as the action name from activity intent filter above. Next, add ```ExportFlowManager``` dependency using Koin and observe for ```ExportResult``` in ```CustomActivity``` ```kotlin class CustomActivity: AppCompatActivity() { private val exportFlowManager: ExportFlowManager by inject() override fun onCreate(savedInstanceState: Bundle?) { super.onCreate(savedInstanceState) exportFlowManager.resultData.nonNull().observe(this) { exportResult -> //... } } } ``` **Optional**. Provide ```ExportNotificationManager``` to manage notifications about exporting process state. There are 3 options: 1. Remove notifications all notifications ```kotlin class EmptyExportNotificationManger() : ExportNotificationManager { fun showExportStartedNotification(){} fun showSuccessfulExportNotification(result: ExportResult.Success){} fun showFailedExportExportNotification(){} } ``` and provide this implementation in [VideoEditorModule](https://github.com/Banuba/ve-sdk-android-integration-sample/blob/main/app/src/main/java/com/banuba/example/integrationapp/VideoEditorModule.kt#L75): ```kotlin single { EmptyExportNotificationManger() } ``` 2. Customize default implementation You can override the following android resources: - R.string.export.notification_started - for message about started export - R.string.export_notification_success - for message about succeeded export - R.string.export_notification_fail - for message about failed export - R.drawable.ic_export_notification - for notification icon in system bar 3. Provide custom implementation of ```ExportNotificationManager``` in [VideoEditorModule](https://github.com/Banuba/ve-sdk-android-integration-sample/blob/main/app/src/main/java/com/banuba/example/integrationapp/VideoEditorModule.kt#L75) ```kotlin single { CustomExportNotificationManger() } ``` ## Export GIF preview Video Editor allows to export preview of a video as a GIF file. Use ```GifMaker.Params``` in your ```CustomExportParamsProvider``` to create params for exporting GIF preview ```kotlin data class Params( val destFile: File, val sourceVideoRangeMs: LongRange = 0..1000L, val fps: Int = 15, val width: Int = 240, val useDithering: Boolean = true, val reverse: Boolean = true ) ``` where - `destFile` - where to store the file - `sourceVideoRangeMs` - is a range of exported video that will be used to create gif image - `fps` - frames per second within gif image - `width` - width of gif image in pixels - `useDithering` - flag that apply or remove dithering effect (in simple words make an image of better quality) - `reverse` - flag to reverse playback inside gif Use ```interactivePreview``` method in ```ExportParams.Builder``` to enable exporting GIF ```diff ExportParams.Builder(sizeProvider.videoResolution) .effects(effects) .fileName("export_video") .videoRangeList(videoRangeList) .destDir(exportSessionDir) .musicEffects(musicEffects) .extraAudioFile(extraSoundtrackUri) .volumeVideo(videoVolume) // highlight-add-next-line + .interactivePreview(gifPreviewParams) .build() ``` ## Get audio used in export You can get all audio used in exported video when export finished successfully. ```ExportBundleHelper.getExportedMusicEffect``` requires ```additionalExportData``` value of ```ExportResult.Success```. The result is ```List``` where ```MusicEffectExportData``` is ```kotlin @Parcelize data class MusicEffectExportData( val title: String, val type: MusicEffectType, val uri: Uri ) : Parcelable ``` `MusicEffectType` contains next values: 1. `TRACK` - audio tracks that were added on the `Editor` screen 2. `VOICE` - voice record track that was added on the `Editor` screen 3. `CAMERA_TRACK` - audio track that was added on the `Camera` screen ## Export metadata analytics Video Editor generates simple metadata analytics while exporting media content that you can use to analyze what media content your users make. Metadata is a JSON string and can be returned from ```ExportResult.Success``` in this way ```kotlin //"exportResult" is an instance of ExportResult.Success object val outputBundle = exportResult.additionalExportData.getBundle(ExportBundleProvider.Keys.EXTRA_EXPORT_OUTPUT_INFO) val analytics = outputBundle?.getString(ExportBundleProvider.Keys.EXTRA_EXPORT_ANALYTICS_DATA) ``` ```ExportBundleProvider.Keys``` includes all constants you can use for parsing. Sample ```JSON { "export_success": true, // defines if the export finished succesffully "aspect_ratio": "original", // aspect ration used in exported video "video_resolutions": ["1080x1920"], // list of video resolutions used in export "camera_effects": [], // list of effects of features used on camera screen while recording video "ppt_effects": { "visual": 2, // num of visual effects i.e. Glitch, VHS used in exported video "speed": 1, // num of speed effects used in exported video "mask": 6, // num of AR masks used in exported video "color": 3, // num of color effects used in exported video "text": 1, // num of text effects used in exported video "sticker": 1, // num of sticker effects used in exported video "blur": 1 // num of blur effects used in exported video }, "sources": { "camera": 0, // num of video sources recorded on camera screen(not PIP) "gallery": 1, // num of video sources selected in the gallery "pip": 0, // num of video recorded with PIP "slideshow": 0, // num of video exported as slideshow "audio": 0 // num of audi tracks }, "export_duration": 12.645, // export processing duration "video_duration": 20.11, // exported video duration "video_count": 1, // num of exported video files "os_version": "11", // OS version "sdk_version": "1.26.6" // VE SDK version } ``` --- ## Face AR and AR Cloud products on Android [Banuba Face AR SDK](https://www.banuba.com/facear-sdk/face-filters) product is used on camera and editor screens for applying various AR effects while making video content. ## Overview Any Face AR effect is a folder that includes a number of files required for Face AR SDK to play this effect. :::tip Make sure ```preview.png``` file is included in effect folder. You can use this file as a preview for AR effect. ::: ## Integrate Face AR :::info ```Banuba Face AR SDK``` integration is included in the [Github sample](https://github.com/Banuba/ve-sdk-android-integration-sample). ::: Follow next steps to integrate ```Banuba Face AR SDK``` into your project. First, add Gradle ```com.banuba.sdk:effect-player-adapter``` dependency in [app gradle file](https://github.com/Banuba/ve-sdk-android-integration-sample/blob/main/app/build.gradle#L74). ```diff def banubaSdkVersion = '1.48.5' ... // highlight-add-next-line + implementation "com.banuba.sdk:effect-player-adapter:${banubaSdkVersion}" ... ``` Add ```BanubaEffectPlayerKoinModule().module``` in the [VideoEditorModule](https://github.com/Banuba/ve-sdk-android-integration-sample/blob/main/app/src/main/java/com/banuba/example/integrationapp/VideoEditorModule.kt#L40) ```diff startKoin { androidContext(this@IntegrationApp) modules( ... // highlight-add-next-line + BanubaEffectPlayerKoinModule().module ) } ``` ## Add and Manage effects There are 3 options for adding and managing AR effects: 1. Store all effects in [assets/bnb-resources/effects](https://github.com/Banuba/ve-sdk-android-integration-sample/blob/main/app/src/main/assets/bnb-resources/effects) folder in the app. 2. Store color effects in [assets/bnb-resources/luts](https://github.com/Banuba/ve-sdk-android-integration-sample/tree/main/app/src/main/assets/bnb-resources/luts) folder in the app. 3. Use [AR Cloud](https://www.banuba.com/faq/what-is-ar-cloud) for storing effects on a server. :::tip You can use both options i.e. store just a few AR effects in ```assets``` and 100 or more AR effects on ```AR Cloud```. ::: ## Integrate AR Cloud ```AR Cloud``` is a cloud solution for storing Banuba Face AR effects on the server and used by Face AR and Video Editor products. Any AR effect downloaded from ```AR Cloud``` is cached on the user's device. Follow next steps to integrate ```AR Cloud``` into your project. First, add Gradle ```com.banuba.sdk:ar-cloud``` dependency in [app gradle file](https://github.com/Banuba/ve-sdk-android-integration-sample/blob/main/app/build.gradle). ```diff def banubaSdkVersion = '1.48.5' ... // highlight-add-next-line + implementation "com.banuba.sdk:ar-cloud:${banubaSdkVersion}" ... ``` Next, add ```ArCloudKoinModule``` module to [Koin modules](https://github.com/Banuba/ve-sdk-android-integration-sample/blob/main/app/src/main/java/com/banuba/example/integrationapp/VideoEditorModule.kt#L67). ```diff startKoin { ... modules( VeSdkKoinModule().module, ... // highlight-add-next-line + ArCloudKoinModule().module, ... ) } ``` Next, override ```ArEffectsRepositoryProvider``` in [VideoEditorModule](https://github.com/Banuba/ve-sdk-android-integration-sample/blob/main/app/src/main/java/com/banuba/example/integrationapp/VideoEditorModule.kt#L91). ```kotlin single(createdAtStart = true) { ArEffectsRepositoryProvider( arEffectsRepository = get(named("backendArEffectsRepository")), ioDispatcher = get(named("ioDispatcher")) ) } ``` ## Change order effects By default, all AR effects are listed in alphabetical order. AR effects from ```assets``` are listed in order. Create new class ```CustomMaskOrderProvider``` and implement ```OrderProvider``` to provide custom order. ```kotlin class CustomMaskOrderProvider : OrderProvider { override fun provide(): List = listOf("Background", "HeadphoneMusic") } ``` :::info These are names of specific directories in ```assets/bnb-resources/effects``` or on ```AR Cloud```. ::: Next, use ```CustomMaskOrderProvider``` in [VideoEditorModule](https://github.com/Banuba/ve-sdk-android-integration-sample/blob/main/app/src/main/java/com/banuba/example/integrationapp/VideoEditorModule.kt#L75) ```kotlin single(named("maskOrderProvider")) { CustomMaskOrderProvider() } ``` ## Disable Face AR SDK Video Editor SDK can be used without Face AR SDK. Remove ```BanubaEffectPlayerKoinModule().module``` from [VideoEditorModule](https://github.com/Banuba/ve-sdk-android-integration-sample/blob/main/app/src/main/java/com/banuba/example/integrationapp/VideoEditorModule.kt#L62) ```diff startKoin { androidContext(this@IntegrationApp) modules( ... // highlight-remove-next-line - BanubaEffectPlayerKoinModule().module ) } ``` and remove Gradle dependency ```com.banuba.sdk:effect-player-adapter``` ```diff ... // highlight-remove-next-line - implementation "com.banuba.sdk:effect-player-adapter:${banubaSdkVersion}" ... ``` --- ## Drawing on Android Lets your users draw freely on screen. It is a convenient tool for highlighting important objects in the video or spicing up the frame with funny doodles or stylish art. There are 5 line variants to choose from for more self-expression opportunities. :::important The Drawing feature is disabled by default. Please contact Banuba representatives to know more about using this feature. :::       --- ## Gallery screen on Android Video Editor SDK includes built in gallery functionality where the user can pick any video or image and use it while making video. :::info Gallery is integrated by default. ::: ## Integration The following guide will help you to integrate gallery to your project if it was not added before. Add module ```com.banuba.sdk:ve-gallery-sdk:1.48.5``` to [gradle](https://github.com/Banuba/ve-sdk-android-integration-sample/blob/main/app/build.gradle#L73) file and specify ```GalleryKoinModule``` module in [VideoEditorModule](https://github.com/Banuba/ve-sdk-android-integration-sample/blob/main/app/src/main/java/com/banuba/example/integrationapp/VideoEditorModule.kt#L39) ```diff startKoin { androidContext(this@IntegrationApp) modules( ... // highlight-add-next-line + GalleryKoinModule().module ) } ``` ## Customizations You can control options ```Videos``` and ```Photos``` by overriding instance of ```EditorConfig``` in [VideoEditorModule](https://github.com/Banuba/ve-sdk-android-integration-sample/blob/main/app/src/main/java/com/banuba/example/integrationapp/VideoEditorModule.kt#L51) . ```diff single { EditorConfig( // highlight-add-next-line + gallerySupportsVideo = ..., // true - show Videos, false - hide. Defaul - true // highlight-add-next-line + gallerySupportsImage = ..., // true - show Photos, false - hide. Default - true ) } ``` ## Implement custom gallery Video editor allows to replace default gallery with your custom. Please follow implementation guide. First, create ```CustomMediaContentProvider``` class that implements ```ContentFeatureProvider, Fragment>```. This class describes a contract between Video Editor and specific Fragment from your project for gallery. ```kotlin class CustomMediaContentProvider : ContentFeatureProvider, Fragment> { private var activityResultLauncher: ActivityResultLauncher? = null private val activityResultCallback: (List?) -> Unit = { activityResultCallbackInternal(it) } private var activityResultCallbackInternal: (List?) -> Unit = {} override fun init(hostComponent: WeakReference) { activityResultLauncher = hostComponent.get()?.registerForActivityResult( ProvideMediaContentContract(), activityResultCallback ) } override fun requestContent( context: Context, extras: Bundle ): ContentFeatureProvider.Result> = ContentFeatureProvider.Result.RequestUi( intent = SelectExternalContentActivity.newGetMediaIntent(context).apply { putExtras(extras) } ) override fun handleResult( hostComponent: WeakReference, intent: Intent, block: (List?) -> Unit ) { activityResultCallbackInternal = block activityResultLauncher?.launch(intent) } } ``` Method `init` is invoked in Video Editor to register `onActivityResult` callback and receive media content. `com.banuba.sdk.core.domain.ProvideMediaContentContract` class is used to manage data bundle that is passed between video editor and your custom media provider implementation. Method `requestContent` is used to create `Intent` for starting your custom Activity with gallery. `extras` argument contains metadata for media content selection and should be passed into your custom media provider. `handleResult` method connects `onActivityResult` callback within Video Editor with the media content provided by `CustomGalleryActivity`. Next, obtain media request params in your `CustomGalleryActivity.onCreate()` ```kotlin override fun onCreate(savedInstanceState: Bundle?) { super.onCreate(savedInstanceState) intent.extras?.let { extras -> val params = ProvideMediaContentContract.obtainParams(extras) ... } ``` `Params` object contains data about request from video editor ```kotlin data class Params( val mode: OpenGalleryMode, val types: List, val maxCount: Int, val minCount: Int, val supportedFormats: List ) ``` `mode` - is a type of request (NORMAL, FEATURE_BACKGROUND, ADD_TO_TRIMMER) `types` - list of requested media types (Video, Image) `supportedFormats` - list of media file extensions that are supported by SDK Deliver media data from `CustomGalleryActivity` to Video Editor SDK ```kotlin val resultIntent = Intent().apply { putParcelableArrayListExtra( ProvideMediaContentContract.EXTRA_MEDIA_CONTENT_RESULT, ArrayList(selectedMedia) ) } setResult(Activity.RESULT_OK, resultIntent) ``` `selectedMedia` is a list of media Uris with **content://** scheme. Finally, provide `CustomMediaContentProvider` in `VideoEditorModule.kt` Koin module. ```kotlin factory, Fragment>>(named("mediaDataProvider"), override = true) { (external: Boolean?) -> CustomMediaContentProvider() } ``` and remove module `GalleryKoinModule` from the list of modules. ```diff startKoin { androidContext(applicationContext) allowOverride(true) modules( ... // highlight-remove-next-line - GalleryKoinModule().module, ... ) } ``` --- ## Green Screen on Android Video Editor SDK brings you real-time background subtraction technology for you to empower your users with the best quality virtual background service. Users can automatically remove, change or augment backgrounds. :::important The feature requires Face AR product with [Background Subtraction](https://www.banuba.com/technology/background-subtraction) feature. Please contact Banuba representatives to know more about using this feature. :::   ```Green Screen``` feature doesn't require physical green screens, the neural networks do all the work. --- ## Open Photo Editor SDK from Camera screen on Android This guide demonstrates how to open [Photo Editor SDK](https://www.banuba.com/photo-editor-sdk) just after taking a photo on Video Editor Camera screen. All recorded video or taken images on Camera screen can be handled using ```MediaNavigationProcessor```. Provide custom implementation of ```MediaNavigationProcessor``` in Koin module to override flow. ``` diff class SampleIntegrationKoinModule { val module = module { ... // highlight-add-next-line + single { object : MediaNavigationProcessor { override fun process(activity: Activity, mediaList: List): Boolean { // Filter media resources to find the target image for Photo Editor SDK val pngs = mediaList.filter { it.path?.contains(".png") ?: false } return if (pngs.isEmpty()) { true } else { // Create ExportResult and close Video Editor SDK. (activity as? VideoCreationActivity)?.closeWithResult( ExportResult.Success( emptyList(), pngs.first(), Uri.EMPTY, Bundle() ) ) false } } } } } ``` Handle received ```ExportResult``` in your ```registerForActivityResult``` or ```Activity.onActivityForResult``` method. ```diff private val createVideoRequest = registerForActivityResult(CustomExportResultVideoContract()) { exportResult -> exportResult?.let { if (exportResult is ExportResult.Success) { // Use exported preview file to open Photo Editor SDK // highlight-add-next-line photoEditorExportResult.launch( PhotoCreationActivity.startFromEditor( applicationContext, imageUri = exportResult.preview ) ) ... } } } private val photoEditorExportResult = registerForActivityResult(PhotoExportResultContract()) { uri -> // Handle exported image result } ``` --- ## Share video screen on Android Share video screen allows users to easily share an exported video using popular social media services and OS specific components. :::info This is an optional screen that you can add to your video editing flow. ::: ## Integration Create new layout ```activity_video_sharing.xml``` file in ```res/layout``` folder. This layout is required for integrating ```VideoSharingFragment``` from the SDK. ``` xml ``` Next, create new ```VideoSharingActivity``` that will handle social media keys and start video sharing screen. ``` kotlin class VideoSharingActivity : AppCompatActivity(R.layout.activity_video_sharing) { companion object { // Set up your Facebook app id const val FACEBOOK_APP_ID = "" } private val exportResult by lazy(LazyThreadSafetyMode.NONE) { intent?.getParcelableExtra(EXTRA_EXPORTED_SUCCESS) } override fun onCreate(savedInstanceState: Bundle?) { super.onCreate(savedInstanceState) val videoSharingFragment = VideoSharingFragment.newInstance( exportResult = exportResult, fbAppId = FACEBOOK_APP_ID ) supportFragmentManager.addFragment( videoSharingFragment, VideoSharingFragment.TAG, R.id.fragmentContainer, false ) } override fun onDestroy() { super.onDestroy() // SampleApp is an implementation available in main Video Editor Sample. // Release video editor dependencies when the user leaves this creen (application as? SampleApp)?.releaseVideoEditor() } } ``` and specify ```VideoSharingActivity``` in ```AndroidManifest.xml``` file. ``` xml ``` Finally, start ```VideoSharingActivity``` to open video sharing screen. ``` kotlin val intent = Intent(getString(com.banuba.sdk.export.R.string.export_action_name, packageName)).apply { putExtra(EXTRA_EXPORTED_SUCCESS, exportResult) } startActivity(intent) ``` --- ## Stickers on Android Guide to using stickers in Video Editor SDK on Android. ## Giphy Video Editor SDK has built in integration with [Giphy service](https://developers.giphy.com/docs/api/) for loading stickers. ```GIPHY``` doesn't charge for their content. The one thing they do require is attribution. Also, there is no commercial aspect to the current version of the product (no advertisements, etc.). Any sticker effect is a GIF file. To use stickers in your project you need to request personal [Giphy API key](https://support.giphy.com/hc/en-us/articles/360020283431-Request-A-GIPHY-API-Key). Override instance of ```GifPickerConfigurations``` and set GIPHY API key to ```giphyApiKey``` in [VideoEditorModule](https://github.com/Banuba/ve-sdk-android-integration-sample/blob/main/app/src/main/java/com/banuba/example/integrationapp/VideoEditorModule.kt#L53). ``` diff factory { GifPickerConfigurations( // highlight-add-next-line + giphyApiKey = .... ) } ``` Stickers will appear in the container on editor screen once ```giphyApiKey``` is set. --- ## Video recording integration guide on Android Guide to modifying video recording feature in Video Editor SDK. ## Quality details Subsequent table describes video quality details used for video recording in various resolutions. | Recording speed | 360p(360 x 640) | 480p(480 x 854) | HD(720 x 1280) | FHD(1080 x 1920) | | --------------- | --------------- | --------------- | -------------- | ---------------- | | 1x(Default) | 1200 | 2000 | 4000 | 6400 | | 0.5x | 900 | 1500 | 3000 | 4800 | | 2x | 1800 | 3000 | 6000 | 9600 | | 3x | 2400 | 4000 | 8000 | 12800 | ## Customize configurations ```CameraConfig``` is a main class used to customize features, behavior and user experience for video recording on camera screen i.e. set min/max recording duration, flashlight, etc. Video editor includes default implementation but you can provide your own implementation to meet your requirements in [VideoEditorModule](https://github.com/Banuba/ve-sdk-android-integration-sample/blob/main/app/src/main/java/com/banuba/example/integrationapp/VideoEditorModule.kt). For example, to sets max video recording duration to 30 seconds. ```kotlin single(override = true) { CameraConfig(maxRecordedTotalVideoDurationMs = 30_000) } ``` | Property | Values | Description | | ------------- |:--------------------------------------------------------------------------------------------:| :------------- | | minRecordedTotalVideoDurationMs | Number > 0; Default ```3000``` | minimum video recording duration *in milliseconds* required to proceed and open video editing screen (i.e. 3000 for 3 seconds) | maxRecordedTotalVideoDurationMs | Number > 0; Default ```120000``` | maximum video recording duration *in milliseconds* available to record | minRecordedChunkVideoDurationMs | Number > 1000; Default ```1000``` | minimum video recording duration *in milliseconds* that is allowed to record on camera | takePhotoOnTap | true/false; Default ```false``` | defines if it is available to take a photo on the camera screen by tap. ```true``` photo is taken by tap and video is recording by long press. | supportsMultiRecords | true/false; Default ```true``` | defines if the use can record multiple video subsequently. ```false``` when the first video recording is done the editing screen will be opened | supportsFlashlight | true/false; Default ```true``` | enables flashlight icon on the camera screen and possibility to take a photo with flashlight | supportsSpeedRecording | true/false; Default ```true``` | enables speed recording icon on the camera screen and possibility to select recording speed | supportsExternalMusic | true/false; Default ```true``` | enables the music icon on the camera screen and possibility to add music track playing over the video recording | supportsMuteMic | true/false; Default ```true``` | enables mute microphone icon on the camera screen and possibility to record video without capturing sound | switchFacingOnDoubleTap | true/false; Default ```true``` | ```true``` allows to switch between front and back camera by double tap | isStartFrontFacingFirst | true/false; Default ```true``` | ```true``` means that ```front``` camera facing is used on the first launch of the camera screen, ```false``` means that ```back``` camera facing is used on the first launch of the camera screen | isSaveLastCameraFacing | true/false; Default ```true``` | defines if the camera facing (```back``` or ```front```) is saved and restored | cameraFpsMode | CameraFpsMode enum values; Default ```CameraFpsMode.FIXED``` | ```CameraFpsMode.FIXED``` means that video recording quality can be degraded to maintain 30 FPS while applying "heavy" Face AR effects (*This behavior is recommended* and allows to reach seamless usage on wide range of devices). ```CameraFpsMode.ADAPTIVE``` means that FPS can be reduced in order to maintain video quality(not recommended). | showCameraInfoAndPerformance | true/false; Default ```false``` | enables debug views for showing camera system details such as current FPS, Iso etc. | supportsSwitchFacing | true/false; Default ```true``` | defines if camera facing switching is available. | supportsAudioRateEqualsVideoSpeed | true/false; Default ```false``` | determines if the audio playback speed is equal to the video recording speed. | supportsGallery | true/false; Default ```true``` | defines if there is an icon on the camera screen at the bottom-right to pick a content from gallery. | videoDurations | List<Long>; Default ```listOf(maxRecordedTotalVideoDurationMs, 60_000L, 30_000L, 15_000L)``` | defines the list of durations available to record. The user can see the option on the camera screen and pick new option. For example, ```60000L``` means that the user can record a number of video with total duration no more than 60 seconds. | supportsVideoDurationSwitcher | true/false; Default ```true``` | defines if video recording time interval swithced is enabled ## Configure microphone state Use ```CameraMuteMicConfig``` if you want to customize default state of microphone on the camera screen. Here is default implementation in [VideoEditorModule](https://github.com/Banuba/ve-sdk-android-integration-sample/blob/main/app/src/main/java/com/banuba/example/integrationapp/VideoEditorModule.kt) ```kotlin factory { CameraMuteMicConfig( // If mic should be muted when open camera screen in normal mode muteInNormalMode = false, // If mic should be muted when open camera screen in picture in picture mode muteInPipMode = true, // If mic should be muted when open camera screen with passed audio track muteWithAudioTrack = true ) } ``` ## Configure recording modes Camera screen includes 3 modes for recording content implemented as ```RecordMode``` - ```Photo``` - ```Video``` - ```Photo``` and ```Video``` - default value Implement ```CameraRecordingModesProvider``` in [VideoEditorModule](https://github.com/Banuba/ve-sdk-android-integration-sample/blob/main/app/src/main/java/com/banuba/example/integrationapp/VideoEditorModule.kt) to customize mode that meets your requirements. Default implementation is ```kotlin single { object : CameraRecordingModesProvider { override var availableModes = setOf(RecordMode.Video, RecordMode.Photo) } } ``` :::danger ```availableModes``` must not be empty, otherwise a crash will happen. ::: ## Picture in picture Picture in Picture or ```PIP``` is video editing technique that lets you overlay two videos in the same video. The multi-layer editing effect is perfect for reaction videos, slideshows, product demos, and more. This feature is similar to TikTok duet feature.   :::info The feature is disabled by default and can be enabled if the license supports it. Please ask Banuba business representatives to include the feature in your license. ::: The subsequent guide explains how to start and customize ```PIP```. First, pass ```pictureInPictureConfig``` in [VideoCreationActivity.startFromCamera](https://github.com/Banuba/ve-sdk-android-integration-sample/blob/main/app/src/main/java/com/banuba/example/integrationapp/MainActivity.kt#L179-L194) method ```kotlin val localVideoUri = ... VideoCreationActivity.startFromCamera( context = this, // set PiP video configuration pictureInPictureConfig = PipConfig( video = localVideoUri, openPipSettings = false // if you want to open pip settings at startup ) ) ``` ```PIP``` includes 4 modes that you can use. - ```Floating``` - ```TopBottom``` - ```React``` - ```LeftRight``` Implement ```PipLayoutProvider``` in [VideoEditorModule](https://github.com/Banuba/ve-sdk-android-integration-sample/blob/main/app/src/main/java/com/banuba/example/integrationapp/VideoEditorModule.kt) to customize the order of modes and other capabilities. ```kotlin single { object : PipLayoutProvider { override fun provide( insetsOffset: Int, screenSize: Size ): List { val context = androidContext() return listOf( EditorPipLayoutSettings.Floating( context = context, physicalScreenSize = screenSize, topOffsetPx = context.dimen(R.dimen.pip_floating_top_offset) + insetsOffset ), EditorPipLayoutSettings.TopBottom(), EditorPipLayoutSettings.React( context = context, physicalScreenSize = screenSize, topOffsetPx = context.dimen(R.dimen.pip_react_top_offset) + insetsOffset ), EditorPipLayoutSettings.LeftRight() ) } } } ``` Please do not forget to update [CameraMuteMicConfig](#configure-microphone-state) implementation if you want to change use of microphone in ```PIP```. You can even customize camera align for each mode and exclude actions for some modes: ```diff ... return listOf( EditorPipLayoutSettings.Floating( ..., excludeActions = listOf( EditorPipLayoutAction.SwitchVertical, EditorPipLayoutAction.Square, EditorPipLayoutAction.Round ), // highlight-add-next-line + isCameraAlignTop = false ), EditorPipLayoutSettings.TopBottom( excludeActions = listOf( EditorPipLayoutAction.SwitchVertical, EditorPipLayoutAction.Original, EditorPipLayoutAction.Centered ), // highlight-add-next-line + isCameraAlignTop = false ), EditorPipLayoutSettings.React( ..., excludeActions = listOf( EditorPipLayoutAction.SwitchVertical, EditorPipLayoutAction.Square, EditorPipLayoutAction.Round, EditorPipLayoutAction.Centered, EditorPipLayoutAction.Original ), isCameraMain = false ), EditorPipLayoutSettings.LeftRight( excludeActions = listOf( EditorPipLayoutAction.SwitchHorizontal, EditorPipLayoutAction.Original, EditorPipLayoutAction.Centered ), // highlight-add-next-line + isCameraAlignLeft = false ) ) ``` --- ## Weatherman on Android Weatherman mode is an improvement on top of Banuba's cutting-edge background replacement. It allows users to drag and drop themselves on any place on the screen to emulate the look of a TV presenter the feature is named after. Weatherman is useful for reactions, learning content, presentations, explainers and other similar videos thanks to the professional feel it creates. :::important The Weatherman feature is disabled by default. Please contact Banuba representatives to know more about using this feature. :::     --- ## Install Photo Editor on Android Guide to installing Photo Editor SDK on Android. SDK modules are stored on GitHub Packages. ## Add repositories Open your project [gradle](https://github.com/Banuba/ve-sdk-android-integration-sample/blob/main/build.gradle#L21) file and add repositories to ```allprojects``` section. ```groovy allprojects { repositories { maven { name = "nexus" url = uri("https://nexus.banuba.net/repository/maven-releases") } } } ``` ## Add dependencies Specify dependencies in the app [gradle](https://github.com/Banuba/ve-sdk-android-integration-sample/blob/main/app/build.gradle#L83) file. ```groovy def banubaPESdkVersion = '1.2.24' implementation "com.banuba.sdk:pe-sdk:${banubaPESdkVersion}" def banubaSdkVersion = '1.48.5' implementation "com.banuba.sdk:core-sdk:${banubaSdkVersion}" implementation "com.banuba.sdk:core-ui-sdk:${banubaSdkVersion}" implementation "com.banuba.sdk:ve-gallery-sdk:${banubaSdkVersion}" implementation "com.banuba.sdk:effect-player-adapter:${banubaSdkVersion}" ``` Add ```kotlin-parcelize``` plugin into plugins section of the [gradle](https://github.com/Banuba/ve-sdk-android-integration-sample/blob/main/app/build.gradle#L63) file. ```groovy plugins { id 'com.android.application' id 'kotlin-android' id 'kotlin-parcelize' } ``` --- ## Install Video Editor on Android Guide to installing Video Editor SDK on Android. SDK modules are stored on GitHub Packages. ## Add repositories Open your project [gradle](https://github.com/Banuba/ve-sdk-android-integration-sample/blob/main/build.gradle#L21) file and add repositories to ```allprojects``` section. ```groovy allprojects { repositories { maven { name = "nexus" url = uri("https://nexus.banuba.net/repository/maven-releases") } ... } } ``` ## Add Packaging Options Settings Specify the following ```packaging options``` in your [build gradle](https://github.com/Banuba/ve-sdk-android-integration-sample/blob/main/app/build.gradle#L46-L53) file: ```groovy android { ... packagingOptions { jniLibs { useLegacyPackaging = true } } ... } ``` ## Add dependencies Specify dependencies in the app [gradle](https://github.com/Banuba/ve-sdk-android-integration-sample/blob/main/app/build.gradle#L63) file. ```groovy def banubaSdkVersion = '1.48.5' implementation "com.banuba.sdk:ffmpeg:5.3.0" implementation "com.banuba.sdk:camera-sdk:${banubaSdkVersion}" implementation "com.banuba.sdk:camera-ui-sdk:${banubaSdkVersion}" implementation "com.banuba.sdk:core-sdk:${banubaSdkVersion}" implementation "com.banuba.sdk:core-ui-sdk:${banubaSdkVersion}" implementation "com.banuba.sdk:ve-flow-sdk:${banubaSdkVersion}" implementation "com.banuba.sdk:ve-sdk:${banubaSdkVersion}" implementation "com.banuba.sdk:ve-ui-sdk:${banubaSdkVersion}" implementation "com.banuba.sdk:ve-gallery-sdk:${banubaSdkVersion}" implementation "com.banuba.sdk:ve-effects-sdk:${banubaSdkVersion}" implementation "com.banuba.sdk:effect-player-adapter:${banubaSdkVersion}" implementation "com.banuba.sdk:ar-cloud:${banubaSdkVersion}" implementation "com.banuba.sdk:ve-audio-browser-sdk:${banubaSdkVersion}" implementation "com.banuba.sdk:ve-export-sdk:${banubaSdkVersion}" implementation "com.banuba.sdk:ve-playback-sdk:${banubaSdkVersion}" ``` Add ```kotlin-parcelize``` plugin into plugins section of the [gradle](https://github.com/Banuba/ve-sdk-android-integration-sample/blob/main/app/build.gradle#L63) file. ```groovy plugins { id 'com.android.application' id 'kotlin-android' id 'kotlin-parcelize' } ``` --- ## Launching Photo Editor on Android Guide to launching Photo Editor SDK. ## Prerequisites :exclamation: The license token **IS REQUIRED** to use Photo Editor SDK in your app. Please check [Requirements](requirements-pe.md#license-token) out guide if the license token is not set. ## Initialize SDK Create an instance of ```BanubaPhotoEditor``` by using the license token ``` kotlin val photoEditorSDK = BanubaPhotoEditor.initialize(LICENSE_TOKEN) ``` ```photoEditorSDK``` is ```null``` when the license token is incorrect i.e. empty, truncated. If ```photoEditorSDK``` is not ```null``` you can proceed and start video editor. Next, we strongly recommend [checking](https://github.com/Banuba/ve-sdk-android-integration-sample/blob/main/app/src/main/java/com/banuba/example/integrationapp/MainActivity.kt#L110) your license state before staring video editor ```kotlin photoEditorSDK.getLicenseState { isValid -> if (isValid) { // βœ… License is active, all good // Start Photo Editor SDK } else { // ❌ Use of Photo Editor is restricted. License is revoked or expired. } } ``` :::danger Photo editing content unavailable screen will appear if the user starts Photo Editor SDK with revoked or expired license. ::: ## Start editor Import classes ```kotlin ``` Start Photo Editor SDK and handle exported results. [See example](https://github.com/Banuba/ve-sdk-android-integration-sample/blob/main/app/src/main/java/com/banuba/example/integrationapp/MainActivity.kt#L79). ```kotlin val photoEditorExportResult = registerForActivityResult(PhotoExportResultContract()) { uri -> // Handle exported image result Log.d(TAG, "Image exported $uri") } // Start Photo Editor SDK photoEditorExportResult.launch(PhotoCreationActivity.startFromGallery(this)) ``` --- ## Launching Video Editor on Android Guide to launching Video Editor SDK on Android. ## Prerequisites :exclamation: The license token **IS REQUIRED** to use Video Editor SDK in your app. Please check [Requirements](requirements-ve.md#license-token) out guide if the license token is not set. ## Update AndroidManifest Add ```VideoCreationActivity``` in your [AndroidManifest.xml](https://github.com/Banuba/ve-sdk-android-integration-sample/blob/main/app/src/main/AndroidManifest.xml#L27) file. ``` xml ``` ```VideoCreationActivity``` is used for brining together and managing Video Editor flow. Each screen is implemented as an Android [Fragment](https://developer.android.com/guide/fragments). Next, add permissions ```xml ``` Network is used for downloading AR effects from [AR Cloud](https://www.banuba.com/faq/what-is-ar-cloud) and stickers from [Giphy](https://giphy.com/). [CustomIntegrationAppTheme](https://github.com/Banuba/ve-sdk-android-integration-sample/blob/main/app/src/main/res/values/themes.xml#L13) is the custom implementation of ```VideoCreationTheme``` and is required for running ```VideoCreationActivity```. Use this implementation for customizing visual appearance of Video Editor SDK i.e. colors, icons and more. ## Configuration Custom behavior of Video Editor SDK in your app is implemented by using Dependency Injection framework [Koin](https://insert-koin.io/). Create new Kotlin class ```VideoEditorModule``` for configuring Video Editor SDK. Next, add new class ```SampleIntegrationKoinModule``` for initializing and customizing Video Editor SDK features. ``` kotlin class VideoEditorModule { ... private class SampleIntegrationKoinModule { val module = module { ... } } } ``` Add [initialize](https://github.com/Banuba/ve-sdk-android-integration-sample/blob/main/app/src/main/java/com/banuba/example/integrationapp/VideoEditorModule.kt#L47) method in ```VideoEditorModule``` class for initializing Video Editor SDK and specify ```SampleIntegrationKoinModule``` in the list of modules. ```kotlin fun initialize(applicationContext: Context) { startKoin { androidContext(applicationContext) allowOverride(true) // pass the customized Koin module that implements required dependencies. Keep order of modules modules( VeSdkKoinModule().module, VeExportKoinModule().module, VePlaybackSdkKoinModule().module, AudioBrowserKoinModule().module, // use this module only if you bought it ArCloudKoinModule().module, VeUiSdkKoinModule().module, VeFlowKoinModule().module, GalleryKoinModule().module, BanubaEffectPlayerKoinModule().module, + SampleIntegrationKoinModule().module, ) } } ``` Finally, [initialize SDK](https://github.com/Banuba/ve-sdk-android-integration-sample/blob/main/app/src/main/java/com/banuba/example/integrationapp/SampleApp.kt#L42) in your Android Application ```onCreate()``` method. ``` kotlin override fun onCreate() { super.onCreate() VideoEditorModule().initialize(this) ... } ``` ## Setup export Video Editor can export a number of media files to meet your requirements. Implement ```ExportParamsProvider``` and provide ```List``` where every ```ExportParams``` is a media file i.e. video or audio. Check [CustomExportParamsProvider](https://github.com/Banuba/ve-sdk-android-integration-sample/blob/main/app/src/main/java/com/banuba/example/integrationapp/VideoEditorModule.kt#L119) implementation. ## Initialize with license Create an instance of ```BanubaVideoEditor``` by using the license token ``` kotlin val editorSDK = BanubaVideoEditor.initialize(LICENSE_TOKEN) ``` ```editorSDK``` is ```null``` when the license token is incorrect i.e. empty, truncated. If ```editorSDK``` is not ```null``` you can proceed and start video editor. :::tip Share the same instance of ```BanubaVideoEditor``` for Video Editor and Photo Editor SDK. ::: Next, we strongly recommend [checking](https://github.com/Banuba/ve-sdk-android-integration-sample/blob/main/app/src/main/java/com/banuba/example/integrationapp/MainActivity.kt#L110) your license state before staring video editor ```kotlin editorSDK.getLicenseState { isValid -> if (isValid) { // βœ… License is active, all good // Start Video Editor SDK } else { // ❌ Use of Video Editor is restricted. License is revoked or expired. } } ``` :::warning Video content unavailable screen will appear if the user starts Video Editor SDK with revoked or expired license. ::: ## Start editor Start Video Editor SDK and handle exported results. [See example](https://github.com/Banuba/ve-sdk-android-integration-sample/blob/main/app/src/main/java/com/banuba/example/integrationapp/MainActivity.kt#L25). ```kotlin val createVideoRequest = registerForActivityResult(IntegrationAppExportVideoContract()) { exportResult -> exportResult?.let { //handle ExportResult object } } val intent = VideoCreationActivity.startFromCamera( context = this, // set PiP video configuration pictureInPictureConfig = null, // setup what kind of action you want to do with VideoCreationActivity // setup data that will be acceptable during export flow additionalExportData = null, // set TrackData object if you open VideoCreationActivity with preselected music track audioTrackData = null ) createVideoRequest.launch(intent) ``` --- ## LLM and Vibe Coding on Android If you use large language models for programming, feel free to use our LLM-ready documentation. Enhance your AI-assisted coding workflow by ingesting the Video Editor SDKs documentation. Provide the ```llms.txt``` and ```llms-full.txt``` files to your LLM of choice (e.g., OpenAI, Claude, or Gemini). Once uploaded, you can improve your integration experience query the model for technical details, code examples, and integration best practices, effectively creating a customized support assistant to guide you through the implementation process drastically reducing time to integrate. - [llms.txt](../../llms.txt) - [llms-full.txt](../../llms-full.txt) This is how they differ from the regular docs: - Plain text, no HTML, CSS or JavaScript - Markdown formatting - Structured in a way to be easily digested by LLMs --- ## Banuba Photo Editor SDK Requirements on Android Guide contains general requirements for using Photo Editor SDK on Android. ## License Token Before you commit to a license, you are free to test all the features of the SDK for free. The trial period lasts 14 days. Send us a message to start the [Photo Editor SDK trial](https://www.banuba.com/photo-editor-sdk#form). We will get back to you with the trial token. Feel free to contact us if you have any questions regarding [Photo Editor SDK](https://www.banuba.com/support). ## Project settings This is what you need to run the AI Video Editor SDK: - Kotlin 1.8+ or Java 17 - Android OS 6.0 or higher with Camera 2 API - OpenGL ES 3.0 (3.1 for Neural networks on GPU) - :white_check_mark: arm64-v8a , :white_check_mark: armv7, :exclamation: x86-64 limited support, :x: x86 - no support ## Supported media formats | Images | | ----------- | | .jpg, .gif, .heic, .png, .nef, .cr2, .jpeg, .raf, .bmp ## SDK size | Photo Editor | Total size | |:------------:|:----------:| |:white_check_mark:| 65 MB | --- ## Android Requirements and Installation Guide # Banuba Video Editor SDK Requirements on Android Learn about the Android requirements and features for the Banuba Video Editor SDK. See supported versions, compatible devices, & other details. ## SDK integration video You can watch a follow along video below demonstrating key steps of SDK integration in a project. ## License Token Before you commit to a license, you are free to test all the features of the SDK for free. The trial period lasts 14 days. Send us a message to start the [Video Editor SDK trial](https://www.banuba.com/video-editor-sdk#form). We will get back to you with the trial token. Feel free to contact us if you have any questions regarding [Video Editor SDK](https://www.banuba.com/support). ## Project settings This is what you need to run the AI Video Editor SDK: - Kotlin 2.1+ or Java 17 - Android OS 8.0 or higher with Camera 2 API - OpenGL ES 3.0 (3.1 for Neural networks on GPU) - :white_check_mark: arm64-v8a , :white_check_mark: armv7, :exclamation: x86-64 limited support, :x: x86 - no support ## Supported media formats | Audio | Video | Images | |-------------------------------------| --------- | ----------- | | .aac, .mp3, .wav, .ogg, .m4a, .flac |.mp4, .mov | .jpg, .gif, .heic, .png, .nef, .cr2, .jpeg, .raf, .bmp ## SDK size Banuba Video Editor SDK with all the core editing features only adds 25 Mb of space to your app’s download size. This parameter can change based on the number of features you want. |SDK| Total size | |:------------|:----------:| |Banuba Video Editor SDK (core editor)| 25 Mb | |Banuba Video Editor SDK (full bundle)| 45 Mb | |Video Editor with Photo Editor | 70 Mb | |Video Editor with face augmented reality features | 45 - 90 Mb | --- ## FAQ on Android These are the answers to the most common questions asked about our SDK. ### I want to start VideoEditor with a preselected audio track To open Video Editor SDK you should create an intent by utilizing any avilable function inside **VideoCreationActivity**: **startFromCamera()**, **startFromTrimmer()** or **startFromEditor()**. All these functions have an argument called **audioTrackData** where you should pass preselected audio track or null (by default). For example, to open an SDK from the camera screen with the track use the code snippet below: ```kotlin startActivity( VideoCreationActivity.startFromCamera( context = applicationContext, audioTrackData = preselectedTrackData ) ) ``` **audioTrackData** is an object of TrackData class ```kotlin data class TrackData( val id: UUID, val title: String, val localUri: Uri, val artist: String? = null ) ``` ### How to add other text fonts that are used in the editor screen To add other text fonts that are used in the editor screen follow the next steps: 1. Add font files to the `app/src/main/res/font/` directory; 2. Add fonts names to the ```strings.xml ``` resource file: ```xml Font 1 Title Font N Title ``` 3. Add `font_resources.xml` with fonts array declaration to the `app/src/main/res/values/` directory. The format of `font_resources.xml` should be the next one: ```xml @array/font_1_resource @array/font_N_resource @string/font_1_title @font/font_1 @string/font_N_title @font/font_N ``` 4. The final step is to pass your custom `font_resources` id to the `MainTextOnVideoTypefaceProvider` in the ```SampleIntegrationKoinModule``` to override the default implementation: ```kotlin single { MainTextOnVideoTypefaceProvider( context = get(), fontsArrayResId = R.array.font_resources ) } ``` ### Optimizing app size The easiest way to gain immediate app size savings when publishing to Google Play is by uploading your app as an [**Android App Bundle**](https://developer.android.com/guide/app-bundle), which is a new upload format that includes all your app’s compiled code and resources. Google Play’s new app serving model then uses your app bundle to generate and serve optimized APKs for each user’s device configuration, so they download only the code and resources they need to run your app. As a result, the final size of our library for one of the platform types (`armeabi-v7a`,` arm64-v8a`, `x86`,` x86_64`) will be **24-26 MB** less than indicated in the documentation ### How do I change the language (how do I add new locale support)? There is no special language switching mechanism in the Video Editor SDK. Out of the box, the VE SDK supports `English` locale. If you need to support any other locales, you can do it according to the standard Android way. See how [Create locale directories and resource files](https://developer.android.com/training/basics/supporting-devices/languages#CreateDirs) for more details. After adding a new locale resource file into your application with integrated VE SDK, you need to re-define the VE SDK strings keys with new locale string values. To do that you need to add all needed string keys in the new locale `strings.xml` file. The newly added locale will be applied after the device language is changed by system settings. If you need to change language programmatically in your application, see the next links how it can be done: [one](https://www.geeksforgeeks.org/how-to-change-the-whole-app-language-in-android-programmatically/), [two](https://medium.com/swlh/android-app-specific-language-change-programmatically-using-kotlin-d650a5392220) ### How do I use the Video Editor several times from different entry points? Before you want to use VideoEditor again, you need to release ```Video Editor SDK```: ```kotlin private fun releaseVideoEditor() { releaseUtilityManager() stopKoin() videoEditor = null } private fun releaseUtilityManager() { val utilityManager = try { getKoin().getOrNull() } catch (e: InstanceCreationException) { Log.w(TAG, "EditorUtilityManager was not initialized!", e) null } utilityManager?.release() } ``` ```EditorUtilityManager``` is NULL when the token is expired or revoked. ### I want to change icons and name for effects. Effect customization is implemented by android resources with well-defined names which follow a strict scheme. | Customization | Resource type | Resource name template | Example | :---: | :---: | :---: | :---: | | Title | string | visual_effect_\{id\}, time_effect_\{id\} | visual_effect_flash | Icon | drawable | ic_visual_effect_\{id\}, ic_time_effect_\{id\} | ic_time_effect_rapid | Color | color | visual_effect_color_\{id\}, time_effect_color_\{id\} | visual_effect_color_vhs To change appearance o the effect, you should place any android resources named according to the scheme presented above into res folder of your app. The resource depends on the item you want to customize (string for the title, drawable for the icon, color for the color on the timeline). Effects identifiers are presented in the table below: | Effect | Type | String identifier | Default icon | | :---------- | :---:| :--------------: | :----------: | | Acid-whip | visual | acid_whip| | Cathode | visual | cathode | | Flash | visual | flash | | Glitch | visual | glitch | | Glitch 2 | visual | glitch2 | | Glitch 3 | visual | glitch3 | | Heat map | visual | heat_map | | DSLR Kaleidoscope | visual | dslr_kaleidoscope | | Kaleidoscope | visual | kaleidoscope | | Lumiere | visual | lumiere | | Pixel dynamic | visual | pixel_dynamic | | Pixel static | visual | pixel_static | | Polaroid | visual | polaroid | | Rave | visual | rave | | Soul | visual | soul | | Stars | visual | stars | | Transition 1 | visual | transition1 | | Transition 2 | visual | transition2 | | Transition 3 | visual | transition3 | | Transition 4 | visual | transition4 | | TV Foam | visual | tv_foam | | DV Cam | visual | dv_cam | | VHS | visual | vhs | | VHS 2 | visual | vhs2 | | Zoom | visual | zoom | | Zoom 2 | visual | zoom2 | | Slow mo | time | slow_motion | | Rapid | time | rapid | For example, to change the title of ```Flash``` visual effect to ```SuperFlash``` add it in ```strings.xml``` in your app. ```xml SuperFlash ``` You can change the color on the timeline for any visual effect. For example, for ```VHS``` you should add color resource with the name ```visual_effect_color_vhs``` into ```colors.xml``` file in your app. ### I want to change the order of masks, video effects or filters. The SDK allows to reorder masks and filters in the way you need. To achieve this, create a class ```CustomColorFilterOrderProvider``` and implement ```OrderProvider```: ```kotlin class CustomColorFilterOrderProvider : OrderProvider { override fun provide() = listOf( "egypt", "byers", "chile", "hyla", "new_zeland", "korben", "canada", "remy", "england", "retro", "norway", "neon", "japan", "instant", "lux", "sunset", "bubblegum", "chroma", "lilac", "pinkvine", "spark", "sunny", "vinyl", "glitch", "grunge" ) } ``` :::important These are names of specific color filters located in ```assets/bnb-resources/luts```. ::: Next, use ```CustomColorFilterOrderProvider``` in [VideoEditorModule](https://github.com/Banuba/ve-sdk-android-integration-sample/blob/main/app/src/main/java/com/banuba/example/integrationapp/VideoEditorModule.kt#L60) ```kotlin single(named("colorFilterOrderProvider")) { CustomColorFilterOrderProvider() } ``` ### Is it possible to disable the transition effects? Transitions are visual effects applying to the segue between two videos. They are provided with the Banuba Video Editor SDK **by default**. To disable or enable transitions, set the `supportsTransitions` flag in the `EditorConfig` class to false or true respectively: ``` kotlin single { EditorConfig(supportsTransitions = false) } ``` :::important Transition effects are not being played if the closest video (either to the left or to the right of the transition icon) is very short. ::: ### How can I convert one or several still images to a video programmatically? If you would like to create a video from the images instances without opening the Video Editor, you can use the methods from the ```Utils``` object in the [GitHub Sample](https://github.com/Banuba/ve-sdk-android-integration-sample/blob/main/app/src/main/java/com/banuba/example/integrationapp/Utils.kt#L72-L131). ### How to change the style appearance of the SDK? Extend ```VideoCreationTheme``` style to customize Video Editor appearance for your app. ```xml ``` Specify your style in ```AndroidManifest.xml``` file for ```VideoCreationActivity```. ```xml ``` [GitHub example](https://github.com/Banuba/ve-sdk-android-integration-sample/blob/main/app/src/main/res/values/themes.xml) ### Is it possible to disable the Trim screen? It is possible to turn off the trimmer screen after the camera screen. The trimmer screen will still be accessible after importing media files from the gallery. To disable it, just change the `supportsTrimRecordedVideo` property to `false` in the `EditorConfig`: ``` kotlin single { EditorConfig(supportsTrimRecordedVideo = false) } ``` ### How to change the video scaling on the editor screen? Yon can change video scaling on the editor screen while playback by providing ```PlayerScaleType``` [VideoEditorModule](https://github.com/Banuba/ve-sdk-android-integration-sample/blob/main/app/src/main/java/com/banuba/example/integrationapp/VideoEditorModule.kt#L60). To ensure that the video will be fully shown - use ```CENTER_INSIDE``` (keep in mind that if device and video resolutions are different black lines will appear), to fill the screen - use ```FIT_SCREEN_HEIGHT``` (it fills the screen only if video has aspect ratio 9:16) ``` kotlin factory(named("editorVideoScaleType"), override = true) { PlayerScaleType.CENTER_INSIDE } ``` The default value is ```PlayerScaleType.FIT_SCREEN_HEIGHT```. ### How to collect logs when I encounter an issue? If you encounter a crash or other issue and are unsure how to provide logs to the support team, please refer to the official Android documentation on how to capture a bug report: [https://developer.android.com/studio/debug/bug-report](https://developer.android.com/studio/debug/bug-report) After following the instructions, please send the generated bug report file to the [support team](https://www.banuba.com/support) for further investigation. ### FFmpeg build issue Below are the steps to resolve the issue while building the project. 1. Add the ```android.bundle.enableUncompressedNativeLibs=false``` in the ```gradle.properties``` ``` properties android.bundle.enableUncompressedNativeLibs=false ``` 2. Add ```android:extractNativeLibs="true"``` in ```AndroidManifest.xml``` file ``` xml ``` ### How to integrate custom FFmpeg dependency. Check out [step-by-step guide](ffmpeg.md) to integrate custom FFmpeg dependency. --- ## Video Templates on Android Templates let users create stunning videos quickly and easily using predefined sets of effects, transitions, and music. All it takes to make a shareable piece is changing the placeholders. With templates, even people who are new to video editing or just lack time can make impressive content in minutes.       :::important The ```Video Templates``` is not enabled by default. Contact Banuba representatives to know more. ::: ## Launch Video Templates Use a new entry point in `VideoCreationActivity` to launch the Video Editor SDK from Video templates. ```kotlin val intent = VideoCreationActivity.startFromTemplates(Context) ``` --- ## Overview Video Editor SDK on iOS Banuba Video Editor SDK has built in UI/UX experience and provides a number of customizations you can use to meet your requirements. **AVAILABLE:** :white_check_mark: Use your branded icons, colors, and text styles. [See details](ve-faq.md#i-want-to-use-custom-icons) :white_check_mark: Localize and change text resources. Default locale is :us: :white_check_mark: Make content you want i.e. a number of videos with different resolutions and durations, an audio file. [See details](guide_export.md) :white_check_mark: Masks and filters order. [See details](ve-faq.md#i-want-to-change-the-order-of-masks-video-effects-or-filters) **NOT AVAILABLE:** :x: Change layout or order of the screens after the entry point. You can, however, [ask](https://www.banuba.com/support) us to customize the mobile Video Editor UI as a separate contract. --- ## AI Clipping on iOS The [AI Clipping](https://www.banuba.com/ai-sdk) feature automates the video creation process by leveraging the power of artificial intelligence. The neural network transforms raw input into ready-to-post videos with various transitions, effects, and a precise music match. :::important The ```AI Clipping``` is based on the [Banuba Face AR SDK](https://www.banuba.com/facear-sdk/face-filters) product. Since this feature uses additional services, it is disabled by default. Please contact Banuba representatives to know more about using this feature. ::: Here is how users can interact with it: 1. User selects clips and music. People can choose their desired clips and accompanying music within the app. 2. AI trims the videos. The AI algorithm intelligently trims the selected clips to match the tempo and rhythm of the chosen track. 3. AI adds various effects. AI Clipping enhances the content by adding a variety of effects to create visually stunning content. 4. User exports, edits, or regenerates the clip. Upon completion, users have the flexibility to export the video as is, make further edits, or regenerate the clip for a fresh perspective. Regardless of the user’s editing skills, with the help of AI Clipping, everyone can get a stunning video within seconds. ## Supported Music Providers - [Banuba Music](guide_audio_content.md#connect-banuba-music) - [Soundstripe](guide_audio_content.md#connect-soundstripe)   ## Integration ### Setup configuration Add ```Banuba Face AR SDK``` dependency by following the [Face AR instruction](guide_far_arcloud.md#integrate-face-ar). Set up ```embeddingsDownloadUrl``` and ```musicProvider``` values in [VideoEditorModule](https://github.com/Banuba/ve-sdk-ios-integration-sample/blob/main/Example/Example/VideoEditorModule.swift#L73) :::important Contact Banuba representative to get trial keys for ```tracksURL``` and ```musicProvider``` ::: #### Config with Banuba Music provider ```swift let videoEditorConfig = VideoEditorConfig() videoEditorConfig.aiClippingConfiguration.embeddingsDownloadUrl = "https://d27n29bgbvbeer.cloudfront.net/index-staging.zip" videoEditorConfig.aiClippingConfiguration.musicProvider = .banubaMusic(tracksURL: URL(string: "https://d27n29bgbvbeer.cloudfront.net/response.json")!) ``` #### Config with Soundstripe provider ```swift let videoEditorConfig = VideoEditorConfig() videoEditorConfig.aiClippingConfiguration.embeddingsDownloadUrl = "..." videoEditorConfig.aiClippingConfiguration.musicProvider = .soundstripe(tracksURL: URL(string: "...")!) ``` ### Launch AI Clipping :::info Video creation with AI Clipping is available on Gallery screen by default. ::: For better experience we added new entry point ```.aiClipping``` to ```VideoEditorLaunchConfig``` for opening AI Clipping as separate mode. In this scenario the user starts from the gallery screen and is taken to AI Clipping screen after selecting media. ```swift let launchConfiguration = VideoEditorLaunchConfig( entryPoint: .aiClipping, hostController: self, animated: true ) videoEditorSDK.presentVideoEditor( withLaunchConfiguration: launchConfiguration, completion: nil ) ``` --- ## Closed Captions on iOS Closed captions(CC) are a textual representation of the audio within a media file. :::important The Close Captions feature is disabled by default. Please contact Banuba representatives to know more about using this feature. ::: Over 80% of videos played on mobile devices don’t have sound turned on. This means many forms of content (e.g. skits, monologues, educational clips, etc.) will be skipped if there are no subtitles. But making captions by hand is tedious. AI-generated subtitles solve this issue, as they are created and placed automatically. The users can then edit the text as well as change its style and color.       [AWS Transcribe service](https://docs.aws.amazon.com/transcribe/) is used to generate captions. ## Supported languages - Arabic - English - Mandarin - Spanish - Portuguese ## Integration Set up ```captionsUploadUrl```, ```captionsTranscribeUrl``` and ```apiKey``` values in the VideoEditorModule. ### Closed Captions V2 (Recommended) :::important Request API V2 key from Banuba representatives. ::: ```swift config.captionsConfiguration.apiV2Key = "..." ``` ### Closed Captions V1 :::important Request keys from Banuba representatives. ::: ```swift config.captionsConfiguration.captionsUploadUrl = "..." config.captionsConfiguration.captionsTranscribeUrl = "..." config.captionsConfiguration.apiKey = "..." ``` --- ## Installation with CocoaPods Learn the [CocoaPods Getting Started Guide](https://guides.cocoapods.org/using/getting-started.html) if you are new to CocoaPods. :::info It is advised to use the latest version of CocoaPods. Please, check your version with ```pod --version``` and upgrade if necessary. ::: The List of the required Video Editor dependencies is present in the sample project's [Podfile](https://github.com/Banuba/ve-sdk-ios-integration-sample/blob/main/Example/Podfile). Complete the following steps to get the Video Editor SDK dependencies using CocoaPods: 1. install CocoaPods using Homebrew: ```sh brew install cocoapods ``` 2. initialize the pods in your project folder: ```sh pod init ``` 3. add the necessary frameworks and podspec sources to the Podfile of your project: ```ruby source 'https://cdn.cocoapods.org/' source 'https://github.com/Banuba/specs.git' source 'https://github.com/sdk-banuba/banuba-sdk-podspecs.git' banuba_sdk_version = '1.48.2' pod 'BanubaVideoEditorSDK', banuba_sdk_version pod 'BanubaSDKSimple', banuba_sdk_version pod 'BanubaSDK', banuba_sdk_version pod 'BanubaARCloudSDK', banuba_sdk_version # optional pod 'BanubaAudioBrowserSDK', banuba_sdk_version # optional ``` 4. install the Video Editor SDK pods: ```sh pod install --repo-update ``` 5. open ```Example.xcworkspace``` in Xcode and run the project. --- ## Audio content on iOS Guide to integrating and customizing audio providers in Video Editor SDK. ## Overview Audio content is the key part of making an awesome video. The Video Editor SDK can play, trim, merge and add audio content to a video. :::info 1. Banuba does not deliver audio content for the Video Editor SDK. 2. The Video Editor can apply audio files stored on the device. The SDK is not responsible for downloading audio content except for [Soundstripe](https://www.soundstripe.com/) and [Banuba Music](#connect-banuba-music). 3. Video Editor SDK supports only one music provider per launch. ::: There are 2 approaches to using audio content: 1. ```AudioBrowser``` - a specific module and set of screens that include the built-in support of browsing and applying audio content within the Video Editor. The user does not leave the SDK while using audio. 2. ```External API``` - the client implements a specific API for managing audio content. The user leaves the SDK and is taken to an app screen when audio is requested. ## Audio Browser Audio Browser is a specific iOS module that allows to browse, play and apply audio content within the Video Editor. It supports 3 sources for audio content: 1. ```Banuba Music``` - includes build in integration with Banuba Music, 2. ```Soundstripe``` - includes the built-in integration with the [Soundstripe](https://www.soundstripe.com/) API, 3. ```My Library``` - includes audio content available on the user's device. Add the dependency below into your [Podfile](https://github.com/Banuba/ve-sdk-ios-integration-sample/blob/main/Example/Podfile#L16) to integrate ```AudioBrowser```: ```swift pod 'BanubaAudioBrowserSDK', banuba_sdk_version ``` If you are using Swift Package Manager, please, visit the [SPM integration page](spm-installation.md). ## Connect Banuba Music Over 35 GB of royalty-free tracks available from within the Video Editor SDK. Your users could check them out through an inbuilt music browser and legally include them in their content. :::info The feature is not activated by default. Please contact Banuba representatives to know more about using this feature. :::   Use ```BanubaMusicProvider``` implementation in [VideoEditorModule](https://github.com/Banuba/ve-sdk-ios-integration-sample/blob/main/Example/Example/VideoEditorModule.swift#L81): ```Swift AudioBrowserConfig.shared.musicSource = .banubaMusic ``` ## Connect Soundstripe [Soundstripe](https://www.soundstripe.com/) is a service for providing the best audio tracks for creating video content. Your users will be able to add audio tracks while recording or editing video content. :::info The feature is not activated by default. Please, contact Banuba representatives to know more about using this feature. :::     Set the ```.soundstripe``` source in the ```AudioBrowserConfig``` configuration to enable ```Soundstripe```. For example, in [VideoEditorModule](https://github.com/Banuba/ve-sdk-ios-integration-sample/blob/main/Example/Example/VideoEditorModule.swift#L81): ```Swift AudioBrowserConfig.shared.musicSource = .soundstripe ``` ## Connect My Library ```My Library``` is a default implementation in ```AudioBrowser``` . It allows a user to apply audio that is available on the device. Set the ```.localStorageWithMyFiles``` source in the ```AudioBrowserConfig``` configuration to enable ```My Library```. For example, in [VideoEditorModule](https://github.com/Banuba/ve-sdk-ios-integration-sample/blob/main/Example/Example/VideoEditorModule.swift#L100): ```Swift AudioBrowserConfig.shared.musicSource = .localStorageWithMyFiles ``` ## Connect External API The Video Editor includes a special API for integrating your custom audio content provider and applying this content to the Video Editor. The user will be taken to your app's specific screen when audio is requested on the Video Editor screen i.e. the camera or editor. Next, once the user picks audio content on your app screen, you need to follow API and return the user to the Video Editor. Any audio file should be stored on the device before applying. To pass audio content to the Video Editor you need to implement a factory that conforms to the `MusicEditorExternalViewControllerFactory` protocol. And put it to the `musicEditorFactory` property in `ExternalViewControllerFactory`. Your factory should implement the following methods: ```swift protocol MusicEditorExternalViewControllerFactory: AnyObject { /// contoller which will be used for presenting otherwise makeTrackSelectionViewController will be used var audioBrowserController: TrackSelectionViewController? { get set } /// should returns controller which provides audio content for Video Editor SDK /// - selectedAudioItem is currently selected audio item func makeTrackSelectionViewController(selectedAudioItem: AudioItem?) -> TrackSelectionViewController? /// should returns controller which provides effects audio content for Video Editor SDK /// Note: MainMusicViewControllerConfig should contains editButton with type .effect func makeEffectSelectionViewController(selectedAudioItem: AudioItem?) -> EffectSelectionViewController? /// should returns view for countdown animation in record button at music editor func makeRecorderCountdownAnimatableView() -> MusicEditorCountdownAnimatableView? } ``` where `AudioItem` is an entity which includes information about the selected audio item in the Video Editor: ```swift // MARK: - AudioItem protocol protocol AudioItem { var uuid: UUID { get } var url: URL { get } var coverURL: URL? { get } var title: String? { get set } var additionalTitle: String? { get set } /// True - display track with which the video was recorded and allow users to edit it. /// False - track will be playing but not displayed. var isEditable: Bool { get set } } ``` Your custom audio browser implementation should conform to the `TrackSelectionViewController` protocol: ```swift class YourCustomAudioBrowser: UIViewController, TrackSelectionViewController { weak var trackSelectionDelegate: TrackSelectionViewControllerDelegate? } ``` Using `trackSelectionDelegate` you can notify the Video Editor about actions in the audio browser with the following methods: ```swift protocol TrackSelectionViewControllerDelegate: AnyObject { func trackSelectionViewController( viewController: TrackSelectionViewController, didSelectFile url: URL, coverURL: URL?, timeRange: CMTimeRange?, isEditable: Bool, title: String, additionalTitle: String?, uuid: UUID ) func trackSelectionViewControllerDidCancel( viewController: TrackSelectionViewController ) func trackSelectionViewControllerDiscardCurrentTrack( viewController: TrackSelectionViewController ) } ``` ## Connect Apple Music You can pass music from Apple Music to the Video Editor SDK. You should export media to a temporary directory and then pass the music url to the Video Editor SDK using `trackSelectionDelegate`. In this sample, we show how to implement it: ```swift let asset = AVURLAsset(url: url) let destination = FileManager.default .temporaryDirectory .appendingPathComponent("\(NSUUID().uuidString).caf") let exportSession = AVAssetExportSession(asset: asset, presetName: AVAssetExportPresetPassthrough) exportSession?.outputURL = destination exportSession?.outputFileType = AVFileType.caf exportSession?.exportAsynchronously() { if let error = exportSession?.error { completion(nil, error as NSError) } else { completion(destination, nil) } } ``` To present your custom audio browser in the Video Editor you need to set the created `ExternalViewControllerFactory` to [externalViewControllerFactory](https://github.com/Banuba/ve-sdk-ios-integration-sample/blob/main/Example/Example/VideoEditorModule.swift#L29). --- ## Camera screen on iOS Guide to modifying camera UI elements and screen configuration. ## Change icons You can use your app specific icons in Video Editor SDK. Add icon files to Xcode ```Assets``` folder with predefined names.   To add custom icons, use the ```AdditionalEffectsButtonConfiguration``` within ```RecorderConfiguration```, ```TimerConfiguration``` and ```SpeedBarButtonsConfiguration```. Refer to the configuration below. To exclude any button, just don't include it in the ```additionalEffectsButtons``` array. ```swift var config = VideoEditorConfig() config.recorderConfiguration.additionalEffectsButtons = [ AdditionalEffectsButtonConfiguration( identifier: .toggle, imageConfiguration: ImageConfiguration(imageName:"toggle.png"), selectedImageConfiguration: ImageConfiguration(imageName: "toggleSelected.png") ), AdditionalEffectsButtonConfiguration( identifier: .flashlight, imageConfiguration: ImageConfiguration(imageName:"flashlight.png"), selectedImageConfiguration: ImageConfiguration(imageName:"flashlightSelected.png") ), AdditionalEffectsButtonConfiguration( identifier: .speed, imageConfiguration: ImageConfiguration(imageName:"speed.png"), selectedImageConfiguration: ImageConfiguration(imageName: "speedSelected.png") ), AdditionalEffectsButtonConfiguration( identifier: .sound, imageConfiguration: ImageConfiguration(imageName:"sound.png"), selectedImageConfiguration: ImageConfiguration(imageName: "soundSelected.png") ), AdditionalEffectsButtonConfiguration( identifier: .muteSound, imageConfiguration: ImageConfiguration(imageName:"muteSound.png"), selectedImageConfiguration: ImageConfiguration(imageName:"muteSoundSelected.png") ), AdditionalEffectsButtonConfiguration( identifier: .pip, imageConfiguration: ImageConfiguration(imageName:"pip.png"), selectedImageConfiguration: ImageConfiguration(imageName:"pipSelected.png") ), AdditionalEffectsButtonConfiguration( identifier: .beauty, imageConfiguration: ImageConfiguration(imageName:"beauty.png"), selectedImageConfiguration: ImageConfiguration(imageName: "beautySelected.png") ), AdditionalEffectsButtonConfiguration( identifier: .effects, imageConfiguration: ImageConfiguration(imageName:"effects.png"), selectedImageConfiguration: ImageConfiguration(imageName:"effectsSelected.png") ), AdditionalEffectsButtonConfiguration( identifier: .masks, imageConfiguration: ImageConfiguration(imageName:"masks.png"), selectedImageConfiguration: ImageConfiguration(imageName: "masksSelected.png") ), AdditionalEffectsButtonConfiguration( identifier: .timer, imageConfiguration: ImageConfiguration(imageName:"timer.png"), selectedImageConfiguration: ImageConfiguration(imageName: "timerSelected.png") ) ] config.recorderConfiguration.timerConfiguration.defaultButton = ImageButtonConfiguration( imageConfiguration: ImageConfiguration(imageName:"timer.png") ) config.recorderConfiguration.speedBarButtons = SpeedBarButtonsConfiguration( imageHalf: ImageConfiguration(imageName:"imageHalf.png"), imageNormal: ImageConfiguration(imageName:"imageNormal.png"), imageDouble: ImageConfiguration(imageName:"imageDouble.png"), imageTriple: ImageConfiguration(imageName:"imageTriple.png"), selectedTitleColor: UIColor, titleColor: UIColor, backgroundColor: UIColor, cornerRadius: CGFloat ) ``` --- ## Cover image on iOS Cover image screen allows users to pick any frame of video as image or choose an image from gallery. This screen can be turned on or off by changing the `isVideoCoverSelectionEnabled` field of `FeatureConfiguration`: ```swift let config = VideoEditorConfig() config.featureConfiguration.isVideoCoverSelectionEnabled = false ``` To change the UI elements of the screen (icons, colors, etc.), please, modify the default values of `VideoCoverSelectionConfiguration`: ```swift let config = VideoEditorConfig() config.extendedVideoCoverSelectionConfiguration.doneButton.textConfiguration?.color = UIColor.red ``` You can override the following string resources in your app: | Key | Default value | | ------------- | ----------- | | Choose Cover Screen Name | Thumbnail | | cover.thumbnail.title | Choose a thumbnail | | cover.gallery.button.title | Choose from Gallery | | cover.delete.button.title | Delete | | Cancel | Cancel | | Done | Done | --- ## Drafts screen on iOS Guide to integrating and customizing drafts screen on Video Editor SDK. ## Configuration Drafts are enabled by default, asks the user to save a draft before leave any VideoEditor screen. If you need to change drafts configuration you should add the code below in the [VideoEditorModule](https://github.com/Banuba/ve-sdk-ios-integration-sample/blob/main/Example/Example/VideoEditorModule.swift#L83): ```swift var config = VideoEditorConfig() config.featureConfiguration.draftsConfig = .enabled ``` You can choose one of these options: - ```.enabled``` - drafts enabled, asks the user to save a draft - ```.enabledSaveToDraftsByDefault``` - drafts enabled, saved by default without asking the user - ```.enabledAskIfSaveNotExport``` - drafts enabled, asks the user to save a draft without export - ```.disabled``` - disabled drafts The default value is ```.enabled``` ## Draft Service The `DraftsService` class is used for managing drafts: ```swift /// External Draft of video session typealias ExternalDraft = VideoSequence /// Allows you to manage drafts class DraftsService { /// Get drafted video sequences func getDrafts() -> [ExternalDraft] /// Remove specific video sequence /// - parameters: /// - externalDraft: Drafted video sequence func removeExternalDraft(_ externalDraft: ExternalDraft) -> Bool /// Get preview for specific drafted video sequence /// - parameters: /// - externalDraft: Drafted video sequence /// - thumbnailHeight: Preview height /// - completion: Completion when preview UIImage generated func getPreviewForVideoSequence( _ externalDraft: ExternalDraft, thumbnailHeight: CGFloat, completion: ((_ preview: UIImage?) -> Void)? ) } ``` To get the instance of `DraftsService` use the `BanubaVideoEditor` instance. ***Example of usage:*** ```swift let videoEditorSDK = BanubaVideoEditor(...) // Initialization of main entity let drafts = videoEditorSDK?.draftsService.getDrafts() // Get drafts list let draft = drafts.first! // Get draft preview videoEditorSDK?.draftsService.getPreviewForVideoSequence( // Choosen draft from list of drafts draft, // Default config, where config is VideoEditorConfig instance thumbnailHeight: config.videoResolutionConfiguration.currentThumbnailHeight, completion: { preview in // Preview usage } ) // Remove draft and use returned value to your own condition usage if needed let _ = videoEditorSDK?.draftsService.removeVideoSequence(videoSequence) // Open Video Editor with preselected draft let draftedConfig = VideoEditorLaunchConfig.DraftedLaunchConfig( // Choosen draft from list of drafts externalDraft: draft, // Any case from DraftsFeatureConfig entity draftsConfig: .enabled ) let config = VideoEditorLaunchConfig( entryPoint: .editor, hostController: self, draftedLaunchConfig: draftedConfig, animated: true ) // Present Video Editor self.videoEditorSDK?.presentVideoEditor( withLaunchConfiguration: config, completion: nil ) ``` ## Transferring drafts BanubaVideoEditor supports archiving drafts into a zip file. It may help you with backing up user data to the server or sharing video content across the devices. For exporting and importing `BanubaVideoEditor` has the following methods: ```swift /// Archive and compress external draft into zip file. /// Returns URL to archive /// Discussion: After performing all actions with zip file your are responsible to remove archive file. func exportExternalDraft(_ externalDraft: ExternalDraft) throws -> URL /// Unarchive ExternalDraft from zip file func importExternalDraft(fromZipUrl url: URL) throws -> ExternalDraft ``` ***Example of usage:*** ```swift let videoEditorSDK = BanubaVideoEditor(...) // Initialization of main entity let drafts = videoEditorSDK?.draftsService.getDrafts() // Get drafts list let draft = drafts.first! // Make archive from draft let archiveURL = try? videoEditorSDK?.exportExternalDraft(draft) // Import draft from archive let importedDraft = try! videoEditorSDK?.importExternalDraft(fromZipUrl: archiveURL) // Open Video Editor with unarchived draft let draftedConfig = VideoEditorLaunchConfig.DraftedLaunchConfig( externalDraft: importedDraft, draftsConfig: .enabled ) let config = VideoEditorLaunchConfig( entryPoint: .editor, hostController: self, draftedLaunchConfig: draftedConfig, animated: true ) self.videoEditorSDK?.presentVideoEditor( withLaunchConfiguration: config, completion: nil ) ``` --- ## Editor screen on iOS Guide to integrating and customizing Editor screen. ## Change icons You can use your app specific icons in Video Editor SDK. Add icon files to Xcode ```Assets``` folder with predefined names.   To add custom icons, use the ```AdditionalEffectsButtonConfiguration``` within ```EditorConfiguration```. Refer to the configuration below. To exclude any button, just don't include it in the ```additionalEffectsButtons``` array. ```swift var config = VideoEditorConfig() config.editorConfiguration.additionalEffectsButtons = [ AdditionalEffectsButtonConfiguration( identifier: .sticker, imageConfiguration: ImageConfiguration(imageName: "stickers"), selectedImageConfiguration: ImageConfiguration(imageName: "stickersSelected") ), AdditionalEffectsButtonConfiguration( identifier: .text, imageConfiguration: ImageConfiguration(imageName: "text"), selectedImageConfiguration: ImageConfiguration(imageName: "textSelected") ), AdditionalEffectsButtonConfiguration( identifier: .captions, imageConfiguration: ImageConfiguration(imageName: "captions"), selectedImageConfiguration: ImageConfiguration(imageName: "captionsSelected") ), AdditionalEffectsButtonConfiguration( identifier: .effects, imageConfiguration: ImageConfiguration(imageName: "effects"), selectedImageConfiguration: ImageConfiguration(imageName: "effectsSelected") ), AdditionalEffectsButtonConfiguration( identifier: .masks, imageConfiguration: ImageConfiguration(imageName: "masks"), selectedImageConfiguration: ImageConfiguration(imageName: "masksSelected") ), AdditionalEffectsButtonConfiguration( identifier: .sound, imageConfiguration: ImageConfiguration(imageName: "music"), selectedImageConfiguration: ImageConfiguration(imageName: "musicSelected") ), AdditionalEffectsButtonConfiguration( identifier: .time, imageConfiguration: ImageConfiguration(imageName: "time"), selectedImageConfiguration: ImageConfiguration(imageName: "timeSelected") ), AdditionalEffectsButtonConfiguration( identifier: .color, imageConfiguration: ImageConfiguration(imageName: "filters"), selectedImageConfiguration: ImageConfiguration(imageName: "filtersSelected") ), AdditionalEffectsButtonConfiguration( identifier: .blur, imageConfiguration: ImageConfiguration(imageName: "blur"), selectedImageConfiguration: ImageConfiguration(imageName: "blurSelected") ), AdditionalEffectsButtonConfiguration( identifier: .autoCut, imageConfiguration: ImageConfiguration(imageName: "autocut"), selectedImageConfiguration: ImageConfiguration(imageName: "autocutSelected") ) ] ``` --- ## NEW Editor screen on iOS Guide to integrating and customizing New Editor screen. ## Overview With the new interface, better controls, and additional quality of life improvements, making stunning videos is easier and more fun than ever. Design and user experience principles are constantly evolving. To keep up with the latest developments and best practices, our team has completely redesigned the Video Editor SDK to be as convenient and enjoyable as possible.       ## Integration Init the Video Editor with argument ```[.useEditorV2: true]``` to enable ```Editor UI V2```: ```swift let videoEditorSDK = BanubaVideoEditor( token: token, arguments: [.useEditorV2: true], configuration: createConfiguration() ) ``` --- ## Export media on iOS Video Editor SDK allows to export a number of media files i.e. video and audio with various resolutions and other configurations. Video is exported as ```.mp4``` file. :::warning Export is a very heavy computational task that takes time and the user has to wait for it to be done. The execution time depends on: 1. the video duration - the longer the video, the longer the execution time; 2. a number of video sources - the more sources, the longer the execution time; 3. a number of effects and their usage in the video - the more effects and the bigger their usage, the longer the execution time; 4. a number of the exported videos - the more videos and audios you want to export, the longer the execution time; 5. the device hardware - the most powerful devices can execute export much quicker. ::: Export only supports the ` Foreground` mode where the user has to wait on the progress screen until the processing is done. The ` Background` export is not allowed on iOS due to the [iOS Restrictions](https://developer.apple.com/documentation/metal/gpu_devices_and_work_submission/preparing_your_metal_app_to_run_in_the_background) Here is a screen that is shown in the ` Foreground` mode: ## Video codecs The Video Editor supports the following video codec options: 1. ` HEVC` - H265 codec. ` Default` if the device supports this codec; 2. ` AVC_PROFILES` - H264 codec with High Profile. You can change the video codec for export by using the ` ExportVideoConfiguration.useHEVCCodecIfPossible` property. In this example, the H264 is set in [VideoEditorModule](https://github.com/Banuba/ve-sdk-ios-integration-sample/blob/main/Example/Example/VideoEditorModule.swift#L56): ```diff let exportConfiguration = ExportVideoConfiguration( fileURL: destFile, quality: .auto, // highlight-add-next-line + useHEVCCodecIfPossible: false, watermarkConfiguration: watermarkConfiguration ) ``` ## Video quality | 360p(360x640) | 480p(480x854) | QHD540(540x960) | HD(720x1280) | FHD(1080x1920) | QHD(1440x2560) | UHD(2160x3840) | |----|---------------|-----------------|--------------|----------------|----------------|----------------| | 1000 kb/s | 2500 kb/s | 3000 kb/s | 5000 kb/s | 8000 kb/s | 16000 kb/s | 36000 kb/s | The Video Editor has the built-in feature for detecting device performance capabilities and finding optimal video quality params for export. You can provide a custom video resolution for the exported video by using the ` ExportVideoConfiguration.quality` property: ```diff let hdExportConfiguration = ExportVideoConfiguration( fileURL: destFile, // highlight-add-next-line + quality: .videoConfiguration(ExportVideoInfo(resolution: .hd1280x720, useHEVCCodecIfPossible: true)), useHEVCCodecIfPossible: true, watermarkConfiguration: watermarkConfiguration ) ``` ## Export storage The Video Editor export method requires the file destination where the exported video will be stored. First, create the destination file: ```swift let manager = FileManager.default let destFile = manager.temporaryDirectory.appendingPathComponent("tmp.mov") ``` and use it in ` ExportVideoConfiguration`: ```diff let hdExportConfiguration = ExportVideoConfiguration( // highlight-add-next-line + fileURL: destFile, quality: .videoConfiguration(ExportVideoInfo(resolution: .hd1280x720, useHEVCCodecIfPossible: true)), useHEVCCodecIfPossible: true, watermarkConfiguration: watermarkConfiguration ) ``` ## Implement the export flow You can create your own flow for exporting media files for your application. Use ` ExportConfiguration` to create an instance with the configurations that meet your product requirements. Below there is a sample that exports 2 video files: 1. a video with the auto resolution **without** a watermark; 2. a video with the HD resolution **with** a watermark: ```Swift let watermarkConfiguration = WatermarkConfiguration( watermark: ImageConfiguration(imageName: "Common.Banuba.Watermark"), size: CGSize(width: 204, height: 52), sharedOffset: 20, position: .rightBottom ) let autoExportConfiguration = ExportVideoConfiguration( fileURL: destFile1, quality: .auto, useHEVCCodecIfPossible: true, watermarkConfiguration: nil ) let hdExportConfiguration = ExportVideoConfiguration( fileURL: destFile2, quality: .videoConfiguration(ExportVideoInfo(resolution: .hd1280x720, useHEVCCodecIfPossible: true)), useHEVCCodecIfPossible: true, watermarkConfiguration: watermarkConfiguration ) let exportConfig = ExportConfiguration( videoConfigurations: [autoExportConfiguration, hdExportConfiguration], isCoverEnabled: true, gifSettings: nil ) ``` Use the created ` ExportConfiguration` to start the export by using the ` BanubaVideoEditor.export()` method: ```Swift public func export( using configuration: ExportConfiguration, exportProgress: ((TimeInterval) -> Void)?, completion: @escaping ((_ error: Error?, _ exportCoverImages: ExportCoverImages?) -> Void) ) ``` ## Handle the export result The ` BanubaVideoEditor.export()` method allows to start the export and track the result. Provide: 1. ` ExportConfiguration` - where you set up all the required media content you want to make; 2. ` exportProgress` - a callback that gets called when the export progress changes. Values are 0.0-1.0; 3. ` completion` - a callback that gets called when the export is finished with an error or not: ```swift videoEditorSDK?.export( using: exportConfiguration, exportProgress: { progress in DispatchQueue.main.async { // Export is in progress. You can show progress view. Progress is 0.0 - 1.0 ... } }, completion: { [weak self] error, exportCoverImages in DispatchQueue.main.async { // Export finishes. Use 'error' value to detect the state of export. // Hide progress view // You can clear exported session if you do no need it anymore //self?.videoEditorSDK?.clearSessionData() } ``` :::tip If the export is finished successfully, you can use the instance of ` ExportConfiguration` as a result and access the media files and its metadata. ::: ## Add a watermark :::warning A watermark is not added to the exported video by default. ::: You can use your custom watermark for a video. First, create the instance of ` WatermarkConfiguration` to ```swift let watermarkConfiguration = WatermarkConfiguration( watermark: ImageConfiguration(imageName: "Common.Banuba.Watermark"), size: CGSize(width: 204, height: 52), sharedOffset: 20, position: .rightBottom ) ``` where ` position` is used for locating the watermark image in a video: ```swfit public enum WatermarkPosition { case leftTop case leftBottom case rightTop case rightBottom } ``` Next, set ` watermarkConfiguration` to every instance of ` ExportVideoConfiguration` where you want to add a watermark in [VideoEditorModule](https://github.com/Banuba/ve-sdk-ios-integration-sample/blob/main/Example/Example/VideoEditorModule.swift#L57): ```diff let hdExportConfiguration = ExportVideoConfiguration( fileURL: destFile, quality: .hd1280x720, useHEVCCodecIfPossible: true, // highlight-add-next-line + watermarkConfiguration: watermarkConfiguration ) ``` ## Export the GIF preview The Video Editor allows to export a preview of the video as a GIF file. The instance of ` GifSettings` is required in ` ExportConfiguration` to export a preview during the export. You can specify this property in [VideoEditorModule](https://github.com/Banuba/ve-sdk-ios-integration-sample/blob/main/Example/Example/VideoEditorModule.swift#L63): ```diff let exportConfig = ExportConfiguration( videoConfigurations: [exportConfiguration], isCoverEnabled: true, // highlight-add-next-line + gifSettings: GifSettings(duration: 0.3) ) ``` ## Get the audio track of exported video You can get the video's audio track after its successfull export using the `BanubaVideoEditor.exportAudio()` method: ```Swift public func exportAudio( fileUrl: URL, audioSettings: [String: Any] = VESettings.audio, completion: @escaping (Bool, Error?) -> Void ) ``` :::tip Learn about the available options on the [Apple Developer Portal](https://developer.apple.com/documentation/avfoundation/audio_settings) to provide custom audio quality settings. ::: ## Get the information about the music tracks used during creation of exported video You can use the `musicMetadata` property of `BanubaVideoEditor` class to get the array of `MusicEditorTrack` entities containing the information about the corresponding music tracks. ```swift /// Video Editor main entity and entry point. /// Can present and hide root view controller. /// Has default export method. public class BanubaVideoEditor { /// Simple metadata of music composition settings public var musicMetadata: MusicEditorMetadata? { get } ... } ``` ```swift // MARK: - MusicEditorTrack public struct MusicEditorTrack: Codable { ///Track URL public var url: URL ///Track original URL public var originalURL: URL ///Track title public var title: String ///Track id public var id: Int32 /// Track volume public var volume: Float ... } ``` ## Export metadata analytics While exporting media content, the Video Editor generates simple metadata analytics that you can use to analyze what media content your users make. Metadata is a JSON string and can be received once the export is finished successfully: ```swift let metadataJson: String? = videoEditorSDK?.metadata?.analyticsMetadataJSON ``` The JSON sample: ```json { "export_success": true, // defines if the export finished succesffully "aspect_ratio": "original", // aspect ration used in exported video "video_resolutions": ["1080x1920"], // list of video resolutions used in export "camera_effects": [], // list of effects of features used on camera screen while recording video "ppt_effects": { "visual": 2, // num of visual effects i.e. Glitch, VHS used in exported video "speed": 1, // num of speed effects used in exported video "mask": 6, // num of AR masks used in exported video "color": 3, // num of color effects used in exported video "text": 1, // num of text effects used in exported video "sticker": 1, // num of sticker effects used in exported video "blur": 1 // num of blur effects used in exported video }, "sources": { "camera": 0, // num of video sources recorded on camera screen(not PIP) "gallery": 1, // num of video sources selected in the gallery "pip": 0, // num of video recorded with PIP "slideshow": 0, // num of video exported as slideshow "audio": 0 // num of audi tracks }, "export_duration": 12.645, // export processing duration "video_duration": 20.11, // exported video duration "video_count": 1, // num of exported video files "os_version": "11", // OS version "sdk_version": "1.26.6" // VE SDK version } ``` --- ## Face AR and AR Cloud products on iOS [Banuba Face AR SDK](https://www.banuba.com/facear-sdk/face-filters) product is used on camera and editor screens for applying various AR effects while making video content. ## Overview Any Face AR effect is a folder that includes a number of files required for the Face AR SDK to play this effect. :::warning Make sure every effect folder includes the ` preview.png` file. This file is used as a preview for the AR effect. ::: ## Integrate Face AR :::info ```Banuba Face AR SDK``` integration is included in the [Github sample](https://github.com/Banuba/ve-sdk-ios-integration-sample/). ::: Add the dependency below to the [Podfile](https://github.com/Banuba/ve-sdk-ios-integration-sample/blob/main/Example/Podfile#L19) to integrate `Banuba Face AR SDK` into your project: ```ruby pod 'BanubaSDK', '1.48.2' ``` ## Manage effects There are 2 options for managing AR effects: 1. ` bundleEffects` folder - use [bundleEffects](https://github.com/Banuba/ve-sdk-ios-integration-sample/blob/main/Example/Example/bundleEffects) folder, 2. ` AR Cloud` Effects - which are stored on the remote server. :::warning Please, keep the name ` bundleEffects`, otherwise, the app will not start. Create the `bundleEffects` folder if it does not exist. ::: :::tip You can use both options i.e. store just a few AR effects in `bundleEffects` and 100 or more AR effects on `AR Cloud`. ::: ## Integrate AR Cloud `AR Cloud` is a cloud solution for storing the Banuba Face AR effects on the server and is used by the Face AR and Video Editor products. Any AR effect downloaded from `AR Cloud` is cached on the user's device. Add the dependency below to the [Podfile](https://github.com/Banuba/ve-sdk-ios-integration-sample/blob/main/Example/Podfile#L11) to integrate `AR Cloud` into your project: ```ruby pod 'BanubaARCloudSDK', '1.48.2' ``` Since the link to your AR Cloud bucket is included into the license token, AR effects will appear once you set the license token with the AR Cloud link. ## Change the effects order By default, all AR effects are listed in alphabetical order. AR effects from `bundleEffects` are listed in the beginning. Provide your ordered list of effects to `preferredMasksOrder` in `VideoEditorConfig`: ```swift videoEditorConfig.recorderConfiguration.recorderEffectsConfiguration.preferredMasksOrder = [ "XYScanner", "Background" ... ] ``` :::warning These are the names of specific directories located in `bundleEffects` or on `AR Cloud`. ::: ## Disable Face AR SDK The Video Editor SDK can work without the Face AR SDK. Change the [Podfile](https://github.com/Banuba/ve-sdk-ios-integration-sample/blob/main/Example/Podfile) to disable the Face AR SDK: ```diff banuba_sdk_version = '1.48.2' // highlight-remove-start - pod 'BanubaSDK', banuba_sdk_version // highlight-remove-end // highlight-add-next-line + pod 'BanubaSDKSimple', banuba_sdk_version ``` :::tip Please, keep in mind that you can remove all AR effects from [bundleEffects](https://github.com/Banuba/ve-sdk-ios-integration-sample/blob/main/Example/Example/bundleEffects) if your license does not include the Face AR product. ::: --- ## Drawing on iOS Lets your users draw freely on screen. It is a convenient tool for highlighting important objects in the video or spicing up the frame with funny doodles or stylish art. There are 5 line variants to choose from for more self-expression opportunities. :::important The Drawing feature is disabled by default. Please contact Banuba representatives to know more about using this feature. :::       --- ## Gallery screen on iOS Video Editor SDK includes built in gallery functionality where the user can pick any video or image and use it while making video. :::info The Gallery screen is integrated by default. ::: ## Customizations The gallery is used in the app when you want to select a photo or video stored on your phone. Implement ` GalleryConfiguration` and set it to ` VideoEditorConfig.combinedGalleryConfiguration` to customize the gallery screen. To setup the visible tabs for the gallery, just configure it in the `CombinedGalleryConfiguration` entity: ```swift var config = VideoEditorConfig() config.combinedGalleryConfiguration.visibleTabsInGallery = [.video, .photo] ``` ## Implement a custom gallery Please, follow these steps to integrate your gallery into the SDK: ### Step 1 Implement custom `UIViewController` inherited from `GalleryViewController`: ```swift @objc open class GalleryViewController: UIViewController { open weak var delegate: GalleryViewControllerDelegate? open var configuration: GalleryConfiguration? open var selectionBehaviour: GallerySelectionBehaviour? /// Setups new album at gallery open func useAlbum(_ albumModel: AlbumModel) {} /// Cancel current export open func cancelExport() {} /// Retry export failed items open func retryExport() {} } ``` - `delegate`- use this property to notify the Video Editor SDK about the user's actions in the custom gallery. Below there is a list of possible actions: ```swift // MARK: - GalleryViewControllerDelegate @objc public protocol GalleryViewControllerDelegate: AnyObject { /// Tells delegate object about starting asynchronous operations at the gallery. /// BanubaVideoEditorSDK showing full-screen spinner by this event. It can help to prevent unnecessary actions from a user. func galleryViewController(_ controller: GalleryViewController, didStartExportWith progressHandler: ProgressHandler) /// Tells delegate object about finishing asynchronous operations at the gallery func galleryViewController(_ controller: GalleryViewController, didEndExportWith error: Error?, hideProgressViewCompletion: @escaping () -> Void) /// Tells delegate object about the closing gallery. func galleryViewControllerDidClose(_ controller: GalleryViewController) /// Tells delegate object about completion picking gallery items. func galleryViewControllerDone( _ controller: GalleryViewController, withGalleryItems items: [GalleryItem] ) /// Tells delegate object that he should present message. /// In BanubaVideoEditorSDK it presents popup message. func galleryViewController( _ controller: GalleryViewController, presentMessage message: String ) } ``` - `configuration` - contains the UI configuration. All the details you can find [here](#customizations); - `selectionBehaviour` - contains the gallery settings. ```swift /// Setups gallery selection behaviour @objc public class GallerySelectionBehaviour: NSObject { /// Maximum possible selected gallery items quantity which can select a user. public let maximumSelectedCount: Int /// Setups already selected items quantity if gallery open as a picker at trimmer sceen or other cases. /// Use this field to control maximum selection items. public let selectedItemsCount: Int? /// Setups picker mode if isMultiselectModeEnabled is false. /// Otherwise, multiselection mode enabled. public let isMultiselectModeEnabled: Bool /// Setups gallery video duration fetched from user gallery supported by BanubaVideoEditorSDK. /// By default is 3.0 public let minimumGalleryVideoDuration: TimeInterval /// Setups allowed media types which user can select in gallery public let allowedMediaTypes: [GalleryMediaType] } @objc public enum GalleryMediaType: Int, CaseIterable { case video case photo } ``` `GalleryItem` is a protocol to which your items should conform in order to pass the gallery selection result to `BanubaVideoEditorSDK`: ```swift @objc public protocol GalleryItem: NSObjectProtocol { /// Video representation url asset var urlAsset: AVURLAsset? { get } /// Preview for gallery item var preview: UIImage? { get set } /// GalleryItem duration var duration: TimeInterval { get } /// Type can be video, photo or unknown var type: GalleryItemType { get } /// Requests preview for displaying in gallery list func requestPreview( size: CGSize, handler: @escaping (UIImage?) -> Void ) /// Requests photo with desired size func requestPhoto( size: CGSize, progressHandler: ((Double) -> (Bool))?, handler: @escaping (UIImage?, Error?) -> Void ) /// Requests video url asset func requestAVURLAsset( progressHandler:((Double) -> (Bool))?, handler: @escaping (AVURLAsset?, Error?) -> Void ) /// Requests video player item func requestAVPlayerItem( progressHandler: ((Double) -> (Bool))?, handler: @escaping (AVPlayerItem?, Error?) -> Void ) } ``` ### Step 2 Implement custom `UIViewController` inherited from `AlbumsViewController`: ```swift @objc open class AlbumsViewController: UIViewController { open weak var delegate: AlbumsViewControllerDelegate? open var configuration: AlbumsConfiguration? open var selectedAlbum: AlbumModel? } ``` - `delegate` - notifies `BanubaVideoEditorSDK` about actions that happen in Albums. These are the possible actions: ```swift @objc public protocol AlbumsViewControllerDelegate: AnyObject { /// Tells delegate object about selection the new album func albumsViewController(_ controller: AlbumsViewController, didSelect album: AlbumModel) // Tells delegate object about close action func albumsViewControllerDidClose(_ controller: AlbumsViewController) } ``` - `configuration` - a simple configuration with the `TextButtonConfiguration` and `BackButtonConfiguration` configurations; - `selectedAlbum` - an entity which contains information about the currently selected album: ```swift @objc public protocol AlbumModel { /// Album name var name: String? { get set } /// Album preview var preview: UIImage? { get set } /// Assosiated asset collection with album var assetCollection: PHAssetCollection { get } } ``` ### Step 3 Provide your custom gallery to `BanubaVideoEditorSDK`. Please, follow these steps: - create your own viewControllerFactory that conforms to `GalleryViewControllerFactory`: ```swift @objc public protocol GalleryViewControllerFactory: NSObjectProtocol { /// Creates GalleryViewController func makeGalleryViewController( withConfiguration configuration: GalleryConfiguration, selectionBehaviour: GallerySelectionBehaviour ) -> GalleryViewController } class MyGalleryViewControllerFactory: NSObject, GalleryViewControllerFactory { func makeGalleryViewController( withConfiguration configuration: BanubaUtilities.GalleryConfiguration, albumsConfiguration: BanubaUtilities.AlbumsConfiguration, selectionBehaviour: BanubaUtilities.GallerySelectionBehaviour ) -> BanubaUtilities.GalleryViewController { let controller = UIStoryboard( name: String(describing: UIViewController.self), bundle: Bundle(for: UIViewController.self) ).instantiateInitialViewController() as! UIViewController return controller } } ``` - paste your factory to the `BanubaVideoEditor` init: ```swift /// Example video editor view controller factory class ViewControllerFactory: ExternalViewControllerFactory { var musicEditorFactory: MusicEditorExternalViewControllerFactory? var countdownTimerViewFactory: CountdownTimerViewFactory? var exposureViewFactory: AnimatableViewFactory? //MARK: - ExternalViewControllerFactory protocols variale var galleryViewControllerFactory: GalleryViewControllerFactory? } ... let viewControllerFactory = ViewControllerFactory() // Paste your custom factory to externalViewControllerFactory viewControllerFactory.galleryViewControllerFactory = MyGalleryViewControllersFactory() videoEditorSDK = BanubaVideoEditor( token: token, configuration: config, externalViewControllerFactory: viewControllerFactory ) ``` --- ## Green Screen on iOS Video Editor SDK brings you real-time background subtraction technology for you to empower your users with the best quality virtual background service. Users can automatically remove, change or augment backgrounds. :::important The feature requires Face AR product with [Background Subtraction](https://www.banuba.com/technology/background-subtraction) feature. Please contact Banuba representatives to know more about using this feature. :::   ```Green Screen``` feature doesn't require physical green screens, the neural networks do all the work. --- ## Open Photo Editor SDK from Camera screen on iOS This guide demonstrates how to open [Photo Editor SDK](https://www.banuba.com/photo-editor-sdk) just after taking a photo on Video Editor Camera screen. All recorded video or taken images on Camera screen can be handled using ```videoEditor``` of protocol ```BanubaVideoEditorDelegate```. Provide custom implementation of ```videoEditor``` method to override flow. ``` diff class ViewController: UIViewController, BanubaVideoEditorDelegate, BanubaPhotoEditorDelegate { ... // highlight-add-next-line func videoEditor(_ videoEditor: BanubaVideoEditor, shouldProcessMediaUrls urls: [URL]) -> Bool { // Filter media resources to find the target image for Photo Editor SDK guard let jpegURL = urls.first(where: { $0.pathExtension.lowercased() == "jpeg" }), let imageData = try? Data(contentsOf: jpegURL), !imageData.isEmpty, let resultImage = UIImage(data: imageData) else { return true } // Close Video Editor SDK videoEditor.dismissVideoEditor(animated: true) { DispatchQueue.main.async { [weak self] in guard let self else { return } // Calling clearSessionData() also removes any files stored in urls array videoEditorModule?.videoEditorSDK?.clearSessionData() // Use this launch config to open Photo Editor SDK let launchConfig = PhotoEditorLaunchConfig( hostController: self, entryPoint: .editorWithImage(resultImage) ) } } return false } } ``` --- ## Share video screen on iOS Share video screen allows users to easily share an exported video using popular social media services and OS specific components. :::info This is an optional screen that you can add to your video editing flow. ::: It can be configured by `SharingScreenConfiguration` which is provided to the `presentSharingViewController` function that shows the screen. `SharingScreenConfiguration` has two key properties: * `sharingModels` - describes what kind of sharing services are available on the sharing screen; * `facebookId` - a required option for Facebook/Instagram reels and stories: ```swift private func showSharingScreen(videoUrl: URL, exportCoverImages: ExportCoverImages?) { let config = SharingScreenConfiguration( sharingModels: [ SharingServiceModel( sharingType: .facebookReels, sharingTitle: "Facebook\nReels", sharingImage: UIImage(named: "FbReels") ?? UIImage() ), SharingServiceModel( sharingType: .facebookStories, sharingTitle: "Facebook\nStories", sharingImage: UIImage(named: "FbStories") ?? UIImage() ), SharingServiceModel( sharingType: .instagramStories, sharingTitle: "Instagram\nStories", sharingImage: UIImage(named: "InstStories") ?? UIImage() ), SharingServiceModel( sharingType: .other, sharingTitle: "Other", sharingImage: UIImage(named: "Share") ?? UIImage() ) ], videoImageViewCornerRadius: 15.0, sharingVideoTextConfiguration: TextConfiguration( font: UIFont.boldSystemFont(ofSize: 22.0), color: .white, text: "Share video" ), backgroundConfiguration: BackgroundConfiguration( cornerRadius: 15.0, color: #colorLiteral(red: 0.1215686275, green: 0.1294117647, blue: 0.1411764706, alpha: 1) ), sharingCellConfiguration: SharingCellConfiguration( titleTextConfiguration: TextConfiguration( font: UIFont.systemFont(ofSize: 10.0), color: #colorLiteral(red: 0.4756349325, green: 0.4756467342, blue: 0.4756404161, alpha: 1) ) ), closeButtonConfiguration: RoundedButtonConfiguration( textConfiguration: TextConfiguration( font: UIFont.systemFont(ofSize: 12.0), color: .white, text: "CLOSE" ), cornerRadius: 20.0, backgroundColor: .clear, borderWidth: 1.0, borderColor: #colorLiteral(red: 0.4756349325, green: 0.4756467342, blue: 0.4756404161, alpha: 1) ), facebookId: "YOUR FACEBOOK APP ID" ) BanubaVideoEditor.presentSharingViewController( from: self, configuration: config, mainVideoUrl: videoUrl, videoUrls: [videoUrl], previewImage: exportCoverImages?.coverImage ?? UIImage(), animated: true, completion: nil ) } ``` --- ## Stickers on iOS Guide to using stickers in Video Editor SDK on Android. ## Giphy Video Editor SDK has built in integration with [Giphy service](https://developers.giphy.com/docs/api/) for loading stickers. ```GIPHY``` doesn't charge for their content. The one thing they do require is attribution. Also, there is no commercial aspect to the current version of the product (no advertisements, etc.). Any sticker effect is a GIF file. To use stickers in your project you need to request personal [Giphy API key](https://support.giphy.com/hc/en-us/articles/360020283431-Request-A-GIPHY-API-Key). Set your personal Giphy Api Key into the `giphyAPIKey` parameter of the `GifPickerConfiguration` entity in the [VideoEditorModule](https://github.com/Banuba/ve-sdk-ios-integration-sample/blob/main/Example/Example/VideoEditorModule.swift#L83): ```swift var config = VideoEditorConfig() config.gifPickerConfiguration.giphyAPIKey = "YOUR GIPHY KEY" ``` Stickers will appear in the container on editor screen once ```giphyAPIKey``` is set. The placeholder of the search bar is specified by the `com.banuba.searchGif.placeholder` key in the `Localizable.strings` file. --- ## Video recording integration guide on iOS Guide to modifying video recording feature in Video Editor SDK. ## Quality details The subsequent table describes video quality details used for video recording in various resolutions with h264 codec. If h265 (HEVC) codec is used for recording a video, the values listed below will be reduced by approximately 34%: | Recording speed | 360p(360 x 640) | 480p(480 x 854) | 540p(540 x 960) | HD(720 x 1280) | FHD(1080 x 1920) | QHD(2560 x 1440) | 4K(3860 x 2160) | | -------------------------- | --------------- | --------------- | --------------- | --------------- | ---------------- | ---------------- | --------------- | | 0.5x, 1x (Default), 2x, 3x | 1.2 Mbits | 2 Mbits | 2.4 Mbits | 3.6 Mbits | 5.8 Mbits | 16 Mbits | 36 Mbits | ## Implement configurations `VideoEditorConfig` is the core class used for customizing all features in the Video Editor SDK. The class includes many internal config classes that are very useful if you want to create your custom experience. `RecorderConfiguration` is the main configuration class in `VideoEditorConfig` and is used for the video recording functionality: | Property | Values | Description | | ------------- |:-----------------------------------:| :------------- | | videoResolution | VideoResolutionConfiguration | defines the resolution configuration for a video recording | loopAudioWhileRecording | Bool; Default `true` | defines if the audio used in the video recording should be looped | isDynamicMusicTitle | Bool; Default `false` | defines if the music title on the screen changes when a new track is applied | isDefaultFrontCamera | Bool; Default `false` | set `true` if you want to open camera in the front mode | useHEVCCodecIfPossible | Bool; Default `true` | enables H265 codec for recording if it is available on the device | isPhotoSequenceAnimationEnabled | Bool; Default `false` | should use animation for photo sequences | isAudioRateEqualsVideoSpeed | Bool; Default `false` | the video speed selected by the user is applied to audio | isGalleryButtonHidden | Bool; Default `false` | defines if the gallery button located in the bottom-right should be hidden. `true` will hide the button | supportMultiRecords | Bool; Default `true` | defines if the user can record multiple video files | takeAudioDurationAsMaximum | Bool; Default `false` | limits the maximum length of a video recording to match the duration of the audio used in the recording The next very handy config class is `VideoEditorDurationConfig` that is responsible for customizing video recording durations: :::warning All values are in seconds. ::: | Property | Values | Description | | ------------- |:-----------------------------------------------------------:| :------------- | | maximumVideoDuration | TimeInterval > 0; Default `120.0` | the maximum allowed video duration | videoDurations | [TimeInterval] not empty; Default `[60.0, 30.0, 15.0]` | the array of durations allowed for the video recording. The user sees a certain button and can change the duration by tapping. For example, `60.0` means that the user can record multiple video sources with the total duration no more than `60.0` seconds. | minimumDurationFromCamera | TimeInterval > 0; Default `3.0` | the minimum allowed video duration required to proceed and open the next screen | minimumDurationFromGallery | TimeInterval > 0; Default `0.3` | the minimum allowed video duration displayed on the gallery screen | minimumVideoDuration | TimeInterval > 0; Default `1.0` | the minimum allowed video source duration that can be recorded on the camera screen | minimumTrimmedPartDuration | TimeInterval > 0; Default `0.3` | the minimum allowed video source duration to trim | slideshowDuration | TimeInterval > 0; Default `0.3` | the slideshow video duration produced by an image In this example, the maximum video recording duration is set to 30 seconds: ```swift var config = VideoEditorConfig() config.videoDurationConfiguration.maximumVideoDuration = 30.0 ``` And `FeatureConfiguration` that helps to customize the use of some specific features on the camera screen where the video recording happens: | Property | Values | Description | | ------------- |:-------------------------:| :------------- | | isDoubleTapForToggleCameraEnabled | Bool; Default `false` | enables switching between the front and the back camera facing by double tapping on the camera screen | isMuteCameraAudioEnabled | Bool; Default `false` | indicates if the "mute microphone" button is visible on the camera screen | isSpeedBarEnabled | Bool; Default `true` | enables the speed selection bar. If the bar is disabled, the speed recording will be interactively switched by tapping | openAutomaticallyPIPSettingsDropdown | Bool; Default `false` | if this property is enabled, the PiP settings drop down view will be presented after opening the camera screen ## Configure the microphone state The `RecorderConfiguration` class includes the `muteMicrophoneForPIP` property you can use to mute the sound in the PIP mode. The default value is `true`: ```swift var config = VideoEditorConfig() config.recorderConfiguration.muteMicrophoneForPIP = false ``` ## Configure recording modes The recording includes 3 modes for recording content implemented in `captureButtonModes` in the `RecorderConfiguration` class: - `Photo`, - `Video`, - `Photo` and `Video` - **default**. In this example, the recording mode is set to `Video` only: ```swift var config = VideoEditorConfig() config.recorderConfiguration.captureButtonModes = [.video] ``` ## Configure the record button appearance The record button is the main UI control on the camera screen which you can fully customize along with the animation that is played by tapping. Implement the `RecordButtonProvider`, `RecordButton`, `RecordButtonDelegate` protocols to create your custom recording button experience: ```swift public protocol RecordButtonProvider { func getButton() -> RecordButton } public protocol RecordButton: UIView { var delegate: RecordButtonDelegate? { get set } var configuration: RecordButtonConfiguration? { get set } func changeViewToIdleState() func changeViewToRecordingState() } public protocol RecordButtonDelegate: AnyObject { var captureButtonMode: CaptureButtonViewMode { get } func recordButtonDidTakePhoto(_ recordButton: RecordButton) func recordButtonDidCancelTakePhoto(_ recordButton: RecordButton) func recordButtonDidStartVideoRecording(_ recordButton: RecordButton) func recordButtonDidStopVideoRecording(_ recordButton: RecordButton) func recordButtonDidZoomingVideoRecording(_ recordButton: RecordButton, recognizer: UILongPressGestureRecognizer) } ``` Set the new implementation of `RecordButtonProvider` to `RecorderConfiguration.recordButtonProvider`: ```swift var config = VideoEditorConfig() config.recorderConfiguration.recordButtonProvider = ... ``` ## Picture in Picture Picture in Picture, or `PIP`, is a video editing technique that lets you overlay two videos in the same video. The multi-layer editing effect is perfect for reaction videos, slideshows, product demos, and more. This feature is similar to the TikTok duet feature.   :::warning The feature is disabled by default and can be enabled if the license supports it. Please, ask Banuba business representatives to include this feature into your license. ::: The subsequent guide explains how to start and customize `PIP`. First, create `VideoEditorLaunchConfig` in [ViewController](https://github.com/Banuba/ve-sdk-ios-integration-sample/blob/main/Example/Example/ViewController.swift#L65) and provide video content for the feature: ```swift let launchConfig = VideoEditorLaunchConfig( entryPoint: entryPoint, hostController: self, videoItems: resultUrls, // highlight-add-next-line pipVideoItem: resultUrls[.zero], animated: true ) self.presentVideoEditor(with: launchConfig) ``` `PIP` supports 4 modes that you can use: - `Floating`, - `TopBottom`, - `React`, - `LeftRight`. Use `PIPSettingsConfiguration` to customize the PIP implementation: | Property | Values | Description | | ------------- |:-------------------------:| :------------- | | backgroundConfiguration | BackgroundConfiguration | BackgroundConfiguration sets up the background view style | dragIndicatorConfiguration | RoundedButtonConfiguration | the cursor color | titleConfiguration | TextConfiguration | the title font for the controls | layoutSettingsButtonsConfiguration | [PIPSelectableCellConfiguration] | the array of the pip cell configurations In this example, 4 supported PIP modes are set: ```swift var config = VideoEditorConfig() config.pipSettingsConfiguration?.layoutSettingsButtonsConfiguration = [ PIPSelectableCellConfiguration(identifier: .floating), PIPSelectableCellConfiguration(identifier: .react), PIPSelectableCellConfiguration(identifier: .topBottom), PIPSelectableCellConfiguration(identifier: .leftRight) ] ``` You can also change the position of the music button. Use `additionalEffectsButtons` and provide custom `AdditionalEffectsButtonConfiguration` with the `.sound` identifier: ```swift let config = VideoEditorConfig() config.recorderConfiguration.additionalEffectsButtons = [ AdditionalEffectsButtonConfiguration( identifier: .sound, imageConfiguration: ImageConfiguration(imageName: ""), selectedImageConfiguration: ImageConfiguration(imageName: ""), titlePosition: .bottom, position: .top ), ... ] ``` The Video Editor supports 3 options for positioning the music button: `bottom`, `center`, `top`. --- ## Weatherman on iOS Weatherman mode is an improvement on top of Banuba's cutting-edge background replacement. It allows users to drag and drop themselves on any place on the screen to emulate the look of a TV presenter the feature is named after. Weatherman is useful for reactions, learning content, presentations, explainers and other similar videos thanks to the professional feel it creates. :::important The Weatherman feature is disabled by default. Please contact Banuba representatives to know more about using this feature. :::     --- ## Launching Photo Editor on iOS Guide to launching Photo Editor SDK. ## Launch Create an instance of `BanubaPhotoEditor` using the license token: ```swift var configuration = PhotoEditorConfig() let photoEditorSDK = BanubaPhotoEditor( token: token, configuration: configuration ) photoEditorSDK?.delegate = delegate photoEditorSDK?.presentPhotoEditor( withLaunchConfiguration: launchConfig, completion: nil ) ``` `photoEditorSDK` is `nil` when the license token is incorrect i.e. empty, truncated. If `photoEditorSDK` is not `nil`, you can proceed and start the Video Editor. Next, we strongly recommend checking your license state before starting the Photo Editor: ```swift photoEditorSDK?.getLicenseState(completion: { [weak self] isValid in if isValid { print("βœ… License is active, all good") } else { print("❌ License is either revoked or expired") } ... completion(isValid) }) ``` :::danger The "Photo editing is unavailable" screen will appear if you start the Photo Editor SDK with a revoked or expired license. ::: :::warning If you have also integrated the Video Editor SDK into your app, make sure to deallocate any instances of `BanubaVideoEditor` that remain in memory before presenting the Photo Editor in order to prevent crashes: ::: ```swift videoEditorSDK = nil ... photoEditorSDK?.presentPhotoEditor( withLaunchConfiguration: launchConfig, completion: nil ) ``` ## Photo Editor Lifecycle Events To receive the notifications when a user has either closed the editor or finished editing and saved the edited photo, create a class that conforms to the `BanubaPhotoEditorDelegate` protocol and assign its reference to the `BanubaPhotoEditor.delegate` property: ```swift photoEditorSDK?.delegate = delegate ``` During handling of both notifications you must dismiss the Photo Editor: ```swift /// User closed photo editor without editing image func photoEditorDidCancel( _ photoEditor: BanubaPhotoEditorSDK.BanubaPhotoEditor ) { photoEditor.dismissPhotoEditor(animated: true, completion: nil) } /// User closed photo editor after saving image func photoEditorDidFinishWithImage( _ photoEditor: BanubaPhotoEditorSDK.BanubaPhotoEditor, image: UIImage ) { photoEditor.dismissPhotoEditor(animated: true, completion: nil) // Do something with received image } ``` --- ## Launching from SwiftUI Guide to launch Video Editor from SiftUI. ## Configuration Custom behavior of the Video Editor SDK in your app is implemented using a number of configuration classes. The `VideoEditorConfig` is the main entity used for the Video Editor configuration: ```swift func createConfiguration() -> VideoEditorConfig { var config = VideoEditorConfig() ... return config } ``` Check out the [demo project](https://github.com/Banuba/ve-sdk-ios-integration-sample/blob/main/Example/Example/VideoEditorModule.swift#L75) for an example of creating `VideoEditorConfig`. The following configuration starts the Video Editor from the camera screen: ```swift func createLaunchConfiguration() -> VideoEditorLaunchConfig { let launchConfig = VideoEditorLaunchConfig( entryPoint: .camera, hostController: UIViewController(), musicTrack: nil, // Paste a music track as a track preset at the camera screen to record video with music. animated: true ) return launchConfig } ``` The Video Editor supports multiple launch methods described in [this guide](ve-faq.md#what-are-the-available-entry-points-of-the-video-editor). The Video Editor can export multiple media files to meet your requirements. Create an instance of `ExportConfiguration` and provide `Array` where every `ExportVideoConfiguration` is a media file i.e. a video or audio: ```swift func createExportConfiguration() -> ExportConfiguration { let url = FileManager .default .temporaryDirectory .appendingPathComponent("video.mov") if FileManager.default.fileExists(atPath: url.path) { try? FileManager.default.removeItem(at: url) } let exportVideoConfiguration: ExportVideoConfiguration = ExportVideoConfiguration( fileURL: url, quality: .auto, useHEVCCodecIfPossible: true, watermarkConfiguration: nil ) let exportConfiguration = ExportConfiguration( videoConfigurations: [exportVideoConfiguration], isCoverEnabled: true, gifSettings: nil ) return exportConfiguration } ``` Please, check out the [export implementation](https://github.com/Banuba/ve-sdk-ios-integration-sample/blob/main/Example/Example/ViewController.swift#L243) in the sample. Learn the [Export integration guide](guide_export.md) to know more about exporting media content features. ## Launch Use `BanubaVideoEditorSwiftUIView` with the license token to present the Video Editor SDK: ```swift @State private var isPresentingVideoEditor = false .fullScreenCover(isPresented: $isPresentingVideoEditor) { BanubaVideoEditorSwiftUIView( token: "", launchConfig: createLaunchConfiguration(), configuration: createConfiguration(), exportConfiguration: createExportConfiguration() ) .onDidCancel { // The user tapped on the cancel button. Dismissing the video editor. isPresentingVideoEditor = false } .onDidSave { videoURLs in // The video export is succeed. Dismissing the video editor. isPresentingVideoEditor = false } .onUpdateProgress { progress in // Current export progress. } .onDidFail { error in // Dismissing the video editor. isPresentingVideoEditor = false switch error { case .exportError(let description): // There was an error exporting the video. case .initError: // There was an error with init of Video Editor SDK. case .licenseError: // There was an error with license. @unknown default: break } } .ignoresSafeArea() } ``` --- ## Launching from UIKit Guide to launch Video Editor from UIKit. ## Configuration The custom behavior of the Video Editor SDK in your app is implemented using a number of configuration classes. `VideoEditorConfig` is the main entity used for the Video Editor configuration: ```swift class VideoEditorModule { func createConfiguration() -> VideoEditorConfig { var config = VideoEditorConfig() ... return config } } ``` Check out the [demo project](https://github.com/Banuba/ve-sdk-ios-integration-sample/blob/main/Example/Example/VideoEditorModule.swift#L75) for an example of creating `VideoEditorConfig`. ## Launch Create an instance of `BanubaVideoEditor` with the license token: ```swift let videoEditorSDK = BanubaVideoEditor( token: AppDelegate.licenseToken, configuration: config, externalViewControllerFactory: viewControllerFactory ) ``` `videoEditorSDK` is `nil` when the license token is incorrect i.e. empty, truncated. If `videoEditorSDK` is not `nil`, you can proceed and start the Video Editor. Next, we strongly recommend checking your license state before starting the Video Editor: ```swift videoEditorSDK?.getLicenseState( completion: { [weak self] isValid in if isValid { print("βœ… License is active, all good") } else { print("❌ License is either revoked or expired") } ... completion(isValid) } ) ``` :::danger The "Video content unavailable" screen will appear if the user starts the Video Editor SDK with a revoked or expired license: ::: The following [implementation](https://github.com/Banuba/ve-sdk-ios-integration-sample/blob/main/Example/Example/ViewController.swift#L48) starts the Video Editor from the camera screen: ```swift let cameraLaunchConfig = VideoEditorLaunchConfig( entryPoint: .camera, hostController: self, musicTrack: nil, // Paste a music track as a track preset at the camera screen to record video with music animated: true ) videoEditorModule.presentVideoEditor(with: cameraLaunchConfig) ``` The Video Editor supports multiple launch methods described in [this guide](ve-faq.md#what-are-the-available-entry-points-of-the-video-editor). ## Implementing export The Video Editor can export multiple media files to meet your requirements. Create an instance of `ExportConfiguration` and provide `Array` where every `ExportVideoConfiguration` is a media file i.e. video or audio. Next, use `BanubaVideoEditor.export()` method and pass the instance of `ExportConfiguration` to start the export. Please, check out the [export implementation](https://github.com/Banuba/ve-sdk-ios-integration-sample/blob/main/Example/Example/ViewController.swift#L243) in the sample. Learn the [Export integration guide](guide_export.md) to know more about exporting media content features. --- ## LLM and Vibe Coding on iOS If you use large language models for programming, feel free to use our LLM-ready documentation. Enhance your AI-assisted coding workflow by ingesting the Video Editor SDKs documentation. Provide the ```llms.txt``` and ```llms-full.txt``` files to your LLM of choice (e.g., OpenAI, Claude, or Gemini). Once uploaded, you can improve your integration experience query the model for technical details, code examples, and integration best practices, effectively creating a customized support assistant to guide you through the implementation process drastically reducing time to integrate. - [llms.txt](../../llms.txt) - [llms-full.txt](../../llms-full.txt) This is how they differ from the regular docs: - Plain text, no HTML, CSS or JavaScript - Markdown formatting - Structured in a way to be easily digested by LLMs --- ## Installation with CocoaPods(Ios) Learn the [CocoaPods Getting Started Guide](https://guides.cocoapods.org/using/getting-started.html) if you are new to CocoaPods. :::info It is advised to use the latest version of CocoaPods. Please, check your version with ```pod --version``` and upgrade it if necessary. ::: Complete the following steps to get the Photo Editor SDK dependencies using CocoaPods: 1. install CocoaPods using Homebrew: ```sh brew install cocoapods ``` 2. create a new Podfile in your project folder or edit the existing one: ```sh pod init ``` 3. add the necessary pods and our podspecs sources to the Podfile. The list of the required Photo Editor dependencies can be found in the [Podfile](https://github.com/Banuba/ve-sdk-ios-integration-sample/blob/main/Example/Podfile) example: ```ruby source 'https://cdn.cocoapods.org/' source 'https://github.com/Banuba/specs.git' source 'https://github.com/sdk-banuba/banuba-sdk-podspecs.git' pod 'BanubaPhotoEditorSDK', '1.2.9' ``` 4. install the Video Editor SDK pods: ```sh pod install --repo-update ``` 4. open the ```***.xcworkspace``` workspace in Xcode and run the project. --- ## Configure Photo Editor on iOS Guide to modifying Photo Editor SDK. ## Initial screen (entry point) selection By default the built-in gallery will be shown first after the presentation of the Photo Editor. In some cases the hosting app may already provide the photo selection functionality using its own custom gallery and it is necessary to directly open the photo editing screen with a preselected photo. To support such use cases there is the `entryPoint` property in `PhotoEditorLaunchConfig`. By using the `editorWithImage` or `editorWithURL` options you can specify the edited image by directly providing either the `UIImage` instance or the *local* `URL` to the on-disk binary image representation. In the following example the image stored in the app's bundle will be opened in the Photo Editor SDK: ```swift let url = Bundle.main.url(forResource: "image", withExtension: "jpg")! let launchConfig = PhotoEditorLaunchConfig( hostController: ..., entryPoint: .editorWithURL(url) ) photoEditorSDK.presentPhotoEditor( withLaunchConfiguration: launchConfig, completion: nil ) ``` Should there be any problem with decoding the image at the provided url, an error message will be logged into the console by the SDK. ## Configuration of the sharing screen After finishing editing an image, a user is navigated to the Sharing screen. If Facebook or Instagram is installed on the user's phone, it will be possible to present an option to share the image as a Facebook or Instagram Story. To enable such a functionality you need to specify the Facebook app id: ```swift var configuration = PhotoEditorConfig() configuration.previewScreenMode = .enabled( previewScreenConfiguration: .init( shareButtonsMode: .enabled( shareButtonsConfiguration: .init(facebookId: "1234567890") ) ) ) let photoEditorSDK = BanubaPhotoEditor( token: "***", configuration: configuration ) ``` Read more about sharing to Instagram and Facebook stories at https://developers.facebook.com/docs/instagram/sharing-to-stories/. The sharing screen can be disabled entirely. In such case a user will be redirected back to your hosting app after tapping the "Save" button at image editing screen: ```swift var configuration = PhotoEditorConfig() configuration.previewScreenMode = .disabled ``` Alternatively, you can also hide the social networks sharing buttons while keeping the sharing screen: ```swift var configuration = PhotoEditorConfig() configuration.previewScreenMode = .enabled( previewScreenConfiguration: .init(shareButtonsMode: .disabled) ) ``` ## Configuration of the image editing screen To prevent the automatic saving of final edited image to the user's photo library set `false` to `saveResultToPhotoLibrary` in `EditorScreenConfiguration`: ```swift var configuration = PhotoEditorConfig() configuration.editorScreenConfiguration = .init(saveResultToPhotoLibrary: false) ``` ## Add localization strings The Photo Editor SDK displays a number of strings that can be freely changed in the Localizable.strings file of your app. Please, make sure that you've included all the strings with the `photoEditor` key prefix in your app's Localizable.strings file from the [Localized Strings](https://github.com/Banuba/ve-sdk-ios-integration-sample/blob/main/Example/Example/en.lproj/Localizable.strings) file provided with the sample project to prevent placeholders from showing up in the photo editor's UI. If you are still seeing placeholders, make sure that the App Language in the Xcode Scheme Options is set to English as this is the only language supported by this demo app. --- ## Banuba Photo Editor SDK Requirements on iOS Guide contains general requirements for using Photo Editor SDK on iOS. ## License Token Before you commit to a license, you are free to test all the features of the SDK for free. The trial period lasts 14 days. Send us a message to start the [Photo Editor SDK trial](https://www.banuba.com/photo-editor-sdk#form). We will get back to you with the trial token. Feel free to contact us if you have any questions regarding the [Photo Editor SDK](https://www.banuba.com/support). ## Project settings This is what you need to run the AR Photo Editor SDK: - Swift 5.10 or newer, - Xcode 16.0 or newer, - iOS 15.0 or newer. ## SDK size | Photo Editor | Total size | |:------------:|:----------:| |:white_check_mark:| 121 MB | --- ## Installation with Swift Package Manager on iOS Learn the [SPM Getting Started Guide](https://developer.apple.com/documentation/swift_packages/adding_package_dependencies_to_your_app) if you are new to SPM. The integration of the PE SDK via SPM is demonstrated in the [spm branch](https://github.com/Banuba/ve-sdk-ios-integration-sample/tree/spm) of our native sample. Complete the following steps to get the Photo Editor SDK dependencies using SPM: 1. open `App project` and navigate to the `Package Dependencies` tab; 2. for every package that needs to be installed click the `plus` button and type its repo url; 3. choose `Exact Version` and type the newest SDK version. The following module must be installed: | Package name | GitHub link | |----------|-------------| | BanubaPhotoEditorSDK-iOS | https://github.com/Banuba/BanubaPhotoEditorSDK-iOS | --- ## iOS Requirements and Installation Guide # Banuba Video Editor SDK on iOS Learn about the iOS requirements and features for the Banuba Video Editor SDK. See supported versions, compatible devices, & other details. ## SDK integration video You can watch a follow along video below demonstrating key steps of SDK integration in a project: ## License Token Before you commit to a license, you are free to test all the features of the SDK for free. The trial period lasts 14 days. Send us a message to start the [Video Editor SDK trial](https://www.banuba.com/video-editor-sdk#form). We will get back to you with the trial token. Feel free to contact us if you have any questions regarding [Video Editor SDK](https://www.banuba.com/support). ## Project settings This is what you need to run the AI Video Editor SDK: - Swift 5.9 or newer, - Xcode 16.0 or newer, - iOS 15.0 or newer. - :white_check_mark: arm64, :white_check_mark: arm64e, :exclamation: x86-64 limited support ## Supported media formats | Audio | Video | Images | | ---------- | --------- | ----------- | |.mp3, .aac, .wav, .m4a, .flac, .aiff | .mp4, .mov, .m4v | .bmp, .gif, .heic, .jpeg, .jpg, .png, .tiff ## SDK size Banuba Video Editor SDK with all the core editing features only adds 14 Mb of space to your app’s download size. This parameter can change based on the number of features you want. |SDK| Total size | |:------------|:-----------:| |Banuba Video Editor SDK (core editor)| 14 Mb | |Banuba Video Editor SDK (full bundle)| 45 Mb | |Video Editor with Photo Editor | 80 Mb | |Video Editor with face augmented reality features | 45 - 140 Mb | --- ## Installation with Swift Package Manager Learn the [SPM Getting Started Guide](https://developer.apple.com/documentation/swift_packages/adding_package_dependencies_to_your_app) if you are new to SPM. :::note The following guide is compatible with the VE SDK v1.31.3 and newer. If you are looking for a way to install v1.31.2 or older with SPM, please, visit the [legacy docs](https://github.com/Banuba/ve-sdk-ios-integration-sample/blob/main/mdDocs/quickstart.md#spm) ::: The integration of the VE SDK via SPM is demonstrated in the [spm branch](https://github.com/Banuba/ve-sdk-ios-integration-sample/tree/spm) of our native sample. Complete the following steps to get the Video Editor SDK dependencies using SPM: 1. open `App project` and navigate to the `Package Dependencies` tab; 2. for every package that needs to be installed click the `plus` button and type its repo url; 3. choose `Exact Version` and type the newest SDK version. ## Mandatory modules The following modules must be installed in all cases: | Package name | GitHub link | |----------|-------------| | BanubaVideoEditorSDK-iOS | https://github.com/Banuba/BanubaVideoEditorSDK-iOS | If you are **not** using the Face AR SDK together with the VE, you must add one additional module: | Package name | GitHub link | |----------|-------------| | BanubaSDKSimple-IOS | https://github.com/Banuba/BanubaSDKSimple-IOS | ## Face AR interoperation modules If your license includes the Face AR SDK, please, add the additional modules listed below: | Package name | GitHub link | |----------|-------------| | BanubaSDK-iOS | https://github.com/Banuba/BanubaSDK-iOS | The Face AR packages (`BNB***`) hosted by the `sdk-banuba` account follow their own versioning. The following table specifies the mapping between the compatible VE and Face AR versions. Please, make sure that you specify the correct Face AR version, otherwise run-time incompatibilities may arise leading to crashes: | Video Editor SDK versions | Compatible FaceAR SDK version | |----------------------------------------|-------------------------------| | 1.48.0 | 1.17.5 | | 1.47.0, 1.47.1, 1.47.2 | 1.17.4 | | 1.44.0, 1.45.0, 1.45.1 , 1.46.0 | 1.17.3 | | 1.43.0 | 1.17.1 | | 1.42.0, 1.42.1 | 1.17.0 | | 1.39.0, 1.40.0, 1.41.0 | 1.16.1 | | 1.38.1 | 1.15.2 | | 1.38.0, 1.37.0, 1.36.6 | 1.14.2 | | 1.36.2 | 1.14.1 | | 1.36.0 | 1.14.0 | | 1.35.1 | 1.12.0 | | 1.35.0 | 1.11.1 | | 1.34.1, 1.34.0, 1.33.3, 1.33.2, 1.33.1 | 1.10.1 | | 1.33.0 | 1.9.3 | | 1.32.2, 1.32.1, 1.32.0 | 1.9.1 | If your license includes [AR Cloud](guide_far_arcloud.md), please, also add the following package: | Package name | GitHub link | |----------|-------------| | BanubaARCloudSDK-IOS | https://github.com/Banuba/BanubaARCloudSDK-IOS | ## Optional modules You may skip the module below only if your app provides custom implementations of [Audio Browser](guide_audio_content.md). Otherwise make sure to add it to your project: | Package name | GitHub link | |----------|-------------| | BanubaAudioBrowserSDK-iOS | https://github.com/Banuba/BanubaAudioBrowserSDK-iOS | --- ## FAQ on iOS These are the answers to the most common questions asked about our SDK. ## How do I start the Video Editor with a preselected audio track? To do so, add the MediaTrack instance as a parameter to `VideoEditorLaunchConfig` which is used for starting the Video Editor method: ```swift let cameraLaunchConfig = VideoEditorLaunchConfig( entryPoint: .camera, hostController: self, musicTrack: MediaTrack(...), animated: true ) videoEditorSDK?.presentVideoEditor( withLaunchConfiguration: cameraLaunchConfig, completion: nil ) ``` ## What are the available entry points of the Video Editor? The Video Editor supports multiple launch entry points that are declared in `PresentEventOptions.EntryPoint` to meet all your requirements: ``` swift public enum EntryPoint: String, Codable { case camera case pip case trimmer case editor case drafts case gallery } ``` To open the Video Editor at the desired screen, use `VideoEditorLaunchConfig` that should be passed to the `presentVideoEditor` method of the `BanubaVideoEditor` class: ``` swift public func presentVideoEditor( withLaunchConfiguration configuration: VideoEditorLaunchConfig, completion: (() -> Void)? ) {} ``` 1. Launch from the Camera screen where the user can record a video or take a picture: ``` swift let launchConfig = VideoEditorLaunchConfig( entryPoint: .camera, hostController: UIViewController, musicTrack: MediaTrack, animated: Bool ) videoEditorSDK.presentVideoEditor( withLaunchConfiguration: launchConfig, completion: nil ) ``` 2. Launch from the Camera screen in the Picture-in-Picture (PIP) mode: :::important The Video Editor will not open in the PIP mode if your license token does not support the PIP feature. ::: ``` swift let launchConfig = VideoEditorLaunchConfig( entryPoint: .pip, hostController: UIViewController, pipVideoItem: URL, animated: Bool ) videoEditorSDK.presentVideoEditor( withLaunchConfiguration: launchConfig, completion: nil ) ``` 3. Launch from the Trimmer screen where the user can trim a video, add transitions and move to the editing screen for adding effects: ``` swift let launchConfig = VideoEditorLaunchConfig( entryPoint: .trimmer, hostController: UIViewController, videoItems: [URL], musicTrack: MediaTrack, animated: Bool ) videoEditorSDK.presentVideoEditor( withLaunchConfiguration: launchConfig, completion: nil ) ``` 4. Launch from the Editor screen where the user can add effects to a video: ``` swift let launchConfig = VideoEditorLaunchConfig( entryPoint: .editor, hostController: UIViewController, videoItems: [URL], musicTrack: MediaTrack, animated: Bool ) videoEditorSDK.presentVideoEditor( withLaunchConfiguration: launchConfig, completion: nil ) ``` 5. Launch from the Drafts screen where the user can pick any non-completed draft and proceed with making a video: ``` swift let launchConfig = VideoEditorLaunchConfig( entryPoint: .drafts, hostController: UIViewController, animated: Bool ) videoEditorSDK.presentVideoEditor( withLaunchConfiguration: launchConfig, completion: nil ) ``` 6. Launch from the Gallery screen where the user can select videos or photos and proceed to the Editor screen: ``` swift let launchConfig = VideoEditorLaunchConfig( entryPoint: .gallery, hostController: UIViewController, animated: Bool ) videoEditorSDK.presentVideoEditor( withLaunchConfiguration: launchConfig, completion: nil ) ``` ## How do I use the Video Editor several times from different entry points? Before you want to use VideoEditor again, you need to deinitialize your current editor instance in your [entry point class scope](https://github.com/Banuba/ve-sdk-ios-integration-sample/blob/d9733e78a6a752dd8fad849f6aa6d5553eb07f56/Example/Example/ViewController.swift#L675). You need to set 'yourVideoEditorSdkInstance' = nil after following the funcs called [done](https://github.com/Banuba/ve-sdk-ios-integration-sample/blob/d9733e78a6a752dd8fad849f6aa6d5553eb07f56/Example/Example/ViewController.swift#L660) and [cancel](https://github.com/Banuba/ve-sdk-ios-integration-sample/blob/d9733e78a6a752dd8fad849f6aa6d5553eb07f56/Example/Example/ViewController.swift#L678): ```swift // Video Editor Delegate implementation example extension ViewController: BanubaVideoEditorDelegate { func videoEditorDone(_ videoEditor: BanubaVideoEditor) { // User finished editing sessoin, need to dismiss video editor and export video videoEditorSDK?.dismissVideoEditor( animated: true ) { [weak self] in self?.exportVideo(...) { ... in self?.'yourVideoEditorSdkInstance' = nil } } } func videoEditorDidCancel( _ videoEditor: BanubaVideoEditor ) { // User canceled editing sessoin, need to dismiss video editor videoEditorSDK?.dismissVideoEditor( animated: true, completion: { self?.'yourVideoEditorSdkInstance' = nil } ) } } ``` Use the following approach if you want to [create the BanubaVideoEditor instance](https://github.com/Banuba/ve-sdk-ios-integration-sample/blob/d9733e78a6a752dd8fad849f6aa6d5553eb07f56/Example/Example/ViewController.swift#L42) again. For example, as your tap button action: ```swift @IBAction func videoEditorButtonTapped(_ sender: UIButton) { if 'yourVideoEditorSdkInstance' == nil { 'yourVideoEditorSdkInstance' = BanubaVideoEditor(...) } } ``` ## How do I add a color filter (LUT)? Color Filters (LUTs) are special graphic files that are placed in the / [luts directory](https://github.com/Banuba/ve-sdk-ios-integration-sample/tree/main/Example/Example/luts) inside the main project folder. To add your own icon to be used in order to represent that specific effect on the list, you must place it in the / [assets folder](https://github.com/Banuba/ve-sdk-ios-integration-sample/tree/main/Example/Example/Assets.xcassets/ColorEffectsPreview). The icon resource name must match the image file name in the / luts directory and end with `_preview`. The display name for the resource is set in the [localization files](https://github.com/Banuba/ve-sdk-ios-integration-sample/tree/main/Example/Example/en.lproj/Localizable.strings#L275). The key for the translation string must start with `com.banuba.filter.name.{lut file name}` and end with the name of the lut file. ## I want to enable the slideshow animation. To be able to turn on the slideshow animation, use the following property of the `RecorderConfiguration` and `CombinedGalleryConfiguration` entities: ```swift let config = VideoEditorConfig() config.combinedGalleryConfiguration.isPhotoSequenceAnimationEnabled = true config.recorderConfiguration.isPhotoSequenceAnimationEnabled = true ``` ## I want to change the cursor color. All you need is just to set your color into the `cursorColor` parameter in the `MainOverlayViewControllerConfig` entity: ```swift let config = VideoEditorConfig() config.overlayEditorConfiguration.mainOverlayViewControllerConfig.cursorColor = .white ``` ## How can I get the track name of the audio used in my video after the export? ```swift /// Video Editor main entity and entry point. /// Can present and hide root view controller. /// Has default export method. public class BanubaVideoEditor { /// Simple metadata of music composition settings public var musicMetadata: MusicEditorMetadata? { get } ... } ``` `MusicEditorMetadata` contains the array of `MusicEditorTrack` which contains the following fields: ```swift // MARK: - MusicEditorTrack public struct MusicEditorTrack: Codable { ///Track URL public var url: URL ///Track original URL public var originalURL: URL ///Track title public var title: String ///Track id public var id: Int32 /// Track volume public var volume: Float ... } ``` or if you want to know what track was played on the camera screen, you can use: ```swift /// Video Editor main entity and entry point. /// Can present and hide root view controller. /// Has default export method. public class BanubaVideoEditor { /// Music track which will be played on camera recording public var musicTrack: MediaTrack? { get } ... } ``` ## I want to change the font. You can change the font for the whole Video Editor by calling in `VideoEditorConfig` this method: ```swift func applyFont(_ font: UIFont) ``` or change the font for each screen separately by calling the appropriate methods: ```swift func updateFullScreenActivityFonts(_ font: UIFont) func updateRecorderFonts(_ font: UIFont) func updateAlbumsFonts(_ font: UIFont) func updateEditorFonts(_ font: UIFont) func updateToastFonts(_ font: UIFont) func updateTextEditorFonts(_ font: UIFont) func updateSlideShowFonts(_ font: UIFont) func updateTrimVideoFonts(_ font: UIFont) func updateTrimVideosFonts(_ font: UIFont) func updateFilterFonts(_ font: UIFont) func updateExtendedVideoCoverSelectionFonts(_ font: UIFont) func updateAlertFonts(_ font: UIFont) ``` Changing the font does not affect its size. The font size will be taken by default or specified by you in the entity configuration. ## I want to use custom icons. Any icon in the mobile Video Editor SDK can be replaced. This is how: 1. load custom images to the Assets catalog; 2. locate the screen with the icon you want to change in the [VideoEditorConfig](https://github.com/Banuba/ve-sdk-ios-integration-sample/tree/main/Example/Example/VideoEditorModule.swift#L75) entity. More examples: [Camera Screen](guide_camera.md), [Editor Screen](guide_editor.md); 3. find the specific element and override it with the resource name or use UIImage if available. ## The β€œluts” file couldn’t be opened because there is no such file. This error occurs because your application bundle doesn't contain the required luts folder. You need to copy the [luts](https://github.com/Banuba/ve-sdk-ios-integration-sample/tree/main/Example/Example/luts) folder to your project. ## I want to add audio filters. The filters' availability depends on the token. However, in order for them to be available, you need to add an implementation of the `VoiceFilterProvider` entity. Here is an example on how to inherit `VoiceFilterProvider` to your own entity: ```swift /// Example voice filter provider struct ExampleVoiceFilterProvider: VoiceFilterProvider { private let filters: [VoiceFilter] // MARK: - VoiceFilterProvider func provideFilters() -> [VoiceFilter] { return filters } init() { filters = [ VoiceFilter( type: .elf, title: NSLocalizedString("com.banuba.musicEditor.elf", comment: "Elf filter title"), image: UIImage(named:"elf") ), VoiceFilter( type: .baritone, title: NSLocalizedString("com.banuba.musicEditor.baritone", comment: "Baritone filter title"), image: UIImage(named:"baritone") ), VoiceFilter( type: .echo, title: NSLocalizedString("com.banuba.musicEditor.echo", comment: "Echo filter title"), image: UIImage(named:"echo") ), VoiceFilter( type: .giant, title: NSLocalizedString("com.banuba.musicEditor.giant", comment: "Giant filter title"), image: UIImage(named:"giant") ), VoiceFilter( type: .robot, title: NSLocalizedString("com.banuba.musicEditor.robot", comment: "Robot filter title"), image: UIImage(named:"robot") ), VoiceFilter( type: .squirrel, title: NSLocalizedString("com.banuba.musicEditor.squirrel", comment: "Squirrel filter title"), image: UIImage(named:"squirrel") ) ] } } ``` Then the instance of ExampleVoiceFilterProvider needs to be passed to the configuration: ```swift var config = VideoEditorConfig() config.musicEditorConfiguration.audioTrackLineEditControllerConfig.voiceFilterProvider = ExampleVoiceFilterProvider() ``` ## I want to change icons and name for effects. The name of the icon for the effect must match the identifier of the effect. Below there is a table with the name, ID and icon of the default effects: | Default image effect | Name | ID | | ---------- | --------- | ----------- | | | Acid whip | 102000 | | Cathode | 102001 | | DV Cam | 102002 | | Flash | 102003 | | Glitch | 102004 | | Glitch 2 | 102005 | | Heat Map | 102006 | | Lumiere | 102007 | | Mirror | 102008 | | Mirror 2 | 102009 | | Pixel Dynamic | 102010 | | Pixel Static | 102011 | | Polaroid | 102012 | | Rave | 102013 | | Soul | 102014 | | Stars | 102015 | | TV-Foam | 102016 | | Transition | 102017 | | Transition 2 | 102018 | | VHS | 102019 | | VHS 2 | 102020 | | Zoom | 102021 | | Zoom 2 | 102022 | | 0.5x | 104000 | | 2x | 104001 In order to change the name of the effect, you need to do it in the [localization file](https://github.com/Banuba/ve-sdk-ios-integration-sample/tree/main/Example/Example/en.lproj/Localizable.strings#L254). ## I want to get the exported video metadata. In order to find out which filter, effects, masks and music were applied to the video, you need to refer to the instance of the `BanubaVideoEditor` entity. The instance: ``` swift let videoEditorSDK = BanubaVideoEditor( ... ) // to get color filter let videoFilter = videoEditorSDK?.metadata?.colorOnVideoMetadata // to get effects let videoEffects = videoEditorSDK?.metadata?.effectsOnVideoMetadata // to get gifs let videoGif = videoEditorSDK?.metadata?.gifOnVideoMetadata // to get texts let videoText = videoEditorSDK?.metadata?.textOnVideoMetadata // to get music track from record screen let videoMusicTrack = videoEditorSDK?.musicTrack // to get music tracks from editor screen let videoTracks = videoEditorSDK?.musicMetadata?.tracks ``` ## I want to change the codec type from h264 to h265. All you need is to just set `useHEVCCodecIfPossible` to `true` in the `VideoEditorConfig, ExportVideoInfo or ExportVideoConfiguration` entities. The first one you need when you're creating `BanubaVideoEditor`, the two last ones - when you're preparing a video for export: ```swift let exportVideoInfo = ExportVideoInfo( resolution: ..., useHEVCCodecIfPossible: true ) let configuration = ExportVideoConfiguration( fileURL: ..., quality: ..., useHEVCCodecIfPossible: true, watermarkConfiguration: ... ) ``` ## How do I specify the video file saving directory? In `ExportVideoConfiguration` set the desired path in the fileURL parameter: ```swift let exportVideoConfigurations: [ExportVideoConfiguration] = [ ExportVideoConfiguration( fileURL: fileURL, quality: .auto, useHEVCCodecIfPossible: true, watermarkConfiguration: watermarkConfiguration ) ] ``` ## How do I change the language (how do I add new locale support)? There is no special language switching mechanism in the Video Editor SDK (VE SDK). Out of the box, the VE SDK includes support for two locales: English (default) and Russian. If you need to support any other locales, you can do it according to the standard iOS way. See how to [create locale directories and resource files](https://developer.apple.com/documentation/xcode/localization) for more details. After adding a new locale resource file into your application with the integrated VE SDK, you need to re-define the VE SDK strings keys with the new locale string values. To do that, you need to add all the needed string keys in the new locale `Localizable.strings` file. You can find the main VE SDK string keys you need in the [Localizable.strings](https://github.com/Banuba/ve-sdk-ios-integration-sample/tree/main/Example/Example/en.lproj/Localizable.strings) file. The newly added locale will be applied after the device language is changed by the system settings. ## How can I change the extension of the exported video? To save the video in the format you want, you just need to add the appropriate `PathComponent` when creating the video URL: ```swift let videoURL = manager.temporaryDirectory.appendingPathComponent("tmp.mov") ``` See an example in the [sample](https://github.com/Banuba/ve-sdk-ios-integration-sample/tree/main/Example/Example/ViewController.swift#L238). See all the formats supported for the video export [here](guide_export.md). ## I want to change the order of masks, video effects or filters. The SDK allows to reorder masks and filters in the way you need. To achieve this, use the `preferredLutsOrder` and `preferredMasksOrder` properties: ``` swift let config = VideoEditorConfig() // Sorting for the record screen config.recorderConfiguration.recorderEffectsConfiguration.preferredLutsOrder = [ "egypt", "norway", "japan" ... ] config.recorderConfiguration.recorderEffectsConfiguration.preferredMasksOrder = [ "XYScanner", "Background" ... ] // Sorting for the post processing screen config.filterConfiguration.preferredLutsOrder = [ "byers", "sunset", "vinyl" ... ] config.filterConfiguration.preferredMasksOrder = [ "XYScanner", "Background" ... ] config.filterConfiguration.preferredVideoEffectOrderAndSet = [ VisualEffectApplicatorType.acid, VisualEffectApplicatorType.dvCam ... ] ``` ## Is it possible to disable the Trim screen? It is possible to turn off the trimmer screen after the camera screen. The trimmer screen will still be accessible after importing media files from the gallery. To disable it, just change the `supportsTrimRecordedVideo` property to `false` in the `FeatureConfiguration` entity: ``` swift func createVideoEditorConfig() -> VideoEditorConfig { var config = VideoEditorConfig() ... // Default is false config.featureConfiguration.supportsTrimRecordedVideo = true ... return config } ``` ## Is it possible to disable the transition effects? Transitions are visual effects applying to the segue between two videos. They are provided with the Banuba Video Editor SDK **by default**. To disable or enable transitions, set the `useTransitions` flag inside the `FeatureConfiguration` class to false or true respectively: ``` swift /// Allows you to use transition effects between videos /// Defaults is true public var useTransitions: Bool ``` Example: ``` swift let config = VideoEditorConfig() config.featureConfiguration.useTransitions = true ``` :::note Transition effects are not being played if the closest video (either to the left or to the right of the transition icon) is very short. ::: ## How can I convert one or several still images to a video programmatically? If you would like to create a video from UIImage instances without opening the Video Editor, you can use our API SDK tailored for solutions with the custom built UI, namely the `exportSlideshowImages` method of the `VEExport` class. ## Is it possible to store localization strings in a file other than Localizable.strings? Sometimes other 3rd party dependencies overwrite the `Localizable.strings` file stored in the app bundle during compilation. In such cases it is also possible to store the localization strings in the `Banuba.strings` file. ## How to change the default animation (exposure view) shown after tapping on the camera preview? Create a class conforming to `ExternalViewControllerFactory` that implements the `exposureViewFactory` property. In the following example the default view factory `DefaultExposureViewFactory` provided by the SDK is used: ```swift class ExampleViewControllerFactory: ExternalViewControllerFactory { var exposureViewFactory: AnimatableViewFactory? var musicEditorFactory: MusicEditorExternalViewControllerFactory? var countdownTimerViewFactory: CountdownTimerViewFactory? init() { exposureViewFactory = DefaultExposureViewFactory() } } ``` Pass the instance of your factory to the `externalViewControllerFactory` argument of `BanubaVideoEditor`: ```swift let videoEditorSDK = BanubaVideoEditor( token: ..., configuration: ..., // highlight-add-next-line + externalViewControllerFactory: ExampleViewControllerFactory() ) ``` ## How to change the default countdown timer shown on the camera screen? Create a class conforming to `ExternalViewControllerFactory` that implements the `countdownTimerViewFactory` property. In the following example the view factory that provides the built-in class `CountdownView` is implemented: ```swift class ExampleViewControllerFactory: ExternalViewControllerFactory { var exposureViewFactory: AnimatableViewFactory? var musicEditorFactory: MusicEditorExternalViewControllerFactory? var countdownTimerViewFactory: CountdownTimerViewFactory? init() { countdownTimerViewFactory = CountdownTimerViewControllerFactory() } /// Example countdown timer view factory for Recorder countdown animation class CountdownTimerViewControllerFactory: CountdownTimerViewFactory { func makeCountdownTimerView() -> CountdownTimerAnimatableView { let countdownView = CountdownView() countdownView.frame = UIScreen.main.bounds countdownView.font = countdownView.font.withSize(102.0) countdownView.digitColor = .white return countdownView } } } ``` Pass the instance of your factory to the `externalViewControllerFactory` argument of `BanubaVideoEditor`: ```swift let videoEditorSDK = BanubaVideoEditor( token: ..., configuration: ..., // highlight-add-next-line + externalViewControllerFactory: ExampleViewControllerFactory() ) ``` ## How to change the global color appearance (palette) of the SDK? Create an instance of `VideoEditorColorsPalette` with the desired colors and assign it to `VideoEditorConfig`: ```swift var config = VideoEditorConfig() config.setupColorsPalette( VideoEditorColorsPalette( primaryColor: .white, secondaryColor: .black, accentColor: .white, effectButtonColorsPalette: EffectButtonColorsPalette( defaultIconColor: .white, defaultBackgroundColor: .clear, selectedIconColor: .black, selectedBackgroundColor: .white ), addGalleryItemBackgroundColor: .white, addGalleryItemIconColor: .black, timelineEffectColorsPalette: TimelineEffectColorsPalette.default ) ) ``` ## How to change the video preview resizing mode at Recording Screen Starting from the 1.35.0 version of the SDK the `RecorderConfiguration` has the property called `previewScalingMode` with two possible options `aspectFill` and `aspectFit`: ```swift /// Specifies options of scaling the video preview from camera at recorder screen public enum RecorderPreviewScalingMode { /// Fill entire preview screen with camera video feed case aspectFill /// Fit camera video in the screen so that entire video is always visible case aspectFit } var config = VideoEditorConfig() config.recorderConfiguration.previewScalingMode = .aspectFit ``` By default the `aspectFill` option is used. It provides the same preview scaling behaviour as in previous releases of the SDK. ## How to collect logs when I encounter an issue? If you encounter a crash or other issue and are unsure how to provide logs to the support team, follow steps provided below: 1. Connect your iPhone to your Mac via USB or Wi-Fi. 2. Open Xcode β†’ go to Window β†’ Devices and Simulators. 3. In the list, select your device. 4. In the right panel, click Open Console. 5. You will now see all system and app logs in real time. 6. To view only logs for your app, use the Filter by process field and enter the app’s Bundle ID. Once you’ve collected the logs, copy them into a separate text file and send it to the [support team](https://www.banuba.com/support) for further investigation. --- ## Video Templates on iOS Templates let users create stunning videos quickly and easily using predefined sets of effects, transitions, and music. All it takes to make a shareable piece is changing the placeholders. With templates, even people who are new to video editing or just lack time can make impressive content in minutes.       :::important The ```Video Templates``` is not enabled by default. Contact Banuba representatives to know more. ::: ## Launch Video Templates "se the ```.videoTemplates``` entry point in ```VideoEditorLaunchConfig``` to launch the Video Editor SDK from Video templates. ```swift let launchConfiguration = VideoEditorLaunchConfig( entryPoint: .videoTemplates, hostController: self, animated: true ) videoEditorSDK.presentVideoEditor( withLaunchConfiguration: launchConfiguration, completion: nil ) ```