What is FaceAR Platform?
Banuba FaceAR Platform is a set of SDKs and technologies that allow developers to build and implement the most immersive and engaging AR experiences in their apps.
Currently, our FaceAR Platform includes the Face Filters SDK, Face Beautification SDK, the Avatar component and designer tools, i.e. the Effect Constructor and Effect Viewer.
The document represents the most common feature use cases and possibilities of our technologies. The general term "the Banuba SDK" is used to refer both to the Face Filters SDK and Face Beautification SDK.
Based on a particular business case developers are introduced to different feature scope. Email us to firstname.lastname@example.org in case you want to access a feature listed in this document yet not covered within your current licensing plan.
Introduction to the Banuba SDK
Welcome to the Banuba SDK. This document is designed to help you get started with the Banuba SDK and guide you all the way through to advanced concepts and examples for developing your augmented reality project.
How to read this Documentation:
- View the samples. The Banuba SDK is delivered with examples featuring complete feature-rich SDK applications for each platform. They cover a variety of real-life use cases and give a comprehensive overview of how to run and use the SDK.
- Setup your project based on the examples provided.
- Create your own effects. You can construct, configure and preview your effects quickly using our Effect Constructor and Effect Viewer tools. The Effect construction section provides a detailed description of the effect construction process and tools' usage.
Architecture of the Banuba SDK
The image below shows the components of the Banuba SDK.
EffectPlayer is a low-level library that provides all core functionality related to effects' playing. The code of the library is generally cross-platform and doesn't depend on the platform-specific APIs and compilers.
EffectPlayer is written in C++, but has bindings to all supported platform-specific languages and run-times.
So you should use Java or Kotlin on Android, and Objective-C or Swift on iOS for dealing with this API.
Main responsibilities of EffectPlayer:
- consumes the camera frames and requests for the frame drawing. Serves it in an asynchronous manner using multithreading if available on a platform.
- runs all recognition operations on the input frames.
- runs and manages all platform-specific modules encapsulated in C++. For example audio playback, video playback, accelerometer, haptic and vibration engines, scripting engine, etc.
- based on the above-mentioned features implements all logic of loading and playing interactive effects (loading from disk, rendering, scripting of effect logic, etc)
The functionality of the platform modules depends on a specific platform, but in general, it includes:
- working with a camera (permission management, configuration, life cycle implementation)
- organization of the effect's rendering context
- video recording
- high-resolution photo taking
- preparing of EffectPlayer's resources on application startup
The source code of the platform modules is included in SDK's distribution archive. You can modify the code according to your particular application needs. So you can freely adapt the SDK up to your specific use case if its default functionality is not enough.