What is FaceAR platform? What does “the platform” mean?
FaceAR Platform is a set of technologies that allow brands and developers to power their apps with augmented reality features.
The “platform” means that our technologies are combined in several SDKs that focus on one specific area, i.e. Filters, Beautification, Avatars, and Analytics. We combine them in turnkey business solutions too e.g. the virtual try-on or makeover.
In addition, we provide the Filter Editor tool for designers to help them construct and test AR effects quickly. We also share our knowledge and expertise gained for our two-year experience creating camera apps.
This way Banuba stands as a platform where developers and brands can find what they needed — the solution, technology, and expertise.
What is Face Filter SDK?
Face filters SDK is a set of modules that enable AR face filters, 3D effects and lens in your mobile app or on your website.
Do you provide a tool for us to create our own filters?
Yes, we provide the Filter Editor engine that allows you to construct, preview and upload your own effects to the app. See more in the Filter Editor section.
I need my branded filter. Can you design it?
Yes, we have the in-house Art Team department who can provide custom filter design services.
What filters does your SDK support?
- Fun, beauty, scary, full-face masks, thematic, e.g. Halloween, Christmas, Love, etc.
- Realistic virtual try-ons (glasses, jewellery, hats, etc.)
- Effects with triggers, e.g. open your mouth to start effect. We support 5 triggers: smile, open mouth, frown, eyebrows up and eyebrows down.
- Effects for photo or video post-processing.
- Background separation — replace the background or add animation.
- Sound effects. Play music on the background.
- Gamification. Create filters where users can make AR objects move with their faces, facial expressions and multi-touch events.
- Camera effects like noir lighting, color corrections.
Can I buy the ready filters and use them in my app?
Yes, you can buy a set of filters that we have ready-made.
What is Face Beautification SDK?
Face Beautification SDK is a set of modules that allow you to fix camera distortion and modify user facial features in the camera.
What features does it have?
- Virtual makeover for lips, eyes, skin, eyebrows.
- Hair recolor.
- Skin smooth. Enhance skin and hide defects like eye wrinkles, pimples.
- Skin tone correction.
- Eyes whitening.
- Teeth whitening.
- Face morphing, i.e. to make cheeks slimmer or wider.
- Facial features modification e.g. increase/decrease eye size, change the shape of the nose and head proportions.
- Improve face symmetry. Correct the shape of the face based on the golden ratio.
What is Avatar SDK?
Avatar SDK is the technology based on Action Units that lets you turn users into live 3D characters or emojis.
What features does it have?
46 Action Units that allow tracking user facial expression and mimics.
What is Face Analytics SDK?
Face analytics SDK is a set of modules and technologies that allow you to analyze the faces of your users and monitor their engagement. Only DEMO version now.
Email us for inquiry at email@example.com and we’ll keep you updated as soon as it’s released.
What features will it have?
- Age, gender, ethnicity recognition
- Gaze tracking
- Remote heart-rate measurement
- Face-device distance estimation
- Detect attributes detection (glasses, beards, mustaches)
- Attention tracking
- 6 expressions (happy, neutral, sad, angry, surprised, disgust, fear)
- 30 anthropometric parameters
Do you provide the trial period?
Yes, before purchasing the license cost you have a 1-month free trial period to validate our SDK performance. To get your free trial period started we need to sign NDA (non-disclosure agreement).
Can I use a separate feature of an SDK?
Yes. If you need only one or several features e.g. background separation or hair recolor, we provide you with the custom build and configuration modules that include the features you need.
What’s the cost of your SDK? What pricing models do you provide?
We offer different license-based pricing models for our clients, e.g. monthly, quarterly or annual payment. As most clients need a specific set of features the cost varies depending on the client’s case. Please, get in touch with us to discuss your pricing.
What platforms does your SDK support?
iOS, Android, Web (with WebAssembly and WebGL), OS X, Windows, Linux
Do you provide Unity support?
Currently no, but Unity3D plugin is coming soon. Email us for inquiry at firstname.lastname@example.org and we’ll keep you updated as soon as it’s released.
What is your minimum SDK size?
Our SDK will take 17.2 MB in your APK (without effects).
Does the user need to download additional data for more masks / features?
To keep the SDK lightweight, masks are downloaded, but also can be included with app as resources. Mask sizes depending on complexity may range from 0.5 to 10 mb uncompressed.
How is the SDK packaged?
Our SDK is packaged as AAR library for Android, iOS Framework for iOS and DLL for Windows.
Can the SDK be used in a desktop app based on QT multi-platform development?
Yes. SDK is used for desktop effect viewer apps for Windows and Mac.
For desktop, it requires OpenGL 4.3 on Windows and OpenGL 4.1 on Mac,
but this requirement could be lowered to OpenGL 3.3.
Do you use iOS ARKit and Android ARCore?
We prefer custom made, self-developed solutions. So we do not use ARKit
What code do you use?
80% of our code is built with C++ including the core components like Audio, Video player, Effect player, FRX recognizer and Renderer.
We also have the custom modules per different platforms e.g. iOS, Android that refer to the aspects of camera performance, OpenGL initialization, Renderer thread logic, video recording logic specific for each platform.
These platform modules allow developers to robustly and quickly enable the needed functionality for their apps with our SDK.
The list of core modules:
- Recognizer - enables all FRX technologies, e.g. face tracking, eye detection and modification, lips segmentation, face modification, etc.
- Rendered - enables the graphical image on the screen and defines its behaviour, e.g. triggers, shadows, tap-based control, etc.
- Effect player - enables the interaction between Recognizer and Renderer for proper performance.
- Audio player - enables sound effects i.e play background music.
- Video player - enables video recording.
- Voice changer - enables the modification of a user voice, e.g. change
Can I modify the code?
The core modules are provided in compiled files. These modules enable the logic of SDK and don’t require any modification.
The platform-specific modules are provided both compiled and with the source code. You can modify the source code to customize the module's functionality per your case.
Do you provide documentation and support?
Yes, we provide documentation with code samples and support.
Face Tracking Technology
How many landmarks do you track?
Our face tracking technology is not built on landmarks. Instead of landmarks, we recognize 37 face characteristics represented as morphs for the default face mesh. This is the most optimal number that allows us to get all the needed data to detect and track a face in the camera.
When demoing our technology, you may see 87 landmarks marked by which you can evaluate the accuracy of our face tracking.
How does your technology differ from competitors?
Since we don’t track landmarks but recognize only 37 face parameters, we achieve a much faster technology performance and better accuracy. The algorithms need to process only 37 numbers saving the capacity resources.
Patented anti-jitter. As we significantly save on the capacity resources we are able to run algorithms several times within one frame. It shows us the constant data (face itself) and the variable one (the noise). We detect and process the noises within one frame due to which the technology provides no lag.
Our face tracking technology is optimized specifically for mobile to function equally well on iPhone-Xs and $100-Androids, at low lighting, and with up to 70% occlusion.
Our device support starts with iPhone 5S, Android 5.0 with Camera 2 API and Open GL ES 3.0 which covers 97% of all iOS devices and 80% of all Android ones.
Are your models trained for all people skin tones?
Yes, we use the balanced datasets that include people's faces of different ages, genders and nationalities.
What is Filter Editor?
Filter Editor is the engine where you can create, test and adjust your own camera face filters before uploading them to your camera app via Banuba face filter SDK.
Do I need any special skills to work in Filter Editor?
No special coding skills needed to construct effects. You design your effects in your regular 3D design tool and upload filters to Filter Editor. You set the effect properties and behavior and can preview them in real-time.
Do you provide effects samples for Filter Editor testing?
Yes, you will have a couple of our effects to test the features like triggers, beautification, physics, morphing and etc.
Which formats do Textures support?
We support KTX, PNG, JPEG.
Can I modify a face? For example, change lips and increase eyes?
Yes, you can modify the face with morphing, i.e. changing the base model of the face. Please, mind keeping the UV coordinates unchanged.
Put the FBX file with the changed geometry in the SRC folder. Go to the Filter Editor, point to the file warp_mesh and turn off the display of the corresponding material. It’s better to morph a face gradually checking the results of your work in the editor's viewport.