Yesterday, we announced a new release of FaceAR Platform, the collection of augmented reality SDKs to integrate camera face filters, AR beauty software and avatars — into your app. Along with the performance improvement, we’ve added new cool features like ASMR tapping and scratching filters, lips and hair recoloring in AR beauty SDK and the background subtraction landscape mode.
We invite you to take a look at these cool features, assess their potential and explore how they can power your augmented reality camera app.
ASMR Tapping and Scratching Effect added to Face Filter SDK
ASMR (Autonomous sensory meridian response) is a new buzzword in the mobile development market and the interest among people looking for relaxation apps is growing. There’re ARMR video apps showing some abstract metamorphoses and ASMR audio apps featuring the sounds of nature. We’ve decided to merge augmented reality with ASMR and empower our Face Filter SDK with an ASMR tapping and scratching filter.
The ASMR filter in the example doesn’t feature the face, so we’ve added the option to switch off face detection and tracking. This way our Face Filter SDK won’t consume the device capacity on face tracking keeping your ASMR (or any other) app lightweight.
You can also develop the ASMR face filter while keeping the option of face tracking turned on and design any concept. The ASMR face filter tracks the face and overlays the graphics on top of that. The filter tracks user tapping and scratching and leaves finger traces.
ASMR effect features:
-
ASMR face filter or general
-
Supports 3D textures and animation
-
ASMR tapping and scratching scenarios
-
Powerful renderer for realistic ASMR effects (supports physics, shadows, ect).
-
Audio support to add music or sounds e.g. сrinkling or rustling.
The plans are to add vibration for the ASMR experience was even more immersive.
AR Beauty SDK: Lips + Hair Recoloring
Virtual lips and hair recoloring have been available as standalone features in our AR Beauty SDK. The new release merges the two separate neural networks to provide hair and lips recoloring in augmented reality. Users can now try on a variety of hair colors with virtual lipsticks simultaneously within one AR Beauty effect.
We’ve also improved the accuracy both of the lips segmentation and hair segmentation neural networks achieving better recoloring results for tips.
The full list of AR Beauty SDK virtual makeup tools now includes mascara, eyeliner, eye-shadow, blusher, highlighter, contouring, lipstick and hair recoloring.
Background Subtraction Improved
Background subtraction in videos has been a demanding feature lately, so we made a couple of improvements aimed to provide better accuracy for separating human from the background including:
-
Sample background separation effect blending improved. The edges of the subtracted silhouette are now more accurate and smooth.
-
Background separation feature respects gyroscope data. A gyroscope measures the rate at which a device rotates around a spatial axis. So, human background extraction now supports the landscape mode.
-
Ignore gyroscope during photo and video processing. The reverse feature that keeps the horizon line in virtual backgrounds when user post-processes images.
Extreme lighting conditions like flare sports have been a soft spot for all real-time background detection and extraction software. We’re on the path to solve it by optimizing our background segmentation neural network to work with dynamically changing lighting.
Real-time background subtraction in videos that separate a human from the background is a frontier for more private video chats and video conferencing. Developers can not only remove the background in videos (aka сhroma key) but replace it with a scenery per their choice for enhanced user experience.
Other technical Fixes And Improvements
- Render passes
- Rendered frames forwarding as byte arrays (Android)
- Ability to debug JS
- Reset effect cache API call
- API to check if the device is compatible with Neural Networks player
- Use background mask transform for background separation layer from config.json
- Option to switch off face recognition in config.json
- Option to set preferred frame-rate on iOS
The full list of add-ons, fixes and improvements of our FaceAR Platform is available in the Release Note history.
We’re excited to see your apps empowered with new FaceAR features! Get in touch for a free trial!
Subscribe to our newsletter and be the first to learn about FaceAR new features!
