Business users would appreciate an even more precise face tracking, and the ability to add makeup to eyebrows. The developers would find many additions that speed up the integration: the SDK is now released as CocoaPods and Maven packages, there is support for M1 simulators and glTF, a new demo scene in Unity, etc. Moreover, there has been a series of improvements to WebAR, so open this article and read the details.
Improved face tracking
We have implemented a new approach to face tracking. Not only does this improve the precision, but it also significantly boosts performance. The most notables change on iOS was RAM consumption - it decreased by 5-7% depending on the device. On Android, this resulted in a massive improvement in FPS - up to 30% (tracking only), depending on the device.
New face tracking in action
The new face tracking works across all platforms that Banuba supports: iOS, Android, Web, and Desktop. But it only works in online mode - for now.
Unity: demo scene with the effects carousel
Now using Banuba Face AR SDK in Unity will be easier. Thanks to the demo scene (a basic part of an app - a 3D environment with some assets) with the effects carousel, your dev team will be able to see how everything should work and quickly modify it to suit your needs.
Eyebrow makeup
The toolbox of our Beauty AR SDK has been expanded to include eyebrow makeup. Now your users will be able to paint them with a realistic replica of a real product or a wild color.
AR eyebrow makeup
This feature works on all platforms: iOS, Android, Web, and Desktop
SDK as CocoaPods and Maven Packages
The SDK versions for Android and iOS will now be distributed as CocoaPods and Maven packages. This will make them significantly easier to integrate.
How the integration works without such packaging:
- Download the SDK
- Unpack the archive
- Copy the framework
- Specify dependencies
- Build the app
- Repeat all the steps above with every new release
How it works now:
- Add two lines of code to specify dependencies
- Build the app
- Repeat step 2 with every new release
Take a look at the new documentation:
The SDK now also supports M1 simulators, glTF, and external textures on Android. Testing the work became easier.
WebAR updates
There are three main additions to the WebAR version of the SDK.
- The ability to load Effect via Request
- The ability to play gif as background texture
- Effect.preload progress listener. You can help your users see the progress as an effect is being downloaded.
We have also expanded the tutorials section of our WebAR documentation with best practices, examples, and insights, so check it out.
Optimization
- iOS: Upgraded lips segmentation neural network
- B&W lips coloring support without additional parameters. Now using black or white colors on lips is simple and straightforward.
- Makeup API: reduced amount of uniforms. Simply put, this will make makeup effects take up less space.
- WebAR: Improved WebAR archive UX. A more convenient, trimmed-down version.
- Rendering Engine: Reduce LUT memory size. LUTs will take use less RAM.
- OEP: extend supported image formats
Bug fixes
- Black screen flash on effects loading
- Android: Memory leak in the OEP
- Android: Pixel 3a crash after applying certain face filters
- Android: deadlock during video_frame draw
- Android: Xiaomi Mi 8 Pro crash
- iOS: iPhone 13 recognized as middle-end hardware
- Fixed invisible entities for some face filters
- Effects API: Api.drawingAreaWidth() & Api.drawingAreaHeight() return 0
- Windows: Impossible to apply effect with path containing extended Unicode
- Windows, MacOS: Banuba Viewer saving video without sound
- MSAA issues in face filters
- WebAR: Fixed processing of RGB images
- WebAR: Fixed MediaStreamCapture crash on Safari 14
- Makeup API: Fixed serialization of empty textures
- Hand AR: hand skeleton false detections
- OEP: crash if imput buffer format changes
- OEP: green frame if texture is not ready
- OEP: fixed rapid camera switch
