Face AR SDK v0.29: Skin Touch Up and Web AR Kit Improvements

Skin smoothing based on neural network (iOS)

Skin beautification is big. And we dare to say it’s the most important feature for any kind of camera app be it a professional makeup simulator, selfie editor or just fun face filter app. 

Previously, skin smoothing was applied using a face mesh and allowed to touch up skin imperfections pretty well. However, we've been seeing a growing demand among users and clients alike for more natural skin beautification. So, our R&D department implemented a new approach to skin retouch based on skin mesh which detects the area of the skin more precisely and blurs it in a sophisticated way delivering a truly natural skin beauty effect almost similar to professional photography.

Skin beautification based on neural network works on any skin color and is compatible with other Face AR features like virtual makeup or face filters. 

We’re designing it to make it lightweight too so it wouldn’t spoil the user experience and consume device capabilities. The current weight is 1.5MB, and the feature is available for offline work for now meaning users can process photos. We’re working on bringing this technology to real-time as we see its potential.

Web AR Kit improvement

Our face tracking Web AR SDK kit has undergone some major technical improvements which facilitate its usage for developers.

1.  Compliance with ES6 standard. It makes it easy for developers to integrate and connect our web augmented reality features as modules. 
2.  Compatibility with bundlers. Our AR browser SDK is now easily integrated with bundlers like Webpack or Rollup which automates and speeds up feature usage.
3.  Memory optimization. The technical upgrades result in lesser memory consumption required for video flow processing and therefore, better Web AR feature performance for the end user. 
4.  Camera operation mode optimized. Developers can now initialize camera operation scenarios, e.g. turn off the camera when a user leaves or closes a web page. This can be pretty helpful for e-commerce sites offering virtual try on as users often keep several tabs opened with multiple products to try. To bring them a feeling of comfort that a website stops “tracking” them after they finished a virtual try on session, you can now automatically turn off the camera when a user leaves a web page and forgets to close the camera.  

All-in-all, Banuba's Web AR kit is much better now.

Features available in Web AR SDK:

• Face filters for web with multiple face detection
• Realistic virtual try on solutions for chrome (glasses, jewelry, headwear, accessories)
• Camera filters like LUT effects 
• Face touch up technology for web
• Avatars and live emojis for web

You are welcome to try some of our web augmented reality features on our website. Browsers Web AR supports include Chrome and Firefox.  

Manual audio session 

This feature brings more freedom for developers to adjust sound in face filters. You can now manually create an audio session setting up the beginning or end of music or sound effect, which can be a small yet nice cherry on top of overall face filter experience.

Other improvements and fixes

• Banuba Viewer color picker reacts to background and lips neural networks
• Make WebAR SDK ES6 module
• Updated llvm backend for WebAR
• Lips segmentation on Android HQ photo
• Memset buffer overflow when using Action Units
• Jaw mesh stretching fix in face tracking algorithm
• Web AR SDK package size decreased

Explore how our Face AR SDK can help you to empower your app with some amazing features that your users will love.