Everything You Need To Know When Choosing Face AR SDK

What is Face AR SDK

A Face AR SDK is a ready-made module that performs many functions related to face tracking and augmented reality effects. This is what it typically includes: 

• Face tracking. This is the core element that all the others revolve around. It detects faces in different positions so that effects can be applied to them. Most SDKs track the so-called “landmarks” (eyes, lips, nose, etc.) via a number of points. More points mean better precision but also higher performance requirements and energy consumption.
  Banuba Face AR SDK has a different approach. It uses a neural network to “see” faces and needs to track just 37 numbers: 30 face types and 7 positions. As a result, this SDK works on iPhone 6S or newer models, as well as 80% of Android smartphones.
• Rendering engine. This element draws the visuals. The quality of filters and effects depends on it.
• Specialized neural networks. For specific tasks like virtual lip coloring or nail polish, a Face AR SDK uses separate neural networks, trained for this kind of thing. 
• Filter editor & viewer. This element converts effects to a format supported by the SDK and launches them in the application that the SDK is integrated with.
• AR assets. The specific filters and effects that are downloaded alongside the SDK. With too many of those, the app could grow bloated, so it is often advisable to use AR cloud. This means storing the filters on cloud servers and letting users download them on-demand.
• Face filters. AR masks, both silly and serious. This can also serve as a basis for virtual try-on of cosmetics, sunglasses, etc.

The point of augmented reality SDKs is to save time and money when developing an app. Building the same features from scratch could take months or even years while integrating an SDK could be done within a day. 

Banuba Face AR Masks

How to choose a Face AR SDK

Now let’s point out the core things to look at when choosing an augmented reality SDK. We will split this section into three parts: 

• Functionality. What exactly this product can do.
• Technical aspects. Compatibility with various software and hardware, cloud support, etc.
• Commercial aspects. Pricing, trial options, and the like.

Functionality

Here we will go into more detail regarding the things mentioned in the first section. 

An important thing to consider is how much functionality would you actually need for your app. For example, if you make a TikTok-like app, you will likely need a variety of filters and effects to one-up the competition, so the augmented reality SDK with more masks would fit better, even if it costs more.

• Beautification. Different augmented reality SDKs have different options here. Some don’t include this feature at all while others can automatically remove wrinkles on the face and neck, make eyes more expressive and whiten the users’ teeth.
• Face filters. Pretty much anything applied to a user’s face is a filter. This includes beautification and virtual makeup try-on.
  
  The more masks you have, the better. However, you should also find out how easy (or expensive) would it be to make new ones. Apps like Chingari (a short video app from India), for example, made extensive use of filters with local feel and style, so they needed many masks created quickly. And having unique and cool effects would be something that sets your product apart from the competition.
• AR assets. Augmented reality SDK vendors generally have some premade filters and effects that they can license out to the customers. Banuba Face AR SDK, for example, has over 2000 effects in the asset store and a new compatible mask could be made in 2-3 days.
• Background replacement. Having this in an augmented reality SDK extra important for video conferencing apps but can be used to great effect in other software as well. In the case of video chats background replacement provides privacy and the feeling of professionalism. In short video apps, they help users to express themselves.
• LUTs (Instagram filters). These are relatively simple to make and they are pretty much expected to be included in any apps with photo/video editing functionality.

Banuba Face AR Masks

Technical aspects

These are the things that are invisible to the user but can make or break your product.

• Rendering engine. The more technologies it supports, the cooler and more realistic effects can it show. This is how the description of a rendering engine’s capabilities looks like for Banuba Face AR SDK: 

• Platform and language compatibility. This is a crucial factor. An augmented reality SDK could have all the functionality you wish for but it would mean nothing if it couldn’t be integrated into your app. The most flexible products would be available for Web, Unity, iOS, and Android and have support for cross-platform frameworks like Flutter or React Native. Using the latter can decrease the development time by about 30% so consider this option when starting the work on your app.
• Performance and optimization. Getting a well-optimized and fast Face AR SDK can increase your audience and provide a better experience for your users. Firstly, it expands the number of devices that your app can run on. Banuba Face AR SDK, for example, supports smartphones starting from iPhone 6S and about 80% of their Android counterparts.
  
  Secondly, the SDK has work well in different lighting and occlusion conditions so that the users don’t uninstall your app due to lags. Thirdly, a well-optimized SDK would not slow down the work of the rest of the app - this is the thing that needs to be tested separately.
• AR cloud. This is a neat feature that could dramatically reduce the size of your app. A single AR mask could take about 2-3 MBs of space. So if you want to add a lot of effects, expect your app to be the size of an AAA game. On the other hand, the AR cloud allows your users to automatically download the necessary filters and effects whenever they need them.
• Integration assistance. The entire point of Face AR SDKs is to make introducing augmented reality features quick and easy. So the vendors should take steps to make this process as seamless as possible.
  
  There are two major ways that this is done. The first is having detailed, step-by-step instructions and examples. We, for example, keep extensive documentation on our site and show code samples on GitHub. The second option is integration by vendor: for an extra fee, the SDK developer could do this and test the product so it performs to specifications. 

Banuba beautification and makeup try-on

Commercial aspects

Everything to do with the business side of things.

• Price and pricing. Each augmented reality SDK is priced differently. Some charge a certain sum per app per platform per year. Others - per processed photos and minutes of video. Still others can sell you a lifetime license for a fixed price. Whether these price and pricing model fit you, depends on your specific situation and the value that the SDK can bring.
• Trial version. Most vendors provide a certain period where you can use their face AR SDK at no cost. We offer 14 days of a free trial. Others can have a different duration or approach (e.g. unlimited free trial but with a watermark placed on all processed photos and videos).
• Support speed and quality. When gathering information about the face AR SDKs, pay attention to how quickly the vendor replies and how well they answer your questions. Developing and running an app is a marathon, so select someone whom you’ll enjoy working with in the long term.

Conclusion

Now that you know what to look for in an augmented reality SDK, you can make the right choice. And while you are here, feel free to check out ours.