Webcam Makeup: Adding AR-Based Browser Makeup

About browser makeup

In essence, webcam makeup/photo editor makeup is similar to other AR filters. It is a virtual object that is placed on the person’s face to change it. So the mechanics (face detection and tracking, effect rendering and placement, etc.) are the same. 

Here’s what can be added with this feature:

• Lipstick/Lip gloss
• Eyeshadow/Eyeliner
• Mascara
• Highlighting
• Contouring
• Foundation
• Blush
• Hair color

Banuba beautification

Webcam makeup filter is often combined with beautification - a feature that removes skin defects (e.g. wrinkles and acne), evens out skin tone, and makes other subtle improvements to the user’s face on the screen. Beauty camera features are very common, as they allow to alleviate camera distortions.

A few points should be made about the use of web-based augmented reality vs its mobile counterpart.

Web AR is inherently more accessible (because of low hardware requirements) and can run on pretty much anything with a browse. It is also cheaper to maintain (one app instead of two - for Android and iOS) and doesn’t require the user to download anything. However, it also has worse performance and requires a constant connection to the Internet.

Webcam makeup use cases

There are several prominent ways in which AR solution companies use this technology.

• Video calls. People want to look good on camera, whether they are calling their friends or participating in a weekly meeting at work. Virtual makeup, especially paired with beauty camera effects, helps users always look their best without spending any time on it.
• Photo editor/video editor. Independent creators don’t need to hire a makeup artist when the software does everything for them. 
• Dating. Looking good drastically increases the number of matches a person can get. 
• eCommerce. With custom-made filters that emulate real makeup, a company can provide a great virtual try-on experience. This is popular among both huge corporations (e.g. Sephora and Ulta) as well as smaller brands like Looke.

Guided Virtual Try-On

Despite the meaningful results shown by AR makeup try-on systems, the product returns still keep growing - up to 30% of purchased retail goods are sent back. So the developers went a step further and created the Guided Virtual Try-On (GVTO) technology.

The core difference is in the "guided" part. An inbuilt AI analyzes the user's facial features, skin tone, and other aspects to recommend individual products and complete looks that fit the person best. Moreover, it compensates for poor lighting conditions and bad camera to achieve realistic looks. 

As an added business benefit, GVTO solutions have an "add-to-cart" button right on the try-on screen, making purchases easier. 

How it helps

Webcam makeup is a powerful tool that can bring significant benefits.

First of all, it helps users look better on camera. Many people, especially younger audiences, are anxious about their appearance. With a few AR touches, they can get a confidence boost. And it, in turn, will motivate them to create content, not just watch it. 

Secondly, in the cosmetics industry, AR try-on has proven to be an effective sales tool. When the pandemic started, the companies invested in three years’ worth of digital transformation in just six months. These investments went towards augmented reality features as well. And as a result, Ulta got a 700% increase in engagement, and Estee Lauder raised their conversions tenfold. Even the niche brands like the abovementioned Looke gained significant benefits from webcam makeup. 

Thirdly, it saves a lot of time on making video content. Instead of spending an hour applying lipstick, foundation, mascara, and other beauty products, the user can just click a couple of buttons and be good to go. This does wonder for user experience and engagement. 

Finally, being web-based has its own set of advantages for AR beauty and makeup. The main one is accessibility: anyone can use it, as long as the user has a device with a camera and Internet connection. However, it is also much cheaper to produce and maintain than mobile apps, even ones made with Flutter or React Native.

Adding AR-based Browser Makeup

Now let’s get to installing everything you need for the webcam makeup functionality. We will use Banuba Face AR SDK as an example.

Technical requirements

This is what you’ll need to get webcam makeup:

• The latest Banuba SDK Web AR release
• Banuba client token
• An AR effect
• Nodejs installed
• Browser with support of WebGL 2.0
  

SDK release and token

The first step is getting the SDK itself and the token that allows it to run. To get both, just send us a message through a contact form.

The first version of the token would likely be a trial one. You will have two free weeks to play around with the SDK and see what works for you. 

Implementation

To add the Web AR functionality, use this code:

[code]

<!DOCTYPE HTML>

<html>

  <head>

    <title>Banuba SDK Web AR</title>

  </head>

  <body>

    <div id="webar"></div>

    <script type="module">

      import { Webcam, Effect, Player, Dom } from "./BanubaSDK.js"

      const run = async () => {

        const player = await Player.create({ clientToken: "PUT YOUR CLIENT TOKEN HERE" })

        player.use(new Webcam())

        player.applyEffect(new Effect("Spider.zip"))

        Dom.render(player, "#webar")

      }

      run()

    </script>

  </body>

</html>

[/code]

Then put BanubaSDK.js, BanubaSDK.wasm, BanubaSDK.data, and Spider.zip next to the created HTML page.

Finally, run the command in the folder to start a web server: npx live-server.

Makeup features

To access specific webcam makeup filters, you will need to access the makeup API. For instructions on how to use it, follow this link: Makeup API. It has detailed descriptions and code samples for each available effect. FYI, there are also instructions on how to implement it in mobile apps so you can compare. 

Beauty API

This API serves two purposes: beautification (obviously) and accessing the hair color feature. There are many elements you can tweak, so follow this link to learn more about them and see code samples: Beauty API.

For the specific hair coloring functionality, see this instruction: hair coloring. It includes ways to have a single-color hair, a gradient, or highlighting different strands with different paints. 

Combining Beauty API and Makeup API

There are three ways you can mary the APIs to use both in your web application: 

• Via the effect’s config.js file
• From within the application
• Through feature combination

The instructions for all three methods can be found here: Combining Beauty API and Makeup API.

Conclusion

Now you should have everything you need to get browser makeup features for your photo editor, video editor, makeup try-on, or other web app and test them out at your convenience. Whether it is still being built or is already live, the SDK can be integrated with either. For a more powerful impression, feel free to combine browser makeup with beautification. And if you are ready, go ahead and request the token and the SDK so you can see everything for yourself!