Locating a person’s visage in a picture is a difficult computer vision task. But it is absolutely necessary for many applications, from biometric identification to autopayments and AR filters.
In this article, you will learn:
- Facial detection vs recognition vs tracking: what’s the difference
- The principles behind face detection algorithms
- 10 real-life use cases and examples of adopting face detecting technologies
- How to save development time by using a face detection SDK
[navigation]
Face Detection vs Face Recognition vs Face Tracking
These terms are often used synonymously but they have different meanings.
Face detection means figuring out whether there are human features in the picture. It is a foundational technology for all face-related applications. It is also used in digital cameras for autofocus – note when they put a rectangle around faces or something they perceive as one.
Facial recognition means comparing a person on camera or image to the picture stored in the database. Face recognition systems are commonly used for identification purposes (e.g. biometric access or criminal investigations).
Face tracking means following the face on the screen, including both its position and angle. It is important for applying AR filters, beautification, background replacement, and other similar use cases.
How Face Detection Algorithms Work
Facial detection is a difficult task even in comparison with other object detection problems.
Firstly, humans can be very different — skin color, haircuts, and tattoos change their looks and affect the accuracy of the detection. Secondly, angles matter — a face can be very different depending on the viewpoint of the camera. Thirdly, different facial expressions can change the way a person looks. Finally, there are matters of lighting and occlusion, as the face might be obstructed by something or the picture can be taken in relative darkness.
Well-written face tracking software can deal with all of those issues. For example, Banuba Face AR SDK can perform face detection even with 70% occlusion, works in very low light, and supports rotation of up to 90 degrees.
Most modern systems use an image-based approach and machine learning. They include neural networks that are trained to detect faces by looking through large numbers of pictures and making decisions about whether a part of an image is also a part of a human face. This is the cutting edge of computer vision, offering better real-time performance and effectiveness.
Older systems use the feature-based approach. They try to locate notable facial regions (eyes, nose, eyebrows, mouth, etc.) and then build a face model around them. This option is simpler but it is very sensitive to lighting and occlusion issues.
There are several popular types of face algorithms in use today:
- Viola-Jones. This algorithm drives a “sliding window” (a square of a predetermined size) over the image to locate Haar-like features. These features can then be recognized as elements of a face.
- Single Shot Detector (SSD). This one places a grid and a number of “anchor boxes” on the image, the latter are created during the training stage. These boxes are used to detect features of the needed objects (e.g. faces) and their position.
- You Only Look Once (YOLO). A similar approach to SSD, it boasts better performance because it only needs one “look” at the picture to find all the objects of interest.
10 Cases of Using Face Detecting Technologies
These applications are examples of practical use of face detection.
1. Clash of Streamers — top-grossing mobile NFT game
Clash of Streamers is the most popular blockchain-powered mobile game in the world, with over 4 million installs on Android and iOS. It is also the top-grossing one in this category.
One of its core features is using player selfies to make the characters unique. To do so, they use selfie segmentation together with AR masks. In both cases, face detection is the first stage.
Read more about the game and how its developers used a face SDK for their benefit in the Clash of Streamers case study.
2. MNFST — Micro Influencer App
Manifest (MNFST) revolutionized influencer marketing by allowing anyone to earn through their Instagram account. It got over a million installs and received $2.5 million in investment.
MNFST users can use branded face filters and effects to create posts advertising a certain company. The payouts will depend on how many engagements these posts got. But the branding itself wouldn’t have been possible without a face detection API that the developers used.
Check out the details about the app and its AR features in the Manifest case study.
3. Looké — Virtual Makeup Try-on
Looké is a niche beauty brand, focusing on halal, vegan, and cruelty-free products. It was the first company to launch a virtual try-on app in its home country of Indonesia, allowing its customers to check out any product in its catalog on their mobile phones.
Virtual cosmetics are a subtype of face filters, designed to look lifelike. Applying them accurately would’ve been impossible without precise face detection algorithms.
Looké got over 50.000 downloads of their app and is using it to guide the people to the stores where the company’s products are sold.
Discover additional information about this project in the Looké case study.
4. FaceYoga — Beautification and Wellness
FaceYoga offers personalized programs to help users keep their face muscles toned, healthy, and attractive. One of their motivational features is “before-after”: the app shows users how they will look if they keep exercising in accordance with the program.
This feature is based on beautification technology. It evens out skin tone, removes wrinkles, and eliminates skin defects. It also starts with detecting a face.
FaceYoga developers used a face AI SDK to achieve this. Read more in the FaceYoga case study.
5. Chingari — TikTok-like Social Network
Chingari is an Indian short video social platform with more than 30 million downloads. It is functionally similar to TikTok and includes prominent features such as AR face filters, beautification, and background replacement. All of them rely on face detection algorithms to some extent.
Many AR masks that Chingari uses are “Indianized” to suit the main target audience.
Today, Chingari is one of the most influential social networks in India. See how it was created in the Chingari case study.
6. Bermuda — Popular Live Chat
With over 4 billion conversations and 20 million installs, Bermuda is one of the most popular live chat apps. To increase engagement, its developers implemented several functions that modify facial features, for example, AR masks, beautification, and background replacement.
Users love these features and launch them about 15 million times per month.
If you want to know more about this app, take a look at the Bermuda case study.
7. Facial Recognition Systems for Payments
A number of technology providers and retail companies are rolling out the pay-by-face feature. These include PayByFace and Mastercard’s “smile-to-pay”. The idea is for shoppers to confirm payments by waving or smiling into a camera, after which the software will match their faces to the picture in the database and the account connected to it.
Even in the cases where people wave, the software still runs a face recognition algorithm, so without face detection, this would’ve been impossible to implement.
8. Face Recognition Access Control
A card can be lost, stolen, or forged. Making a copy of a face is far harder. This is the core principle behind face recognition access control. Simply using a photo of the person won’t work, as the anti-spoofing algorithms in the software will detect it.
Besides making unauthorized access far harder, this approach also helps monitor the exact entrance/exit times and is more convenient for people who don’t need to look for cards or fobs.
9. Customer Segmentation
Cameras inside the shops have many uses. One of them is monitoring and segmentation of the people who visit the store. Advanced systems have the ability to determine age, sex, ethnicity, mood, and other parameters. This can help owners make informed business decisions regarding marketing, inventory, and other aspects.
This is already in use by major retailers like Walmart and Target.
10. Theft prevention
US retailers alone lose hundreds of millions of dollars to shoplifting each year. Face recognition technologies can help combat this problem to some extent. Cameras can help identify known thieves and alert the security to watch out for them.
Face Detection SDK and API Integration
We have already mentioned that face detection alone is a complicated task. It is only one element of a system that could bring actual business benefits. In addition to it, you will likely require face tracking, face recognition, or both, as well as the relevant features for augmented reality, virtual try-on, emotion detection, etc.
Building all of this from the ground up is lengthy and expensive. You will have to hire a team of machine learning engineers, and these professionals alone can command salaries starting from $100K/yr. That is, if you can outbid other companies competing with them and if your offer is interesting enough. Add the benefits, taxes, and other overhead costs, and you might be looking at twice the salary in total expenses. Expect to spend at least 6 months working on the minimum viable product (MVP) version of your software.
There is, however, a cheaper and simpler way to get the functionality you need — a face detection SDK or API. It is a premade product that can perform face detection as well as other specialized functions: apply AR masks, change backgrounds, remove wrinkles, put on virtual makeup, etc.
It is much cheaper in the short term, as the license payments for it are much lower than the initial investment in development would’ve been. And it is much faster to implement, taking under a week at most, and under a day at best.
For example, this is how you can integrate Banuba Face AR SDK.
The first step is getting the SDK itself and the unique token to make it operational. You can try this face detection SDK for free before committing to purchase.
Doing it is simple: send us a message through the contact form.
Face Detection SDK for Android
The token needs to be put here:
[code]
BanubaClientToken.java
final class BanubaClientToken {
public static final String KEY = "YOUR_TOKEN_HERE";
private BanubaClientToken() {
}
}
[/code]
The second step is adding the Maven repo to your build.gradle
[code]
allprojects {
repositories {
google()
mavenCentral()
maven {
name "GitHubPackages"
url "https://maven.pkg.github.com/sdk-banuba/banuba-sdk-android"
credentials {
username = "sdk-banuba"
password = "\u0067\u0068\u0070\u005f\u0033\u0057\u006a\u0059\u004a\u0067\u0071\u0054\u0058\u0058\u0068\u0074\u0051\u0033\u0075\u0038\u0051\u0046\u0036\u005a\u0067\u004f\u0041\u0053\u0064\u0046\u0032\u0045\u0046\u006a\u0030\u0036\u006d\u006e\u004a\u004a"
}
}
}
}
[/code]
Face Detection SDK for iOS
First, add a custom source to the Podfile:
[code]
source 'https://github.com/sdk-banuba/banuba-sdk-podspecs.git'
[/code]
Then specify Banuba Face AR SDK as a dependency:
[code]
target 'quickstart-ios-swift' do
use_frameworks!
pod 'BanubaSdk', '~> 1.4'
end
[/code]
That’s it.
Face Detection SDK for Web
In this case, you will get an archive along with the token. Unpack the archive and place the token into index.html.
[code]
// ...
const player = await Player.create({ clientToken: "PUT YOUR CLIENT TOKEN HERE" }) // <- put your Banuba Client token here as the hint suggests
// ...
[/code]
Then run npx live-server and open localhost:8080.
Why Banuba Face Detection SDK
Banuba Face AR SDK is your best bet for face detection if you want to launch a video-based social network, a virtual try-on app, or an AR-powered game. This is why:
- Precise face detection algorithm, one of the most accurate on the market.
- Works in up to 70% occlusion and in low-light conditions.
- Operates on 80% of Android devices and iPhones starting from model 5S.
- Quick to integrate, requires just a few lines of code.
- Supports many types of effects: masks, stickers, LUTs (“Instagram filters”), beautification, custom backgrounds, 3D avatars, virtual contact lenses, etc.
- More than 1000 ready-made effects in the asset store.
- Used by companies like Gucci, Samsung, Agora, Vidyo, RingCentral, and others.
Conclusion
Face detection algorithms form the basis for other face-related technologies. They are used in social media, marketing, wellness, eCommerce, security, and other domains. While developing them from scratch is hard, a face detection SDK/API can make the process much faster and easier. So if you need to locate faces in the image and modify them with cool AR effects, feel free to contact us and try our SDK for free.
-
Modern face detection algorithms divide the image into squares and place bounding boxes in places where the faces are most likely to be. If they find a match to features that were on their datasets, they define the object as a face. This is an oversimplification because the detailed explanation would require a scientific article.
-
In short, yes. Facial detection, tracking, and recognition are all based on deep learning. This is a subset of artificial intelligence that trains on hierarchical models with different abstraction levels. So yes, facial recognition uses AI.
-
A Haar cascade classifier algorithm is an algorithm for object detection. It uses combinations of elements with different intensities to detect object features — facial features in the case of face recognition. Haar cascades are an element of the Viola-Jones face detection algorithm.
