Background segmentation is a must-have feature in videoconferencing and is often being used in other niches. Despite its popularity, it remains a somewhat complex technology that few companies can build from scratch. This means that smaller businesses or ones that don’t focus on R&D turn to third-party providers for their background separation. In this article, we will compare the performance of two background subtraction SDKs: Banuba Face AR SDK and Vectorly so that the prospective users will have information to base their decision on.
Who Is Who
Banuba is a leading AR-focused lab with 250+ employees, mostly engineers. It has a presence in the USA, Europe, and Asia, and its products are used by world-renowned brands (e.g. Gucci, Samsung, and AKQA).
Vectorly is a California-based startup that focuses on AI image upscaling and virtual backgrounds. It was recently acquired by Hopin, a company specializing in software for virtual events.
What Is The Purpose Of This Comparison?
Recently Vectorly has released a beta version of a new WebGL-based product that is supposed to use much fewer resources than Google Selfie or CPU-based products like Banuba Face AR SDK.
This seemed like an interesting approach to the problem, so we decided to test it ourselves. We took a look at how well background subtraction works in both the “old” and “new” ways and will present the results below.
What’s This All About?
Traditionally, the calculations needed for background segmentation run on CPU (Central processing unit - the chip that does the core calculations in a computer or a mobile phone). Vectorly thought that this was inefficient, as graphics cards (including integrated ones) were better suited for such tasks. So they created their own custom library that works on GPU (Graphics processing unit - the chip that is designed to create images). Here’s what they claim to have achieved.
CPU usage by AI model. Source
The team at Vectorly had issues with the more popular GPU-based computation method which included several iterations of CPU-GPU data exchange. They considered it inefficient, and so decided to bypass it by having the graphics processor handle everything. At first, they tried to build their solution on Tensorflow.js, but it had too much overhead. So Vectorly made a custom convolutional neural network from scratch.
Methodology & Criteria
The focus of this comparison is background subtraction.
We have run both SDKs on several systems with different hardware, browsers, and OS. There was a single person on camera in all cases. The details can be found in the description of each test.
We evaluated real-time performance, FPS, stability, browser compatibility, and resource usage.
Performance
Firstly, we ran our own tests to compare the performance of CPU- and GPU-based approaches.
This is a good time to show what matters in such things. Firstly, there is FPS (frames-per-second). This metric shows how smooth the video feed will look. If it’s below 30, the image will lag and stutter. Overall, the higher the number, the better.
Then there is the resource consumption. This is shown in MBs of RAM (random-access memory - a “place” where the data currently needed is temporarily stored) used and percentage of CPU and GPU load. Low resource consumption means less battery drain and heat emissions on mobile devices, as well as the ability to keep more applications and browser tabs open simultaneously. The fewer resources the app consumes, the better.
Environment:
MacBook Pro (13-inch, 2017, Intel Core i5), MacOS (11.4)
Chrome 91.0.4472.114, (x86_64)
Performance comparison results.
While it’s true that the background segmentation technology of Vectorly SDK uses fewer CPU resources than Banuba, they also use a lot more RAM. Moreover, even on a reasonably powerful MacBook, the CPU load was at least 16%, not 3% as was claimed.
A lower load is still a compromise if the image quality and performance suffer as a result. As you will see in the videos below, Banuba offers a lot more precision and stability, as well as higher FPS.
In addition, there is a matter of specialization vs. generalization. Banuba SDK can perform many functions simultaneously (background subtraction, AR face masks, beautification, virtual makeup, etc.) while Vectorly is focused only on background replacement.
Browser Compatibility
Banuba Face AR SDK is compatible with all major browsers (Chrome, Safari, Firefox, Edge, Opera). Vectorly demonstrates bugs and glitches in browsers other than Chrome. Note that this regards both mobile devices and desktop/laptop computers.
Given that Apple device users are considered to be bigger spenders, getting access to them is important. In absolute terms, the number of potential users that are lost due to browser incompatibility is massive (18.3% Internet users worldwide).
For example, this is how Vectorly’s new background subtraction works on Safari 14.1.1.
Image Quality
The higher the resolution, the more resources the software requires. This is a pretty obvious correlation. The solution from Vectorly consumes less but also has visible imperfections.
Background separation in Banuba SDK
Background separation in Vectorly
Background subtraction in Vectorly (WebGL)
Note the holes in the image and slow reactions.
The cause for this issue could be the “GPU trap”. In the cases of similar GPU-based systems, the tests sometimes show great FPS while in reality, they are below the basic minimum.
Why Choose Background Subtraction From Banuba
Given the results of the comparison, you should opt for Banuba’s product if you prioritize the following qualities.
- Quality and performance. The mature technology offers precise tracking of the face and upper body and does it smoothly, without lags or glitches.
- Expanded feature set. This includes many background types (blur, static picture, video, 3D environment), the ability to combine backgrounds with AR effects, multiple face tracking, etc.
- Browser compatibility. Access to a large number of Apple product users.
- Stable third-party integrations. This makes Banuba easier to use with certain other SDKs (e.g. Agora for videoconferencing).
Conclusion
There are reasons Banuba Face AR SDK is used by companies like Gucci, Chingari, and RingCentral. Thanks to thorough optimization and powerful background subtraction technology, it is compatible with all major browsers, provides top-notch image quality, and is easy to integrate. The test results consistently show high performance. And if you want to see it for yourself, feel free to request a trial version - you will be able to play around with it for two weeks at no charge.
