Get Free Demo
Wanna give our technologies a whirl?
  • It takes 7 min to install our SDK demo kit. Support documentation and guides provided!
  • Have other questions? Need more info or consultancy? Email us! We'd love to hear from you.

3D Face Tracking Software For Android: Banuba vs ARCore Compared

Share
banuba 3D face tracking

Developing the best face tracking software for Android requires time, efforts and non-stop optimization. That’s why it’s important to keep an eye on what tech giants like Google do in face tracking and align yourself with them. 

We compared our 3D face tracking software for Android with Google’s ARCore examining different performance criteria.

This post explains how each performance parameter impacts the user experience on the example of face filters and approaches the choice of real-time face tracking software from the business perspective.

Test Banuba’s Face Tracking Software

Face tracking cycle explained

banuba 3D face tracking

Face Tracking Cycle: SSD and FRX Dependency 

The face tracking cycle involves two stages: Face Detection stage (SSD) and Face Tracking stage (FRX).

Face Detection or SSD (single-shot detection) shows how fast the algorithms detect a face in a single shot. Once the face is detected, the algorithms step into Face Tracking (FRX) stage to track the facial feature points located in the screen.

In case the face is lost, the algorithms switch back to SSD, and the process goes round in circles.

Having two stages in the face tracking cycle makes sense — in case of the face loss, the application switches to power-saving face detection mode.

Why SSD metrics are important

The SSD stage is the starting point of face tracking software performance. Faster and more stable SSD provides a better and smoother user experience.

In practice, the device will heat lesser, the FPS will be more stable and the functionality of the app built on top of this face tracking software will perform better from the end-user experience perspective.

If SSD is tuned to detect the face quickly, at different angles, lighting, occlusion and on different types of people, the starting point of the camera experience runs smoothly. The poorly optimized SSD results in AR masks lagging and frequent losses.

Why FRX metrics are important

The FRX stage aka face tracking enables the software performance in the context of real-time user experience.

It’s natural for users to move, turn and tilt when using the camera app. They may wear glasses, have poor lighting when using the app not to mention the limited device capabilities. All these factors influence the performance of face filters or any other AR camera experience built on top of face tracking.

It's important to consider different FRX metrics when choosing the face tracking software. With FRX, optimized for different conditions such as low lighting, camera shakes and rotates, the user experience feels better, making AR camera apps more enjoyable. 

3D face tracking software comparison: SSD

To compare the SSD criteria we considered 5 key criteria including its speed, angles, distance support, multi-face performance and technical features.

Device used: Pixel 2 

Comparison summary:

  • Both Google's ARCore and Banuba face tracking software are pretty fast with face detection when looking straight into the camera.

  • Both Google and our face tracking detect a face with 80-90° angles and different camera positions (under and above the face). ARCore performs more accurately.

  • Banuba supports longer distance being able to detect a face from 1.8-2.0 meters while Google supports  1.6-1.7 meters.

  • Banuba’s face tracking software has shown better performance with multi-face detection at different angles compared to ARCore. 

ssd

 

How speed impacts user experience

The speed of face detection shows how fast the face is detected when the user opens the camera or after the face loss.

Fast SSD: Face filter appears on a face within milliseconds once the user opens the camera. The mask sticks well from the very first moment.

Slow SSD: The user may need a couple of seconds to have a face filter located on his face. Face filter may “blink” for some time when adjusting to the face.

How angles impacts user experience

The angles metric shows how fast and accurate the face is detected at different angles or when the user is not looking straight into the camera.

Wide angles (80-90°): The user may hold the device at any angle without looking straight to the camera. Face filter sits fast and well or with insignificant inclinations. 

Narrow angles (15-45°): The user needs to look straight in the camera to have a filter placed on his face. The filter may disappear if the user inclines or turns the head. 

How distance impacts user experience

The distance metric shows how fast the face is detected at a long distance. Usually applied for the back camera.

The longer distance support makes it possible to apply filters beyond the selfie mode using the back camera.

How multi-face impacts user experience

The multi-face criterium shows how fast and stable the face detection is with several faces in a shot.

Fast multi-face: Face filters can be used with several people simultaneously with equally good performance. 

Slow multi-face: Face filter doesn’t fit properly with the second face in the camera, blinks and falls apart.

3D face tracking software comparison: FRX

To compare the FRX, we replaced ARCore’s Augmented Faces mask with the 3D face mesh which provides a better visual representation of face tracking performance.

We tested a variety of use cases e.g. camera jitter, different head position, low lighting and others and paid attention to mask stability and performance. The table below explains the performance differences.

Device used: Galaxy S9

frx-2

NOTE: The 3D face mesh doesn’t fully show how Google’s face tracking performs on the mouth. It’s important for filters with mouth triggers, virtual lips recoloring or 3D avatar performance. 

Banuba and ARCore 3d face tracking comparedFRX comparison on Galaxy S9: Banuba vs ARCore

FRX technical features

FRX technical features cover the device outreach that face tracking software supports. Broader device coverage makes face detection and tracking solutions accessible to a wider user audience. 

When comparing our face tracking software with ARCore, we found ARCore is supported only on relatively new devices, with minimal Android version 7.0 (in practice, 8.0 and later), and an API version of at least 24. The price range of such devices is $ 250 -300 on average, which makes ARCore face tracking inaccessible to users who stick to low-end phones.

Banuba: Min device price — $100 

ARCore: Min device price — $250-350 

View the full list of the devices ARCore supports. Many on the devices require Android 8.0 or later to enable face tracking.

Besides, in the content of AR camera apps, users will need to install an additional Android package Google Play Services for AR to unlocks augmented reality experiences.

Open source 3D face tracking vs proprietary software

Google provides its tracking as an open-source solution meaning you, as a developer, can use it for free. The proprietary face tracking software comes on paid terms meaning you need to buy the licence. 

So, why to pay?

In the context of apps, 3D face tracking usually stands as the core technology to enable specific functionality. 

With face tracking, you can implement face AR experiences, camera effects or face analytics in your app. But you don’t need an app that only detects the square of the face, right? Moreover, you want your app to be accessible for as many users as possible.

From a business perspective, implementing such experiences involves significant development and design efforts. 

Open source technology fails to fulfill them while the proprietary software provides them out-of-the-box.

5 advantages of Banuba’s 3D face tracking software over ARCore

1. Cross-platform coverage

Even though Android devices are currently the most popular out there, with an astonishing 81.7% share of the global market, cutting off iOS devices narrows down the potential audience reach.

ARCore face tracking works only on Android while Banuba covers both Android an iOS also giving developers the maximum user coverage without any coding hassles.

2. Broader device reach 

ARCore face tracking is supported only on mid and top-level devices with an average price of $350. Banuba’s face tracking supports low-end $100-devices being the only face tracking software able to run even on Android Go. It significantly increases the audience outreach.

3. Power and capacity saving

To enable the ARCore performance, an additional package needed which takes additional space on a user device. Our face tracking is lightweight and integrates seamlessly as a mobile SDK or API.

4. 3D Rendering 

To augment a user, you need to develop a separate renderer component which is a timely and costly endeavor. Banuba’s 3D renderer includes the most powerful and feature-rich user modification capabilities and it’s shipped with ready-to-deploy packages and examples.

5. Time to market

The open-source software usually requires significant customization and optimization efforts not to mention the design costs for AR content creation. Banuba has all that’s needed to enable a quick MVP launch including the ready to use AR filter packs.

Summing up

From a technical perspective, Google’s face tracking is great. It’s super fast being able to detect faces instantly. With different angles and camera rotation, the mask sticks well allowing for the seamless end-user experience. 

However, from a business perspective, ARCore has a couple of serious drawbacks i.e. no iOS support and narrow device coverage. 

We were quite satisfied to see our face tracking software for Android being on par with Google’s ARCore. Banuba's 3D face tracking has shown better performance with occlusion, low lightning and camera rotates.

From the business side, our face tracking software has a range of advantages. Brands and developers benefit of broader audience reach, quality performance on low-end devices and ready-to-deploy features that enable the quick face AR app launch.

 

Tags
Share
Next up for you
FaceAR 0.23.0 Release: Improved Face Tracking
October 11, 2019

New SDK release pushes the boundaries of our Face AR technology, making its features accessible on more platforms. We’ve also impr...  

#Release notes
Banuba Joins Forces with Agora to Empower User Engagement
October 8, 2019

Banuba is proud to be selected as an Agora technology partner delivering Face AR experiences to businesses across the globe. Banub...  

#News
Augmented Reality Face Filters To Growth Hack User Acquisition
October 3, 2019

Augmented reality face filters can be an effective growth hacking tool for driving new users to your app. The best part? It won’t ...  

Join the conversation