Ever wondered what you would look like as the opposite sex? And we’re not just talking about smearing your face with makeup or putting on a fake moustache.
We’re referring to the real deal – a full transformation that convincingly alters your facial features.
Snapchat has once again successfully gotten our attention – let’s find out more about what they’ve done!
What is Snapchat’s Augmented Reality (AR) filter doing to the world?
The gender swap filter was released earlier last month, and if the photos on my social media feed are anything to go by, almost everyone’s intrigued by how they would look as a person of a different gender.
Just about everyone is getting in on the fun and trying out the filter – and the results are amazing. It’s like having a twin – without the hassle of sharing everything.
Source: Twitter @wheeler_anna
Take a look at Twitter user wheeler_anna. Assuming you can figure out which one she is. Thankfully, her earrings are a pretty big tell. Things could have gotten a lot more confusing otherwise.
Source: Twitter @mark_a_valdez
If looking like a celebrity of a different gender has always been your dream – your time is now – as this user demonstrates, the Anne Hathaway look is timeless. Talk about striking the genetic lottery.
Tinder users, beware!
The infinite ingenuity of mankind is matched only by their infinite capacity for mischief.
It’s not just social media that’s been exhibiting a sudden surge in fraternal twins – Snapchat users have also been exploiting the filter to wreak havoc on dating applications such as Tinder. By all accounts, they seem to have been quite successful.
Here’s some of them:
Source: Twitter @MohimenMahbuba
Men, amirite?
Source: Twitter @DylanParkerNash
We’ve heard about drunk texts and calls but setting up a whole new tinder account with a filter? That’s a whole new level of inebriated dedication.
Source: Twitter @J_Askew
Nothing is sacred.
AR and Snapchat are back at it again
Despite the initial splash that Snapchat made when it was first launched, a change to its UI in February 2018 and increased competition from similar applications such as Instagram left the beleaguered social media platform gasping in the dust. The gender swap filter looks like a step in the right direction for the business– with some users even going so far as to re-install Snapchat to give it a shot.
While there have been some mutterings on the interwebs about how the filter continues to perpetuate a pernicious idea of gender as a binary, this will no doubt work to Snapchat’s advantage – after all, few things are as effective at capturing public attention as controversy.
Now, we don’t know if these issues were part of Snapchat’s consideration set when they published the filter, but we do know that Augmented Reality (AR) played a huge role in making this possible.
Computer Vision
As we’ve mentioned in earlier articles, Augmented Reality works by overlaying digital elements on real-world environments. This presumes that the camera is first able to recognize and lock on to an image or a specific feature.
The gender swap filter is impressive precisely because of its ability to accurately identify such facial features as jawlines, eyebrows, noses and lips – allowing for the necessary alterations to be carried out.
And Snapchat’s got Looksery to thank for that – a Ukrainian startup that created an application to allow users to modify their own facial features during a video call. It was acquired for $150 million USD in 2015.
Now, let’s look at Looksery’s facial recognition process in a little more detail.
Face detection
With the understanding that most human faces have similar properties – for instance, all noses should stick out of the face – the computer’s job is to first figure out what it is looking at – specifically by running some numbers. This can be achieved by proxy – a camera may have no concept of an “eye socket” or a “nose” but it can definitely tell that the sockets of your eyes are darker than your nose because of how light reflects off different depths. Once it has gotten enough points of data, it can decide if it’s looking at a face, or not.
Facial landmarks
Once a face has been detected, the processor goes into greater detail to figure out where everything is. We’re talking nose, eyes, mouth, lips and ears – or facial landmarks. At this point, many little coordinates will amount to a facial feature and together – they adumbrate the shape of your face.
Image processing
Now that the detection and differentiation is over and done with, it’s time to process the images with the use of Active Shape Models (ASMs). Through the use of Machine Learning (which is really a fancy way of saying you expose a computer to thousands upon thousands of faces, get it to process and synthesize the data into an “average human face”), the computer has got a pretty good idea of what your face looks like. That’s when it overlays a generic mould on your face. Think of this as a store-bought Halloween mask designed for ages 12 and up.
And this is when the magic begins. This Halloween mask can sense that it might not be a perfect fit for your face and it corrects itself to achieve a better fit. Once the fit is achieved, the effects are slapped on the mesh – which in turn adheres tightly to your face. This is why your dog ears and your dog nose remain nicely stuck to your human features even if you’re shaking your face vigorously (within reason) from side to side.
Other brilliant filters by Snapchat:
From dogs to babies, the filters on Snapchat run across quite a spectrum of different lifeforms.
Sure, you say – that’s real entertaining, Snapchat, but what value do these frivolous filters truly create?
To be honest, we’re not entirely sure as well – but we daresay that having such a large audience upon which to perfect your machine learning algorithms would go a long way towards helping you to perfect your facial recognition technology.
And this, as we all know, might find some non-frivolous applications in such fields as security and forensics.
And this should give you something to think about late at night.
