Snapchat Filters

Snapchat filters are incredibly cool and way too much fun, but how do they work? Well, through with application called Computer Vision, they can use pixel data from a camera to identify objects and interpret 3D space. As you can see below, my friend Leah and I love snapchat filters and are so glad Snapchat paid $2.5 million to get the technology for them!

How Does It Work?
A computer cannot detect facial feature like the human brain can, so the program first step the computer does is detection. It uses the Viola-Jones Algorithm to repeatedly scan the image, getting data in numbers that tell it about the pixel shade. It looks for different shades of color and compares the light and dark parts of the image. For example, the bridge of your nose is lighter than the area around your nose, and your eye sockets are darker than your forehead.

To find specific features of the face, an "active shape model" is used. This is a model of the face that has been made manually by people who have marked the borders of facial features on hundreds of thousands of sample images. The take the average face from the data they've collected and align it with the image of your face on your camera. They tweak the pixel data and adjust the points by once again examining the brightness and darkness of the pixels around your lips, eyebrows, etc., as you can see above. Once they've perfected the points, they connect them to make a mesh, which is a 3D mask that can move and rotate with the movement of your face (right).


So interesting! Check out this cool vid for more! http://www.digitaltrends.com/photography/snapchat-selfie-lens-computer-vision-technology/


References:
http://www.digitaltrends.com/photography/snapchat-selfie-lens-computer-vision-technology/
Snapchat