The MTN Giant Emoji
For MTN’s 2017 end-of-year party, client wanted an engaging activation to bring an interactive element to the event.
Enter, the Giant Emoji…
Brave was tasked to implement the Giant Emoji project developed by Jonas Jongejan & Stewart Smith as an Android experiment for the Google Creative Labs initiative. This was to serve as an activation installation, but, because of the timeframe and lack of hardware, we decided to convert the project to use the standard Openframeworks toolkit that would be able to export the project as a desktop app.
The functionality and technology behind the app is best described in this paragraph quoted from the original developers:
“The app is running several algorithms to detect facial landmarks, and run sentimental analysis on these. The openFrameworks app delivers these landmark points and sentimental analysis results via JavaScript injection into a local WebView and/or over WebSockets to a remote browser, ready for HTML5 canvas animation. Within a local WebView or remote browser a Paper.js app creates the persistent emoji vector objects for animation upon initialization. A lightweight JavaScript animation loop parses the incoming JSON facial landmarks and models the corresponding X/Y coordinates, viewable in debug mode. The landmarks and confidence scores are then translated into preset emoji models (such as kissy face, cheesy smile, etc.), augmented by live expressions like mouth shape or lip geometry.”
How we did it
The desktop version of this application was created by creating a basic Openframeworks MacOS app, and refactoring the code used in the android app to work with the default branch of Openframeworks, as apposed to the Android specific fork used for the android application.
Because the original application depended on the ofxAndroidMobileVision library that used the Google Mobile Vision API to detect eye blinks and other facial features, custom logic had to be written to replicate this functionality for the desktop app.
Technologies
- Openframeworks
- Nodejs
- Paper.js
- Computer Vision (ofxCV)
- Facial Recognition and Tracking (ofxFacetracker2)
Links
And if you’re in the mood for more, here’s a cool video from Google’s Jonas Jongejan showing the Giant Emoji in action:
