Do you have trouble expressing yourself without a small picture of a cat? Can’t lol without a tiny yellow face doing it for you? Skip the middle man and get Facelooked – the world’s first human-to-emoji converter – turning you into thousands of tiny animals and similes mashed together in a mosaic of social media madness.
By day, use the Facelook photobooth and have your emoji monstrosity emailed to you; by night, publicly express your mess of emotive clusterings, projected on to the wall for all to see.
- Event
- Kingston Arts Centre Pop-up Bar
- Hardware
- Kinect 2
- Arduino
- Software
- Max/Jitter
How Does It Work?
Using some of the visual processes that I recently built in Max/Jitter, the patch takes a down-scaled video input and replaces each “pixel” of video with the closest matching image from a large library of images. It all happens in real-time on the graphics card – turning whatever the input is into a mosaic make from thousands of tiny images.
With the Kinect 2, the background is masked out using the depth image, leaving only the images of the people in the photo-booth. To keep in the colourful and bright theme of the piece, the background is replaced with a vibrant, evolving rainbow of colour generated using simplex noise. This keeps the piece constantly evolving with a constantly shifting, psychedelic wash of colour.
Image Content
A collection of over 600 emojis is used as the library of images. After each photo is taken, the patch selects 300 emojis at random, making each capture totally unique. Emojis work extremely well; not only thematically (turns out kids absolutely love them!), but visually, since emojis have very little detail by design. The images that replace the original video stream are very small (anywhere from 10 to 30 pixels squared), so images with high amounts of detail are difficult to identify.
Photo-booth
I made a simple button with an Arduino that allows the user to trigger the countdown for the photo. After the image is taken, the user is prompted to enter an email address using a keyboard so their image can be delivered to them. I didn’t get time to make an automatic system for posting the photos, so I had to send the emails manually.
In order to run the conversion in real-time at over 30fps the entire patch has to run at 1280 x 720. However, just before the image is captured, the system switches resolution and stores a super crisp 4k image. This creates a great take-home experience for the users, as they can then crop the images as they see fit – or even print it out.
Night-time
The great thing about running this in real-time, is that it can work both as a photo-booth, but also as an interactive projection. So, as the night set in, we dragged the Kinect out of the booth and projected the real-time emoji mayhem on to the building.
It ended up looking like some kind of freaky 80s/millennial crossover video clip, fusing a green-screen vibe with a social media theme.
Photo-booth Images
Here’s some captures from the day (click to see the 1920×1080 resolution image for more detail of the emojis):
