Mixed Reality Prototypes

1/5 Yummy!

Yummy! is a collection of AR filters to explore playful engagement with digital foods and food exposure. This is part of ongoing research in feeding issues in young people with autism.

Tools: Lens Studio

2/5 Shared AR

Multi-player in Augmented Reality using WebXR API.

After I conducted a research process in distance learning and assessed tools teachers are using in the virtual environment, I noticed that there is space for shared experiences in AR, especially on accessible platforms. This could be used to explore models or shared brainstorming sessions.

Learn more on this Medium post

Tools: WebXR, Three.js, Socket.io

3/5 Animate by gestures

How about easing the process of animation by just animating with our hands? This series of experience prototypes explore the “feel” of animated using hand gestures.

Tools: P5.js, mediapipe / Aero

4/5 Brushing teeth

Learning Through Habits exploration is based on brushing teeth. It is based on a known physical activity, which is augmented through a digital interface and multimodal interaction.

When the child is brushing their teeth in front of the camera, bubbles will appear on the screen, following and moving around their face. They can also pop the bubbles and make them fall.

This experience helps parents control the time and support habit creation.

Tools: P5.js, ml5.js

5/5 Puppet

I used puppeteering as an open-ended toy for freedom of interaction. The character is presented on the screen and the animation responds to the movement of the hand in a fluid way.

The character is enhanced by the actions of its users like the opening of the mouth, winking and hand shapes. The goal is for children to explore their movement while receiving animated feedback.

Tools: Paper.js, ml5.js