Move.Play
Video call and play with your loved ones.
01
A new way to play our very first game
Play Peek-a-Boo as if you were sharing the same space. Move and hide for a “feels like you’re there” experience.
02
Move and Play
Move your body to hide behind objects, find others, or reveal yourself.
03
Multiple forms of interaction.
Multi-modal interaction enhances the experience and improves engagement.
Overview
Role - Solo Project
Interaction Designer, Researcher, Coder
Toolkit
DESIGN: Figma, Ps, Ai, After Effect
PROGRAMMING: WebRTC, Javascript(Three.js, Socket.io, ml5.js)
Highlights
Part of my thesis project on Body-centric experiences for the web
Duration
5 months
▦
Design Process
The design process was conducted through Human-Centered and Prototype-driven framework. Methodologies include:
In-depth research
Define context and users
UX and UI practices
Functional prototyping
Testing and Improvements.
-
Discovery
User Research
Technology Research
Context Map
Precedent Analysis
-
Define
Personas
(Children & grandparent)User Empathy - Interviews
(Educators and Parents)Problem Statement
-
UX Practice
Sketching
Storyboard
User Flow
Information Architecture
-
Prototyping
Lo-Fi & Hi-Fi Wireframes
Functional Prototype
User Testing
-
Making Improvements and Testing
Usability Testing
Observations/Interviews
Discovery
Research the Problem
Children are engaged in long-distance communication more than ever.
However, these digital interactions have developed in ways that mostly constrain the contextual physicality surrounding them, challenging the way they shape relationships: through play.
The context map is around the subjects of play and long-distance real-time communication. Kinesthetic interactions and play allow spontaneity, stimulate multiple modes of interaction, and social engagement.
User Research
Interview experts - I spoke with teachers, education therapists, and parents about the challenges they perceive in the relationship that children have with digital devices.
Surveyed Parents - A survey conducted on more than 15 parents, revealed that the lack of social interaction was a concern and they would like their children to use digital devices in a more creative and dynamic way while engaging more in meaningful connections.
From these insights, I created two personas: the grandma and granddaughter.
Define
How might we create a meaningful interaction that supports play and physical engagement in video calls?
My goal was to explore how to improve the experience of interacting over digital applications for children and their families.
Ideation
Peek-a-Boo is a important game for infants' development, who play this game instinctively. Here, the game is used to enhance social and playful interaction for video calls.
Storyboard
The storyboard goes through the video call and game experience. Each user is on the “other side” of the virtual toy room and can use their body to move, hide and reveal themselves.
Design and Sketches
This work explores physical body movement in digital spaces. The idea is to elude the flatness of the screen through motion and the perception of perspective. The level of participation and repetitiveness can vary depending on the age of the child.
Application Diagram
By laying out of the Information Architecture, I establish how the interaction would unfold, from the Landing page and onboarding process, to adding new contacts and starting the video call and play experience.
Visual Design and A11y
During the design process, it is important to keep accessibility at the forefront. When designing the User Interface and visual design, I use tools like Adee in Figma to evaluate my decisions. In the images below, I’m evaluating the button states for different forms of color blindness.
User Testing
The goal of user testing was to gain insight into the type of kinesthetic interactions that children prefer.
Each testing session was conducted remotely in a home environment with their personal computers and their standard setup. After discussing the plan with the parents, I introduced the children to each of the activities.
What Children are Saying
“It would be a good idea to be able to move the toys to create spaces to hide and find other people”
— 5 years old playtester
Findings
Peek-a-Boo has a moderate level of physical engagement. I noticed that children reach for the mouse or touchscreen expecting more ways interact with the scene.
Insight
Future iterations could make use of multimodal interaction, where the children can use their bodies to move, but also tap on objects and use gestures to perform the action of hiding. It is important to leverage multimodal interaction while balancing discoverability and difficulty.
User Feedback
New Interactions + New Possibilities
During playtesting, I noticed a correlation between digital engagement and physical demand. Children change the way they interact with the app depending on the amount of physical activity. Physical activity seems to influence how much they look for other ways to interact.
If the activity is passive in terms of the physical interaction, the child would tend to try to find other digital interactions in the game.
The diagram shows the balance between the digital and physical experience based on level of complexity. The level of complexity is determined by the amount of physical vs digital interactions.
-
Move Around
Move in front of the screen to hide.
-
Gestures
Use your hand to create hiding experiences.
-
Tap and Move
Move objects to find people or create new hiding places.
Play Flow
It is possible to create a video call experience that reinforces the unique way that children learn and explore the world through play and social relationships. This flow shows the integration of Peek-a-Boo into the video call experience.
Low Fi wireframes of play.move app
Let’s Start
Open the application, create your account and start connecting with friends and family.
Onboarding Process
The onboarding will guide the user through the interactions, metaphors, settings and setups.
Video call with your loved ones.
Video call your loved ones with an easy-to-use interface.
Let’s Move and Play
Jump into play mode in the same conversation. Invite other participants, change the game .
▦
Functional Prototyping
The system is implemented in javascript and makes use of ML5 (a machine learning library based on Tensorflow), WebRTC, three.js, and socket.io. The system allows users to access the app from different devices while connected to the web.
Prototyping process in Three.js using ml5
Prototyping process in Three.js using ml5
Prototyping process in Three.js using ml5
Prototyping process in Three.js using ml5
Experience Prototyping
The goal of the functional prototype was to safely test the experience with children and families during the Covid-19 pandemic. This meant it had to be remote.
Technical Feedback
Since children do not have a clear awareness of how the system is tracking and are dynamic with their movement, the system is not always able to follow, which leads to imprecise interaction. Fast and small movements also challenge the system. This led me to understand that more robust tracking is necessary when it comes to children and body tracking.
Video recording of a playtesting session.
Lessons Learned
Since Peek-a-Boo is a game that evolves throughout childhood until it turns into hide-and-seek, I would like to develop further multi-modal interactions, including moving objects in the scene and hand gestures.
Conclusion
The combination of kinesthetics and digital play could enhance the quality of experiences in the life of a child.
This exploration suggests that children are open to exploring new ways of interacting and playing with digital interfaces, making it possible to create an interaction system that reinforces the unique way that children engage with the world.
Acknowledgments
Thanks to the families that offered their time to playtest these experiences. Thanks to Madeline Gannon for all her guidance and feedback during this research and Aidan Nelson for providing open-source for real-time social environments on the web. Additional resources and credits are detailed in this medium post.