XR KITCHEN
Creating a kitchen that is accessible for people in wheelchairs by utilising computer vision and mixed reality headsets.
Motivation
I decided to use my knowledge of Interaction Design and Creative technology to create a kitchen that is accessible for people in wheelchairs.
The goal is for people like my mom, who has polio and uses a wheelchair, to be able to prepare their own meals safely and easily. This project was inspired by a personal experience where my mom couldn't access the kitchen to make meals.
Afterall, the ability to make yourself food is a human right.
What I did
- Conducted research on Machine Learning and XR
- Created a system map of the kitchen and its objects
- Designed the user interface for the headset and mobile
- Prototyped the interaction in Unity
- Conducted motion study analysis with the motion
capture suit
" Within this project, Lucas used rigorous technology centered research to discover how mixed reality technologies will enable the future tracking of everything and then distilled her insights into a well-considered user experience and visual language that clearly speaks to the intended user. "
Jenny Roddenhouse, Associate Professor and Faculty Director of Immersion Lab
Key Features
Safety Warnings
Kitchens can be hazardous for wheelchair users due to hot surfaces and sharp objects. To help prevent accidents, I used visual and auditory cues to warn users when their hands were detected in hazardous areas.
Action Prediction
It can be beneficial for a computer to predict the next move of a wheelchair user in the kitchen because it can reduce the amount of motion they need to make, helping them stay more alert and aware. After being trained on the user's movement patterns, it can also suggest ways to reorganize the kitchen for further efficiency.
Peer Guidance
Having a wheelchair-using guide in the kitchen can be beneficial because they can provide valuable tips and tricks for safer and more efficient navigation. For example, they can show the best way to reach for items on high shelves and how to avoid hot surfaces and sharp objects.
Enhanced Vision
One reason kitchens can be inaccessible is that the lower point of view of wheelchair users can make it difficult to see through opaque pots and pans, leading to food burning or undercooking. This can be frustrating and potentially dangerous. Reaching for items out of sight can also strain their arms and make the kitchen less safe overall.
Learning Points
01. Motion as Research
02. Think like a Machine
03. Prototype for Inspiration
04. Design can be Personal
01.
Motion as Research
Motion capture suits can track a user's movement patterns in the kitchen.
I used perception neuron suits to record my motion while cooking in the kitchen. I then imported the motion clip into Unity and used the trail renderer component to visualize the common paths of motion.
Video recording from multiple angles was used to identify repeated motions.
I used perception neuron suits to record my motion while cooking in the kitchen. I then imported the motion clip into Unity and used the trail renderer component to visualize the common paths of motion.
02.
Perceiving like a Machine
Mapping relationship
between objects
The goal of technology-centered research is to understand how the technology can work in the background before determining its potential value to the user. To begin, I studied the relationships and states that objects can have with each other.
Mapping relationship between objects and space
After creating a diagram of movement in the kitchen, it was important to also map which items belonged in which part of the kitchen. This helped to identify any organizational inefficiencies and provided insight into the psychology behind the user's behavior when organizing items.
Mapping relationship between objects,space and context
After I established the relationships between objects and the space, it was time to consider these relationships in the context of cooking. Cooking typically involves five stages: storing ingredients, preparing them, cooking, serving, and cleaning.
Analysing micro motions
I created micro motion diagrams to understand how the system would analyze the different states and relationships of objects during a complete motion, such as searching, cutting vegetables and frying. These diagrams helped me to study how the system would function.
03.
Prototyping for Insights
Understanding what AI can learn can provide inspiration.
By training gestures and objects using Runway ML, I was able to develop the idea that the system could predict the user's next move and learn more about their behavior.
Prototyping in Unity inspired the use of trigger zones.
One thing I learned is that tools can also inspire us. I knew that Unity has colliders and trigger zones, so I thought about using these features to display warning zones. If the system can predict the user's movement, it can also turn colliders on and off based on whether the user has completed previous actions in the micro motion flow. These collider meshes can also serve as visual warnings for the user.
04.
Design is Personal
Reimagining the design for a
different audience.
Initially, my designs were very average and followed the conventions of XR. The UI would have included arrows and floating holographic text. However, after realizing that the user was my 65 year old mom, who is not familiar with text but used to rely on books for information, I decided to completely change the design to better suit her past and culture. This involved a radical redesign.
Details add a personal touch.
I enhanced the mobile app interface with textures and different interactions. To add a personal touch and make the models more meaningful, I used a watercolor shader inspired by my grandma's love of watercolor painting and the popularity of this style for illustrations in China at the time. The resulting system combined modern and traditional elements, creating a familiar experience for the
target audience.
Future Improvements
- Experimenting with more prototypes
- User testing the prototypes
- Conducting qualitative research with users