Motion Lab: Learning Experience Design
Motion Lab: Learning Experience Design
Motion Lab: Learning Experience Design
01
RESPONSIBILITIES
UX RESEARCH
UX RESEARCH
UX DESIGN
UX DESIGN
UI DESIGN
UX DESIGN
UX DESIGN
DELIVERABLES
Unity Application
OVERVIEW
Making hands-on, biology lab experiment more affordable and accessible
Making hands-on, biology lab experiment more affordable and accessible
Making hands-on, biology lab experiment more affordable and accessible
Hands-on, lab-based science experiments are critical in STEM education but are often expensive and inaccessible, particularly for students with learning disabilities (LDs).
Motion Lab aims to create an interactive virtual simulation of a biology lab experiment that is relatively cheaper, with a focus on an accessible experience for students with LDs.
Our final product is a Unity application that users can interact with through the Leap Motion sensor, covering the main concepts of the micropipette experiment.
Hands-on, lab-based science experiments are critical in STEM education but are often expensive and inaccessible, particularly for students with learning disabilities (LDs).
Motion Lab aims to create an interactive virtual simulation of a biology lab experiment that is relatively cheaper, with a focus on an accessible experience for students with LDs.
Our final product is a Unity application that users can interact with through the Leap Motion sensor, covering the main concepts of the micropipette experiment.
Hands-on, lab-based science experiments are critical in STEM education but are often expensive and inaccessible, particularly for students with learning disabilities (LDs).
Motion Lab aims to create an interactive virtual simulation of a biology lab experiment that is relatively cheaper, with a focus on an accessible experience for students with LDs.
Our final product is a Unity application that users can interact with through the Leap Motion sensor, covering the main concepts of the micropipette experiment.
THE GOAL
Building a prove of concept for scalable system
Starting with Micropipette: essential, expensive, and challenging to learn digitally.
Measuring success through:
Accessibility
Affordability
Gesture transferability.
This is what a Radar look like, but there are more than 20 more them!
This is what a Radar look like, but there are more than 20 more them!
This is what a Radar look like, but there are more than 20 more them!
BACKGROUND
In need of a virtual solution to reach more students
In need of a virtual solution to reach more students
In need of a virtual solution to reach more students
Our client, PittBio Outreach, has developed Pitt-Kits, which include essential equipment and video instructions to help K-12 students perform lab experiments. However, due to the limited availability of these kits and safety concerns associated with certain experiments, PittBio is seeking to expand their reach by offering virtual learning solutions.
Our client, PittBio Outreach, has developed Pitt-Kits, which include essential equipment and video instructions to help K-12 students perform lab experiments. However, due to the limited availability of these kits and safety concerns associated with certain experiments, PittBio is seeking to expand their reach by offering virtual learning solutions.
Our client, PittBio Outreach, has developed Pitt-Kits, which include essential equipment and video instructions to help K-12 students perform lab experiments. However, due to the limited availability of these kits and safety concerns associated with certain experiments, PittBio is seeking to expand their reach by offering virtual learning solutions.
Discover Bigger Picture
Discover Bigger Picture
Discover Bigger Picture
Build Depth for Personalized Path
Build Depth for Personalized Path
Build Depth for Personalized Path
Simultaneously, the Learning Disabilities Association of Pennsylvania (LDA of PA) recognizes the unique challenges students with LDs face in traditional laboratory settings. They see this initiative as an opportunity to enhance educational access for these students through highlighting the accessibility of the new virtual learning solution.
Simultaneously, the Learning Disabilities Association of Pennsylvania (LDA of PA) recognizes the unique challenges students with LDs face in traditional laboratory settings. They see this initiative as an opportunity to enhance educational access for these students through highlighting the accessibility of the new virtual learning solution.
Simultaneously, the Learning Disabilities Association of Pennsylvania (LDA of PA) recognizes the unique challenges students with LDs face in traditional laboratory settings. They see this initiative as an opportunity to enhance educational access for these students through highlighting the accessibility of the new virtual learning solution.
Small, precise movements based on tactile feedback
Operating a real micropipette relies on small, precise thumb movements, with tactile feedback distinguishing the first and second stops.
Small, precise movements based on tactile feedback
Operating a real micropipette relies on small, precise thumb movements, with tactile feedback distinguishing the first and second stops.
Small, precise movements based on tactile feedback
Operating a real micropipette relies on small, precise thumb movements, with tactile feedback distinguishing the first and second stops.
From expensive to affordable
Better understand what students need through research
Better understand what students need through research
Better understand what students need through research
Galaxy Visualization: Enables open-ended exploration, revealing hidden connections through an intuitive, visual map.
Galaxy Visualization: Enables open-ended exploration, revealing hidden connections through an intuitive, visual map.
Galaxy Visualization: Enables open-ended exploration, revealing hidden connections through an intuitive, visual map.
Personal Dashboard: Adapts to individual roles and goals, surfacing timely, relevant insights that support day-to-day decisions.
Personal Dashboard: Adapts to individual roles and goals, surfacing timely, relevant insights that support day-to-day decisions.
Personal Dashboard: Adapts to individual roles and goals, surfacing timely, relevant insights that support day-to-day decisions.
From passive watching to hands-on practice
Based on these research insights, we proposed the following features to make the virtual lab experience more accessible:
Multimodal instructions: Offer multimodal instruction combining video, text, and narration to accommodate varying learning preferences.
Streamlined Interactions: Minimize task-switching by integrating instructions and immediate feedback directly into the workflow to guide user focus.
Open and Observable Environment: Create a shared, open environment where peers can observe each other’s progress, enabling body doubling while maintaining an immersive and realistic practice experience.
Based on these research insights, we proposed the following features to make the virtual lab experience more accessible:
Multimodal instructions: Offer multimodal instruction combining video, text, and narration to accommodate varying learning preferences.
Streamlined Interactions: Minimize task-switching by integrating instructions and immediate feedback directly into the workflow to guide user focus.
Open and Observable Environment: Create a shared, open environment where peers can observe each other’s progress, enabling body doubling while maintaining an immersive and realistic practice experience.
Based on these research insights, we proposed the following features to make the virtual lab experience more accessible:
Multimodal instructions: Offer multimodal instruction combining video, text, and narration to accommodate varying learning preferences.
Streamlined Interactions: Minimize task-switching by integrating instructions and immediate feedback directly into the workflow to guide user focus.
Open and Observable Environment: Create a shared, open environment where peers can observe each other’s progress, enabling body doubling while maintaining an immersive and realistic practice experience.
Traditional lab-based experiments can feel overwhelming due to fragmented instructions, constant attention-switching, and inaccessible teaching methods.
Traditional lab-based experiments can feel overwhelming due to fragmented instructions, constant attention-switching, and inaccessible teaching methods.
Traditional lab-based experiments can feel overwhelming due to fragmented instructions, constant attention-switching, and inaccessible teaching methods.
Technology View
What it does: Groups projects by the technologies they apply or explore.
Hierarchy: Tech field → Specific methods/models → Projects
Why it’s helpful: Ideal for identifying innovation trends, technical overlaps, or evaluating tech adoption maturity.
Technology View
What it does: Groups projects by the technologies they apply or explore.
Hierarchy: Tech field → Specific methods/models → Projects
Why it’s helpful: Ideal for identifying innovation trends, technical overlaps, or evaluating tech adoption maturity.
Technology View
What it does: Groups projects by the technologies they apply or explore.
Hierarchy: Tech field → Specific methods/models → Projects
Why it’s helpful: Ideal for identifying innovation trends, technical overlaps, or evaluating tech adoption maturity.
Domain View
What it does: Groups projects based on problem space within SLB, application area, or impact field (e.g., “Subsurface,” “Grid Modernization”).
Hierarchy: Domain → Product → Projects
Why it’s helpful: Useful for strategists or external partners to see applied impact areas and identify gaps or overlaps in research.
Domain View
What it does: Groups projects based on problem space within SLB, application area, or impact field (e.g., “Subsurface,” “Grid Modernization”).
Hierarchy: Domain → Product → Projects
Why it’s helpful: Useful for strategists or external partners to see applied impact areas and identify gaps or overlaps in research.
Domain View
What it does: Groups projects based on problem space within SLB, application area, or impact field (e.g., “Subsurface,” “Grid Modernization”).
Hierarchy: Domain → Product → Projects
Why it’s helpful: Useful for strategists or external partners to see applied impact areas and identify gaps or overlaps in research.
Team View
What it does: Organizes projects by contributing teams (e.g., AI Lab, Frontend, Robotics).
Hierarchy: Team → Sub teams within a Lab → Projects
Why it’s helpful: Great for internal alignment, performance tracking, and collaboration mapping across the org.
Team View
What it does: Organizes projects by contributing teams (e.g., AI Lab, Frontend, Robotics).
Hierarchy: Team → Sub teams within a Lab → Projects
Why it’s helpful: Great for internal alignment, performance tracking, and collaboration mapping across the org.
Team View
What it does: Organizes projects by contributing teams (e.g., AI Lab, Frontend, Robotics).
Hierarchy: Team → Sub teams within a Lab → Projects
Why it’s helpful: Great for internal alignment, performance tracking, and collaboration mapping across the org.
From passive watching to hands-on practice
Due to the limitation of time, we didn't get to implement all the features. Below are the proposed features that are possible with available data from radar.
What it is like to use a real micropipette?
What it is like to use a real micropipette?
What it is like to use a real micropipette?
Inspiration Prompt:
Inspire user about what question to ask
Presentation of popular questions
Build mental model of how the system work
Inspiration Prompt:
Inspire user about what question to ask
Presentation of popular questions
Build mental model of how the system work
Inspiration Prompt:
Inspire user about what question to ask
Presentation of popular questions
Build mental model of how the system work
Intelligent Search & Filters:
Help user ask better questions by:
Evaluating if the system has enough parameters from user input
Ask follow up questions and give suggestions on refinement
Intelligent Search & Filters:
Help user ask better questions by:
Evaluating if the system has enough parameters from user input
Ask follow up questions and give suggestions on refinement
Intelligent Search & Filters:
Help user ask better questions by:
Evaluating if the system has enough parameters from user input
Ask follow up questions and give suggestions on refinement
Issues with translating it to a digital environment
Issues with translating it to a digital environment
Issues with translating it to a digital environment
The interactive timeline serves as both a filter and a line graph that indicates change in activity over time.
The interactive timeline serves as both a filter and a line graph that indicates change in activity over time.
The interactive timeline serves as both a filter and a line graph that indicates change in activity over time.
Moving from real to digital interaction in low fidelity
Moving from real to digital interaction in low fidelity
Moving from real to digital interaction in low fidelity
See projects with related domains, tech, team easily
See projects with related domains, tech, team easily
See projects with related domains, tech, team easily
From passive watching to hands-on practice
Make it bigger to make it clear:
Using larger motions to better distinguish between stops
Make it bigger to make it clear:
Using larger motions to better distinguish between stops
Make it bigger to make it clear:
Using larger motions to better distinguish between stops
Choosing the appropriate medium and technology
Choosing the appropriate medium and technology
Choosing the appropriate medium and technology
visual parameter
Mapped to
What’s needed to implement
Planet size
effort invested (time or money)
Actual effort metrics based on e.g. time and money
Star Brightness
Impact or visibility
quantitive impact score, or stakeholder priority
Rotation speed (within the trail or self-rotation)
Update frequency within stages, major updates
More granular way of documenting updates in projects
Split off trails
if multiple spin-off projects emerged from an original one
More relationship data to indicate relationships between projects
Satellite (Level 3 Category)
work (tech evaluation, partner, spin off) that supported that specific projects
Evaluation of DataOS
Planet shape
Entity type
Project, tech evaluation, partner
Domain star cluster
Product sub concepts
product line-features
Result
After an exploration of different technologies that enable replicating the pipetting motion interaction, we chose Leap Motion for its flexibility to capture a wide range of lab experiment gestures.
After an exploration of different technologies that enable replicating the pipetting motion interaction, we chose Leap Motion for its flexibility to capture a wide range of lab experiment gestures.
After an exploration of different technologies that enable replicating the pipetting motion interaction, we chose Leap Motion for its flexibility to capture a wide range of lab experiment gestures.
Our approach enables adaptive interfaces through conversational feedback. Users can directly express what they want to track or explore.
By entering their goal or question into a prompt input box, the system performs intent tagging to categorize the query and automatically surfaces the most relevant UI components.
Step 1: Intent Tagging: Understanding the ask
On interpreting the user input side, I defined 5 parameters to tag a questions.
On interpreting the user input side, I defined 5 parameters to tag a questions.
On interpreting the user input side, I defined 5 parameters to tag a questions.
For example ☝️
How did I come up with these parameters? Why?
How did I come up with these parameters? Why?
How did I come up with these parameters? Why?
During the user interview, I asked each interviewee: " If you can ask the galaxy page any question, what would you ask?" and looked for recurring patterns in what they were fundamentally asking for.
While an LLM could infer UI responses directly, defining a parameter tagging framework adds a structured, interpretable layer between user intent and system response. Putting LLM on rails as oppose to letting it range free.
During the user interview, I asked each interviewee: " If you can ask the galaxy page any question, what would you ask?" and looked for recurring patterns in what they were fundamentally asking for.
While an LLM could infer UI responses directly, defining a parameter tagging framework adds a structured, interpretable layer between user intent and system response. Putting LLM on rails as oppose to letting it range free.
During the user interview, I asked each interviewee: " If you can ask the galaxy page any question, what would you ask?" and looked for recurring patterns in what they were fundamentally asking for.
While an LLM could infer UI responses directly, defining a parameter tagging framework adds a structured, interpretable layer between user intent and system response. Putting LLM on rails as oppose to letting it range free.
Step 2: UI Matching: How to Answer with Different UI Components
On the system output side, for each of the potential intent, I have a set of corresponding UI components to answer.
On the system output side, for each of the potential intent, I have a set of corresponding UI components to answer.
On the system output side, for each of the potential intent, I have a set of corresponding UI components to answer.
01
01
01
02
02
02
03
03
03
04
04
04
For example, for "Discover" intent, an UI component like this be fetched☝️
Step 3: Curate Dashboard Space with Feedback
For hi-fi user testing, we wanted to make an effort to recruit students who had LDs in order to assess the target audiences' needs. To do this, we created a functional needs survey that asked users whether they identified with common needs that are present in students with LDs, such as ADHD and dyslexia.
We were able to test 5 college students, 2 of which self-reported as having light ADHD/executive dysfunction, and 3 students who scored above average on the survey. None of our users had previous experience with micropipetting.
For hi-fi user testing, we wanted to make an effort to recruit students who had LDs in order to assess the target audiences' needs. To do this, we created a functional needs survey that asked users whether they identified with common needs that are present in students with LDs, such as ADHD and dyslexia.
We were able to test 5 college students, 2 of which self-reported as having light ADHD/executive dysfunction, and 3 students who scored above average on the survey. None of our users had previous experience with micropipetting.
For hi-fi user testing, we wanted to make an effort to recruit students who had LDs in order to assess the target audiences' needs. To do this, we created a functional needs survey that asked users whether they identified with common needs that are present in students with LDs, such as ADHD and dyslexia.
We were able to test 5 college students, 2 of which self-reported as having light ADHD/executive dysfunction, and 3 students who scored above average on the survey. None of our users had previous experience with micropipetting.
Evaluation Highlights:
Evaluation Highlights:
Evaluation Highlights:
Superior accessibility, affordability, and gesture transferability compared to competitors
Superior accessibility, affordability, and gesture transferability compared to competitors
Superior accessibility, affordability, and gesture transferability compared to competitors
During the process of brainstorming how to leverage the power of LLM to empower better search, I talked to a start up company called glean, which provides a software that combines all source of information across platforms and domain and fuse them into smart search that's specific to the company.
Reflection
Final User Testing Result: Successful concept transfer and memory reinforcement through gestures
Effectively transfer conceptual learning from digital to real micropipette.
This project taught me the importance of bridging the senses—finding ways to connect tactile, visual, and auditory cues to help users build fluency and confidence in their learning. It was a delicate balancing price, conceptual understanding, and motion precision, but it was rewarding to see how these elements came together to create a seamless connection between digital learning and real-world practice.
This project taught me the importance of bridging the senses—finding ways to connect tactile, visual, and auditory cues to help users build fluency and confidence in their learning. It was a delicate balancing price, conceptual understanding, and motion precision, but it was rewarding to see how these elements came together to create a seamless connection between digital learning and real-world practice.