Shaip
IoT Ecosystem for Smart Home Aging in Place
Proposing a senior care ecosystem within Samsung SmartThings that focuses on the well-being of seniors and offers peace of mind to them and their loved ones.
Project Objectives
My Role
Schedule
Project Challenge
-
Propose a senior care ecosystem in Samsung SmartThings with a proof-of-concept for gathering valuable information about daily activities from minimal IoT devices.
-
Develop compelling senior care use-cases for differentiation from competitor home automation services.
-
Develop a project vision with a proof-of-concept to evaluate the company's IoT integration capabilities, aiming to extend senior care services with natural language AI.
-
UX Designer and Researcher and co-led project management at Samsung SRA, Think Tank Team (R&D lab in Mountain View).
-
Part of a R&D team consisting of 2 UX designers,1 project manager, and 5 developers.
-
Led comprehensive UX research, encompassing user surveys, in-depth interviews with a group of older adults, persona development, user flow creation, and user scenario design.
-
Participated in interaction design and information architecture.
-
Overall 4 month duration January-April 2023
-
Kickoff and Initial Planning in December 2022
-
Research, Design, Development from January 2019
-
Testing Sensor, ML Reading in April 2023
-
Approval delays for external interview and testing due to confidentiality.
-
Feature changes due to limited resources, time, and technical limitations.
-
Creating a new product system tailored for senior users within the framework of the existing business model.
Missing opportunity
Smart IoT Solutions for Enhanced Senior Care Ecosystem
By integrating smart IoT devices and sensors, the aim is to extend the Senior Care ecosystem to SmartThings. Emphasizing the significance of Activities of Daily Living (ADL), the proposal suggests enhancing ADL analysis through data integration from smart appliances.
Our Mission
Propose new use case possibilities involving ADL reading and analysis that can extend the Senior Care ecosystem to SmartThings, supporting the independent living of older adults.
Shared Smart Home Control with Collaborative UI
Enable collaborative smart home management and monitoring through multi-home profiles. Streamline smart appliance setup and maintenance to foster control over data and environment. Implement privacy-preserving data analysis for enhanced security.
Leverage Universal UI for Older Adult's Proactive Engagement
Foster seniors' proactive engagement through an interactive interface with visual and audible feedback. Improve accessibility with voice and visual interfaces on devices like TV, tablets, or smart screens. Utilize conversational AI for proactive assistance, ensuring active engagement and easy control.
Proactive Wellness and Accident Prevention for Older Adults
Support independent living for older adults with proactive accident prevention. Track vital signs, monitor daily activities, and provide personalized recommendations using an intuitive map and device user interface. This reduces the risk of falls and adapts user behaviors for seamless automation.
Work Process
The project began with a strategy proposal for a senior care IoT service, requiring extensive UX research to identify unmet needs and validate the concept. I worked closely with a senior product designer on researching and brainstorming approaches for designing interactions with voice AI technology. Due to limited resources and time, our team decided to focus on proposing a UX vision direction while testing technology availability with developers. We then planned to integrate the prototype with the machine learning model at a later stage.
01. Demographic Research
Addressing ADL Needs in an Aging Population
Our team researched demographics and found that due to longer life expectancy, more people preferred to age in place. However, there was a shortage of caregivers, and younger family members could not provide enough support. The primary need was addressing Activities of Daily Living (ADL). It was essential to find a solution to assist seniors in meeting their ADL.
02. Competitor Research
Analyzing Current Services
Examining the functionalities of Samsung SmartThings and competitor IoT apps was crucial for a comparative analysis and uncovering new opportunities in senior care services. While some apps prioritized IoT connectivity, they often lacked comprehensive support beyond emergencies, mainly focusing on family caregivers. In the ever-evolving IoT landscape, the emphasis shifted from mere connectivity to smooth device operation, scalability, and user experience.
One of the significant features of SmartThings was its ability to create harmonious automations and routines, particularly with Samsung's various appliance portfolio. We discovered that incorporating senior ADL analysis into these services could provide significant benefits.
03. Internal Survey
Uncovering Key Aspects of Senior Care
Due to time constraints, we started with questionnaires with 11 employees in caregiver roles to anticipate interview topics. The online survey delved into user challenges, daily routines in senior care, and problem-solving approaches. The results guided us in pinpointing areas for focus, testing hypotheses for user services, and discovering potential new opportunities.
04. Affinity Diagram
Categorizing
Senior Care Features
We organized our quantitative research findings through affinity mapping, which helped us identify the key feature categories of senior care for our proposals. We categorized these into maintaining senior's environment, healthcare, preventing and detecting catastrophic events, working around physical limitations, and visibility and sharing of status metrics.
05. Focus Group Interview
Navigating Senior Perspectives on Smart-Home Solutions
Our aim was to thoroughly investigate the experiences and requirements of older adults and caregivers, in order to identify features that would fully address their challenges and needs. We conducted two Zoom-based focus groups, one with seniors (8 participants) and another with caregivers (7 participants). The seniors mostly fell within an age range conducive to independent living, while caregivers shared their perspectives based on experiences with older relatives who generally couldn't live alone.
*Redacted photo
Key Takeaways
✓
✓
✓
Both groups showed interest in a smart home system analyzing behaviors and providing alerts.
→ Seniors want holistic wellness data but with control over sharing.
→ Caretakers are eager for data sources for peace of mind.
A successful senior care platform should gain enthusiastic adoption from seniors.
→ Capture users early in the aging process and gradually shift focus to caretakers' needs as seniors' independent living diminishes.
Both groups expressed reservations about a smart-home data analysis system.
→ Privacy concerns were a common worry.
→ Skepticism was voiced regarding the system's intelligence in understanding their unique circumstances.
→ Value was considered limited if the system flagged problems already known or lacking solutions.
06. Expert Interview
Overcoming Stereotypes
Our team consulted with a strategic advisor from a senior care public benefit corporation to validate our smart home automation hypotheses for supporting ADL requirements. The advisor's insights into senior users' behaviors and interaction needs guided our service direction.
Key Takeaways
✓
✓
✓
Tech companies typically fail in senior care product design by focusing on younger caregivers.
→ Older adults want autonomy and a user-friendly interface for control over technology in their homes.
Personalization is critical- each household is unique with different needs and wants.
→ Seniors are more tech-friendly than assumed, given a user interface designed with their needs in mind.
Tech companies often stereotype older adults as disabled, but many are healthy and want solutions beyond age-related health issues.
→ Success requires complete solutions, achieved through a dialogue to understand users' root causes.
07. Usability Research
Usability testing for an IoT data mining prototype
We worked with developers to build a prototype for mining data from IoT devices in senior housing. Our goal was to create a straightforward and user-friendly system that allows seniors and caregivers to specify how data should be visualized using natural language. During the testing phase, we found that older adults preferred actionable insights in addition to data highlights. We faced the challenge of tailoring suggestions and notifications to accommodate diverse comfort levels with digital applications.
08. Research Analysis
Unmet Needs for Senior Care System
Multi-Stakeholder Value
✓
Because aging is a progression rather than a condition, a senior-focused tech solution needs to evolve along with the user.
→ A successful platform might start out providing everyday value to the senior and then evolve to address more caretaker concern over time.
Anomaly Detection and Long Term Trend Tracking
✓
✓
✓
Existing literature on ADL classification and monitoring in smart homes often uses a limited sensor suite.
The SmartThings ecosystem, particularly its appliances, provides more ADL-specific data.
→ Refrigerator door actions are specifically related to eating/drinking activities.
→ Washers, dryers, dishwashers, etc. can report their status and usage, providing specific activity data.
Integrating smart appliance data enables richer ADL analysis beyond current research.
Need of Dialogue
✓
✓
✓
Understanding the root cause of an issue is key to solving it. and this requires a conversation.
Consumers value continuous physical and mental wellbeing throughout their lives.
→ To some extent, smart-home monitoring and a dialog with AI conversational agents are simply tools that contribute additional data about wellness.
The value of the digital agent depends on its evolving role with the aging adult.
→ Should be easy and intuitive to use.
→ Utilize vocal biomarkers to track depression during conversations for emotional wellbeing monitoring.
→ Implement a lifelike interaction agent for dialog in understanding smart home behavioral trends.
→ Act as a companion to address loneliness, particularly in isolated or dementia-affected individuals.
Design Challenge
Empowering Seniors and Providing Peace of Mind for Caretakers
Addressing the challenge of transitioning from senior user management to caregiver oversight involved ensuring seniors maintained control over data sharing preferences. Simultaneously, the system had to offer valuable insights to caregivers, meeting the unmet needs of both seniors and caregivers.
09. User Persona
Deepening User Empathy
Personas were created to capture the personal traits and insights of seniors and family caregivers, which helped identify higher-level feature needs.
Mary's User Value (older adult)
✹ Reduced home maintenance burden
✹ Improved health and wellbeing
✹ Enhanced social connections
✹ Increased safety and security
David's User Value (family caregiver)
✹ Availability to monitor Mary's home and well-being with less intriguing privacy
✹ Less concern about Mary's falls and injuries
✹ Feeling like he can help remotely
10. User Journey Map
Understanding the Journey
Two persona journey maps were created to reveal emotional nuances in daily activities. The senior individual aims to reduce the family's load and seeks independence in handling daily tasks. Meanwhile, the family caregiver desires the ability to monitor their loved ones and prefers notifications or actionable insights when they are unable to maintain constant attention.
11. Sketch Wireframe
Control Home Together
The design team created a conceptual sketch for an interface that connects seniors with their caregivers, encouraging them to maintain a connection despite physical distance.
Shared Dashboard
The Shared Dashboard prioritized privacy protection by utilizing IoT-connected sensors in a senior's home, ensuring a continuous connection while presenting only filtered information necessary for their well-being. By monitoring the senior's daily patterns over an extended period, the service analyzed and evaluated data, creating a summary of events and trend reports related to health and safety while upholding privacy.
Family Account Setup
Seniors often faced challenges when it came to setting up new technology and navigating unfamiliar interfaces. To address this, we came up with a solution: a single family account that could create multiple profiles for their loved ones. This simplified approach eliminated any complications and made it easier for family members to remotely set up or upgrade devices and appliances.
12. Design system
Enhance Senior Interaction through a Universal User Interface
A user-friendly and intuitive interaction method was crucial for seniors to use a system for an extended period with a sense of control. The envisioned interface seamlessly integrates visualization and interactivity for managing home maps, appliances, and devices. It provided user-friendly visual and audible feedback, with accessibility for diverse needs. Tailored features, like Bixby voice interaction for visually-impaired users and a visual interface for those with hearing issues, enhanced usability. The design ensured a cohesive experience across various platforms with its fluid system and responsive components.
Universal UI (TV/ Tablet)
✹ Voice conversational AI
✹ Speech to text
✹ Dialog flow Q&A conversation format
Visual Interactive Contents
✹ Analyzed graphic chart
✹ Visuals for displaying changes over time and time-series data effectively.
13. Story Board
Story Telling Features
Based on the requirements and features that we proposed, the UX design team crafted a storyboard outlining crucial use cases derived from user needs. It served as an internal tool for stakeholders to better understand the requirements and facilitate high-level discussions. Once the requirements and storyboard were aligned with the teams, the engineers began planning the development process and testing sensor readings and analysis.
Design Mission
Developing seamless interaction with dialogue
Designing systems that effectively convey valuable information has been a challenge. We have explored new approaches to interface design for seniors by recognizing the effectiveness of language-based interactions and utilizing Natural Language Processing (NLP).
14. Flow Chart
Navigating the Shaip Experience
To understand the essential steps for a Shaip experience, we developed a user flow chart outlining the user journey. This visualization underscored key steps, high-level tasks, and specific sensor/data connections to enhance the overall user experience. The primary use cases included ADL monitoring for healthcare, home environment monitoring for safety, and fall prevention.
ADL Monitoring Health Care
Home Safety for Peace of Mind
Fall Prevention
15. Flow Chart Summary
Organizing Key Application Features
After examining the flow chart of each use case, we identified common themes across all stages. As depicted in the chart below, we have categorized the key application features according to each stage of the customer experience.
Stages
Monitoring
Measuring / Tracking
1st response to user
2nd response to user
Report history
Applications / Experiences
Mobility/route trajectory monitoring
-
Analysis of location, duration
-
Mobility route traffic
Home appliance activity monitoring
-
Kitchen appliance usage data
-
Water dispenser activity data
-
Cleaning appliance usage data
Daily activity analysis
-
Mobility analysis
-
Essential daily routines checkup
-
Check the living environment condition
Notification / alert
-
Daily schedule / missing routine reminder
-
Home safety / maintenance alert
-
Emergency contact / caregiver connection
→ Give immediate urgent alert to support for a loved one and give peace of mind.
Contextual recommendations
-
Encourage water and healthy meals
-
Encourage physical activity
-
Guide safe living environment
→ Provide actionable recommendations for personalized support.
Extended 3rd party services
-
Food service, online grocery shopping
-
Home maintenance service
-
Medical support
→ Connect outsourced physical ADL support to reduce the pressure on reaching out to caregivers.
Visualize history of activity & long-term trends
-
Activity trends (day, week, month)
-
Predictive risk analysis
-
Share reports to caregiver
→ Find and track unusual points, and share a
visualized exploratory data analysis with caregiver.
16. Information Architecture
Timely-manner Support and Conversational AI Guidance
We enhanced the conversational AI, showcasing a use case focused on timely support linked to activity trends. The initial response follows a Q&A format with yes, no, and ok for quick solutions, supported by a visual UI. The subsequent response suggests actionable tips and facilitates connections with third-party services for physical support. Finally, the system sends reports with trends and support recommendations to caregivers, offering adaptable and proactive assistance based on the user's preferred level of detail.
My Take Away 📝
What I learned
Next Steps
Looking back, I was fortunate to engage in extensive user research, typically reserved for A/B testing in fast-paced work environments. This brought valuable insights into senior care. During this research, I encountered numerous IoT devices designed for seniors, primarily focused on detecting catastrophic events and providing notifications—yet, they often fell short of addressing fundamental concerns for seniors. Many companies tend to prioritize younger adults as the primary users, assuming they will purchase the service and oversee seniors. It is crucial for these companies to understand the true client base and pivot their targeting strategies, considering the increasing life expectancy and shifting demographics.
Creating a user experience with new technology to support seniors, providing peace of mind and fostering connections within society, was a rewarding experience for me. I am committed to persistently developing a universal design experience that takes into account the needs of all users.
Unfortunately, the project was put on hold during the planning of testing sensor installations in a focus group of senior living environments due to time and resource constraints. If there is a chance for further improvement, I would be interested in observing test results to validate our UI prototype and implement enhancements.
Developing the information structure for conversational AI presented a new challenge for me. I am keen to learn and explore in detail how dialogue interfaces should be crafted to deliver a natural interactive experience to users while considering psychological aspect.
Peer Review
