Jan - May 2025 | UX Researcher
OVERVIEW
Public-access automated external defibrillators (AEDs) are critical, life-saving medical devices that broadly guide users through the steps to provide an electrical shock to the heart to restore normal rhythm following sudden cardiac arrest (SCA) in an emergency situation. They also provide varying levels of instruction for cardiopulmonary resuscitation (CPR). Despite their importance, no systematic in-situ usability testing of these devices has been conducted, particularly with untrained users. Usability differences between devices may affect their real-world deployment during emergencies.
To address this, I partnered with Philips Healthcare as part of the University of Washington direct research group led by PhD Ruican Zhong to work on comparative usability study of 4 on-market AED devices with 89 participants, to surface current usability issues and design optimization opportunities for Philips products.
CONTRIBUTION
I worked on the data analysis of this study as an UX researcher alongside 6 researchers from UW, and contributed to the final analysis manuscript that was submitted to emergency medicine and healthcare related journal.
WHY THIS STUDY
Although some studies in the past have examined the usability and accessibility of public-access AEDs on stimulated patients (e.g. dummies and manikins), that overly complex and confusing user interface, distracting graphical, audio instruction, or components have negatively impacted user behavior in terms of speed, accuracy, and ease of use, they were all conducted in quiet lab-like settings where there were no distractions or bystanders as in a real-world setting.
This study was conducted in a way that stimulate rescue in a real-world environment to understand how untrained bystanders would interact with AED devices under stress and in the presence of eternal distractions, to further highlight implications for how device feature design impact user behavior.
CHALLENGES
Large dataset with a tight timeline
With 89 participants, each session generated a usability testing video, a 30-minute post-interview recording, images of pad placement, likert scale notes, and an Excel sheet. Working through this data across 4 different devices, taking notes, calculating time spent, revisiting past sessions to check for anything missed, compiling notes, and cross-referencing across participants, all while keeping wording consistent with other researchers, was time-consuming and challenging.
I developed a template for quickly logging keywords and timestamps from the videos, making them easier to categorize and cross-reference during analysis. Since we regularly moved between different devices and participants, I also added context summaries so other researchers could quickly understand each session. This sped up the analysis process and made our workflow more efficient.
Quantifying and aligning across different devices
Analyzing four different devices and quantifying metrics like time spent and task completion rates was difficult to compare, since each device delivered voice instructions differently, at different points, and with more or fewer required steps.
To address this, we broke down each device’s workflow into aligned task segments (device activation, user’s first interaction attempt, start of audio prompt, second pad placement, end of shock delivery audio prompt, user’s first chest compression) common across all devices. This alignment made it easier to identify corresponding points between devices and calculate time spent between tasks.
PROCESS
We started with scattered data across participants and different devices with multiple synthesis.
I was in charged of working on device design analysis, while working on this section it was a bit overwhelming at first as lots of comments were confusions about liner use (the sticker that needs to be peeled off from the pads), the problems were clearly stated, but the more I reviewed these verbatim with the videos and interviews, I started to notice "liner design" was not the sole problem.
Here's an interesting break down:
⭐️THE AHA! TRAIN OF THOUGHTS
What is the root cause here, really?
1. Visualizing this confusion…
For the Heartsine device, the two pads were attached on the same liner on the same side, causing confusion of where or how to place or separate them. Many participants were witnessed placing the one-piece pad directly on the manikin without peeling off the liner, then when it did not stick, they looked confused then gradually started to realized they had to peel the liner off. And that made me think, why is that? Was it only about the defective design of the liner or pads?
2. How AED works
2. How AEDs work and assist users
To successfully assist users, especially untrained bystanders, in operating AEDs, audio prompts, graphic instructions, and device design with clear visual cues must all work together to guide users step-by-step, without extraneous features that draw unnecessary attention. Now, although the physical design was clearly what confused users, not realizing they needed to peel two pads off the same liner, something was missing here: what about the instructions? Why weren't the instructions at the moment filling that gap?
2. How AED works
3. How the user's mental model reacted
Many users reacted in almost the same way. They did not expect the one-piece liner design, where two pads share a single liner instead of each having its own, so they paused and examined it, looking confused. Based on their mental model of two separate pads, they tried tearing the liner down the middle to split it in two. When that didn't work, they simply stuck the entire one-piece liner onto the manikin. Since it wouldn't adhere properly, this left them even more confused, breaking their workflow and adding to task completion time.
Additionally, even when users did peel off one pad correctly, no instructions clarified that they still needed to peel off the remaining liner attached to the second pad.
2. How AED works
3. The actual root causes: simultaneous factors
Device design vs. mental model
Users did not expect the one-piece liner design, where two pads share a single liner instead of each having its own, a mismatch with their mental model of how AED pads should be packaged.
Lack of clear instruction
No clear instructions clarifying what users should do with the pads, most instructions simply tell users to "peel the pads", when it comes to design like one-piece pads, no instructions are made for the remaining pad on the same liner.
Unclear visual cues/affordance
The tabs on the liners were intended to serve as affordances, guiding users to peel the pads. However, they were not noticeable, and there were no clear instructions specifying where or how to peel the pads from the liners.
RESULT
(Objective task performance measures and subjective usability ratings were collected and analyzed using one-way ANOVA with post-hoc testing)
Significant differences were observed across devices, particularly during pad placement, shock delivery, and initiation of cardiopulmonary resuscitation (p < 0.0001). Devices providing clearer visual and verbal instructions were associated with faster task completion, higher accuracy, and greater user confidence.
REFLECTION
Dig deeper for the root cause!
Sometimes the obvious answer isn't the whole answer, we hear it and immediately assume that's the problem. I ran into several moments like this during the study. Learning to grasp the meaning behind participants' words, digging into the implications, and unpacking their mental models was genuinely fun. It also taught me to look past the surface and refine my understanding of the real problem.
Visualizing my research analysis process
There were many times I needed to explain my thought process and findings to the rest of the team. At first, I found it hard to communicate without losing context, since the study involved so much visual material and data. I've always been a visual person, and this study showed me that research can be visual too. I started sketching out the problem causes and my thought process to give the team clearer context and a shared reference point, and it turned out to be incredibly helpful!
back to top