Lessons from Real-World Device Testing

Design teams work hard to understand workflows, anticipate user behavior, and build usable devices. But as Eric and Paul discuss in this episode of Before the Build, even the most carefully planned usability work can’t fully replicate what happens in real clinical and emergency environments. The stories they share make the case for why real-world observation matters.

When the Room Changes Everything

One example involves a circulating nurse preparing a specialized cutting device during a glioblastoma resection surgery. The procedure can run five to ten hours. The room is dark. The surgical team is under pressure. The nurse was working through the instructions for use while the surgeon demanded the device immediately. The stress of that moment raised a direct question about usability testing: was this device ever tested in a dark room, under that kind of pressure? And more fundamentally, how do you replicate that level of stress in a simulated environment when a person’s career isn’t actually on the line?

The Gap Between Design Intent and Real-World Access

AEDs offer a different kind of example. Modern AEDs are designed for bystander use, with voice prompts and automation that remove the need to read instructions or press a button at the critical moment. In practice, however, facilities often install AEDs without adequate signage, train only one or two employees on their location, and store them in locked cabinets. The device may be well-designed for use, but the context of use creates its own barrier entirely separate from the device itself.

Assumptions That Don’t Survive Contact with Reality

A third example comes from an epidural observation session. An anesthesiologist described the first patient as a nurse, someone familiar with clinical environments and likely to be cooperative throughout the procedure. The situation turned out to be the opposite, with the patient in significant distress, the procedure completed quickly, and the observation session effectively lost. The second patient, expected to be more difficult, was the one described as potentially ideal. Neither assumption held.

What Edge Cases Actually Tell You

The thread running through each story is that unexpected situations aren’t necessarily outliers. As one host puts it, these edge cases seem to follow you around, which raises the question of whether they’re really edge cases at all. Understanding what happens at the fringes of expected use, alongside the predictable workflow, is what ultimately leads to better products.

What this episode covers

• The limits of design planning and usability simulation: some stressors and scenarios simply cannot be replicated in a lab environment
• A neurosurgery observation where a circulating nurse struggled to operate a specialized cutting device in a dark room under direct pressure from the surgical team, raising questions about whether the device had been tested under realistic conditions
• The AED example: a device designed for intuitive bystander use that in practice is often stored in locked cabinets, without signage, with only one or two employees aware of its location
• An epidural observation session where a patient expected to be cooperative turned chaotic, and a patient expected to be difficult was anticipated to be the clearest case, with neither assumption holding
• The hosts’ view that what appears to be an edge case may actually reflect consistent patterns in how devices get used, and that observing these situations produces some of the most valuable design insights