5 Things Medical Device Engineers Should Know About User Research
By Debbie McConnell and Scott Danhof, Battelle
When designing a medical device, nothing beats direct observation of, and feedback from, the people who will be using it. Ideally, this research is performed as a close partnership between the human factors (HF) researchers and the engineers who will be working on the device.
Directly participation by engineers in the user research sessions, observing users firsthand, provides a deeper understanding of user needs and user interface requirements than simply reading the final user research report. Building effective partnerships during the user research process also will ensure that engineers get answers to their most critical questions surrounding design decisions.
To get the most out of user research, engineers need to understand how the process works and how they can participate productively. Here are five things engineers should know about user research before starting on their next medical device development project.
1. There Is A Structure To All User Research
The earliest user research often is conducted in the homes of target population patients. Gathering information in a home visit allows researchers to make observations about the home environment, as well as the ways in which patients behave in their environment, that go above and beyond what users actually say.
A home interview may look like an informal conversation; HF researchers work hard to put subjects at ease and establish rapport. However, no matter how unstructured the discussion may look to an untrained eye, there actually is a great deal of structure and skill involved in conducting a successful home interview.
All user research interviews are conducted according to a strict protocol, with specific research questions established well in advance. This protocol is designed to ensure that the research questions the study seeks to answer are addressed during the interview, and that participants are treated according to guidelines established by the Institutional Review Board (IRB) overseeing the study.
During the interview, the HF researcher will have a set of predetermined questions to ask. However, he or she also must be able to follow the threads of conversation as they evolve, know how to elicit additional information when clarification is needed, and understand how to gently steer an off-track conversation back to the subject of interest.
When HF researchers and engineers partner during user research sessions, the engineer must understand the protocol being followed and the research questions to be explored during the interview. Engineers also should prepare their own research questions before interviews begin. During the interview itself, they should follow the researcher’s lead and allow the researcher to guide the conversation with participants.
2. You Will Have A Chance To Ask Your Questions
There will be multiple opportunities over the course of the interview for engineers to ask additional questions. It is best to limit these to one or two good questions at each opportunity to ensure that all topics of interest are covered during the interview.
When asking these questions, it is important to avoid the introduction of bias in the wording of the question. Don’t ask leading questions, such as, “Did you notice the flashing green light that told you the therapy was completed?” Instead, ask open-ended questions, such as, “How did you know when therapy was completed?”
3. Observation Yields The Most Reliable Data
Some of the most valuable and reliable information comes not from listening to what participants say, but from observing what they actually do. The rule of thumb here is to “watch first, ask next.” This is true at all stages of research, from early user studies using illustrations or 3D models to summative HF validation studies.
Participants may tell researchers one thing, but do something quite different when they actually complete the task. Participants are not intentionally misrepresenting facts or omitting information from researchers; humans simply are unreliable narrators — to themselves, as well as to others. They may forget to describe a step that comes automatically to them when they conduct the task, or omit details about their environment or how they interact with a device, because they think such details are not important.
For example, in one study, researchers asked participants to describe when and how they took their medication in a typical day. It was observed that most participants took their medication in the bathroom, even when the participants named a different location. This was an important distinction because the device had to be charged, creating a risk for patients who might accidentally drop the device into a sink or toilet while it was plugged in. This observation led to design revisions to better protect patients from this risk.
In another study, participants were asked to try several prototypes of a drug delivery device during a formative research study. After demonstrating how they would take a dose using each prototype, participants were asked which model they preferred. Researchers observed that the model participants selected did not always match the model they appeared to use most easily and accurately. Reconciling these differences through further questions provided valuable insights that were used to guide the next stage of development.
4. Look For Themes
A single observation may offer a clue into a potential usability issue or user need. But, to have confidence in the results of a user study, researchers must look for trends across multiple participant sessions. While one observation may be an anomaly, a pattern that emerges across four or five sessions strongly suggests that the observation has validity that device designers should take into account.
The number of participant sessions needed to establish a trend with confidence will depend on how strong the pattern is. Eight sessions usually are sufficient to establish a pattern but, sometimes, a strong pattern can be determined with fewer sessions. When the data is inconsistent, you may need 10 or 12 sessions to determine a pattern.
In one formative research study, the first participant was observed holding a prototype drug delivery device the wrong way. When asked how she knew how to hold it, the participant said the prototype was similar in appearance to another device she used. After seeing five participants in a row make the same use error, researchers determined that changes would need to be made to the device’s form factor to break this established mental model, and to ensure safe and effective use.
Not every pattern is equally important. Sometimes, patterns emerge that have no impact on safe and successful use of the device; for example, patterns that emerge in how patients plan to store a device may not be meaningful if storage methods will not affect drug efficacy or device safety. But when trends are observed that have a potential impact on patient safety or device efficacy, designers should examine the trends carefully and make sure they have collected enough data to have confidence in the results.
5. Refine The Design And Head Back To The Field
User research is not a “one and done” proposition. Ideally, medical device manufacturers will have several chances to collect user feedback and observations over the course of device development.
The number of user research rounds can vary widely from project to project. In general, the more complex and novel a device is, the more rounds of user research manufacturers should plan to incorporate into their timelines and budgets.
In an Agile development model, the design team may go back to users at frequent intervals as they refine the device design. More commonly, the team plans user research at a few key points in the development cycle. These may include early formative research and evaluation of design concepts, early prototypes, and final prototypes. A new set of participants should be recruited for each stage of user research to avoid the emergence of biases that can result when the same participants give input during multiple user research studies.
If time and research budgets are tight, manufacturers should put more emphasis on early research. Identifying potential problems and validating the fundamentals early in the design process can help the team reduce the likelihood that they will need to make extensive — and expensive — changes later in the design process.
However, whenever significant changes are made in the design, additional research should be completed to gain confidence in the new design. If too many design iterations are allowed to go untested, there is a greater likelihood of unwelcome surprises when it comes time to conduct final HF validation testing prior to regulatory body submission. An iterative approach to user research will help ensure that the final design fully meets the needs of users.
Following these guidelines will help engineers establish effective partnerships with HF researchers, leading to better user research and, ultimately, better medical devices.
About The Authors
Debbie McConnell is a Human Factors Technical Lead in Battelle’s Human Centric Design group. She brings 25 years of experience with human factors research for private industry, public service and government agencies. For the last 12 years, her work has focused on the medical device and healthcare industries. Her focus is on identifying and delivering medical device design solutions that exceed customers’ expectations and ensure safe use.
Scott Danhof is a Mechanical Engineering Research Leader in Battelle’s Consumer, Industrial and Medical Technologies group. He provides leadership for project teams for a wide range of products for government and commercial clients. For the last 20 years, much of this work has focused on medical product development, including participant/user safety, regulatory compliance, design controls and usability.