Out With The Old, In With The New: Considering Legacy Systems Early In The Design Process

By Andy Schaudt

It is important to understand the product that your medical innovation is replacing. While disruptive innovation can be positive, humans become comfortable with the old way they did things or the old systems with which they interacted. As a medical device innovator, you may indeed be positioning your product to replace a legacy system. If that is the plan, then you should be aware of how exactly that target legacy system is designed and is currently being used in the healthcare environment. If you ignore how your device will affect an established workflow, you could negatively impact user experience and safety with your new design. This article will discuss these negative consequences and show the importance of understanding the legacy system workflow early in your design process — to help reduce use errors and achieve higher acceptance/retention rates.

The Prevalence Of Legacy Systems In Healthcare
Let's face it, healthcare is a complex system. A cursory investigation into the term complex resulted in a couple definitions I present for context: 1) "consisting of interconnected or interwoven parts", or 2) "not easy to understand or analyze."¹ A complex system, as defined by Ladyman and Lambert (2012), is "an ensemble of many elements which are interacting in a disordered way, resulting in robust organisation and memory."² Complex systems tend to have characteristics in common such as numerous elements, interactions, operations, diversity, environments, and activities.¹ Regardless of which definition and set of characteristics resonates best with you, it is clear that healthcare is a complex system and is growing in complexity each day.³

Increased levels of complexity make implementing change difficult. Because of this, many healthcare facilities avoid redesigning and/or replacing existing systems — even if these systems are in dire need of replacement. A legacy system is exactly this, an old piece of technology or a computer program limping along when there are better solutions available. There are numerous reasons why a legacy system is not replaced, ranging from simple perceptions that it still meets the basic functions required, all the way to the most common explanation: that the costs would be prohibitive. But eventually, legacy systems are replaced. When this change occurs, users can experience frustration and poor acceptance, and in some cases the result could be an irrecoverable use error.

Negative Transfer
The reason users stumble when switching to a new system usually is because they have acquired skills from significant training and/or experience with the legacy system, and that inhibits the transfer to the new system.⁴ This negative transfer is also observed when users switch back and forth between systems. In the case of healthcare, you can imagine this during transition periods of new system rollouts, where healthcare professionals are required to use a legacy device (e.g., older infusion pump) in one unit of a hospital, but are required to use a newly implemented device (e.g., smart infusion pump) in another unit. Switching back and forth could inhibit their performance with both systems.

The issue of negative transfer is not completely new to healthcare, although it has been primarily focused on negative transfer of training and simulation (sometimes referred to as negative learning)^5,6, rather than on system design. Much of the concept is the same, though, especially when training and simulation involve interaction with devices and how initial interactions and experiences can have an impact on future performance.  

Using The Legacy System To Your Advantage
There are numerous methods and techniques that medical device designers can implement in order to launch a safe and usable product that is replacing a legacy system. The first recommendation is to do a thorough workflow assessment of how the legacy system is being used in the current healthcare environment with the target user groups. Do this early in your design process. The earlier the better, especially if you can return to it multiple times. This baseline understanding of how a legacy system is currently being used will allow you to identify opportunities for improved design characteristics (e.g., removing unnecessary and inefficient steps), while at the same time identifying areas that could be maintained for a more seamless integration into the environment upon launch.

One method that other industries use is to build their product to be different/distinguishable on some aspects of the design, but similar in others. Car manufacturers, for example, often make their controls very similar in position and function, but their displays differ significantly. This allows users to quickly and seamlessly interact with the system, while at the same time acquire the information and feedback in a new way that still benefits them. The key is identifying where and when to make these design decisions, and it is difficult to make informed decisions without results from a proper workflow assessment.

Using results from the workflow assessment, you can next build out all of the tasks that users would perform with the old device (i.e., task analysis). This will become the backbone of a risk analysis. The most common risk analysis used in human factors engineering for medical devices is a failure mode and effects analysis of usage (use FMEA). Performing a use FMEA will help you identify gaps that your new design may be able to close, opportunities for improving upon inefficiencies, and areas of critical concern that may end in irrecoverable use errors. These irrecoverable use errors can then be given special consideration in order to proactively design out potential effects of negative transfer.

When you have ventured far enough into your product development cycle and have one or more functional prototypes, you can design formative usability studies to perform with end user groups. I recommend that you recruit users that have experience with one or more of the identified legacy systems to participate in your study. These studies should make sure to exercise the tasks that could potentially lead to irrecoverable use errors, in order to validate whether your design features did indeed help eliminate — or at least mitigate — negative transfer effects.

In parallel to the formative usability studies, the critical areas identified from the risk analysis can also be used when designing training programs or drafting training materials. The goal is to provide the end users with a correct mental model of how the new system works, especially where it differs from legacy systems. It is important to note, however, that training should not be relied upon for eliminating the occurrence of use errors. It is crucial to implement proper human factors techniques to design the interaction features with the goal of reducing errors.

Conclusion
In summary, it is crucial to understand how the implementation of your medical device that is targeted to replace a longtime legacy system could negatively affect healthcare professionals. Human factors engineering and the use of standard usability techniques can really set you up for success by eliminating the potential frustrations and safety issues that arise from implementing new systems into the healthcare environment. My message to medical device manufacturers, which you will hear from me time and time again, is that incorporating human factors early and often throughout the design can result in a better, safer, and more usable design. Out with the old method of design, and in with user-centered design.

References

1. Bar-Yam, Y. (1997). Overview: The Dynamics of Complex Systems — Examples, Questions, Methods and Concepts. In Dynamics of complex systems (pp. 1-15). Cambridge, MA: Perseus.
2. Ladyman, J.L.; Lambert, J.; Wiesner, K. (2013) What is a complex system? Euro. J. Phil. Sci., 2013 Jan; 3(1):33–67.
3. Plsek, P.E. &Greenhalgh, T. (2001) Complexity science: The challenge of complexity in health care. BMJ, 2001 Sep; 323:625-628.
4. Wickens, C.D., & Hollands, J.G. (2000). Engineering Psychology and Human Performance. Third Edition. Prentice Hall. Upper Saddle River, New Jersey.
5. Fritz, PZ, Gray, T., & Flanagan, B. (2008). Review of mannequin-based high-fidelity simulation in emergency medicine. Emerg Med Australas, 2008 Feb;20(1):1-9.
6. Bond, W.F., Lammers, R.L., Spillane, L.L., Smith-Coggins, R., Fernandez, R., Reznek, M.A., Vozenilek, J.A., & Gordon, J.A. (2007). The use of simulation in emergency medicine: a research agenda. Acad Emerg Med, 2007 Apr;14(4):353-63.

About The Author
Andy Schaudt is the director of usability services for the National Center for Human Factors in Healthcare at the MedStar Institute for Innovation. In this role he plans, coordinates, and manages the projects, programs, and daily operations for the usability division, which is chartered to conduct medical device and health IT usability evaluations, both for the industry and for MedStar Health (a 10-hospital healthcare system in the Washington, DC / Maryland region). The author can be contacted through www.medicalhumanfactors.net.