Five Qualities Of A Good Use Risk Assessment (uFMEA)

By Bryon Calawa, Design Science

Not all use risk assessments are created equal. As a human factors researcher, I have worked with a variety of Use Failure Mode and Effects Analysis (uFMEA) assessments, for a wide range of medical products, to identify hazards and ways to mitigate them. If there is one key takeaway from these experiences, it would be that there is a right way and a wrong way to complete an assessment.

Your use risk assessment can make or break your regulatory submission. Here are five qualities of an assessment that is on the path to success.

A Good Use Risk Assessment Is Alive

Risk assessment is intended to be an ongoing process, not a one-and-done document. Multiple revisions are to be expected over the course of the product’s development. Your use risk assessment process should start early in development, and be reviewed and revised after each round of usability testing to ensure that all potential risks are identified, and that all risk controls are properly documented.

The absence of a revision history is almost certainly a red flag in your regulatory submission. This conveys that little or no thought has been given to mitigating potential sources of harm. By regularly updating your use risk assessment documents, you provide a clear history of the ways your product’s use safety has improved throughout development.

A Good Use Risk Assessment Is Collaborative

Don’t go it alone! Risk assessment is best completed by a collaborative, cross-functional team that can identify all potential risks associated with the product. Consider including product engineers, clinicians, and intended users on your initial assessment team. Each group can provide valuable insight into potential use-related risks:

• Product engineers can identify reasonably foreseeable risks that relate to misuse of the product.
• Clinicians can provide insight regarding the efficacy of the intended therapy within the various failure modes identified by the engineers.
• Intended users provide a unique perspective and may identify risks that relate to their expected use of the product in their specific environments.

It is very unlikely that a single person will be able to identify all possible use errors and outcomes for a product. Preliminary product evaluations should always be used to help fill any gaps in the risk assessment, but a cross-functional team can get you on the right path earlier in development.

A Good Use Risk Assessment Identifies All Potential Use Errors And Outcomes

The FDA is concerned with ensuring that potential high-severity risks have been mitigated as much as possible. However, this should not be taken as an invitation to stop your assessment at the highest possible severity for each potential use error. Instead, your risk assessment should show each possible level of severity for each potential outcome.

The goal of any good risk assessment is to be so thorough that every potential outcome is recorded in the document. If someone accidentally sticks himself with a needle, it should be easy to determine what could happen in any scenario where needlesticks are possible: A needlestick with a dirty needle could lead to infection and hospitalization, but a needlestick with a sterile needle may be nothing more than an inconvenience or minor injury. By limiting a risk assessment to the highest possible severity, you inadvertently make your product seem more dangerous to use than it is. Being thorough and explaining all possible outcomes is a great benefit to your regulatory submission.

A Good Use Risk Assessment Is The Blueprint To Your Usability Study

One of the primary goals of any use risk assessment is to identify critical tasks in the normal use process of a product. The FDA expects that these critical tasks “[are] used to structure the human factors validation test to ensure it focuses on the tasks that relate to device use safety and effectiveness” (Guidance for Industry and Food and Drug Administration Staff: Applying Human Factors and Usability Engineering to Medical Devices). In other words, the risk assessment is the basis for determining which activities will be included in the human factors validation study.

An incomplete or poorly executed risk assessment could spell disaster for your regulatory submission. It can affect the FDA’s confidence in the validity of your human factors testing. Most importantly, you might miss critical tasks. This can result in a study that does not demonstrate your device is safe and effective for use. Fortunately, this outcome can be easily avoided using thorough task analysis and regular revision based on formative evaluations.

A Good Use Risk Assessment Helps You Address Residual Risk

Imagine you have just completed your human factors validation testing, and your results show that some participants made errors on critical tasks. Before you press the panic button, consult your risk assessment! If your risk assessment has successfully documented all potential use errors, outcomes, and severities, this result may not end with a failed submission.

It is easy to become distracted by the term “critical task.” But, as we discussed above, tasks are considered “critical” based on their highest-severity potential harm. The error you observed in the validation study may not have a serious safety implication after all. You may be able to demonstrate that the highest severity harms have been mitigated, and that the residual risk is acceptable, despite the error occurring in a critical task.

Without a complete risk assessment, it becomes much harder to make that argument. Revising your assessment to justify an error in your validation study is likely to attract scrutiny from your FDA reviewer. Your best chance at success is to be diligent with your risk assessments prior to the validation study. Don’t let critical task errors be the death of your submission; invest effort, time, and care in your use risk assessment!