By Susan Gould, Halloran Consulting Group
I remember sitting through a quarterly management review meeting early in my career. There was a presentation of about 100–150 slides, with graphs and a few “dashboards” featuring traffic-light colors of red, yellow, and green.
Seemingly innocuous yellow dots dominated each dashboard, along with a few scattered green dots, and just one or two red splotches dotting the page. The red items each had extensive backup slides describing the issue, the impact, and a few action items with due dates.
The slides were presented by a quality manager and the appropriate cross-functional leadership was present. At the end of meeting, the CEO asked, “Is there anything I need to worry about?” “No” all around. In the days following that meeting, the notes were issued, documenting our conclusions that we were operating well and within the bounds of our quality expectations. Hearty congratulations all around!
Three months later, the company had the pleasure of hosting an FDA inspection — resulting in multiple 483s indicating our quality system was not operating as FDA expected. What happened? We were running a quality management review and routinely assessed our own performance. Sure, we had areas for improvement, but certainly nothing to warrant those 483s! How could everyone have missed the warning signs? Did we even know what the warning signs looked like?
I believe that experience is similar to what many companies face: The organization feels like it is working well and is “doing all the right things,” yet the Agency drops in and finds compliance issues that seemingly were not identified by the internal system. It is rarely, if ever, an issue of gross negligence or incompetence, as most firms are filled with dedicated and motivated personnel who believe in their products and come to work wanting to do the right thing. So, how is it that we don’t see the warning signs when the company is heading towards trouble?
There are three key warning signs that indicate a company may be headed in the wrong direction.
Not Seeing The Forest For The Trees (Over-Reliance On Standard Compliance Metrics)
Most companies rely on data from a variety of sources:
This essential data is the basis for developing a “standard” set of quality system reporting slides, often shown in graphs and tables with little context or actionable information.
These metrics often fail to provide early signals of product and process quality issues. Human nature and many human resource-driven performance management programs create an environment where individual operators are motivated to fix the metric and not the root cause. If the time required to close a CAPA is more important than proper, effective analysis and remediation, then a nice metric on a quality slide will not be an accurate indicator that the organization is operating in control. Decreasing the number of complaints is a noble goal, but if that is achieved solely through user interface, and no root cause analysis is performed on the product, the metric may not really tell you anything about product quality.
A focus on the individual trees (metrics) is an important element in any quality organization. However, metrics for metrics’ sake can keep us looking in the rear-view mirror at what happened last month, rather than looking ahead at signs of impending danger.
Gathering and presenting metrics is not sufficient. The value of the metric is to frame a discussion — and, ultimately, a broad organizational agreement — on what each metric means, what is revealed by the story each tells about the current state of compliance, and where the metrics predict the company is likely headed.
Meeting For The Sake Of Meeting (Ineffective Management Reviews)
Rarely do I meet people who look forward to a management review meeting. However, the frequency and timing of management review meetings can be a warning sign of compliance issues.
In 21 CFR 820.20c, it is stated that a quality system should be reviewed “at defined intervals and with sufficient frequency according to established procedure.” Regulatory inspectors expect to see evidence of an active and effective management review process, and almost always ask for the schedule, agenda, and attendees for management review meetings.
They want to see that you are regularly checking the health of your quality system (and products), asking the right questions, and challenging assumptions, as well as verifying that the firm has committed the necessary resources to collect, analyze, and interpret quality system data. Most significantly, inspectors want to see that you are taking timely action on any areas that show signs of potential non-compliance.
Many companies define “sufficient frequency” as once or twice a year. How effective can it be to review data that is six to twelve months old? How timely can your actions be to address those concerns? Quarterly management review meetings may be better, but the data still is too old to allow for proactive steps to be taken. Frequency should be determined by what is happening at the company and with the products, not solely by a set schedule. Some questions that help determine the right meeting schedule might include:
Any of these events may (and should) impact the frequency of management review. Some firms object to adjusting management review for ongoing events, claiming it will “point FDA right to our problems.” This could not be further from the truth; a well-documented meeting, called to assess quality before product launch or due to a new regulation, shows regulators that the firm is actively monitoring itself and puts them at ease.
Top management engagement (meaning attendance + participation) in management review meetings is vital to developing and maintaining an effective quality system. While management must be involved, per the guidelines, actual management engagement is an opportunity for company leadership to directly assess the organization’s compliance and demonstrate the organization’s commitment to quality and compliance. If setting and meeting quality goals is a high organizational priority, viewed as essential to meeting business and financial goals, it is far easier to identify and manage compliance risks before they become compliance problems.
Here, as in the opening example, the answer to the CEO’s query, “Is there anything I need to worry about?” is “Everything.” The red dots require immediate action. Management also should dig into each yellow indication’s root cause to understand the “why,” so appropriate action can be taken to prevent a citation or observation. And, while green is something to celebrate, it is a not a license to ignore that aspect of the operation. Every segment of the quality management system should be important to leadership. Again, “Everything” is the answer to “What is worthy of concern?”
If You See Something, Say Something! (Quality Is A Function, And Not A Culture)
It is very easy to describe quality as a specific function in an organization with a Head of Quality, a department code, a staff, etc. However, when quality is viewed only as a specific function, and not a core part of the organization’s culture, spanning functional areas and embedded in each decision, employees start to believe that quality is not each individual’s responsibility. It disengages and disempowers all levels of the organization from taking ownership for quality. At its worst, such a state discourages employees from speaking up and raising concerns.
An organization that does not support quality as a cultural value, and instead sees it as a separate function dedicated solely to ensuring compliance, is much more likely to suffer regulatory enforcement actions (483s, warning letters) down the road. In fact, FDA cites “corporate culture” as the root of most compliance problems.
When the culture is such that everyone in the organization believes quality is his or her responsibility, concerns are more likely to be voiced, eyes are on the lookout for systems or processes that are not working, and continuous improvement is at the forefront.
Where The Rubber Meets The Road (Internal Checks Lead To External Approval)
The warning signs may not always be easy to see. Unfortunately, corporate “self-reflection” often does not occur until after a major event happens, be it a regulatory enforcement action or a major recall. It is in your company’s best interest to self-analyze proactively.
An external set of eyes may be needed to evaluate how the organization approaches quality and to provide critical insight — including on the collection and analysis of metrics, management review practices, and steps taken (or not taken) to instill a culture of quality. Regardless of who internally evaluates you, though, such hard introspection is worth its weight in gold when it serves to keep your product on-track toward FDA approval.
About The Author
Susan Gould, a principal consultant at Halloran Consulting Group, joined the company in 2015. She brings over 20 years of experience in the medical device and pharmaceutical industry. Her career has focused on clinical quality, compliance, and technical execution, including quality systems regulation (QSR), Phase I-IV clinical trials, and human factors evaluations.
Prior to joining Halloran, Susan held senior-level global positions at Baxter and Boston Scientific, both in clinical and medical affairs, where she supported the development of device and drug-device products. Susan started her career in the pharmaceutical industry, taking multiple products to market.
Susan holds a master's in business administration from Lake Forest Graduate School of Management, with advanced certification in business analytics and strategy development, as well as a master's degree in clinical microbiology from Thomas Jefferson University and a bachelor's degree from Mount Holyoke College. She is certified in quality systems from AAMI.