CAPA Effectiveness Checks 101: How To Verify That Your Actions Plans Are Successful

By Danielle DeLucy, ASA Training & Quality Consulting

In the highly regulated life sciences industry, serious quality issues can and do occur. Once an investigation team convenes and performs a thorough root cause analysis (RCA), the focus then turns to remedying the issue. Perhaps the most commonly used method to fix quality issues — and ensure they don’t rear their ugly heads again — is to create thorough corrective and preventative action (CAPA) plans. These plans take each root cause that was investigated and spell out specific steps to address them.

However, before you can consider the issue corrected, the CAPA needs to be verified as effective and complete, per international regulations and standards. For example, step 3.2.2 of ICH Q10 states: “CAPA should be used and the effectiveness of the actions should be evaluated.” Thus, quality units must perform effectiveness checks on their CAPA plans — measuring and determining that the corrective action has (or has not) eliminated the problem.

There are main three reasons for taking CAPA plans through effectiveness checks:

1. To build confidence that the solution will work
2. To validate that the solution did work
3. To minimize any risks or potential problems that might cause reoccurrence

Verifying the effectiveness of a CAPA closes the loop between identifying a problem and completing the actions to solve a problem.

Choosing The Appropriate Measurement

Determining the best verification approach for the wide range of problems that could occur can often be challenging. Both qualitative and quantitative methods can be used to measure success of a CAPA. Qualitative methods explain and gain insight and understanding of an issue through intensive collection of narrative data, generate a hypothesis to be tested. Quantitative methods explain, predict, and/or control phenomena through focused collection of numerical data and hypothesis testing.

Some firms struggle to use quantitative methods in performing effectiveness checks, because certain aspects of CAPAs can be harder to measure than others. However, you can measure just about anything. For example, you can measure entry errors for operators who are deviating from good documentation practices by identifying areas where quality records are reviewed and documentation errors are measured. You can measure by operator, by type of documentation (e.g., lab record, batch record, or cleaning log), by month, etc.

Another example is the measurement of transcription errors made when copying information from sample ID labels to laboratory notebooks. Say an investigative team determines that the labels made on the current label printer are frequently unreadable and suggests a CAPA plan that includes replacing the current label printer with one that produces legible labels. A proper effectiveness check would include returning to the area within a week to determine if the new printer is operating correctly and the labels can be read. In addition, the team could also monitor the number of errors made during the week and compare the rate with the number of errors made prior to solution being implemented.

To help determine the most appropriate measurement for your effectiveness check, answer some/all of these questions:

1. Who will measure it? The task should be assigned to the quality unit or other group in the organization that ensures all quality system issues are resolved according to regulations.
2. What will be measured? Measurements can take on many different appearances. You can measure the number of errors made once the solution is in place versus the number made prior to its implementation. You can measure the types/amount of deviations that occur after the solution is implemented, or you can measure the effectiveness of a new process on a specific product. You can also measure the amount of over-the-limit environmental monitoring results after incubation.
3. Where will it be measured? In most cases, the measurement can be documented using the CAPA form or electronic tracking system.
4. When will it be measured? The timing of the measurement depends on the severity of the defect. CAPAs with low severity defects, which are not harmful to the patient, can be put in the “parking lot.” While more significant defects, such as those that may cause the patient life-threatening harm, should be investigated and checked immediately.
5. How will it be measured? Measurement of effectiveness checks can be done qualitatively or quantitatively, as discussed in the previous examples. Metric-based checks can account for the quantitative aspect, whereas the qualitative approach can include revisions to a standard operating procedure (SOP) or training of impacted employees. For example, once the SOP has been revised, you can monitor employee performance and compliance to the SOP revision by tracking the number of deviations to the step.
6. How will measurements be analyzed? Analysis of the measurements is conducted by the quality unit, who will determine if the solution was effective and the CAPA plan can be closed. For instance, quality staff can conduct measurements by heading out onto the shop floor to determine if corrective actions are being followed and if they are indeed beneficial to the process.
7. Who will data analysis be communicated to? As in most organizations, this data analysis should be discussed and presented to the quality unit executive management team in the form of monthly metrics. Of course, the analysis should also be communicated to the owners of the CAPA plans.

Once these questions have been answered, you can move on to developing ways to verify your CAPA effectiveness.

Methods For Verifying Effectiveness

There are many highly effective tools and principles you can use to make your effectiveness checks successful. Here are a few common approaches:

• Trend analysis — In cases of human error, training, environmental monitoring excursions, cleaning deviations, testing errors, etc., trend analysis can help you determine if the corrective action has remediated the issue. Review data over a predetermined timeframe and determine if the problem or deviation has occurred again (or as frequently) after the corrective action was implemented. In most cases, the data will tell you if the fix was successful, or if another corrective action should be established.
• Periodic checks — This method involves scheduling a time for the quality unit to review the process that was remediated. For example, if there was a CAPA implemented to improve gowning practices, the auditor can observe the newly improved gowning practices prior to the operator entering the clean room.
• Surprise audits — Audits are intended to ensure that the operators, process, or equipment is following the prescribed corrective action documented in the CAPA. One other benefit of surprise audits is that the reviewer gets to observe the daily occurrences in the area and the how the solution is working in the area affected.
• Sampling — Sampling is a successful method to verify corrective actions pertaining to environmental monitoring or lab testing. For example, if there was a deviation for an area exceeding environmental action limits, and more robust cleaning practices were installed, additional samples could be taken to verify that the new cleaning protocol was effective.

When To Schedule Effectiveness Checks

Another vital piece to ensuring a robust and properly managed CAPA effectiveness verification is establishing the appropriate timeframes to perform the checks. When settling on a due date, it is important to allow enough time for the corrective action to take effect, while also maintaining a sense of urgency around ensuring that the issue is fixed and will not recur.

Some deviations or problems only require a simple procedure update, retraining of staff, or recalibration of a piece of equipment. However, in many cases, CAPAs are opened to address more difficult issues, such as changing the facility layout, altering gowning practices, increasing sample sizes, or implementing a new piece of equipment. In these cases, it is best practice to set up several effectiveness checks over a period of time. Perhaps the first check can be done rather quickly after the CAPA plan has been approved, but others will be spread out over months.

Conclusion

CAPA effectiveness checks are a surefire way to track, trend, and remediate deviations that occur during manufacturing and laboratory testing. By developing solid corrective action plans and appropriately timed effectiveness checks, firms can begin the process of remediating issues that could cause deviations. Having a strong quality unit in place to perform these checks is the key to ensuring that these effectiveness checks are conducted in a timely fashion and are improving the process. When done right, these checks demonstrate your firm’s commitment to quality and compliance.

About The Author:

Danielle DeLucy is owner of ASA Training and Consulting, LLC, which provides pharmaceutical- and biologics-based companies with training and quality systems assistance in order to meet regulatory compliance. Prior to this role, she spent 15 years in the industry serving in numerous quality management roles. She has helped lead many regulatory health inspections and was instrumental in the coaching process of her peers prior to any inspection. Currently, Danielle assists companies who are faced with warning letters or consent decrees, and those wishing to improve compliance, establish more robust quality systems.