Guest Column | January 30, 2017

How To Establish Sample Sizes For Process Validation When Destructive Or Expensive Testing Is Required

By Mark Durivage, ASQ Fellow

How To Establish Sample Sizes For Process Validation When Destructive Or Expensive Testing Is Required

The first article in this series, Risk-Based Approaches To Establishing Sample Sizes For Process Validation (June 2016), provided and established the relationship between risk and sample size. This article will discuss how to establish sample sizes for process validation when the testing required is expensive or destructive. Of all the approaches discussed in this series, this one is probably the most difficult to address and statistically justify. Guidance documents or formal requirements on this topic from regulatory authorities, accrediting bodies, standards organizations, and governmental agencies simply do not exist. Additionally, there is limited literature available for the practitioner to use for justification. 

I would like to suggest that, whenever possible, variable sampling plans should be utilized, for two reasons. First, variables data will yield much more information than a comparable sample using attribute data. Second, with fewer samples, associated costs of inspection, measurement, and testing can be substantially reduced.

It should also be noted that a simple random sample is meant to be an unbiased representation of a group. A sampling error can occur with a simple random sample if the sample doesn't end up accurately reflecting the population it is supposed to represent. In other words, each piece that is to be inspected, measured, or tested must have the same chance of being selected and be representative of the batch or lot.

The Guideline for Sampling in the Commercial-Grade Item Acceptance Process (TR-017218-R1) was developed to address specific sampling issues in support of U.S. Nuclear Regulatory Commission (NRC) Regulatory Guide DG-1070.  The report addresses sampling plans for nondestructive and destructive tests and inspections. The sampling plans developed in this report are highly dependent upon the lot being homogeneous (selected sample is representative of the entire lot) with good traceability.

Acknowledging that the NRC is not part of the life sciences industry, safety is still highly important to both industries. Therefore, it is reasonable to consider adapting the sampling used by the nuclear industry in the life science industry — until life science regulatory agencies develop a comparable scheme.

There are requisite strategies that should be deployed to support validation activities when expensive or destructive testing is required.  Figure 1 depicts the sampling plan selection factors that should be considered when sample testing and inspection is either destructive or expensive, as recommended in the guidance.

One strategy that can be used to provide supporting data is the use of standard material coupons from the same lot of raw material used to produce the component or finished device. The coupons would have the same mechanical properties for testing but without the value-added costs associated with manufacturing the component or finished device. The testing data could be recorded within the batch record to support the reduced testing and inspection on the component or finished device. Reduced sampling strategies should also be reinforced with lower cost non-destructive tests, as well as visual inspection.

Reduced sampling strategies should be used sparingly. It will be difficult to defend a validation program if reduced sampling strategies are used for every validation.

Figure 1: Sampling plan selection factors (Source: EPRI's Guideline for Sampling in the Commercial-Grade Item Acceptance Process, Figure 2-5)

 

Table 1 provides sample sizes for reduces, normal, and tightened inspections. The inspection levels could be linked to risk (low, medium, high) or supplier status (approved, probationary) as determined by failure modes and effects analysis (FMEA).

 

Table 1: Recommended Set of Nondestructive Test and Inspection Sampling Plan (Partial)

 

Table 2: Recommended Destructive Test and Inspection Sampling Plan Factors (Partial)

 

The Link Between FMEAs And Inspection Levels

Before we begin, we must establish our definitions of risk. These definitions can and should vary based upon the organizational needs.  A good place to determine the risk level is from the FMEA. FMEA (design, process, user) is a systematic group of activities designed to recognize, document, and evaluate the potential failure of a product or process and its effects. FMEA uses a risk priority number (RPN), which is based on the frequency, detection, and severity of a potential failure mode. The higher the RPN, the higher the risk. However, a high severity in conjunction of low probability of occurrence and high probability of detection may still necessitate the appropriate controls for high risk.

Once the risk level has been determined (low, medium, high), the appropriate acceptance quality limit (AQL) can be selected using Table 3.  Figure 2 depicts the linkage between FMEA, risk, and AQL.

Figure 2: Risk process for determining the appropriate AQL

 

Table 3 shows an example of risk level definitions with accompanying defect classifications. These definitions can and will vary based upon the product(s) produced and their intended and unintended uses.

 

Table 3: Example of Risk Level Definitions

 

Table 4 depicts example AQLs based upon risk. Different AQLs can and should be utilized based upon the organizations risk acceptance determination threshold, industry practice, guidance documents, and regulatory requirements. A note of caution when using this method: Lot sizes used for validation activities should be consistent with the lot sizes anticipated for production.

 

Table 4: Example AQL Based upon Risk Acceptance for Expensive or Destructive Testing

 

Acceptance sampling is used to assess the quality based on a sample size, acceptance number, and desired quality level. C=0 sampling plans are based on the premise of accepting the lot if zero defects are found during the inspection, and rejecting the lot if one or more defects are found. Another way to define C=0 is the acceptance number (a) is zero (0); in other words, we accept the lot if zero (0) defects are detected.  C=0 sampling plans provide more protection to the consumer, which is especially important when health and human welfare are involved. Using the C=0 sampling plan is the easiest method to determine a statistically valid sample size to support process validation activities because all that is needed is the lot size (N) and a risk-based AQL.

Table 5 is a partial C=0 sampling plan that could be used for sampling when the testing is destructive or expensive. High AQLs with reduced sampling should be used as the exception and not the rule!

 

Table 5: C =0 Sampling Plan (Partial)

 

It should be noted that by strategically selecting lot sizes, significant cost savings can be realized, due to decreased inspection, measuring, and testing costs. Also, one should realize that the smaller the AQL, the more inspection, measuring, and testing must be performed. Conversely, the larger the AQL, the less inspection, measuring, and testing is required.

Conclusion

This article has provided some possible techniques to utilize for destructive or expensive testing is required for process validation activities. I want to reinforce that different inspection levels should be utilized based upon an organization’s risk acceptance determination threshold, industry practice, guidance documents, and regulatory requirements.

This article series has introduced several different methods for establishing sample sizes for process validation. The articles in the series include:

About the Author

Mark Allen Durivage is the Managing Principal Consultant at Quality Systems Compliance LLC and an author of several quality-related books. He earned a B.A.S in Computer Aided Machining from Siena Heights University and a M.S. in Quality Management from Eastern Michigan University. Durivage is an ASQ Fellow and holds several ASQ certifications including; CQM/OE, CRE, CQE, CQA, CHA, CBA, CPGP, and CSSBB. He also is a Certified Tissue Bank Specialist (CTBS) and holds a Global Regulatory Affairs Certification (RAC). Durivage resides in Lambertville, Michigan. Please feel free to email him at mark.durivage@qscompliance.com with any questions or comments, or connect with him on LinkedIn.

References:

  1. American Society for Testing and Materials. ASTM F302-78. Field Sampling of Aerospace Fluids in Containers. Philadelphia, Pennsylvania.
  2. Durivage, M.A., 2014, Practical Engineering, Process, and Reliability Statistics, Milwaukee, ASQ Quality Press
  3. Durivage, M.A. and Mehta B., 2016, Practical Process Validation, Milwaukee, ASQ Quality Press
  4. Durivage, M.A., 2016, Risk-Based Approaches To Establishing Sample Sizes For Process Validation, Life Science Connect
  5. Guideline for Sampling in the Commercial-Grade Item Acceptance Process, EPRI, Palo Alto, CA: 1999. Report TR-017218-R1.
  6. Squeglia, Nicholas K. 2008, Zero Acceptance Number Sampling Plans. 5th ed. Milwaukee: ASQ Quality Press.