You Can't CAPA What You Can't Locate
By Kayla Britt, Britt Biocomputing

An AI agent reviews executed batch records by exception and identifies and reports exceptions to QA. It checks a process limit against a source document that is current and approved, but for the wrong product strength. The record is not routed to QA, because the result falls within the other strength’s limit. QA then approves the record in the quality management system (QMS).
Recording that the agent missed the excursion describes what happened, but not what caused it, or where the control needs to be applied. The piece “You Can’t CAPA What You Can’t Classify”,1 which introduces the two-dimensional probabilistic failure mode taxonomy, gives investigators a shared GxP vocabulary and operational map for probabilistic failures. The corrective and preventive action (CAPA) process typically assumes a dominant assignable cause of the identified failure. For a deterministic system, this frequently holds true: The root cause and the point of correction typically coincide. A probabilistic system is stochastic by design. Intrabatch variation is intrinsic to the system design, analogous to process validation. The two-dimensional taxonomy, which classifies failures by type (Dimension 1) and origination point (Dimension 2),2 is tailored to the probabilistic AI system, defined as the model × harness × human-in-the-loop or human-on-the-loop (HITL/HOTL). Within the defined context of use, the system has the capacity for output variation via the model and/or harness, given the same input. Intrinsic variation is what remains after locatable defects have been excluded.
The challenge is distinguishing failure that is intrinsic to the architecture and therefore cannot be corrected at a single point (similar to the concept of “irreducible” failures)3 from failure that results from a locatable, correctable defect in a component of the system. The flowchart outlined below in “The Operational Test” is meant to give QA teams a structured, streamlined method for assessing whether a failure stems from intrinsic variation or a locatable cause.
Before The CAPA: The Data Inheritance Gate And The Validated Baseline
The validation of a probabilistic AI system is outside the scope of this article. The International Society for Pharmaceutical Engineering (ISPE) GAMP® 5, Second Edition;4 GAMP® AI Guide;5 and the preprint research paper on the methodology of assurance and control of probabilistic AI systems6 mapped to existing regulatory guidance cover it. This piece covers the data inheritance gate, the validated baseline, and subsequent failure mode classification and analysis.
To classify a failure as either intrinsic or locatable, the organization must establish a validated baseline for the probabilistic AI system. Shewhart’s statistical control chart documented in his book, Economic Control of Quality of Manufactured Product7 (1931), was built on this concept; he emphasized statistical control of a system, not perfect reproducibility. The flowchart, therefore, only applies to probabilistic AI systems that have been statistically validated and assured to be in a state of control prior to post-deployment monitoring. The two mechanisms used to accomplish this are the data inheritance gate and the baseline failure mode rates per type and input class.
The Data Inheritance Gate
Data governance (DG) should work cross-functionally with validation and the business process owner (BPO) to complete the data inheritance gate activities: DG produces the evidence, validation verifies and inherits it, and the BPO owns the gate decision. QA, system owners, security, and suppliers also have roles at the gate, outside this article’s scope.
Postdeployment, DG errors may look like AI errors and be classified as such under the failure mode type axis. The defect surfaces at an origination point, although its cause lies upstream. Shewhart’s state of control requires that “no cause produces a predominating effect.” Inherited data defects are absorbed silently and may appear equivalent to chance variation; these inherited defects are predominating causes disguised as chance. Give data inheritance gate activities proportionate focus to the risk of the proposed system’s context of use. The gate’s regulatory anchor is step 4 of the FDA’s draft credibility framework.8
Validated Baseline
The baseline rates (occurrence) per failure mode type and input class are the output that the remainder of the methodology uses as the reference point. Europe’s draft Annex 22 text9 defines occurrence as the frequency of a failure mode in the class, reported with an uncertainty bound. The same draft Annex 22 defines input class, or subgroup, as a group of inputs that share common characteristics relevant to the specified context of use. The following should be specified by a cross-functional team:
- A failure mode classification guide
- A limits table per failure mode, with alert and action levels tiered by severity
- Capturing inputs, retrieved specifications, and as-run configuration, at execution
The Operational Test: Applying The Error Flowchart
The flowchart provides the roadmap for the identified failure modes to investigate and assign the origination point(s), and, ultimately, to locate the point of correction(s). The two-dimensional taxonomy’s scope is limited to probabilistic-specific errors.

When a system failure is identified, the flowchart is incorporated into the root cause analysis to determine whether the observed failure mode is intrinsic to the system. If the failure mode is intrinsic, case monitoring is the appropriate response; if due to a locatable cause, a corrective action is appropriate; and if indeterminate, the capture gap is treated as a data integrity finding before any disposition.
The Question Logic
The operational test’s regulatory logic is grounded in the out-of-specification (OOS) investigation framework,10 adapted to stochastic systems. Question Q0 maps to Phase I: Check for laboratory error first, to verify that the result is real. The SOP fixing the retest count in advance maps to Q2’s SOP-fixed replay count. A passing retest does not invalidate the original result without an assignable cause. The question logic progresses from cheapest and most deterministic to most expensive and most judgment-dependent:
- Q0: Is the failure real, and is the evidence preserved?
- If yes to both, proceed to Q1. If the failure is not real, this question logic is not the right tool. If the evidence is not preserved, the priority should be recovering the available evidence.
- Q1: Has the configuration been changed?
- If yes, replay the validated configuration. A change is a suspect, not a cause. Configuration includes the model version, prompts, harness settings, index contents, tool definitions, and thresholds. Determine whether the configuration lives on the sponsor or supplier side. Trace the replay to identify any anomalies. If an anomaly is identified, trace the earliest place where it appears to identify the primary origination point. If the validated configuration also produces the failure, proceed to Q2.
- If not, continue to Q2.
- Q2: Under SOP-fixed replay conditions, is this input’s failure rate systematically elevated versus the validated baseline?
- If yes, the failure may be locatable, and the origination point(s) can be identified using the taxonomy. The cause may also sit in the model weights, where the fix would be either a compensating control or supplier escalation. A yes sets severity and urgency; proceed to Q3.
- If not, the failure is consistent with the validated rate for this input class, but that alone doesn’t establish it as intrinsic. Proceed to Q3 at standard severity.
- Q3: Does the execution trace show a step whose correction (verified by counterfactual replay where feasible) prevents the failure, and does that step lie in a sponsor- or supplier-controlled harness component?
- This is only a reliable method when execution-time traces are present. Without a trace, the answer cannot be determined.
- If there is no trace, it is indeterminate.
- If yes, it is locatable.
- If not, it is intrinsic.
A missing trace is itself a data integrity finding.
The Example: AI-Assisted Batch Record Review
Note: Draft Annex 22 excludes large language models (LLMs) and other probabilistic models from critical GMP use. The example used here is illustrative of non-critical use with qualified human review.
Returning to the opening scenario of the batch record routing agent:
- Preconditions
- The specification corpus passed the gate at onboarding.
- Validation sets a baseline: The failure mode taxonomy is used to assess potential risks prospectively. The risk identified, and tested for during assurance testing, is contextual misapplication. For this input class (multistrength products), the validation testing shows contextual misapplication in 10 of 500 (2.0%; 95% confidence interval, 1.0% to 3.6%) outputs, which is within acceptance criteria.
- The post-deployment failure
- During a routine reconciliation check, an approved batch record is identified as having been incorrectly approved. A QA professional is assigned to investigate the deviation and identify why the record was approved when the batch failed acceptance criteria.
- Running the flowchart
- Q0: Yes to both. The reconciliation confirmed the result was outside the limit for the batch’s actual strength. Execution-time capture preserved the assembled input, the retrieved specifications, and the as-run configuration.
- Q1: No. The model version, prompt, and index version all match the validated configuration and the specification index snapshot.
- Q2: Run five SOP-fixed replays against the frozen captured context. Two fail. If this input behaved like the class baseline, even at the upper bound of 3.6%, two or more failures in five would happen less than 2% of the time. The input is failing at an elevated rate. Proceed to Q3.
- Q3: Yes. The trace shows retrieval returned the current specifications for both strengths, with no strength filter. A counterfactual replay that supplies only the correct strength’s specification doesn’t reproduce the failure. Retrieval configuration is a sponsor-controlled harness component, so the outcome is locatable.
- Multicell attribution
- Primary cell: Contextual misapplication × Retrieval/retrieval-augmented generation (RAG) layer. Retrieval is where the trace first went wrong. This cell is used for trending.
- Secondary cell: Contextual misapplication × Model inference. Given both strengths’ specifications, the model sometimes applied the wrong one, which explains why only two of five replays failed. This is the intrinsic part.
- Secondary cell: Confidence miscalibration × Human-AI interface. The final record was presented to QA in the QMS without any cognitive forcing functions11.
- Dispositions, one per cell
- Retrieval: CAPA. Filter retrieval on the batch’s product-strength metadata. Effectiveness check: rerun the validation set for the input class, with the strength filter in place, and confirm that the contextual misapplication rate falls within the baseline interval.
- Model inference: no CAPA. Record it and trend it.
- Human-AI interface: add a control. Show which strength’s limit was applied on the review screen.
- Control location separate from origin
- A compensating control also goes in a different place: a deterministic, rule-based check after generation that the limit applied matches the strength in the batch record header.
- That control sits in the output-verification step, not in retrieval where the failure began.
- Counting
- The incident counts once, under contextual misapplication, for trending. All three cells are recorded for routing.
- Why capture matters
- If the trace hadn’t recorded which document was retrieved, the failure would have defaulted to “intrinsic.”
The Multicausal Problem
While primary-cell failure mode attribution is suitable for trending purposes, each identified origination point should be routed to its respective control. The primary cell doesn’t change; it keeps its meaning as the first observable point. The secondary cells record contributing origination points, each with a required disposition: its own control, a compensating control, or an accepted risk with a justification. The notable conceptual shift is from identifying a single root cause and mapping it to a single corresponding control to identifying both the primary and secondary origination points and mapping each to one of three downstream actions:
- Corresponding control
- Compensating control
- Accepted risk (with justification)
Don’t Reinvent The Wheel: Adapt It
The take-home here is twofold. First, don’t reinvent the wheel — adapt it. And second, locate before you correct, because where a failure appears isn’t necessarily where it gets fixed. One failure can need several dispositions. QA professionals already have the tools they need. The two-dimensional taxonomy with the data inheritance gate and multicell attribution adapts these tried-and-true tools to the intricacies of probabilistic AI systems.
The agent applied the wrong strength’s limit and failed to route the record. QA approved it. A single-root-cause CAPA would likely have retrained the reviewer or blamed “the model” generically. The located version produces a retrieval fix, a trended intrinsic rate, and an interface control, each in a different place.
You can’t CAPA what you can’t locate, but when you do locate it, you can.
Author’s note: Failure Mode 6 Population Bias, originally labeled in the taxonomy as well as the preprint, has since been renamed Subgroup Performance Failure.
References:
- Britt, K. (2026a, September 16). You can’t CAPA what you can’t classify. Pharmaceutical Online. https://www.pharmaceuticalonline.com/doc/you-can-t-capa-what-you-can-t-classify-0001
- Britt Biocomputing. (2026). Probabilistic failure mode taxonomy [Web page]. https://brittbiocomputing.com/probabilistic-failure-mode-taxonomy
- Tancev, G. (2026). Governing large language models in regulated pharmaceutical environments: A risk engineering framework [Preprint]. ResearchGate. https://www.researchgate.net/publication/404873511_Governing_Large_Language_Models_in_
Regulated_Pharmaceutical_Environments_A_Risk_Engineering_Framework - International Society for Pharmaceutical Engineering. (2022). ISPE GAMP® 5 guide: A risk-based approach to compliant GxP computerized systems (2nd ed.).
- International Society for Pharmaceutical Engineering. (2025). ISPE GAMP® guide: Artificial intelligence.
- Britt, K. (2026b, October 11). Establishing AI state of control: An error taxonomy and assurance framework for transformer-based probabilistic systems in regulated life sciences [Preprint]. Research Square. https://www.researchsquare.com/article/rs-10476545/v1
- Shewhart, W. A. (1931). Economic control of quality of manufactured product. D. Van Nostrand.
- U.S. Food and Drug Administration. (2025). Considerations for the use of artificial intelligence to support regulatory decision-making for drug and biological products [Draft guidance]. https://www.fda.gov/media/184830/download
- European Commission. (2025). EudraLex Volume 4, Annex 22: Artificial intelligence [Consultation draft]. https://health.ec.europa.eu/document/download/5f38a92d-bb8e-4264-8898-ea076e926db6_en?filename=mp_vol4_chap4_annex22_consultation_guideline_en.pdf
- U.S. Food and Drug Administration. (2022). Investigating out-of-specification (OOS) test results for pharmaceutical production (Rev. 1). https://www.fda.gov/media/71001/download
- Buçinca, Z., Malaya, M. B., & Gajos, K. Z. (2021). To trust or to think: Cognitive forcing functions can reduce overreliance on AI in AI-assisted decision-making. Proceedings of the ACM on Human-Computer Interaction, 5(CSCW1), Article 188. https://doi.org/10.1145/3449287
About The Author:
Kayla Britt is the founder of Britt Biocomputing, a consultancy advising regulated life sciences companies on validation of AI-enabled systems, including large language models and agentic AI. She is a member of the International Society for Pharmaceutical Engineering (ISPE) and the Parenteral Drug Association (PDA), and serves as founding Co-Chair of the Chesapeake Bay Area chapter of the ISPE GAMP Community of Practice (CoP) and as a member of the AI CoP Workforce Preparedness subcommittee. Before founding the firm, she conducted adversarial testing of frontier AI models. She is a co-author of a public comment to FDA Docket FDA-2026-N-4390 on the AI-Enabled Optimization of Early-Phase Clinical Trials Pilot Program. She can be reached at kayla@brittbiocomputing.com.