Validating Large Language Models in GxP: What 21 CFR Part 11 Requires From Your AI Audit Trail

The first time a quality director hands me a validation protocol for an AI-assisted batch record review system, I can usually tell within two pages whether it was written by someone who understands language models or someone who copied a template from a 2015 LIMS validation.

The difference shows up fast. The old template asks: does the system produce the same output every time for the same input? For a language model, the honest answer is: not necessarily. And that single fact dismantles roughly 40% of the traditional validation logic that FDA-regulated manufacturers have been relying on for decades.

This isn’t a reason to avoid LLMs in GxP environments. They offer real value — faster deviation investigations, more consistent literature reviews, smarter anomaly detection in LIMS data. But deploying them responsibly means building a validation framework that actually reflects how these systems work, not pretending they’re just another Category 3 application under an outdated playbook.

The Protocol Problem: Your Validation Template Wasn’t Written for GPT

Computer System Validation (CSV) frameworks have been evolving since FDA first codified electronic records requirements under 21 CFR Part 11 in 1997. Nearly 30 years later, the core logic is still sound: document intended use, define acceptance criteria, and run enough tests to demonstrate the system performs within defined limits.

The problem is that “defined limits” assumes determinism. When you validate a LIMS module that converts a raw absorbance reading to a concentration, you expect — and get — the same answer every time. An LLM doesn’t work that way. The same prompt, run twice against the same model, can produce outputs that differ in wording, structure, and sometimes substance. Temperature settings, tokenization, and model versioning all introduce variability that a traditional OQ script simply wasn’t designed to catch.

FDA’s September 2022 draft guidance on Computer Software Assurance (CSA) acknowledged this tension, even if it predated the current generation of LLM deployments. The agency pushed toward “critical thinking and risk-based approaches” over documentation-for-documentation’s-sake — a reasonable direction, but one that leaves significant interpretive space when you’re trying to validate a system whose outputs are, by design, probabilistic.

And here’s what I see in practice: manufacturers are deploying LLM-based tools for deviation categorization, SOP question-answering, and audit preparation support — then writing validation protocols that treat the model like a deterministic lookup table. That gap is going to generate 483 observations.

What 21 CFR Part 11 Actually Requires — And Where LLMs Complicate It

Let’s be specific about what the regulation actually says, because it matters.

21 CFR Part 11.10 lays out the controls required for closed electronic systems: accurate and complete audit trails, computer-generated timestamps, authority checks, and limitations on who can alter records. These requirements don’t disappear because your software uses a transformer architecture. If an LLM is generating, reviewing, or classifying records that fall under 21 CFR Part 11 — batch records, deviations, change controls, stability reports — the audit trail obligation attaches to that output.

The question your validation package has to answer: what exactly gets logged, and is it sufficient for a human reviewer (or an FDA investigator) to reconstruct what happened?

For a traditional application, that’s straightforward. For an LLM-assisted workflow, you need to capture at minimum:

• The exact prompt submitted to the model (or prompt template version plus input variables)
• The model identifier and version, including fine-tuned weights if applicable
• The full model output, before any downstream processing or formatting
• The human review decision: accepted, rejected, or modified — and by whom
• Any modification made to the AI-generated output before it entered the official record
• Timestamp and user identity for every step in that chain

That’s not a small lift. Most off-the-shelf LLM integrations don’t log at this granularity out of the box. When I audit teams using wrapped third-party APIs, the audit trail is often limited to the final accepted text — which tells you what ended up in the record, but not what the AI actually generated or what the reviewer chose to change. That’s a data integrity failure waiting to happen.

ALCOA+ principles — Attributable, Legible, Contemporaneous, Original, Accurate, plus Complete, Consistent, Enduring, and Available — apply to AI-assisted records just as rigorously as to manual entries. FDA’s 2018 data integrity guidance made that expectation explicit for computerized systems. There’s no carve-out for AI.

GAMP5 Second Edition’s Risk-Based Framework for AI Systems

ISPE published the second edition of GAMP5 in 2022 with meaningful updates to how custom-developed software should be categorized and validated. Under the updated framework, custom AI/ML applications — systems developed specifically for your processes, not commercial off-the-shelf tools — fall squarely into Category 5. That’s the highest risk designation, requiring the most thorough specification, testing, and documentation.

What GAMP5 Second Edition added, helpfully, is explicit recognition that validation rigor should be proportionate to patient risk and regulatory impact. A language model that drafts a first-pass deviation description for human review sits in a different risk tier than one that autonomously approves batch disposition. That distinction should drive your validation depth — but it doesn’t eliminate the obligation.

For Category 5 AI systems, the GAMP5 framework expects:

• Documented user requirements that include performance expectations, including accuracy thresholds and acceptable output variability
• A design specification describing how the model was trained, fine-tuned, or configured
• Formal testing covering not just typical inputs but adversarial cases and edge conditions
• Ongoing monitoring with defined triggers for revalidation

That last point is critical and consistently under-addressed. LLMs change. Model providers push updates. Fine-tuned models drift when retrained on new data. A validation package accurate in Q1 2026 may not reflect the system’s actual behavior in Q3 2026. Building in periodic performance reviews — I typically recommend quarterly sampling against defined acceptance criteria — is the difference between a defensible validation program and one that only looked right on paper.

Building a Defensible Validation Package for LLMs in GxP

There’s no FDA-approved template for LLM validation yet, which is actually freeing: well-reasoned, documented thinking carries weight. Here’s the practical framework that holds up under regulatory scrutiny.

Step 1: Define intended use with precision. Not “AI-assisted deviation management” but “LLM-based initial categorization of deviations into one of six defined categories, with mandatory human review and approval before entry into the quality management system.” The narrower the intended use, the narrower — and more defensible — the validation scope.

Step 2: Establish performance specifications with a reference dataset. Pull a sample of at least 50–100 representative records (depending on deviation volume), have qualified reviewers classify them manually, and define an acceptable concordance threshold. For high-risk classification decisions, I’d rarely accept below 90% agreement with human expert judgment on that reference set.

Step 3: Version-lock the model. If you’re using a third-party API, identify the specific model version being called and control it in your configuration management system. If the provider pushes a new model version, treat it as a change control event requiring a revalidation impact assessment. This sounds obvious; fewer than half the teams I work with have this actually controlled when I first audit them.

Step 4: Build the audit trail at the integration layer. Don’t rely on the LLM provider’s native logging. Build middleware that captures prompts, raw outputs, model version identifiers, and human reviewer decisions into your quality management system’s audit trail. Ensure the data is timestamped, user-attributed, and read-only once created — meeting the same 21 CFR Part 11.10 requirements as any other electronic record.

Step 5: Define a drift monitoring protocol. Every quarter, pull a blinded sample of 30–50 AI-classified records and have a qualified reviewer re-classify them independently. Compare against what the model produced at the time of original classification. If concordance drops more than 5 percentage points below your baseline, trigger a formal revalidation review. Document the protocol. Document the results. That record is what an auditor will ask for.

This is regulatory compliance consulting work that requires genuine process knowledge — not just software testing. The teams that get it right treat LLM validation as a quality systems problem, not an IT deployment problem.

What FDA Auditors Are Starting to Ask

Based on warning letters issued in 2024 and 2025, and conversations with former FDA investigators now in private consulting, the questions appearing in pre-approval inspections and for-cause audits are getting specific.

Auditors are asking to see: the complete audit trail for AI-generated records, evidence that the validation protocol matches the system actually in production (version alignment), change control records for any model updates, and — increasingly — evidence of ongoing performance monitoring. The “we validated it at go-live” answer isn’t landing well anymore.

One question that’s caught several teams off guard: “Show me the most recent instance where the AI produced an incorrect output and what your procedure required the reviewer to do.” If your validation package doesn’t include a documented procedure for handling AI errors, and a log showing that procedure was actually followed, that’s a gap — and it’s exactly the kind of gap that turns a routine inspection into a Form 483.

FDA hasn’t yet issued specific guidance on LLM validation in GxP manufacturing environments, distinct from its more active work on AI/ML-based Software as a Medical Device (SaMD), where more than 521 AI/ML-enabled devices received clearance or approval through early 2024. But the underlying expectations from 21 CFR Part 11, 21 CFR Part 820, and GAMP5 Second Edition apply regardless. Waiting for LLM-specific guidance before building a defensible framework isn’t a strategy — it’s a risk acceptance decision that most quality leaders haven’t formally made.

Start with the audit trail. Make sure it captures what the AI actually did — inputs, version, raw output, human decision — not just the final text that entered the record. That one requirement, implemented properly, gives you something solid to stand on when an investigator walks in and asks the question you weren’t expecting.

Written by Sam Sammane, Founder & CEO, Aurora TIC | Founder, Qalitex Group. Learn more about our team

Reserve early access to our AI audit tools — including ChatGMP and DeepGMP, purpose-built for validated GxP environments. Contact us

Related from our network

• Accredited Laboratory Testing for FDA-Regulated Products — Qalitex Laboratories provides ISO 17025-accredited analytical testing and compliance support for US manufacturers.
• GMP-Compliant Testing for Canadian Regulated Industries — Androxa supports Health Canada compliance with pharmaceutical, NHP, and environmental testing across Canada.