When AI Isn't Enough: What the FDA Warning Letter Teaches Us About AI Literacy in the Regulatory Industry

In early April 2026, the FDA published a warning letter to Purolea Cosmetics Lab of Michigan. CGMP violations are not uncommon in warning letters, and they usually don't make big waves outside of regulatory circles. But this time, something is different. For the first time, an FDA warning letter includes a dedicated section titled: "Inappropriate Use of Artificial Intelligence in Pharmaceutical Manufacturing."

Everyone is already talking about it. So I am too.

What Exactly Happened There?

In September 2025, FDA investigators arrived at the Purolea facility for a three-day inspection. What they found was a grim picture: insects, dirt, leaves, and trash in production areas. A loading dock door that, when opened, exposed the production line directly to the outside environment. Products released to the market without microbiological testing. Raw material suppliers who were not qualified. And overall, a quality control system that was simply non-functional.

But the detail that caught everyone's eye was something else: when investigators asked the company owner how they managed to meet FDA requirements, she explained that they use an AI agent that creates their specifications, procedures, and production records for them.

When the investigators noted that the firm had never performed process validation before releasing products, the owner's response was: "We didn't know that was required, because the AI didn't tell us it was needed".

Two Problems, Not One

The FDA defined two separate failures in the letter, and it's important to understand the difference between them.

The first failure: Lack of output review. The company used AI to generate documents, but no one verified if they were correct. The FDA stated this explicitly: if AI is used to create documents, the output must be reviewed to ensure it is accurate and compliant with CGMP. Failure to do so is a violation of 21 CFR 211.22(c). This is a procedural failure, and its correction is less complex.

The second, and more serious, failure: overreliance. The FDA uses this word specifically. It's not just that no one reviewed the documents. It's that the AI was given a role it is neither capable of nor qualified to fill: the company's regulatory gatekeeper. When the owner said: "The AI didn't tell me validation was needed", she revealed something deeper than negligence. She revealed that she had delegated professional judgment itself to a technological tool.

The Connection to AI Literacy

This is where the concept that has been widely discussed in recent years comes in: AI literacy.

AI literacy is the ability to understand, evaluate, and use artificial intelligence tools in an informed and critical manner. It doesn't mean knowing how to write code. It means knowing what AI can do, what it cannot do, and precisely where its output requires professional human eyes before moving forward.

The connection between the two is simple: overreliance is a direct result of a lack of literacy.

Someone with AI literacy understands that AI is a tool that executes instructions, not an expert who bears responsibility. They know how to ask it the right questions, review the output, and identify what's missing. They know how to use it as an accelerator for their work, not as a substitute for their judgment.

Someone without AI literacy doesn't know what they don't know. The output looks professional, the document looks complete, the procedure looks correct. They lack the regulatory foundation to recognize that something is missing. From this, overreliance is born.

In Purolea's story, the owner simply didn't know enough about CGMP to understand that the AI didn't cover everything necessary. The lack of regulatory literacy led directly to blind trust in the tool. And the overreliance led to disaster.

AI literacy is what prevents overreliance. When you understand the tool's limits, you don't delegate more to it than it can handle.

What Exactly Did the FDA Say?

The FDA did not ban the use of AI. It said something simpler and more precise:

"If you use AI as an aid in document creation, you must review the AI generated documents to ensure they were accurate and actually compliant with CGMP. Your failure to do so is a violation of 21 CFR 211.22(c)."

In other words, the AI itself is not the problem. The problem is that no one checked. No one knew enough to know that something was missing.

Even a lawyer who commented on the story put it clearly: "I don't think the FDA is opposed to using AI to draft documents. Their problem is that no one reviewed it and that the result was completely inaccurate".

Three Principles Derived From This Story

1. AI is a tool, not a regulatory expert

No AI tool, no matter how sophisticated, can serve as a substitute for the professional responsibility of the subject matter expert. The AI can generate a CAPA procedure that looks excellent. It doesn't know if it's suitable for your specific process, your regulatory environment, or your product's risk level.

The document produced by the AI might look professional. That doesn't mean it's correct.

2. The quality unit is responsible for the outputs, not just the inputs

According to 21 CFR 211.22, the quality control unit must approve procedures, review production records, and ensure compliance with CGMP. This doesn't change because the document was created by an AI instead of a human. The responsibility remains human. Period.

3. Lack of knowledge is not a defense principle

"The AI didn't tell me validation was needed" may be a candid statement, but it is not a regulatory defense. The FDA does not accept "I didn't know" as an answer, whether the ignorance stems from delegating responsibility to a hired person or to a technological tool.

So, What Should We Do with AI?

I use AI in my work every day. I've built entire tools around AI capabilities. And I say it clearly: AI is an amazing thing for professionals who know what they're doing. It can be dangerous in the hands of someone who uses it as a substitute for knowledge.

What AI can do for quality and regulatory professionals:

  • Generate a first draft of a procedure, which you will review and adapt

  • Summarize a long regulatory document into key points

  • Help draft a CAPA that you know in-depth

  • Search for information, suggest a structure, identify gaps

  • Translate, format, organize

What AI cannot do:

  • Decide if your product meets requirements

  • Verify that your process is validated

  • Check if the generated procedure matches actual practice

  • Take on regulatory responsibility

The 30/70 Rule:

No more than 30% of the final output should come directly from an AI, without professional processing and review.

The remaining 70%? That's your knowledge, judgment, and qualification.

And What About Medical Devices?

This story came from the pharmaceutical world, but the lesson is entirely relevant to the medical device industry as well.

In January 2026, the FDA published a principles document for the good use of AI in drug development. The EU MDR already requires that every process be documented and proven, and every procedure be controlled and approved. ISO 13485 requires that quality assurance approve quality system documents before their release.

Whether it's a CAPA, a specification, a validation plan, or an assessment of MDR compliance, AI tools can produce a draft. Never the decision.

The Most Important Point

This sentence, from the FDA's letter, will stay with me for a long time:

"You replied that you were not aware of the legal requirement, as the AI agent you used never told you it was required".

This is not a failure of the AI. It is a failure of literacy, which led directly to overreliance.

Someone who understands what AI can and cannot do doesn't get to that point. They know to ask, to check, and to keep the professional responsibility with themselves.

The AI is not designed to know what you haven't asked it. It is not your regulatory gatekeeper.

Your regulatory gatekeeper, in the end, is you.