Alternative Method Audit Preparation A Comprehensive Checklist to Ensure Inspection readiness for your MMM

Is there a better way to prepare for new technologies than by using new technologies?

The Modern Microbial Methods (M³) group strives to prepare pharmaceutical quality groups for using modern technologies.

That preparation includes audit readiness – the ability to defend decisions and actions taken onboard a new method. Much documentation is generated as new methods are installed and qualified on-site. We have created a checklist to help you organize your documents to satisfy any regulatory body asking you to see them.

The M³ team started this project with a twist- Why not prompt an artificial intelligence (AI) tool to generate a checklist? From there, the microbiologists could refine the list into a usable document. We are now sharing both versions to highlight how far these AI tools have come while pointing out important nuances that still need a human review.

The Good (and extremely impressive)

The version of ChatGPT, publicly available on November 19^th, 2024, was challenged with a short prompt: “Create audit prep checklist to defend rapid method.” Within seconds, the model responded with a five-page, bullet-pointed list that read like it was written by someone who spent some time in our field.

• Industry acronyms like ALCOA (Attributable, Legible, Contemporaneous, Original, and Accurate) SOPs (Standard Operating Procedures) and CAPA (Corrective and Preventive Actions) were used correctly in context.
• Department stakeholders like Regulatory Affairs, Quality Assurance and Quality Control were correctly attributed to their respective responsibilities.
• The basic content to support the audit was near complete. No major checklist items were missing.

Overall, the AI checklist was a fantastic starting point for our team. With this baseline available, our team spent the next few months coordinating difficult schedules to fine-tune the checklist to be applicable to your site.

The Bad (and why Microbiologists are still needed)

It is worth noting first that the M³ team is aware of a basic tenet of current AI Large Language Models: they are only as good as their prompt. Our prompt was minimal and did not include follow-up instructions to fine-tune the result. However, since the initial prompt yielded a substantial amount of content, we determined it was adequate for our purposes.

Our team first noticed that the checklist did not have a cohesive order that naturally explained a site’s thought process for evaluating, implementing and routinely running a new system. For example, providing information on how you validated a system before sufficiently laying out how a system should be validated could lead to the inspector asking more questions than you want.

With that in mind, the team constructed a checklist aligned with the site’s alternative method journey. You can see in Table 1 the final order of the checklist sections. You can also compare that order to what the AI-generated. Overall, this order provided a useful structure to the entire document.

• AI Section: Verify that all personnel performing the rapid method have completed the required training and competency assessments.
• M³ Rewrite: Verify that all personnel using the modern method have completed the required training and proficiency assessments.
• Training can include instrument operation, result interpretation and actions taken when a discrepancy occurs.
• Proficiency assessments can include operation of equipment to demonstrate absence of false positive/false negative results, aseptic technique and adherence to SOPs.

The M3 group has already started to solicit feedback from industry experts on the checklist. One of the first reviewers noticed an important section missing from both the AI version and our edit- a description of the alternative method technology. This description should highlight how the technology works, potential drawbacks, and important considerations about the technology that drove the implementation plan.

Conclusion

It is critical to be well-prepared for audits, including those that are tied to new methods. While AI-generated tools can be very helpful in establishing baseline requirements for audit readiness, it is important to have additional insight from experienced microbiologists. This will lead to a more comprehensive checklist that can be readily used in your audit preparations. The original AI “checklist” and our final version are available here for your review. We love your thoughts and feedback!