It used to be adequate enough to show training records as evidence of trained employees. And then regulatory inspectors began asking about the effectiveness of our training programs. So we added the ubiquitous “quiz” conducted at the conclusion of training, and for a while this was acceptable as a practice. These “quizzes” maybe measure knowledge retention, and possibly, comprehension, if they include challenge questions about real workplace situations. But having a quiz is no guarantee that the employee transformed the knowledge itself into a skill that can be performed correctly back on the job, especially when the original training was conducted via the “Read and Understand” technique.

As a means of accessing information, it is effective. But reading a procedure is not the same thing as training and qualifying a SOP. Reading the procedure is reading what the SOP contains. Training is closing a knowledge and skill gap and then applying that information back on the job. Qualifying is performing a procedure accurately without coaching. Given that GMP regulations are very clear, i.e., “thou shall follow the procedure,” why would management take a less effective approach to ensure compliance?

The next part of the training effectiveness evolution came about when regulatory inspectors began asking how we qualify our trainers. And to the surprise of a lot of firms, the standard of “if they are trained, they can train others” is no longer sufficient. How do you know they are qualified? “Do you have a procedure/process” for that has become the new refrain.

Most Train-the-Trainer courses focus on presentation skills and managing a classroom environment. On-the-job qualified trainers deliver training on the floor, at the lab station or at the work space but not in a classroom. What they need is a workshop that focuses on the demands facing on-the-job trainers within the workplace environment (Figure 1).

Figure 1 On-the-Job Training: Train the Trainer Curriculum Key Topics

Industry Will Need to Evolve Again

At the 2013 PDA Pharmaceutical Quality Metrics Conference, CDER Director Janet Woodcock, MD, announced that the U.S. FDA will shift their inspection focus to performance and away from compliance. What will this mean for industry? While industry task forces and committees have formed to address that question, I believe that the current 100% trained reports and SOP quizzes will not be enough to satisfy the performance challenge for training effectiveness. Industry will need to document how they qualify their employees. It’s called the Final Performance Demonstration and it’s documented as a Qualification Event.

During the final performance demonstration, the Qualified Trainer observes performance and determines if the employee is qualified or requires more time to practice. These Qualification Events are “moments of truth” when the effectiveness of the training up to this point can actually be verified. This is the true measure of SOP training effectiveness, not a quiz or 100% trained report.

Qualification Events: Level 3 Evaluation

When you look at the well-known Kirkpatrick’s Four Levels of Evaluation Model (Figure 2), you may recognize qualification events (final performance demonstration) as Level 3 – Behavior Change (1).

Figure 2 Kirkpatrick’s Four Levels of Evaluation

When seen through the lens of human performance improvement, Level 3 can be seen not strictly as behavior change but as effective transfer. Thomas Gilbert, author of Human Competence: Engineering Worthy Performance, writes “behavior is a necessary and integral part of performance, but we must not confuse the two,” (2). He explains that behavior is a means; not the end goal. Inherent in a behavior change is the presumption that it will lead to a change in the output(s); thus, an improved result. Without a concerted effort to go one step more and link the behavior change to performance results, the connection is often lost. And so is the effectiveness of all that learning and on the job practice sessions.

So, when employees demonstrate procedures correctly, the Qualification Event is a successful knowledge and skill transfer back to the job. Thus, removing doubt about their ability to perform as expected and raising everyone’s confidence that employees are complying with the required SOPs. This is the end goal achievement.

We’ve evolved with our training practices and will continue to enhance our training programs/processes to meet regulatory expectations regarding qualified to perform their job functions. Is your organization ready to step up and address the new regulatory performance challenge? Just how qualified are your employees? Can you back that up with “proper” evidence?

References

1. Kirkpatrick, D. and Kirkpatrick, J. Transferring Learning to Behavior: Using the four levels to improve performance, San Francisco: Berrett-Koehler, 2005.
2. Gilbert, T. Human Competence: Engineering Worthy Performance, San Francisco: Wiley, 2007.

About the Author

Vivian Bringslimark has 26 years of a unique mixture of education, life sciences industry experience and consulting engagements enabling her to provide human performance consulting services for improving people strategies.

Hear more about this topic from Vivian at the PDA Education course, “Qualifying Your SMEs as Trainers,” April 20, during “Train the Trainer” week at PDA’s Training and Research Institute in Bethesda, Md. To learn more, visit www.pda.org/train-the-trainer-week.