Modern Quality Professional, Part 3: Using the Scoring System

In Part 1 of our article, we described how AstraZeneca designed a Modern Quality Professional (MQP) development tool to frame a vision of the quality organization that the company aspires to achieve. In Part 2, we presented two case studies that illustrate how capability target levels were set for the capability categories relevant to the Product Quality Leader role and how the level descriptions were revised to fit PQL-specific job tasks.

We conclude the series here by pulling together Parts 1 and 2, discussing the scoring system we devised to identify capabilities and how we used those scores in planning job-specific capability uplifts.

Scoring and Capability Uplift Planning

Once we had developed the Product Quality Leader (PQL) job-specific capability framework, identified the target levels, specified the actionable items, and established review task-specific repetitions we scored the members of the group, both individually and as a team. To facilitate a transparent process and more accurately reflect the current PQL capability levels, as opposed to future levels, we ensured all participants understood that a current lower level is not considered problematic.

In addition, while the PQL target levels are set at one significant figure (e.g., 1, 2, 3), the PQL scoring was performed using “half levels” (e.g., 2, 2.5, 3). This allows for greater flexibility in scoring and provided a more graded approach to monitoring improvement. For example, if the PQL had completed all expectations for a given target level (e.g., 1) but was missing one or more expectations for the next higher level (e.g., 2), a half-level score was used (i.e., 1.5). Like most risk-scoring exercises, the scores (grading) used should strike a balance between too few and too many level options. Therefore, further fractionalization of levels (e.g., quarter-level scores) was not considered beneficial to the process.

All six team members then scored themselves on the PQL-specific capability-level descriptions and relevant actionable review task repetitions. The overall team scores are summarized for the PQL job-relevant capability categories in Table 1. An average deficit level of 1.0 for individuals, which corresponds to a total score of 6.0 for the team (n=6), was considered a major team deficit that could be a limiting resource-flexibility factor for the PQL team. Not surprisingly, the quality risk management (QRM) and change control capabilities had high deficit scores (9 and 8, respectively) for those scores. The similarity of these scores highlights their interdependency. Exceeding an average deficit level of 0.5 (3.0 for the team) was considered a minor deficit and far less likely to affect resource options. Any team score lower than 3.0 was considered insignificant from a resourcing perspective, although capability uplifts may still be required for some PQLs.

Figure 1 visually captures the proportional overall capability deficits for the PQL team. The fact that these deficits are not necessarily problematic for individual performance expectations is important to recognize. It simply made the current capability and experience level of specific capabilities more transparent, and it allowed us to focus our future development in the areas where significant deficits exist. Given the current scoring results in Figure 1, starting at the 12 o’clock position and moving clockwise, capability uplifts could be tackled by providing more coaching and training on the capability categories of QRM, change control and analytical microbiology. As statistics & trending is a core capability for higher-level QRM and change control capabilities, an adequate level would need to be attained in order to reach high levels in these interdependent capabilities.

Line managers need to understand that learning and capability uplifts will take a significant amount of time. We have set up relatively high review/approval repetition numbers (for the 70% learning category) to reach a level 3. Practically, we will only be able to reach the desired capability level as fast as these review tasks come to us in real-time.

Figure 1 Proportional PLQ Team Deficit

What is Next ?

Given the typical behaviors and attributes associated with the vision of a successful PQL job performance, several “soft skills” not currently captured within the MQP development tool, should also be considered when advancing toward an ideal uplifted skill level. Understandably, setting measurable uplifting target levels for soft skills is much more difficult (see Figure 2). Often, these can only be evaluated qualitatively, based on evaluation relies, interviews and formal or informal feedback from PQL partners, such as CMC counterparts. However, target level(s) for soft skills ultimately could be applied wherever we interact with partners. The feedback from partners would need to be directed toward relevant soft skills, and the responses could be standardized by asking focused questions. Similar to the current MQP capabilities, PQLs will work towards reaching relevant soft-skills maturity levels.

In addition, should an individual change their role within the Quality function, they should consider adjusting the MQP capabilities and target levels appropriate to them to ensure alignment with the requirements and job tasks associated with their new role.

Figure 2 Uplifting PQL “Soft Skills”

Conclusion

After we reported in 2020 on how we started the PQL function within AstraZeneca, and how we improved our influence, reputation and overall PQL job function success, we reported in the PDA Letter how we developed and used an MQP development tool to create a capabilities development plan specific to the PQL function (1-4). We adapted this tool to our job functions, which allowed us to develop clear objectives regarding the capabilities needed to uplift. The knowledge of where each PQL currently falls within our adapted capability framework can support suitable PQL job descriptions intended to close gaps when recruiting additional team members. Besides identifying short- and long-term individual and PQL team capability gaps, among many other benefits, this framework can assist in setting individual PQL development goals, and it can frame (and standardize) PQL job grade-level expectations.

References

1. Krause et al., “The PQL Team Part I: Building the PQL Role,” PDA Letter, January/February 2020, 50-51.
2. Krause et al., “The PQL Team Part II: Getting Ahead,” PDA Letter, March/April 2020, 28-33.
3. Krause et al., “The PQL Team Part III: Staying Ahead,” PDA Letter, May/June 2020, 26-29.
4. Krause S. “The Modern Quality Professional Development Tool – Capability Uplift for Product Quality Leaders,” Presented at the 2021 PDA Annual Meeting (online), 17 May 2021.