The Risk Evaluation Method
Aseptic processing of sterile
can and should be improved. The same
challenges, problems and issues seem to
appear, reappear, or never really disappear
from year to year. These problems persist
despite more awareness of the issues due
to increased training, conference sessions
on the topic, guidance documents,
quality system management approaches
and metrics. Each year, regulatory audit
observations, 483s, and warning letters
continue to cite the same problems and
issues over and over. Admittedly, aseptic
processing is challenging, and there are
obstacles to improvement, but it is the
job of those working in this area to resolve
Numerous documents stress the need for
science and risk-based decision making in
aseptic process design and performance.
These include PDA Technical Report No.
22 (1), PDA Technical Report No. 44 (2), PDA Points to Consider for Aseptic Processing
(Parts 1 and 2) (3–4), as well as the
U.S. FDA aseptic processing guidance
and the planned revision to EU’s Annex
1. As stated in Part I of the PDA Points
to Consider for Aseptic Processing, decisions
based on product risk can, and should,
be used to improve aseptic process. Risk
assessments should provide the information
needed to make informed decisions
throughout the process lifecycle. The objective
is not merely to identify risk but to
improve the process. Yet, the industry still
struggles with developing effective risk assessment
methods for aseptic processing.
Aseptic process risk assessments have
some unique challenges. The severity of
sterility-related issues, primarily the loss
of sterility, is high and certainly worthy
of consideration. Yet, the causes of aseptic
process failures, which are directly correlated
with the loss of sterility, are relatively
rare; hence, very low occurrence rates
are noted. Detection of process failures
is not always reliable, and the correlation
between what we can detect and the desired
or undesired outcome may not be
quantifiable. An issue with many aseptic
processing risk assessments is the lack of
objectivity. Since it is not easy to measure
or quantify people or environmental risk
factors, risk assessments tend to be subjective
and therefore of limited benefit.
This subjectivity comes from the lack of
data that can be correlated to sterility
or aseptic process failure. For example,
interventions can add microbiological
contamination to the environment.
However, we may not know when, or to
what extent, those interventions can result
in a level of contamination necessary
to contaminate the product. In common
practice, boundaries and limits are set,
then the results are observed. Too often
this leads to approaches based largely on
the assessor’s experience and bias rather
than science. An objective method for
considering the correlation and evaluation
of risk to aseptic processes is needed
to facilitate process improvement.
Objective Defines the Method
The Risk Evaluation Method or REM
described in this article was initially developed
to satisfy a need for an objective,
simple, and more accurate method
to rank aseptic process interventions, in
order to better plan media fills (5). This
first article will present the basic objective
and steps for the REM. Subsequent
articles will explore its specific use for
aseptic process improvement.
For the aseptic processing REM to be effective
and useful, it should be:
Objective: Objectivity reduces the potential
for bias. Bias leads to disagreement,
loss of confidence in the method, and
risk assessments without benefit. Objectivity
is achieved if different people with
similar knowledge about the process and
access to information are able to use the
REM and conclude a similar risk rating.
For example, if the Quality lead and the
line mechanic use the REM, they should
obtain similar rankings. For this to happen,
meaningful, measurable data or information
must be available and accessible.
It may be that different people have
different perspectives and varying levels
of knowledge. This knowledge becomes
valuable input into setting the ranking criteria.
Once the criteria are set, the evaluation
of information should be objective.
Simple: Simplicity allows for the use of
the REM by a larger group. This leads to
better acceptance of the REM outcome.
The REM should be performed by the
people engaged in the process itself. The
REM should not be burdensome to use.
The people using the method should be
able to explain the basis of the method
and be able to perform the assessment.
Robust: A robust method is one that is
applicable to most relevant applications
or processes. It also avoids the need for
multiple or changing methods which
may add confusion, bias and complexity.
Logical: The REM must be based on
sound science in order to be viewed as
truly objective, to gain user acceptance
of results and convince regulators that
the method and its results are credible.
The REM relies on two principles to
achieve effectiveness. It uses a Line of
Sight approach. Line of Sight focuses
and links actions to the overall objective
of the assessment. That objective is defined
and stated in a problem statement.
The use of a problem statement linked
to the objective assures that all actions
are directly beneficial and useful to accomplishing
the objective and that actions
of little value are omitted.
For this to work, the definition of the
objective as reflected in the problem
statement must be well thought out.
For example, is the objective of the environmental
monitoring program to find
particles, achieve a particle free environment,
meet regulatory requirements, or achieve an environment suitable for
aseptic processing? Is the objective of the
aseptic process simulation to qualify the
operators, the interventions or the aseptic
process? As the objective varies, so
will the focus and use of the assessment.
The second principle is the use of a Key
Word approach, a tactic in which those
assessing process risk identify the words
and values most relevant and meaningful
to them. Rather than use general
terms or risk tables, the participants decide
which words best describe or assess
their situation and tie these to risk scales.
Aseptic processes may seem similar
throughout the industry, but companies
have unique cultures and individuals
have different backgrounds, biases,
perspectives, and experience. Trying to
match one set of criteria to all aseptic
processes would prove difficult. The Key
Word approach applies the company’s
and the involved individuals’ knowledge
to the problem at hand. Also, by asking
the involved individuals to select the risk
criteria, it better assures ownership, respect
and use of the method. It reduces
the chance of the model becoming a
Because the Key Word approach encourages
individual assertion of criteria
and factors, one company/process may
not be completely applicable to other
companies/processes. The temptation
to develop a one-size-fits-all template
should be resisted, as it is the process of
developing the factors and criteria for
the assessment which is most valuable in
ensuring its effectiveness.
While not necessarily an exhaustive list,
those factors should fit the four basic
requirements of the REM—objectivity,
simplicity, robustness and logic. All of
the elements should be objectively measured.
No one factor alone fully defines
risk. It is the combination of the factors
that presents risk.
Taking these principles into consideration,
the REM comprises eight steps:
- Step 1: Problem Statement. The team develops a problem
statement that defines the objective
and boundaries of the REM. This
helps the team stay focused. The objective
should be meaningful to those performing
and analyzing the assessment
and should not be broad beyond reasonable
usefulness, yet it should not be
too narrow as to be limiting.
- Step 2: Team Selection.
The REM team should consist
of subject matter experts in
the field defined by the problem statement.
They should represent multiple
departments and levels of authority.
The team does not have to include all
affected stakeholders. In this phase, the
objective is to define risk elements and
criteria, rather than inform stakeholders.
Team size is important. While it should
include all relevant stakeholders, large
groups can stifle participation. It should
include diverse participation, thus reducing
the risk of bias or preconceived
outcome. A facilitator may be of benefit.
- Step 3: Risk Factors Determination. The team identifies the parameters,
actions, events, conditions
or items that affect the objective or problem.
These should have measurable criteria.
The data for those criteria should
be accessible and understandable by anyone
doing the assessment. Brainstorming
may be used to select risk elements.
- Step 4: Criteria Setting. The criteria, limits or ranges
used to rank the parameters or
elements should be set by the team prior
to the assessment. The criteria should
be meaningful, logical, attainable, useful,
verifiable and measurable. The data should be accessible. Rules should be set
for accepting levels of risk. For instance,
are high levels of risk acceptable or must
steps be taken to reduce those risks? Are
less-than-low levels of risk accepted without
attempts to further reduce them?
- Step 5: Assessment Tool Development. The team should choose a tool
that evaluates the parameters
and meets the objective, both from a
product quality and process performance
perspective. One-size-fit-all tools
may not work for all applications. Customizing
a tool or model for a particular
use may be useful.
- Step 6: Risk Evaluation. The team performs the assessment
and obtains the risk rankings.
The meeting in which the evaluation
is performed should have a duration
target to help keep things on track. All
should have a voice at the meeting.
- Step 7: Mitigation Actions. The team or additional stakeholders
should determine, evaluate
and implement actions and steps based
on the ranking in an effort to improve the
process and/or meet the objective, considering
the rules set out in Step 4. Remember
that the purpose of the evaluation is
to improve the process, not merely identify
risk. Where actions or steps are to be
taken, consideration should be given to
the addition of risk from any unintended
consequences of the action or change.
- Step 8: Review. Follow up on actions to confirm
that they a) were implemented
properly, b) were effective, and c) did not
add any unintended consequences or additional
risk. Communicate results to affected departments. Perform periodic review of the REM to
confirm accuracy and effectiveness of results.
Now, that a case for aseptic process improvement has been made
and the REM introduced, Parts II and III of this article series will
show how the method is used to improve aseptic process interventions
and cleanroom environmental monitoring through examples
and case studies.
- Agalloco, J. et al. PDA Technical Report No. 22: Process Simulation
of Aseptically Filled Products. Bethesda: PDA, 2011.
- Ahmed, R., et al. PDA Technical Report No. 44: Risk Management
for Aseptic Processes. Bethesda: PDA, 2008.
- PDA Points to Consider for Aseptic Processing: Part 1. Bethesda:
- PDA Points to Consider for Aseptic Processing: Part 2. Bethesda:
- Baseman, H., and Long, M. “Risk Management of Microbial
Contamination Control in Aseptic Processing and Interventions
Risk Assessment Model (IREM): The Use of Critical Thinking
to Make Informed Decisions.” In Contamination Control in
Healthcare Product Manufacturing, Vol. 3, eds. Russell Madsen
and Jeanne Moldenhauer, 341-404. Bethesda: PDA/DHI, 2014.