Utilizing Run Rules for Effective Monitoring in Manufacturing

Sorendls/Getty Images

To maintain a state of control and comply with regulatory authorities, many pharmaceutical, biotech, and medical-device companies have adopted continued process verification (CPV) initiatives for manufacturing processes (1). By adopting a proactive approach to monitoring, life-science manufacturing companies can identify changes in their manufacturing processing prior to a quality event such as a batch failure. Control charts are the predominant monitoring tool used to monitor parameter data across batches with control limits defined for the normal manufacturing process. Control chart run rules generate signals when non-random processes occur and flag potential process issues. The maximum partial pressure of CO₂, for example, may have an undesired upward trend due to a machine calibration issue, and a signal would prompt a process engineer to investigate before drug potency was affected. Valid signals can reduce costs, improve process understanding, and enhance operational reliability (2).

Despite the well-defined benefits of run rules, many life-science companies misuse them due to lack of guidelines and/or statistical expertise regarding statistical process control, which may lead to incorrect signals (i.e., the inability to differentiate between signals and random noise) and, ultimately, failure of the monitoring system. Essentially, the goal when using run rules is to ensure that valid signals are generated, correct signals are not overlooked, and false signals are not created.

This article discusses the life-science manufacturing industry’s current use of monitoring techniques and provides guidance on how to improve the value obtained from monitoring programs and run-rule signals.

Current industry practices
Life-science manufacturing companies with mature CPV initiatives apply run rules to critical/key process parameters and quality attributes for trending purposes (4). Despite small variation among companies, run rules are typically applied to parameters based on risk assessments (e.g., process failure mode and effects analysis, estimated process capabilities, etc.). Two common issues life-science companies often face in the early stages of adopting a CPV initiative are over-alerting (e.g., generating false, non-value-added signals) or under-alerting (e.g., overlooking valid signals). Over-alerting is often the result of monitoring every parameter with every run rule. Conversely, under-alerting can occur when companies only evaluate if critical release parameters fall within specification limits and adequate monitoring is not performed. The objective of run rules in a mature CPV program is to generate valid signals that provide useful information for engineers and process experts.

One of the principal benefits companies receive from using run rules is the enablement of a monitoring-by-exception solution. Instead of manually reviewing control charts on a regular basis, selected software programs can automatically alert users to run-rule signals (i.e., when a run rule is violated). Then, resources may be allocated appropriately to evaluate parameters with signal violations. A monitoring-by-exception solution that can scale up to multiple sites and products is essential to a CPV model.

Regardless of their specific practices, companies should clearly document the proper use of run rules in their monitoring procedures to avoid significant regulatory risk, and responsible individuals should be assigned to create and maintain these crucial documents. Implementation will also likely require ongoing evaluations and support from the company.

Getting the right signals
Generating valid run-rule signals provides a variety of benefits, primarily through cost reductions and deployment of an early-warning system to prevent product quality issues. Obtaining valid signals to drive these business benefits, however, is dependent on five mathematical assumptions: parameters must be baselined from an in-control process, baselined on an acceptable sample size, detailed regarding data precision, normally distributed, and independent.

Receiving correct value-added signals requires the use of historical data to establish baseline control limits for ongoing monitoring. Many companies overlook the criticality of baselining parameters, which results in inflated control limits and missed valid signals. To ensure the process is in control, special cause variation needs to be identified and removed during the baseline process. Special cause variation results from variability caused by events outside of the normal manufacturing process, such as operator error, power failure, etc. Eliminating special cause variation provides limits that more accurately reflect nominal manufacturing processes. Baselining should occur after a known process change occurs (e.g., equipment change, supplier change, etc.) or on a regular basis (e.g., annually or semi-annually) depending on the availability of staff, the number of batches produced in a year, and other parameters. When one is baselining parameters, at least 25–30 batches should be available to generate targets that accurately reflect the manufacturing process. The monitoring system should allow engineers to differentiate valid signals from random process noise, as well.

To produce valid signals that generate value, an organization needs to measure and record process variability accurately with sufficient data precision. This is especially important when the decimal precision for parameters results in a small number of unique values
an issue that often plagues the life-sciences manufacturing industry. For example, if pH is reported at a single decimal precision and values are always 6.7, 6.8, or 6.9, run rules cannot be appropriately applied and false signals may be generated or valid signals may be overlooked.

For an organization to effectively use control charts and run rules, parameters need to follow a normal distribution. The assumption of a normal distribution relies on the fact that only 1 out of 370 observations will fall beyond control limits just by chance (5). Abnormal data results in an inflated likelihood that data will fall beyond control limits (e.g., a one out of 40 chance) (6). Non-normal distributions are highly prevalent in life-science manufacturing processes, and they are often expected when processes have lower or upper bounds (e.g., yield, hold time, etc.). Not correcting for this assumption violation, however, may lead to false signals, which have the effect of devaluing the significance of a violated control limit. Over-alerting by generating false signals can result in employees simply ignoring the monitoring system.

CLICK TABLE TO ENLARGE Table I: Monitoring assumptions checklist.

Non-independent parameters have narrower control limits and are more likely to generate false signals, resulting in wasted resources and an inefficient monitoring system. The assumption of independence assumes that previous batches do not affect subsequent batches, and non-independence often indicates there is a pattern in the data. It is crucial that CPV and monitoring plans provide directions on how to test assumptions and correct for assumption violations so that monitoring-by-exception solutions generate appropriate signals.

To achieve the most value from a monitoring system and generate valid signals, one needs to continuously evaluate and correct the assumptions underlying a system. Without these corrections, false signals will waste company resources and valid signals may be overlooked. Table I provides assessments and suggested actions for the aforementioned assumptions.

Proper response to signals
Life-science companies with mature CPV models derive value from run-rule signals by implementing appropriate follow-up practices. When a monitoring system is configured correctly, incorporating all the steps defined in Table I, signals should be valid and indicate that the process has changed in some way. Thus, when a run-rule violation occurs, it is important to appropriately follow up on the signal. Conversely, life-science manufacturing companies with immature CPV initiatives spend a lot of non-value added time chasing false signals, or they revert to the other extreme of not following up on signals at all. Limited resources and statistical expertise can make following up on violations a struggle for many life-science manufacturers.

CLICK FIGURE TO ENLARGE Figure 1: Examples of signals denoting process events/changes.

It is crucial to understand that run-rule signals should be treated differently than out-of-specification quality events. Quality events require immediate corrective and preventive actions and investigations, whereas not every run-rule signal should be investigated. If a parameter has been relatively stable with infrequent and inessential signals for example, it is acceptable to take no immediate action. However, it is important to track parameter signals over time so that multiple violations or problematic patterns are not overlooked. If a signal does require follow-up, it is crucial to identify the type of violation and follow-up with appropriate investigative techniques to detect a root cause. Although a multitude of process changes can occur, run rules are designed to detect some of the more common signals: single events, shifts, trends, oscillations, increasing/decreasing variability, and cycles (see Figure 1).

CLICK TABLE TO ENLARGE Table II: Common run rules used in the manufacturing industry.

Identifying the type of signal and appropriate follow-up techniques will bring significant value to a business by reducing resource and time requirements. Each signal indicates that a different process change/event occurred, and investigations should be tailored based on the signal (see Table II. Once a signal has been detected and a decision has been made to pursue it, typical follow-up steps include:

• Confirm the value is correct and not a data entry error.

• Determine if there is a readily apparent root cause or if other issues occurred in the process.

• Discuss the signal with subject matter experts.

• Examine the lot traceability and review parameters related to the parameter of interest.

• Escalate to other groups for further statistical investigation.

While run rules provide a useful tool for detecting some non-random processes, not all patterns can always be flagged. Additionally, signals can be overlooked for parameters that are being trended with few run rules. Thus, it is vital to review all parameters regularly (e.g., annually) to ensure that trends are not missed. Individuals responsible for configuring, maintaining, and reporting on the monitoring system should be clearly identified in company documents. Properly responding to signals will result in more efficient use of resources and time. Additionally, investigating signals can enhance process understanding, which should improve process monitoring and operational reliability.

Conclusion
Proper CPV monitoring is a crucial business value investment for a company. A monitoring-by-exception system that generates valid signals can reduce resource requirements, proactively identify issues prior to a quality event, and create a regulatory compliant environment. Today’s life-science industry can improve productivity and compliance by developing more mature CPV initiatives and adopting practices to ensure that they get the right signals. Furthermore, performing appropriate follow-up on valid signals is vital to an effective and continuously improving monitoring solution. 

References
1. FDA, Guidance for Industry: Process Validation: General Principles and Practices (Rockville, MD, January 2011).
2. BioPhorum Operations Group, Continued Process Verification: An Industry Position Paper with Example Plan, BPOG-Biophorum Operations Group (2014).
3. R.J. Seely, L. Munyakazi, and J. Haury, BioPharm Int. 14 (10), pp. 28-34 (2001).
4. J. M. Juran, Juran on Quality by Design: The New Steps for Planning Quality into Goods and Services (Simon and Schuster, New York, NY, 1992).
5. D. Montgomery, Introduction to Statistical Quality Control (John Wiley, Hoboken, NJ, 5th ed., 2005).
6. S. Steiner, B. Abraham, and J. MacKay, Understanding Process Capability Indices (University of Waterloo, Ontario, 1997).

About the Author
Aaron Spence is manager of analytic consulting at BIOVIA.

ALL FIGURES ARE COURTESY OF THE AUTHOR.

Article Details

BioPharm International
Vol. 28, No. 9
Pages: 48–51, 57
Citation: When referring to this article, please cite it as A. Spence, "Utilizing Run Rules for Effective Monitoring in Manufacturing," BioPharm International28 (9) 2015.