A Risk-Based Approach to Deviation Management

ABSTRACT

A well-designed and implemented deviation management system offers a mechanism for obtaining critical quality data in a timely manner to enable quick response to failures, early warning of potential failures, and redeployment of resources to problematic areas. This article presents the key features of deviation management that can lead to early detection and resolution of problems and uncover gaps and weaknesses at a system's level that can help prevent potential problems in the future.

Each year, the US Food and Drug Administration issues multiple warning letters to establishments citing failures in managing deviations, for example, failure to thoroughly investigate unexplained discrepancies or products that did not meet specifications; failure to determine the scope of a deviation; failure to verify that corrective actions were effective; or failure to recognize, report, investigate, and correct serious discrepancies and deviations later discovered by the FDA during inspections.^1,2 Warning letters arise from inspection observations that are not satisfactorily resolved in a timely manner. In 2004 and 2005, "failure to thoroughly review any unexplained discrepancy" was among the 20 most common inspection observations and "failure to perform a proper investigation" was among the 10 most common observations in 2006 and 2007.^3–5

(PacificGMP)

Failure to understand and control process deviations exposes organizations to adverse regulatory action and financial loss, and compromises the organization's brand.

A well-designed and implemented deviation management (DM) system offers a mechanism for obtaining critical quality data in a timely manner to enable quick response to failures, early warning of potential failures, and redeployment of resources to problematic areas. It is one of the most valuable tools available to management to help maintain a state of control. To be successful, the DM process must work for the organization rather than the organization working for the system. It must be designed to perform at the correct level to meet the organization's needs and to deliver optimal results. This requires incorporating risk-management principles, prioritization, and an understanding of conflicting interests among the consumer, regulatory agencies, and the business.

This article presents the key features of DM, which if incorporated into working policies and procedures, will not only lead to early detection and resolution of problems but also will uncover gaps and weaknesses at a system's level that will help prevent potential problems. The result will eliminate redundancies in processes for investigating nonconformances; promote comprehensive and permanent solutions; strengthen management's oversight capability; reduce the probability of product recall; and enhance the company's bottom line.

DISCOVERY AND REPORTING OF DEVIATIONS

In this article, the term deviation encompasses events often referred to as nonconformances, errors, discrepancies, failures, or problems and is defined as unexpected or unplanned departures from current good manufacturing practices (cGMPs), regulations, standards, procedures, or specifications that may affect product safety, quality, identity potency, or purity.

The point of entry into the DM system is the discovery and documentation of a deviation on a standard report form (including automated reporting systems). Other systems can feed into the DM process at this point, e.g., discrepancies and nonconformances discovered during equipment calibration, stability testing, complaints management, production, labeling, validation, and so on. Integrating systems in this way eliminates the need for redundant processes such as investigation and corrective action planning and ensures all deviations, regardless of the system in which they occur, are handled in a consistent manner and generate trending data based on the same parameters.

A deviation should be documented with a description that is clear and concise and briefly states the who, what, where, and when information. Extraneous information that confuses the investigator and does not add value to the description of the problem should be avoided.

A good example of a deviation description is: "On Friday Feb 3, 2008, during the manufacturing of drug ABC, in room B-2 the technician failed to take the hourly in-process sample at 4 pm as per SOP 1234."

Compare the above example with the following deviation description that leaves the reader confused about the event reported: "Cycle count from machine hours 4:20 to 9:36 is not correct. The 24-hour inspection not performed. Cycle count from bin 9 to 11 is not possible." It is difficult to tell from this statement what the actual problem is or why the cycle count is not correct, how this is known, or who discovered the problem and when.

At the point of discovery, a knowledgeable subject matter expert should evaluate and assess the risk associated with the event. Risk is commonly defined as the combination of the probability of occurrence of harm and the severity of that harm. Deviations range in degree of criticality or potential risk; many are minor and can be corrected on the spot while others present a higher safety risk and require more work. Therefore, deviations must be handled in a manner that is commensurate with the level of risk. Higher risk deviations, that are a risk to the customer, (i.e., health or safety), risk to the business (e.g., loss of product or production), and regulatory risk (e.g., warning letters, recalls) may require immediate or containment actions to stop the deviation from continuing, to contain the damage or to gain control of all potentially affected products.

PERFORM A THOROUGH INVESTIGATION

The level of investigation should be commensurate with the level of risk. Applying risk-assessment principals helps ensure resources and efforts are used where they are needed most. Instituting a risk-based approach to manage deviation investigations is key to focusing on problems that could result in patient harm. A major biotechnology company recently described how it implemented this type of program.⁷ Deviations are categorized according to the level of associated risk and investigation techniques are applied proportional to the risk. It is expected that the highest-risk deviations, which are fewer in number, will consume the major part of resources dedicated to DM. The concept of varying the thoroughness of investigation according to the nature of the problem, and the definition of the elements of a thorough failure investigation have been recorded in case law by the United States District Court for the District of New Jersey in United States v. Barr Laboratories, Inc.⁸

Investigations begin with fact-gathering activities designed to collect as much information as possible to enable proper evaluation of the event. The end product of the investigation is an in-depth analysis of the deviation that will lead to determination of the root cause, not merely a restatement of the problem.

To produce the best results, a team of subject matter experts from the affected department(s) should conduct the investigation according to established procedure(s).

Information sought includes, but is not limited to:

• the scope of the deviation (i.e., the period over which the deviation occurred, the number of times the deviation occurred, the number (and identity) of products involved)

• facts and events surrounding the deviation

• the involvement of other facilities.

Investigation methods include:

• interviewing people closest to the problem

• reviewing records and documents

• inspecting and testing products and materials

• inspecting equipment and facilities

• observing operations.

• reviewing past deviations of the same type

• experimentation or simulations.

As information is gathered, investigators should resist the temptation to jump to conclusions about what happened and how it happened because this could prevent the discovery of additional information leading to a different or more comprehensive conclusion. There are also pressures to complete the investigation quickly because production and product release may be affected. However, an incomplete investigation affects the ability to analyze risk and the data, which will be used to formulate the preventive action. If done haphazardly, this demonstrates that the process is not in a state of control and the problems will probably recur.

PURSUE THE REAL ROOT CAUSE

The importance of performing a good root cause analysis (RCA) cannot be overstated because the actions taken to correct or prevent the deviation from recurring are directly related to and depend on finding the right cause. However, investigators often fail to dig deeply enough to find the cause and apply the wrong corrective action, thereby risking recurrence of the problem. By prematurely ending the search, investigators may incorrectly focus on placing blame on an individual involved or providing unneeded retraining rather than seeking an opportunity to design safety into a process. Other notions that impede successful RCA are:

• believing that when more than one root cause, or potential root cause, is identified, only one may be selected

• settling on a best guess of the cause as the quickest way to solve the problem and close the deviation.

Several techniques that support comprehensive root cause analysis include:

• Brainstorming: an interactive group-thinking process to identify all possible causes. The exercise can be structured or unstructured. This is a process carried on without criticism or judgment that stimulates thinking and generates many enhanced ideas. ⁹

• Cause and effect or fishbone diagrams:⁹ a technique which expands the scope of thinking beyond a single type of cause and also allows the focus to be centered on the core of the issue and not the history or personal opinions. This process prompts the problem solvers to consider the five basic components of a process (people, procedures, materials, equipment, and facilities or environment) as possible sources of root causes.

• Failure mode and effects analysis (FMEA):¹⁰ a step-by-step approach for identifying all possible failures in a design, a manufacturing or assembly process, or a product or service. Failures are prioritized according to how serious their consequences are, how frequently they occur, and how easily they can be detected. The purpose of the FMEA is to take actions to eliminate or reduce failures, starting with the highest priority ones.

• Five why matrices: a process that has been determined to uncover the real root cause by asking "why" five times. This expands the depth of thinking beyond the first thought or impression.

The root cause of a deviation can usually be attributed to a defect in the system design (e.g., inadequate procedures, materials, equipment, facilities, or unqualified personnel), system implementation, or individual performance.

In circumstances where multiple potential root causes have been identified, investigators should narrow the list from possible causes to most probable cause(s). They can use various analytical tools and techniques such as voting on the more probable causes based on objective evidence, rather than gut feeling.

Generally, industry practice dictates that deviation investigation and root cause determination are completed within 30 days of discovery of the deviation. In fact, in the U.S. v. Barr Laboratories case, the Court declared that “…all failure investigations must be performed promptly, within thirty business days of the problem's occurrence, and recorded in written investigation or failure reports”.¹¹ If this is not possible, justification for the delay should be documented in the investigation report along with the date of expected completion. The deviation investigation process is shown in Figure 1.

Figure 1

DEVELOP AND IMPLEMENT CORRECTIVE AND PREVENTIVE ACTIONS

After the determination of root cause(s), corrective actions can be determined. To narrow the field of possible corrective actions and select the best solution, the following techniques can be used:

• Matrix diagram:⁹ a method in which the team selects the best of alternatives of the solutions and rates them.

• Force field analysis:⁹ a method of weighing the pros and cons of change to identify the forces and factors that support or work against the problem.

• Affinity diagram:⁹ a method for sorting multiple ideas into categories.

To be effective, corrective action(s) must address the root cause. In fact, a strong correlation or linkage should be evident throughout the deviation report from one section to the next, i.e., the description should relate to the investigation, which should relate to the cause and that should relate to the corrective action, and so on. In some circumstances, the identified root cause may require several corrective actions. In these cases, the corrective actions may be implemented consecutively, rather than simultaneously, to determine the impact of any single action on the cause. Consideration should be given to the severity and frequency of the occurrence, demands on resources, and priority of the deliverables.

The team that is involved in determining and implementing the best solution(s) should be one that has the authority and responsibility to make it happen. If the team determining the action to be taken lacks the authority, even the most logical or rational corrective action may not be implemented or implemented incorrectly because the time, money, and resources were not authorized.

Corrective and preventive action (CAPA) plans require that individual tasks and deliverables, timelines, roles, and responsibilities be documented. This provides a mechanism for tracking completion of all activities associated with the action plan. Progress reports should be sent to the affected department managers and the quality unit on a timely basis to ensure that timelines are met and any problems are addressed in a timely manner. If timelines cannot be met, justification for the delay should be documented and forwarded to the quality unit for review and concurrence. Senior management must be informed of corrective actions that have passed their target dates. Timely notification enables the management to deploy resources where needed.

The benefits derived from a well thought out corrective action based on in-depth root cause analysis are multiple and include the following:

• prevention of deviations from recurring and prevention of potential deviations from occurring

• reduction in recalls and market withdrawals

• reduced safety and regulatory risk

• increased customer satisfaction resulting from consistently produced quality products

• redeployment of resources from fixing problems to other projects and commitments

• increased employee satisfaction (reduced frustration) in resolving problems in an efficient, effective manner

• cost savings from a decrease in discarded or reworked products.

MONITORING

Monitoring the performance of the organization's quality system is essential for continuous improvement and the functionality of the DM system plays an important role in this process. Useful information may be obtained in a variety of ways including monitoring the process outcomes (e.g., effectiveness of CAPA); trending deviations; and establishing performance measures to ensure system objectives are met (e.g., timeliness of investigations and completion of corrective actions).

In some circumstances, it may be necessary or desirable to monitor the effectiveness of CAPA. Generally, this need is based on the criticality of the deviation and the magnitude of the corrective action taken (i.e., time, money, or resources consumed). Methods for checking effectiveness may include:

• reviewing the DM tracking system to determine whether the expected decrease or elimination of like deviations has occurred

• using the internal auditing program to determine whether the problem has been eliminated

• examining relevant documentation during annual product review to determine whether corrective action has effectively addressed the problem

• looking for unanticipated adverse consequences resulting from a change.

The method and schedule for performing effectiveness monitoring should be determined at the time corrective action is developed. Typically, effectiveness is confirmed within three to six months postimplementation.

Deviation trending is an important way to monitor the overall performance of operations and identify potential systems-based problems. It should be performed on a routine basis (e.g., monthly or bimonthly). A useful way of trending involves categorizing deviations based on root cause. Root causes may be classified by function or by quality system and sub-system. Deviations also can be categorized by process or product line. Trend reports must be reviewed and analyzed by the management and the quality unit to identify the existence of significant trends and determine the need for action to address adverse trends.

Performance measurements for the DM system should be established and monitored on a routine basis to ensure the system is performing as designed and meets expectations. Select the elements that are most critical or appear to be problematic and design metrics around these. Useful metrics do not need to be overly complex and the number should not exceed three or four. Limiting the number of metrics is more likely to result in useful data that will aid in identifying potential problems and trends, i.e., too many measuring points equals too much information equals loss of focus. Examples of performance measures are:

• number of times CAPA plans fail to correct the deviation (note: this is often an indicator of how well RCA is performed)

• number (or percent) of times established timeframes for completing a deviation investigation are not met

• number (or percent) of times target dates of CAPA are not met

• number of times status reports are not created and circulated.

THE ROLE OF THE QUALITY UNIT

The primary role of the quality unit in deviation management is collection and analysis of quality data and providing independent oversight. The quality unit staff are involved in assuring that each key step is performed appropriately from discovery and reporting through corrective action completion and effectiveness verification. The quality unit assures that investigators have explored all aspects of the problem, assessed the potential risk, and used the appropriate investigative techniques. Through their independence, the quality unit has direct, uncompromised access to senior managers to communicate critical quality information. They possess the ability to balance the conflicting interests that threaten to undermine the DM system. Often, the quality unit fulfills the role of system administrator or system owner. Some of the tasks the quality unit performs include:

• initial review of deviations after discovery for evaluation of criticality

• review and approval of investigation report

• preside over material review board for product disposition problems

• review and approval of CAPA plans

• Monitoring and ensuring that DM timelines are met

• Trending deviations

• Generate and issue DM status reports

• Expedite communication to senior management about hazardous deviations

• Provide training on the DM system to all staff.

DEFINING MANAGEMENT'S ROLE AND RESPONSIBILITIES

The senior management bears ultimate responsibility for establishing, implementing, and resourcing all of the organization's systems. Therefore, they have direct accountability for brand integrity and increased risks arising from inefficient, ineffective, and unsafe systems. The management must balance multiple objectives including supplying a safe and effective product, maintaining profitability, sustaining regulatory compliance, and supporting continuous quality improvement. They have both the authority and the responsibility to ensure these objectives are met. A well-designed and implemented DM system that incorporates risk-management principals provides an indispensable tool to help managers fulfill their obligations.

One major biotechnology company recently reported a "...major improvement in the company's corrective and preventive action process that significantly reduced the company's perceived liability exposure and saved millions of dollars..." when they instituted a management review.¹²

In addition, active engagement of senior management is essential for the successful functioning of a DM system. Managers demonstrate their involvement by:

• engaging staff routinely in interactive process design and assessment work.

• encouraging open communication without fear of reprisal, listening to staff input, and soliciting feedback in the context of just culture.

• supporting value-added training that enhances job knowledge and skills

• monitoring systems performance through inquiries, review of status and progress reports, and reacting to critical information in a timely manner.

• ensuring line staff participate in deviation investigations. Too often a manager or supervisor performs the deviation investigation and determines the root cause and corrective action without staff input despite the fact that staff may have daily involvement and extensive experience. The involvement of staff also encourages them to anticipate potential problems and communicate them to the appropriate manager.

• Senior management is also responsible for exploiting the benefits offered from continually monitoring and analyzing the system's data output. This enables more informed decisions about the need for changes and timely actions to prevent serious and costly problems. The DM system will provide a mechanism for timely and effective escalation of quality issues to the appropriate management level. It feeds key information into the management review process and helps ensure continual compliance with regulatory requirements.

CONCLUSION

In summary, the DM system that integrates risk management throughout the process will help the organization discriminate between critical and noncritical issues and better manage the sometimes conflicting interests of business, regulatory, and customer requirements.

The design of deviation management should encompass key elements including timely reporting, assessing risk, taking necessary immediate containment actions, performing thorough investigations and RCA, completing appropriate CAPA, and monitoring the effectiveness of corrective actions. The DM system must also provide for deviation trending to predict performance changes and allow for preventive action, tracking and status reporting on a routine basis; and an escalation process to communicate critical issues to appropriate management.

Finally, the deviation management system with strong management support can be advantageous, not only to the staff (by providing a process that works for them), but also to the organization's bottom line by offering benefits such as reducing product rework and destruction, and time spent on failed corrective actions. Collectively, these help assure regulatory compliance and protection of the organization's brand.

Gail Bredehoeft is principal consultant and Judy O'Hara is senior consultant at Parexel Consulting, St. Lowell, MA 01851, Gail.Bredehoeft@parexel.comJudy.O'Hara@parexel.com

REFERENCES

1. US Food and Drug Administration. Warning Letters. Available from: http://www.fda.gov/foi/warning.htm.

2. The Gold Sheet: Pharmaceutical & Biotechnology Quality Control. 2007;41(4):13–17.

3. US Food and Drug Administration. 483 Observations from 1/04 to 3/05. Available from: http://www.fda.gov/ora/frequent/483s/boston_sci_2005_var/watertown_ma.html.

4. Pharmaceutical Technology, December 2, 2006. In the Field Section: Table 1, Causes of Form 483 and Warning Letter Citations based on 818 FY 2006 Citations .

5. US Food and Drug Administration. Turbo Establishment Inspection Report.

6. International Conference on Harmonization. Q9, Quality Risk Management. Geneva, Switzerland; 2005.

7. The Gold Sheet: Pharmaceutical & Biotechnology Quality Control. 2008;42(1):14.

8. United States v. Barr Laboratories, 812 F. Supp. 458, 467–68 (D.N.J. 1993).

9. Brassar M, Ritter D. The Memory Jogger II, A Pocket Guide of Tools for Continuous Improvement and Effective Planning. 1st ed.

10. The Quality Toolbox, 2nd ed. ASQ Quality Press;2004:p.236–242.

11. Barr Labs, 812 F. Supp. at 468.

12. The Gold Sheet: Pharmaceutical & Biotechnology Quality Control. 2008;42(1):13.