21 CFR Part 11: Choosing a Risk Assessment Methodology

FDA issued the 21 CFR Part 11 Scope and Applicability Guidance Document partly due to industry concern that the breadth of applicability and the cost of Part 11 compliance have hindered the use of new technology. The guidance states that records must still be maintained in compliance to the underlying predicate rules, but FDA will take a "risk-based" approach to enforcing some of the technical controls for Part 11, such as validation, audit trails, record retention, and record copying. FDA will also include Part 11 in its formal review of current Good Manufacturing Practice (cGMP) regulations and follow a more subjective course in taking regulatory action. FDA's intent is to return emphasis to the predicate rules that govern Good Clinical Practice (GCP), Good Laboratory Practice (GLP), and cGMP, together referred to as GxP.

Due to these predicate rules, a risk-based approach to protecting product quality and public safety is not new for drug manufacturers. For example, FDA expects a firm subject to GxP to develop a risk evaluation of its product and to mitigate the identified risks. Identified risks may be effectively eliminated, or their likelihood of occurrence or severity of consequences may be reduced to acceptable levels. There may be risks for which there are no technical fixes. Such risks may be addressed by including warnings in product labeling. Some risks may be so minimal that no specific action is required to maintain them at acceptable levels.

What is Risk?

Today, a risk analysis must be included in a firm's Part 11 remediation plan. It should address how systems that generate regulated electronic records can potentially affect consumer safety. A useful definition of risk is available in the 1999 ISO/IEC Guide 51: a combination of the probability of occurrence of harm and the severity of that harm. Whether applied to Part 11, or to other safety-related aspects of FDA-regulated products, risks to product quality or public safety are central. Such products obviously include foods and cosmetics, blood products and drugs, medical devices, and any other regulated products that are ingested or consumed by or applied to a living creature. When a system generates electronic records that can greatly impact product safety and quality or the integrity of other regulated records, it is considered a "high-risk" system, and the technical controls of Part 11 that protect electronic record integrity apply. Otherwise, the system is considered to be "low-risk," and the agency will simply enforce the GxP requirements for record integrity instead of the more stringent Part 11 controls.

Table 1: What is risk?Table 1 compares some of the high vs. low risk systems that generate GxP records according to a recent ISPE White Paper submitted to FDA.¹

In fact, applying a risk-based approach to Part 11 compliance should be nothing new for regulated firms. Quality System Regulation (QSR) requires a firm to perform a risk analysis of the various systems that generate or maintain electronic records and implement electronic signatures. This analysis allows companies to determine which records have high impact on consumer safety. The firm can then rank the identified risks according to their criticality.

Part 11-Triggering Systems

For example, quality data from a Part 11-compliant database may be included in a Corrective and Preventative Action (CAPA) report, typically generated by a spreadsheet program. The spreadsheet formulae should be validated according to GxP. However, the overall relative risk to public safety is low, and therefore the typical Part 11 technical controls (for example, audit trails) are not required to protect the integrity of the spreadsheet.

On the other hand, adverse event reporting and clinical trial data that fall under GCP regulation potentially can have a high impact on public safety. Programs that analyze and visualize clinical data subsequently have an impact on record integrity. Such systems would be considered high risk and therefore should incorporate the technical controls for Part 11 compliance as well as maintaining compliance with predicate rules.

In summary, Part 11 remediation has not changed for high-risk GCP-related systems such as adverse event and case report form (CRF) data management systems, SAS analysis software, web trial systems, electronic patient diaries, patient randomization, and trial supply labeling systems. Both GCP and Part 11 definitely apply to these high-risk systems. (In addition, FDA's Guidance to Industry for Computerized Systems Used in Clinical Trials remains in effect and applies to these systems as well.)

Keep in mind that while Part 11 is an enforceable law, an FDA guidance document is not a law. Guidance documents present FDA's current thinking on a subject and are only a recommendation on how to address a law's requirements. Guidance documents are not binding on either industry or the agency.

Risk Assessment Methodologies

There are many risk assessment protocols or methodologies available. What follows is a discussion of the most common.

A fault tree analysis (FTA) is a deductive, top-down method of analyzing system design and performance. It involves specifying a "top event" to analyze, followed by identifying all of the associated elements in that system that could cause that top event to occur. Fault trees provide a convenient symbolic representation of the combination of events resulting in the occurrence of the top event. Events and "gates" that allow events to occur are represented by graphic symbols. Sometimes certain events must occur together in order for the top event to occur. In this case, these events are arranged under an "AND" gate. If the individual events each can trigger the top event alone, they are grouped under an "OR" gate. The entire system, including human interactions, is analyzed when performing a fault tree analysis.

Failure Mode Effects and Analysis (FMEA) categorizes and ranks potential process failures, or critical issues, and targets their prevention. When the analysis quantifies the probability and criticality of potential failures, it is referred to as failure mode effects and criticality analysis (FMECA). FMEA prioritizes potential failures according to their risks and then implements actions to eliminate or reduce the likelihood of their occurrence. FMEA is a tool that should identify product and process failures before they occur, identify appropriate risk mitigation measures to prevent or otherwise control the failure, and ultimately improve product and process design. FMEA assumes that all product and process failures (and the actions required to control these failures) are predictable and preventable. Surprisingly, organizations still frequently experience predictable and preventable failures with costly consequences, and FMEA can help address these. Such failures can lead to product recalls, death or injury, poor quality, and unanticipated cost. Although the aerospace and defense industry have used FMEA for decades, this methodology recently has been making significant inroads into the biomedical device industry.

Hazard Analysis and Critical Control Points (HACCP) is embraced by FDA. HACCP was first used in food production in the 1960s. In 1996, the US Food Safety and Inspection Services Task Force (FSIS) developed a HACCP-based regulatory proposal that became the Pathogen Reduction/Hazard Analysis and Critical Control Point Systems (HACCP) Rule. With this rule, FSIS determined to reduce the risk of food borne illnesses from meat and poultry products by ensuring that appropriate and feasible measures were taken at each step in the food-production process where hazards are present and where procedures and technologies exist (or can be developed) to prevent or reduce the likelihood of these hazards.

HACCP has seven basic steps (see "The Seven Basic Steps of HACCP" sidebar). It is understood that implementation of HACCP does not mean the absolute elimination of risks, but rather, the reduction of hazards to reduce risk to an acceptable level.

Where to Start

It is necessary to analyze computer systems and information-handling processes to assess not only risk but also the cost of converting paper-based information to an electronic format. A good place to start is to perform a system assessment. Plot your various systems and processes on a simple X-Y matrix that measures (from low to high) the risk to security of the data (X-axis) and the cost of remediation (Y-axis). Then you can prioritize the systems and processes needing upgrades or replacement based on their position in the matrix. Systems that fall in the "high data security risk, low conversion cost" area of the matrix could be targeted first for compliance validation.

Because they had to address Y2K issues, many organizations have already generated an inventory of all their computer systems and hopefully evaluated the risks associated with computer error or failure. Companies with cost considerations and many non-compliant computer systems must, of course, prioritize which systems to remediate first.

The following is a list of systems to be reviewed in a typical criticality assessment (high to low) for nonclinical laboratory systems.²

• systems for quality processes and standard operating procedures
• lab spreadsheets and databases for data collection
• systems for other R&D data
• central database for inventory management
• systems for liquids processes
• systems for company financials
• systems for customer relationship management
• systems for packaging
• systems being decommissioned

Conclusions

21 CFR Part 11 is not going away and the FDA intends to enforce it. What has recently changed is the adoption of a narrower scope for the rule, a new understanding of agency enforcement discretion, and the application of a risk-based approach to compliance. When choosing a risk assessment protocol or methodology for Part 11 remediation, it is important to use basic common sense. Identify the greatest potentials for risk to product quality (and ultimately to public safety) and implement measures to mitigate those risks. Finally, document the entire endeavor. Whether you choose to adopt a standard risk assessment methodology or develop your own, remember that FDA will show enforcement discretion if you have a well-documented plan in place and if you are making true progress toward implementing your plan.

The Seven Basic Steps of Hazard Analysis and Critical Control Points

1. Conduct a hazard analysis:

• Define terms of reference
• Select the HACCP team
• Describe the product
• Identify intended use
• Construct a flow diagram
• Conduct on-site verification of flow diagram
• List all hazards and control measures

2. Determine the critical control points (CCPs) using a decision tree. CCPs are the points where hazards must be eliminated or minimized.

3. Establish critical limits that must be met to ensure CCPs are under control.

4. Establish a system for monitoring the control at the CCPs.

5. Establish the corrective actions to be taken when monitoring indicates that a particular CCP is not under control.

6. Establish procedures for verification to confirm that the HACCP system is working correctly.

7. Establish documentation for all procedures and records.

References

1. ISPE. Risk-based approach to 21 CFR Part 11. Available from URL:

www.21cfrpart11.com/files/library/compliance/isp_riskbasedapproach21cfr.pdf

.

2. Clarkston Consulting. 21 CFR Part 11 compliance: an enterprise issue, not a point solution. Available from URL: www.clarkstonconsulting.com/WhitePaper/Part11Compliance.pdf.