Validation of Commercial-Off-the-Shelf Calibration Management Systems: Managing Instruments Cost-Effectively

Biotech companies are among the most intensive instrumentation users of any FDA-regulated business. Because of the complex nature of these companies' research and manufacturing, they typically have a larger number of instruments per employee. Most biotech startups begin the process of managing calibration by developing simple spreadsheets or databases. As these companies grow, they add hundreds, or even thousands, of instruments and typically have difficulty managing the calibration process.

As time wanes and resources evaporate, companies (specifically in FDA-regulated industries) cannot afford to develop in-house systems that improve productivity and manage compliance. Organizations must look to software vendors that focus all their resources on the development of such products.

In the biotech industry, the use of commercial-off-the-shelf (COTS) calibration management software (CMS) is a growing trend. GMPs are a major driver for CMS functions such as multi-tiered security, audit trail, and log files (1). Under GMPs and 21 CFR Part 11 requirements, a software package must be validated, as it is a part of the quality assurance of the manufactured product (1, 2, 3, 4).

Validation provides assurance that the system is reliable and suitable for its intended use, and it demonstrates that the system operates in a state of control. There are three important steps to validating a software system: user requirement specification (URS), vendor qualification, and user validation.

User Requirement Specification

To validate a software's functionality, biotech companies must verify the software's functionality against the URS. The URS should explain what the software needs to do, not how to do it. Carefully defining requirements in advance makes it possible to determine later that they have been met.

Good Automated Manufacturing Practices Guide 4 (GAMP) gives guidelines for establishing a URS (5). It states that

• Each requirement statement must be uniquely referenced and no longer than 250 words.

• Requirement statements should not be duplicated or contradicted.

• The URS should express requirements, not design solutions.

• Each requirement should be testable.

• Both the customer and supplier must understand the URS; avoid ambiguity and jargon.

• When possible, the URS should distinguish between mandatory and regulatory requirements and desirable features.

• A formal review of the URS by the customer and supplier may be necessary to check that both parties understand it and that the functional specification's requirements have been met.

These URS guidelines are especially important when selecting a commercial software package. Since it is not a custom-designed system, the user must verify, through the URS, that it is suitable for its intended application. Ultimately, the URS is used to customize the validation protocol.

Vendor Qualification

In the highly regulated biotech environment, the software package user must verify the vendor's qualifications. The user should not only evaluate the history, financial status, and market share of the vendor, but also the Software Quality Assurance (SQA) program used in developing a package.

Once a software package has been selected, there are a number of qualification techniques to ensure vendor integrity, including a vendor audit (both on-site and via surveys) and vendor documentation, such as an SQA. These can be costly, but ultimately it is the biotech's responsibility to verify that the vendor uses appropriate methodologies.

A well-documented software-development lifecycle (SDLC) is the vendor's responsibility. A typical SDLC begins with functional requirements that are translated into design requirements, leading to the initiation of development activity. The next step is testing and internal validation and finally, controlled release of the software. The SDLC also tracks additional maintenance activities, such as bug fixes and enhanced functionality.

The SQA process is an important step for a vendor to authenticate practices. As an internal validation method, the SQA documents the procedures and tests against a software package's functional requirements.

User Validation

Once the software and vendor have been qualified, the implementation process begins. It typically involves installation, training, procedure writing, and - most important for a regulated industry - validation. Although other companies successfully use the same software package, validation is required because it is used under different conditions within industry.

Creating a master plan is key to effectively validating a software package. This plan gives a step-by-step outline of the validation process. In the end, it produces documentation that the system is in a state of control.

A validation protocol (provided by vendors of most COTS packages) specifically defines the software's qualification procedures. The validation protocol is typically made up of five different elements including team assignments, protocol approvals, installation qualification (IQ), operation qualification (OQ), and performance qualification (PQ).

• IQ documents the installation of software, such as the hardware and software configuration, documentation library, and procedures.

• OQ ensures the software runs as intended.

• PQ tests customizations created especially for installation, and specific procedures used at a particular site. It should include test scripts incorporating procedures defined in the URS.

Most of the validation time is spent testing the OQ. It includes pre-written test scripts from the vendor confirming that the software package functions as initially designed. The PQ focuses on the software's actual use in the facility, including integrating SOPs and creating custom templates. While OQ is vendor driven, PQ is typically customer driven.

Validation protocols generally contain the IQ and OQ of the validation process. The vendor or the user can do these parts of the validation. However, it is common practice, due to many biotechs' lack of resources, for the vendor to execute this part of the validation. While the PQ is normally carried out by the vendor, it requires assistance from the user because it is specific to that facility.

Validation efforts should include worst-case testing or stress testing. This may include data-entry, concurrent-user, and record-locking tests. Ensuring that the software works in these real-world scenarios should be documented as a part of the PQ.

Revalidation may be required when the decision is made to implement a new version of a software package. If the new release is minor, the vendor often identifies the updated functions, and validation can be limited to these specific areas.

And finally, upon completion of the user requirements, vendor qualification, and validation plan, all the documentation should be organized in a central location for easy access during an audit.

This entire process ensures the software meets application requirements; the vendor meets industry-accepted development criteria; and the implementation meets the validation plan. With these methods, biotech firms can be confident that the calibration software will meet users' requirements and hold up under regulatory scrutiny. And they can take full advantage of off-the-shelf calibration management software's ability to provide superior compliance more productively and at a lower cost.

References

FDA, "Good Manufacturing Practices for Finished Pharmaceuticals,"

Code of Federal Regulations

Title 21, Part 211 (U.S. Government Printing Office, Washington, DC, 2000).

FDA, "Electronic Records; Electronic Signature: Final Rule," Code of Federal Regulations Title 21, Part 11 (U.S. Government Printing Office, Washington, DC, 1997).

FDA, "Guideline on General Principles of Process Validation," (U.S. Government Printing Office, Washington DC, 11 January 2002).

FDA, "Quality System Regulation," Code of Federal Regulations Title 21, Part 820 (U.S. Government Printing Office, Washington DC, 1996).

GAMP, "GAMP 4 The Good Automated Manufacturing Practices (GAMP) Guide for Validation of Automated Systems in Pharmaceutical Manufacture" GAMP Forum (Amsterdam, The Netherlands, December 2001).

Nettleton, D. and Gough, J., Commercial Off-The-Shelf (COTS) Software Validation for 21 CFR Part 11 Compliance (PDA, Bethesda, MD, 2003). BPI