Performing a Protein Purity Analysis Comparability Study

For protein-based drugs to be approved for human use, companies must perform testing to demonstrate compliance with requirements set forth by regulatory agencies such as FDA. Such guidelines require cGMP-regulated laboratories to validate and subsequently gain approval for any changes to current product testing or production methods, which may affect the final protein product. Changes to a method of production or testing introduce the risk that the consistency or results may differ from a previous method, thereby requiring significant additional work to justify the subsequent change in the pass criteria.

These risks and the additional work required to validate a new method have prevented many biopharmaceutical developers from adopting new procedures and technologies, such as less expensive reagents or faster, more efficient workflows. While pharmaceutical companies may be hesitant to make any changes to current protocols despite the aforementioned gains, a well-designed comparability or bridging study can quickly determine if changes to a method maintain the same accuracy and highly reproducible results while at the same time allow researchers to perform a significant portion of the initial protocol validation required for cGMP approval.

The design of a comparability study depends on the method or process that is being changed. This article focuses on a model comparability study based on a common analytical method for purity analysis (1). The method includes all the steps and products necessary to evaluate the composition of protein biologics including: SDS-polyacrylamide gel electrophoresis (SDS-PAGE) using Criterion TGX (Bio-Rad Laboratories) precast gels, staining of the gel with one-part QC Colloidal Coomassie stain (Bio-Rad Laboratories), image acquisition on the GS-900 calibrated densitometer (Bio-Rad Laboratories), and quantitative image analysis using the Code of Federal RegulationsTitle 21 (21 CFR) Part 11 compliant Image Lab acquisition and analysis software (Bio-Rad Laboratories).

Experimental Design
FDA’s Guidance for Industry: Comparability Protocols—Protein Drug Products and Biological Product—Chemistry, Manufacturing, and Controls Information and the International Conference on Harmonization of Technical Requirements for Registration of Pharmaceuticals for Human Use (ICH) Validation of Analytical Procedures: Text and Methodology Q2 (R1) outline guidelines for conducting a comparability study (2, 3). To gain cGMP approval for a new purity assay, the compatibility study must compare the following characteristics: accuracy, precision, specificity, limit of detection (LOD), limit of quantitation (LOQ), linearity, and range. Table I defines the performance characteristics and methodology used to evaluate changes to a purity analysis method.

Table I: Summary of Performance Characteristics

Performance Characteristic

Description

Methodology

Accuracy

Difference between the mean and accepted true value, defined by the mean value and standard deviation

Nine determinations over a minimum of three concentrations covering the specified range (e.g., three concentrations / three replicates)

Precision

Reproducibility as defined by the standard deviation, coefficient of variation, and confidence interval

Nine determinations over a minimum of three concentration levels covering the specified range (e.g., three concentrations / three replicates)

Specificity

No specified recommendation

Linearity

Defined by the fit of the method of least squares along the correlation coefficient, y-intercept, slope of the regression line and residual sum of squares.

Minimum of five different concentrations / three replicates

Limit of Detection (LOD)

Lowest amount of target that can be detected with a specified confidence interval, which is defined by two parameters, the rate of false positives and the rate of false negatives

No specified recommendation

Limit of Quantitation (LOQ)

Lowest amount of target that can be quantified within a specified confidence interval

No specified recommendation

Range

The limits over which acceptable degree of linearity, accuracy and precision are achieved within a specified confidence interval

Minimum of five different concentrations / three replicates

With appropriate experimental design, researchers can generate the data needed to evaluate these characteristics with as few as two sets of samples, thus minimizing the amount of work needed for cGMP approval. Further, users can increase efficiency by making multiple changes at once and evaluating all changes (e.g., changes to the gels, stain, and imager) in a single comparability study. The performance characteristics and pass criteria for the new method (e.g., the sample must be greater than 90% pure as defined by the percent band) are defined by the criteria established for the previous cGMP-approved protocol. For a replacement method to be approved, it must perform as good as or better than the previous assay.

How it works
The first sample should be a validated target protein that reflects the purity of a representative lot, as defined by the percent band (the volume of a given band divided by the sum total volume of all bands) or percent lane (the volume of a given band divided by the sum total volume in the lane). A dilution series of the first sample determines the linearity, LOD, LOQ, and range, in which sample amounts range from saturation to beyond the LOD. There will be a region within this range in which the band quantitation (volume measured in OD) is linear with respect to the amount of protein loaded, and the percent purity is constant. The region in which both conditions are satisfied defines the range over which the results (percent band or percent lane) are accurate and precise. A minimum of ±20% of the target biologic protein load should be included in the region and will define the valid range of the protocol.

The second sample set should include stressed samples that contain impurities. In this case study, four samples were generated consisting of a second lot of reduced bovine serum albumin (BSA), a BSA sample that had undergone partial degradation, a BSA sample containing aggregates, and a BSA sample with impurities (approximately 70% impurities by percent lane). These samples are characterized in three different amounts, covering the target load and both the high and low limits of the range, as defined from the analysis of the first sample set. These samples further define the accuracy, precision, and specificity of the assay for detecting protein impurities.

In the model study, the samples were separated by SDS-PAGE, stained with a colloidal Coomassie stain, imaged with a calibrated densitometer, and analyzed by two different protocols, as outlined in Figure 1. One set of experiments represented an existing validated QC protocol (Protocol A, NuPage 4-12% Bis-Tris gels, two-part colloidal Coomassie stain from Life Technologies, the GS-800 calibrated densitometer, and Quantity One Bio-Rad Laboratories software). Another set of experiments was then performed using a Biologics Analysis Workflow (Bio-Rad, Protocol B, Criterion TGX precast gels, one-part QC Colloidal Coomassie stain, the GS-900 calibrated densitometer, and Image Lab Security Edition software).

Results
To determine the linearity, range, LOD, and LOQ of protocols A and B, the first set of samples (a dilution series of BSA [~90-95% pure]) were run in triplicate for both protocols (Figure 2). The band volume and percent purity (both percent band and percent lane) of BSA were measured from these data for each lane, averaged across the three gels (N = 3), and fit with a least square linear regression. The molecular weight of the BSA was determined using the three samples in the target range (2.5 µg, 1.25 µg, and 625 ng) across the three gels (N = 9).

To further test the accuracy, precision, and specificity of the protocols, the second set of samples were run at three different concentrations bracketing the target load for the assay (Figure 3). These samples covered different ranges of purity, contained different contaminants of low and high molecular weight, and were used to compare the performance of the different protocols and the specificity of the stain to different protein contaminents.

The existing validated QC protocol and the Biologics Analysis Workflow protocol performed similarly across all seven characteristics. The linear range (2.5 µg to 2.4 ng) and coefficient of correlation (all greater than 0.99) were indistinguishable, with minor differences in the slope and y-intercept. BSA amounts above 2.5 µg were excluded from the linear fit because of a reduction in the coefficient of correlation below 0.99. The LOD and the LOQ were statistically equivalent. The LOD allows routine detection down to 2.4 ng of BSA in each of the methods, and the LOQ allows reproducible quantitation of BSA above 9.8 ng to within ± 20%, with a 95% confidence interval (CI).

The mean percent bands for each protocol were within a single standard deviation of each other and were statistically equivalent in the region from the maximum load of 2.5 µg down to 625 ng. For a target load of 1.25 µg BSA, the percent purity was within 1% of the 2.5 µg and 625 ng load (±50% of target load), allowing for robust purity determination.

The BSA band began to saturate at loads above 2.5 µg BSA, while signals from impurities continued to increase, resulting in a decrease in the measured relative purity of the dominant band. The percent band quickly reached 100% at amounts lower than 625 ng as the signal from the impurities dropped below the LOD. The additional complexity of the stressed samples provided a more stringent test of the accuracy and precision and further demonstrated the equivalence of the two protocols.  

The percent band values for a given sample over the tested range were statistically equivalent across the two protocols. This consistency demonstrates that the protocols resolve, stain, and detect a variety of protein contaminants equivalently and, therefore, have similar specificities. The error in the percent band increased as the purity of the dominant band decreased and the amount of impurities increased. A larger percentage of the total volume was made up by the variation in the contaminants and, therefore, resulted in more variation in the measured relative purity.

Conclusion
The experimental design presented here can be used to efficiently generate the data needed to evaluate the seven characteristics required for approval of changes to a protocol in cGMP laboratories. Additionally, this experimental design measures all the required performance criteria and provides the framework for measuring the intermediate precision of the protocol (i.e., user-to-user variability, instrument-to-instrument variability, etc.) for future validation work.

The comparability study also demonstrates the advantages that can be gained when changing an existing protocol. Protocol B was faster and more efficient compared to the existing protocol (Protocol A). First, the larger format of Criterion TGX gels (18 wells compared to 10) allowed more samples (dilutions) to be run on a single gel (a two-fold dilution series could be run uninterrupted from 10 µg down to 1.4 ng). This provided greater resolution across the dilution range, thereby improving the determination of the linearity and range, allowing for fewer iterations to bracket the desired range for the validation study (requiring six gels as opposed to twelve gels). The Criterion gels also ran faster than the NuPAGE gels (1/3 time), and the one-part stain and protocol was easier to use (no mixing/assembly were required, and the protocol fixing had fewer steps).

Making the changes described in protocol B, therefore, resulted in a more efficient workflow and demonstrated that a comparability study encapsulating multiple changes to an existing protocol did not require significantly more work than what is already required for changing a single parameter.

References
1. P. Elms, P. Lui, K. Smith, and M. Nguyen, Bulletin 6387, (2013).
2. FDA, Guidance for Industry: Comparability Protocols--Protein Drug Products and Biological Products—Chemistry, Manufacturing, and Controls Information (2003).
3. ICH, Q2 (R1)Validation of Analytical Procedures: Text and Methodology (2005).

About the Author
Phillip Elms is senior scientist in the Lab Separations Division at Bio-Rad  Laboratories, 2000 Alfred Nobel Drive, Hercules, CA, 94547