ISPE Targets Five Quality Metrics for Round Two of Pilot Program, Releases Data From First

ISPE’s quality metrics team has narrowed from 16 to five the number of metrics it will ask drugmakers to collect data on during phase two of a pilot program aimed at helping the FDA boost quality compliance.

The five metrics the pilot will focus on are:

• Lot acceptance rate, normalized by lots produced or packaged, at the site level;
• Lot acceptance rate, normalized by lots produced or packaged, at the product level within a site;
• Critical complaints, normalized by packs released, at the product level by application and not broken down by site;
• Critical complaints, normalized by packs released, at the site level and not differentiated by product; and
• Deviations rate at the site level.

While the team said these five metrics are certain, they are also considering three others that drugmakers think are good indicators of future quality: recurring deviations rate, total product complaint trends and unconfirmed out-of-specification rate.

The second phase of the program also aims to harmonize definitions of the quality metrics, since certain ones may mean different things to different types of drugmakers.

Data and Correlations From First Wave

The first round of the pilot program collected data from 18 drugmakers at a total of 44 sites and analyzed it to determine whether certain metrics are predictive of quality — the first step toward better targeting of the FDA’s inspection resources.

Paul Rutten, a partner with McKinsey & Company, which analyzed the data for ISPE, said the data show interesting patterns that suggest some metrics could be more relevant to overall quality performance than others.

In all, 10 correlations were found. Among these was that a higher amount of critical complaints is consistent with an increased number of U.S. recalls, whereas total complaints go down, and that sites with lower lot acceptance rates have higher rates of critical complaints. By contrast, sites with higher deviation rates have higher critical complaints rates.

The pilot also found that higher deviations recurrence correlates with lower lot acceptance rate, that sites with a higher number of U.S. recalls have a high rate of deviation recurrence, that a stronger site culture correlates with fewer action limits exceeded in sterile product production and that stronger culture scores are associated with high lot acceptance rates.

Sites with higher quality culture scores also have lower recurrence of deviations and higher CAPA effectiveness rates, and those rates correlate with more annual product quality reviews being completed on time, the pilot showed.

The quality metrics measured were grouped into three main categories: external (market) quality outcomes, internal quality outcomes and site culture. External quality outcomes included metrics for total complaints, critical complaints and total U.S. recalls.

Internal quality outcomes consisted of the number of lots that were investigated for exceeding environmental monitoring action limits, deviation rates, lot acceptance rates, rework and reprocessing rates, confirmed out-of-specification rates, unconfirmed OOS rates and stability failure rates. Site culture took into account the firm’s quality culture, recurring deviations rate, APQRs being completed on time and CAPA effectiveness rate.

The data were presented at ISPE’s Quality Metrics Summit in Baltimore, Md.

ISPE will begin enrolling companies in the second wave of the pilot in June. While the hope is the initial 18 firms will continue in the program, the team wants to add more sterile and API manufacturers.

The FDA is promoting the use of Quality-by-Design as an approach for helping pharmaceutical and biotech companies speed up production, fine-tune continuous manufacturing and improve quality. Learn how to build quality into the product at every stage of production with Using Quality by Design to Improve Manufacturing - Webinar CD/Transcript.