FDA to Unveil Quality Metrics for Devicemakers by Oct. 1

The FDA plans to unveil by Oct. 1 its own quality metrics for devicemakers based on three metrics from a pilot program conducted by Xavier Health and the Medical Device Innovation Consortium.

The three metrics developed under the MDIC/Xavier Health pilot are:

• The total number of product changes related to product or process failures during pre-production activities compared to the total number of projects;
• The number of units manufactured without nonconformances (i.e., right-first-time rate) during production; and
• Post-market indicators such as complaints, service records, installation failures, medical device reports and recalls.

Pre-production metrics were identified as the most difficult to track because very few companies track the number of changes necessitated by inadequate product and process development.

That is partly due to the nature of continuous trial and error in the pre-production phase that loops back to research and development.

Ideally, the pre-production metric is designed to track the number of changes that occur during the transfer stage that were triggered by the product or process inadequacies, according to a best practices document released after the pilot program.

Additionally, it was difficult to segregate which of the planned changes were due to inadequacies versus improvements. Tracking the number of changes can also be an indication of the effectiveness of design controls, the best practices document says.

This metric provides an indication of the reliability of the R&D process and increases overall awareness of those shortcomings within a company. The metric also provides estimates for the overall time and cost of getting a product to market.

Definitions were found lacking for the second metric — the number of units manufactured without non-conformances compared to the number of units started — because the term “units” could refer to finished items, in-process material or sub-components.

For this reason, participants revised the metric to compare the number of units manufactured right the first time (RFT) within or across lots compared to the number of units started.

The revised metric would be more useful in feeding information back to R&D to improve development. The ability to track and trend within and across lots on a rolling basis would allow devicemakers to identify the highest-risk areas and to assess production efficiencies.

The post-production metric was found to be problematic because it was unclear if measuring in the aggregate was more informative than tracking separately. Also unclear was the impact to tracing complaints and MDRs for products that don’t have service and installation.

Participants suggested several multi-step options instead, such as:

• Calculating each post-production indicator separately with defined equations provided;
• Aggregating post-production indicators using weighting factors based on product and process risk profiles; and
• Using a comparative analysis through dashboards, score cards or heat maps.

The FDA anticipates announcing a voluntary pilot program in December. It hopes to officially roll out the metrics a year later. The agency will also focus on how third-party servicers fit into the scheme.

The current focus is leading indicators of quality, and an aggregate scoring system still needs to be developed. Such a scoring system could also make public the list of firms that fall into the highest category. Top-level firms would also not have to go through inspections, suggested William MacFarland, director of the Division of Manufacturing and Quality within CDRH. Those firms that met a “quality floor” would warrant less extensive inspections than companies that don’t meet the quality floor.

Stay up to date on regulatory stories like this one by subscribing to The GMP Letter. For 30 years, devicemakers have relied on The GMP Letter for the latest on the FDA’s interpretation and enforcement of cGMPs and the Quality Systems Regulation.