Draft Guidance Outlines Steps for Clinical Evaluation of Software as a Medical Device

The International Medical Device Regulators Forum has developed guidance outlining the steps required to generate clinical evidence of effectiveness and safety of software as a medical device (SaMD).

The draft guidance, issued by the FDA for public comment, addresses stand-alone software designed to produce or extract data, including diagnostic information, in tandem with a medical device. SaMD is not part of a device, nor is it used to operate a device, and would run on general-purpose computing platforms.

The first step in generating evidence would be to determine whether the scientific validity of the SaMD is already well-known. If not, scientific validity evidence would need to be generated.

Next would be an evaluation of the SaMD’s analytical validity, followed by an assessment of its clinical performance.

Measures of the analytical validity of software would include: accuracy (how close a measurement is to a quantity’s true value); precision (related to repeatability and reproducibility); limits of detection (discerning between informationbearing patterns and random patterns); linearity (behavior of output across a range of input data); and analytical sensitivity (degree to which output is affected by input data).

The guidance recommends that clinical performance of SaMD be characterized by demonstrating:

• Sensitivity — ability of the software to correctly identify across a range of available measurements patients with the intended clinical disease or condition;
• Specificity — ability of software to correctly identify across a range of available measurements patients that do not have the intended disease or condition;
• ROC curve — a graphic plot that shows the tradeoff between sensitivity and specificity as the decision threshold that separates negatives and positives is varied;
• Positive predictive value — which indicates the likelihood of the patient having a disease or condition given that the software’s output is positive;
• Negative predictive value — which indicates the likelihood of the patient not having a disease or condition given that the software’s output is negative;
• Likelihood ratio — the likelihood that a given result would be expected in a patient with the target condition compared to the likelihood that the same result would be expected in an individual without that condition; and
• Cut-off thresholds, indices or scales — should be meaningful for the intended use of the software and established prior to validation.

The risks posed by the software would involve decisions based on inaccurate software output. Those risks would be weighed according to the significance of the information provided by the software and the state of the healthcare condition in question.

In some instances software will be used in a clinical setting to measure the effectiveness of treatment. It also could be used for diagnostic purposes, including in vitro diagnostics. Other software may have a more general functionality, with potential uses in multiple settings.

Are you ready to transform your clinical development through advances in RBM? By watching this 90-minute webinar CD you’ll find out how to design and implement a risk-based clinical trial monitoring program: Using Risk-Based Clinical Trials Monitoring to Lower Data Errors.