Software as medical device clinical trials as per US FDA

What is SaMD?

Software as Medical Device, or SaMD, can be described as “a class of medical software designed to carry out one or more medical functions without the need for actual hardware. This can comprise of software or applications intended to treat diagnose, cure, mitigate or prevent disease.”.

The International Medical Device Regulators Forum (IMDRF) defined SaMD as, “Software intended to be used for one or more medical purposes that perform these purposes without being part of a hardware medical device.” To put it another way, SaMD as a software on its own is a medical device. For example, the medical device software used to view images from MRI on mobile would be the SaMD.

But the software that enables an MRI machine to run its test wouldn’t be. SaMD is used with non-medical computing platforms connected to virtual networks. IMDRF develops globally agreed-upon credentials for a range of topics having to do with medical devices. The IMDRF presents information on SaMD classification, including the responsibilities of a manufacturer as well as labelling and instruction standards for these types of software.

Other examples of SaMD:

• Software as a Medical Device may be interfaced with other medical devices, including hardware medical devices, and other software as a medical device software, as well as general-purpose software. Software that provides parameters that become the input for a different hardware medical device or other Software as a Medical Device. For example, treatment planning software that supplies information used in a linear accelerator is Software as a Medical Device.
• Software with a medical purpose that operates on a general-purpose computing platform, i.e., a computing platform that does not have a medical purpose, is considered Software as a Medical Device. For example, software that is intended for the diagnosis of a condition using the tri-axial accelerometer that operates on the embedded processor on a consumer digital camera is considered Software as a Medical Device.
• Software that is connected to a hardware medical device but is not needed by that hardware medical device to achieve its intended medical purpose is Software as a Medical Device and not an accessory to the hardware medical device.
• Software as a Medical Device is capable of running on general-purpose (nonmedical purpose) computing platforms. Software as a medical device running on these general-purpose computing platforms could be located in a hardware medical device.

SaMD Clinical Evaluation Process Flow Chart

Clinical evaluation is a systematic and planned process to continuously generate, collect, analyse, and assess the clinical data6 pertaining to a SaMD in order to generate clinical evidence verifying the clinical association and the performance metrics of a SaMD when used as intended by the manufacturer. The quality and breadth of the clinical evaluation is determined by the role of the SaMD for the target clinical condition and assures that the output of the SaMD is clinically valid and can be used reliably and predictably. This section uses simple steps to help SaMD manufacturers through the approach to generating evidence for the clinical evaluation of a SaMD and provides links to techniques, definitions and sources that are available to help a SaMD manufacturer generate appropriate evidence.

Note: The examples used are not intended to be comprehensive. All data sources and statistical methods should be tailored to the specific SaMD and its intended use.

1. Valid Clinical Association: Is there a valid clinical association between your SaMD output, based on the inputs and algorithms selected, and your SaMD’s targeted clinical condition?

Step 1: Verify that the association between the SaMD output and the targeted clinical condition is supported by evidence.

Note: All SaMD should demonstrate a valid clinical association.

Question: How do I “generate evidence”? You can verify by using existing evidence or you can verify by generating new evidence.

Examples of existing evidence :

• Literature searches
• Original clinical research
• Professional society guidelines

Examples of generating new evidence :

• Secondary data analysis
• Perform clinical trials

2. Analytical Validation: Does your SaMD meet technical requirements?

Step 1: Generate evidence that shows that the output of your SaMD is technically what you expected. Note: All SaMD should demonstrate analytical validation.

Question: How do I “generate evidence”?

You can generate evidence during verification and validation activities as part of your quality management system or as part of your good software engineering practices, or by generating new evidence through use of curated databases or the use of previously collected patient data.

Verification –

Confirmation through the provision of objective evidence that specified requirements have been fulfilled.

Source: GHTF SG3 N18:2010

Validation –

Confirmation through the provision of objective evidence that the requirements for a specific intended use or application have been fulfilled.

Source: GHTF SG3 N18:2010

3. Clinical Validation: Does your SaMD generate clinically relevant outputs?

Step 1: Generate evidence that shows your:

SaMD has been tested in your target population and for your intended use; and that users can achieve clinically meaningful outcomes through predictable and reliable use.

Note: All SaMD should demonstrate clinical validation.

Question: How do I “generate evidence”?

You can generate evidence to validate clinical significance during verification and validation activities as part of your quality management system or as part of your good software engineering practices, referencing existing data sources from studies conducted for the same intended use. Where available data references studies conducted for a different intended use, extrapolation or generation of new clinical data may be required.

Examples of measures of clinical validation

• Sensitivity
• Specificity
• Positive predictive value (PPV)
• Negative predictive value (NPV)
• Number needed to treat (NNT)
• Number needed to harm (NNH)
• Likelihood ratio negative (LR-)
• Likelihood ratio positive (LR+)
• Odds ratio (OR)
• Clinical usability / User Interface
• Confidence interval

Considerations for Generating and Assessing Evidence

Being able to generate evidence to demonstrate the valid clinical association, analytical validation and clinical validation of a SaMD is essential to establishing the SaMD’s value for users. The degree of evidence generation needed for a given SaMD will depend on parameters including but not necessarily limited to the:

• Maturity of evidence underlying the clinical association
• Confidence in the evidence, as applied to a specific SaMD.

The purpose of the assessment of the evidence is to select information based on its merits and limitations to demonstrate that the clinical evaluation evidence is high-quality, relevant, and supportive of the SaMD intended use.

For example, an assessment of clinical association would classify a SaMD as having either a:

1. a) Well-established clinical association: These SaMD have outputs with the well-documented association as identified in sources such as clinical guidelines, clinical studies in peer-reviewed journals, consensus for the use of the SaMD, international reference materials or other similar well-established comparators in terms of previously marketed devices / SaMD; or
2. b) Novel clinical association: These SaMD may involve new inputs, algorithms or outputs, new intended target population, or new intended use. An example may include the combination of non-standard inputs such as mood or pollen count, with standard inputs such as gait, blood pressure or other physiological and environmental signals using novel algorithms to detect early onset of a deterioration of health or diagnosis of a disease.

What if I can’t generate evidence to validate 1,2 or 3?

• Perform ongoing data analysis
• Modify your intended use to one that can be supported by available evidence
• Modify the target clinical association
• Make changes to the software.

Importance of Independent Review of a SaMD’s Clinical Evaluation

Comparison of SaMD Clinical Evaluation Process to Process for Generating Clinical Evidence for IVD Medical Devices in GHTF/SG5/N7:2012

Know More on SaMD

SaMD applications help to boost the discovery, management and treatment of a variety of medical problems, automating specific aspects of care for saving time.

By being able to collect huge amounts of data, and doing so quickly, SaMD can also solicit user feedback through its availability on various devices.

The biggest advantages include:

1) Improved health outcomes through more accurate data as well as quicker production and feedback.

2) leading to faster innovation.

Companies using or evolving SaMD can use a feedback loop to enable faster product iterations, drive faster innovation and get to market quicker. It also makes collecting data fast and simple, providing high-quality data leading to better outcomes. Some of the uses of SaMD platforms can be seen below.

• Screening and diagnosis
• Monitoring and alerting
• Chronic condition and disease management
• Digital therapeutics

One of the groups regulating and monitoring the emergence of SaMD is the U.S. Food and Drug Administration. The FDA has released various guidelines relating to SaMD that focus “To help control the process of using this type of software in the field”. In 2013, IMDRF formed the Software as a Medical Device Working Group (WG) and which is chaired by the US FDA. The intention to formed it was to create guidance supporting innovation, access to safe and effective SaMD’s to the whole world.

Guidelines

The US FDA has developed guidelines for software classified under Software as a Medical Device.

• “The Software as a Medical Device must support clinical vocabulary for its use; this has to do with proper instruction and linguistic design in the interface. “
• “Addressing clinical evaluation methods and clinical evidence relevant to the use of the SaMD software.”

The US FDA says “Software as a Medical Device products should also have various recommendations attached to the software for analytical purposes and that manufacturers should outline potential adverse consequences.”

Life Cycle

In general, the manufacturers of SaMD products are supposed to gather exact kind of information, analyse the data, and deliver it along with the software as evidence of safety and effectiveness of software.

In whichever medical field SaMD is used, whether it’s oncology, radiology, or general patient care, the interface must support clinical work in specific ways. The software should be fully documented to identify its functions and its place within the clinical environment.

Outcomes for Software Sellers

The bottom line is that when a piece of healthcare or medical software is categorized as Software as a Medical Device, it’s going to generate its own unique regulatory requirements. Vendors need to know how this classification applies and what it means for vendor products. This is in addition to other major healthcare regulatory programs such as the Health Insurance Portability and Accountability Act or HIPAA, which closely governs a patient’s medical data and information. Think about how guidance and standards from the US FDA and other groups impact a company’s use of SaMD products, and how it drives the conversation between the customer and the vendor. Ask relevant questions before purchase to understand how these tools will help clinicians to provide a higher quality of care to patients.

Challenges

While SaMD is able to improve health outcomes using data and feedback loops to drive quicker product iteration, there are some underlying fundamental challenges faced by many SaMD builders. The biggest challenge involves integrating modern product development methodology — designed for an ever-evolving Internet of Everything world — with patient safety and regulatory compliance. While this can be difficult, companies who can successfully do this will benefit immensely while those who cannot are likely to fall behind.

For the US FDA, the primary concern or challenge is how to ensure patient safety and clinical effectiveness without sacrificing faster innovation. It’s worth noting that there is no “one-size fits all” approach when it comes to SaMD. For larger organizations, in particular, starting off with pilot programs to implement best practices and fast feedback loops is usually the preferred approach.

While it may be a little confusing in comparison to other software, SaMD provides unique features that extend beyond those of the traditional medical device or hardware. Unlike other devices, SaMD is able to leverage technology and connectivity to devices as well as people that can continuously monitor safety, effectiveness and performance.

References :

1. https://www.selecthub.com/medical-software/software-medical-device-samd/
2. https://www.fda.gov/medical-devices/software-medical-device-samd/what-are-examples-software-medical-device
3. Software-as-a-Medical-Device-(SAMD)–Clinical-Evaluation—Guidance-for-Industry-and-Food-and-Drug-Administration-Staff.pdf