SBIR Banner

You are here

NIH/NCI 444 - Evaluation Datasets as Medical Device Development Tools for Testing Cancer Technologies

Fast-Track proposals will NOT be accepted.

Direct-to-Phase II proposals will NOT be accepted.

Number of anticipated awards: 3-5

Budget (total costs, per award):

Phase I: up to $400,000 for up to 12 months

Phase II: up to $2,000,000 for up to 2 years

PROPOSALS THAT EXCEED THE BUDGET OR PROJECT DURATION LISTED ABOVE MAY NOT BE FUNDED.

 

Summary

Oncology data science and analytics is a burgeoning area of machine learning (ML) and artificial intelligence (AI) technologies that have fueled unprecedented levels of interest across the industrial and academic sectors. The past few years have witnessed many startups and large companies focusing on ML/AI technologies with the aim of reducing complexities in clinical workflow or increasing accuracy in detection, diagnosis, and treatment of cancer. To that end large, well-characterized datasets with the best available ground truth/reference standard and relevant metadata are essential for developing machine-based applications in cancer. While tremendous amounts of data are generated through clinical practice, significant gaps remain to leveraging the data for device development and evaluation, including: 1) generation/acquisition of patient outcome data; 2) truthing of images by clinicians; 3) correlation of combined imaging, comprehensive clinical, and genomic data in common repositories for developers; 4) extraction of information from unstructured electronic health records (EHR) data; and 5) availability of infrequent, but clinically relevant, variants. The goal of this topic is to promote and support an unmet need for the development of large, well-curated, and statistically robust datasets that can be used for the evaluation of cancer medical devices subjected to regulation by Center for Devices and Radiological Health (CDRH). Such datasets may be used in scientific research, to develop new devices as a measure of device performance, and have a regulatory use appropriate for the FDA Medical Device Development Tool program. A tool eligible for consideration by the MDDT Program is one that reduces the regulatory burden of industry and the FDA.

FDA’s mission is to protect and promote public health by helping to speed innovations that make medical products safer and effective for the public. To qualify a dataset as an MDDT, the FDA evaluates the dataset and concurs with the available supporting evidence that the dataset produces scientifically plausible measurements and works as intended within the specified context of use. More information about the FDA’s MDDT Program can be found here. FDA’s MDDT program collaboration with the NCI SBIR Development Center can help incentivize the small business community to develop and qualify innovative tools for oncology-related regulatory decision-making. These tools can be sold to industry or academia developing new device technologies that would benefit from using the MDDT in their regulatory submission thus stimulating and supporting translation of innovative devices to the clinic. Given these similar areas of interest, FDA CDRH and NCI SBIR have developed this joint contract topic to stimulate and support innovation across our overlapping communities.

 

Project Goals

The goal of this contract topic is to stimulate the participation of small businesses in the FDA's MDDT program to develop and demonstrate the utility of qualified datasets as MDDTs to assess medical devices subject to regulation by CDRH. An MDDT can be a method, material, or measurement used to assess the effectiveness, safety, or performance of a medical device. The functionalities of such medical devices run the gamut in the cancer care continuum including prevention, detection, diagnosis, treatment planning etc. Datatypes of interest cover a broad range of data produced by those devices, and include, but are not limited to, imaging (radiology and pathology), cancer genomics, proteomics, structured data extracted from unstructured EHR, and treatment outcome data. In order to achieve the goal of developing datasets as MDDTs for a specified context of use, each dataset may have the following technical characteristics:

  • Focused on a specific cancer (i.e., disease site), a specific clinical application (e.g., diagnosis, therapy), and a specific modality (e.g., radiologic imaging systems, microscopy, spectroscopy, genomics, proteomics, laboratory testing, therapeutic or surgical devices, etc.).
  • Structured and well-characterized, to include the best available ground truth or reference standard and the relevant metadata and data model to help in device development and evaluation. The truthing process must be clearly described and include an appropriate number of qualified experts.
  • Contain a diverse patient demography and an appropriately broad range of data acquisition systems, follow well-described reconstruction and processing methods, include full details of the imaging systems, protocols, reconstruction methods, etc., and be presented in formats that follow the latest standards, when available.
  • Anonymized with respect to the protected health information (PHI) and patient-identifying information (PII).
  • Stored and tabulated as an organized collection of data and metadata electronically accessible and searchable by a computer system, and include a concise data descriptor, covering the above requirements.

 

Offerors are expected to follow the above requirements and conform to the two phases of the MDDT process. Please note that the MDDT process phases are separate from the SBIR phases.

Proposal Phase: The goal is to determine if the MDDT is suitable for qualification consideration through the MDDT Program by submitting a Qualification Plan that includes MDDT description, context of use, and an appropriate plan for collecting evidence to support qualification of the tool for the defined context of use. The FDA makes a decision on whether to advance the tool to the qualification phase.

Qualification Phase: The goal is to determine whether, for a specific context of use, the tool is qualified based on the evidence and justifications provided. The data collected according to the Qualification Plan is submitted as the Full Qualification Package and reviewed by FDA for qualification decision.

During the NCI Phase I contract time period, companies will engage with FDA in the proposal phase and develop their Qualification Plan for the MDDT. By the end of the Phase I contract, companies will submit their Qualification Plan to FDA, and FDA review will determine if the tool is accepted into the MDDT Program. During the NCI Phase II contract time period, companies will complete activities in the qualification phase.

Examples of technologies considered responsive to this solicitation include, cancer diagnostics (e.g., laboratory in vitro, imaging in vivo) and therapeutics (e.g., chemo, radiation, surgery, and immunotherapy).

Activities that would not be responsive under this announcement include datasets solely for the purpose of algorithm training and acquired without proper statistical considerations, or datasets that are applicable to assessing performance of only a single manufacturer’s device design.

 

 

Phase I Activities and Deliverables

  • Develop a pilot dataset that demonstrates how the data will be collected and what it will look like. In addition to truth data (from the clinician, an alternate modality, or patient outcome), include important patient sub-group information (demographics, disease type and stage, therapies) and information about the source of the data (site, date, sample prep, imaging device make and model, imaging protocol, and post-acquisition image processing, like reconstruction methods).
  • Develop an algorithm-assessment plan and corresponding software. Use the pilot dataset to demonstrate the algorithm-assessment plan: performance metric, uncertainty estimation, hypothesis test. This may require simulation or modeling of the dataset and a hypothetical algorithm. This should explore different levels of hypothetical algorithm performance, sources of variability from the algorithm, sources of variability from the dataset, and expected missing data.
  • If truth data is from a clinician or alternate modality, characterize the related uncertainty and account for it in all analyses. Multiple clinicians or multiple replicates are needed.
  • Identify precision and performance-level parameters necessary for the dataset to become a clinically relevant tool that can be used for testing and evaluation of novel medical devices. This includes a sizing analysis to determine the size of a pivotal dataset following the algorithm-assessment plan. Develop a dataset and a statistical analysis plan for algorithm assessment. The plan should estimate the expected uncertainty of the algorithm assessment results for a range of algorithm performance levels using modeling and simulation.
  • Prepare an MDDT Qualification Plan Submission Template using the MDDT Qualification Plan Submission Template which includes specific requirements and activities with respect to the proposed MDDT. For additional details review ‘Qualification of Medical Device Development Tools - Guidance for Industry, Tool Developers, and Food and Drug Administration Staff.’
  • Demonstrate suitability of the dataset for the targeted test population and planned reference standard(s).
  • Submit a complete Qualification Plan to the FDA’s MDDT Program. The plan to collect evidence for qualification of the dataset should include details on the data source and planned patient population for the specified context of use. Use the MDDT Qualification Plan Submission Template for this submission.
  • Specify the quantitative technical and commercially relevant milestones that will be used to evaluate the success of the dataset.

 

Phase II Activities and Deliverables

  • Collect the pivotal dataset and prepare it for sharing: plan, establish, and demonstrate the sharing platform and methods. Fully document the data.
  • Characterize the precision and performance-level parameters of the dataset. If truth data is from a clinician or alternate modality, characterize the related uncertainty and account for it in all analyses. Multiple clinicians or multiple replicates are needed.
  • Compare and contrast the pivotal dataset against the simulated and modeled results related to the algorithm-assessment plan and sizing analysis from Phase I.
  • Demonstrate clinical utility and value of the dataset for use in testing and assessing novel medical devices.
  • Validate the dataset according to the specifications approved by the MDDT program.
  • Prepare a Full MDDT Qualification Package Submission Template which includes specific requirements and activities with respect to the proposed MDDT.
  • Submit a Full Qualification Package to the FDA’s MDDT Program including the data collected according to the FDA-approved Qualification Plan. Use the MDDT Qualification Package Submission Template for this submission.

 

Frequently Asked Questions

1. Who are the potential customers for an MDDT?

MDDTs can be used by other developers, researchers, small businesses, and other industry and research groups who are working to develop technologies in the same space as the MDDT technology. These tools will facilitate the regulatory decision-making process and expedite the development of new technologies, benefiting both FDA and companies with technologies under FDA review.

2. Will FDA or NCI purchase the MDDT?

Offerors must identify the eventual customers for their tool. NCI and the FDA are not potential customers for this product.

3. Are there examples of MDDTs?

Yes, the MDDT page (https://www.fda.gov/medical-devices/science-and-research-medical-devices...) lists some examples of MDDTs. There are no examples in the biomarker or the dataset spaces, which is one reason that the FDA and NCI are interested in supporting offerors working in these areas.

4. What happens if my tool is not qualified as an MDDT?

You must submit your qualification plan to the FDA by the end of the Phase I contract. CDRH will review Full Qualification Packages submitted at the end of the Phase II contract and make a qualification decision regarding the tool’s acceptance as an FDA-qualified MDDT. This risk is mitigated by a company developing their Qualification Plan in accordance with CDRH feedback prior to submitting their final Qualification Plan to FDA. If awarded, companies are highly encouraged to engage FDA early on when developing their Qualification Plan for the MDDT Program.

 

Full PHS2022-1 Contract Solicitation

Receipt date: October 28, 2021, 5:00 p.m. Eastern Daylight Time

Apply for this topic on the Contract Proposal Submission (eCPS) website.

Inquiries: Cherie Wells (ncioasbir@mail.nih.gov)

 

Updated Date: 
August 2, 2021