Skip to main content
An official website of the United States government
Email

NIH/NCI 460 – Evaluation Datasets as Medical Device Development Tools for Testing Cancer Technologies

Fast-Track proposals will NOT be accepted.

Direct-to-Phase II proposals will NOT be accepted.

Number of anticipated awards: 3-5

Budget (total costs, per award):

Phase I: up to $400,000 for up to 12 months

Phase II: up to $2,000,000 for up to 2 years

PROPOSALS THAT EXCEED THE BUDGET OR PROJECT DURATION LISTED ABOVE MAY NOT BE FUNDED.

Summary

Oncology data science and analytics is a burgeoning area of artificial intelligence (AI) and machine learning (ML) technologies that have fueled interest across the industrial and academic sectors. During the past few years, several startups and large companies have focused on AI/ML technologies with the aim of reducing complexities in clinical workflow or increasing accuracy in detection, diagnosis, and treatment of cancer. While tremendous amounts of data are generated through clinical practice, significant gaps remain to leverage the data for device development and evaluation, including: 1) generation/acquisition of patient outcome data; 2) truthing of images by clinicians; 3) correlation of multi-modal imaging, comprehensive clinical, and genomic data in common repositories; 4) extraction of information from unstructured electronic health records (EHR) data; and 5) availability of clinically infrequent variants. This topic supports an unmet need for the development of large, well-curated, and statistically robust datasets that can be used for the evaluation of cancer medical devices subject to regulation by Center for Devices and Radiological Health (CDRH) of the Food and Drug Administration (FDA). Datasets that may be used to develop new devices as a measure of device effectiveness or performance, and support regulatory decision-making may be eligible for CDRH’s Medical Device Development Tools (MDDT) program. A tool eligible for consideration by the MDDT Program is one that reduces the regulatory burden of industry and the FDA.

CDRH’s mission is to protect and promote public health by assuring that patients and providers have timely and continued access to safe, effective, and high-quality medical devices. To qualify a dataset as an MDDT, CDRH evaluates the dataset and concurs with the available supporting evidence that the dataset produces scientifically plausible measurements and works as intended within the specified context of use. More information about the FDA’s MDDT Program can be found here. CDRH’s MDDT program collaboration with the NCI SBIR Development Center can help incentivize the small business community to develop and qualify innovative tools for oncology-related regulatory decision-making. These tools can be sold to industry or academia developing new device technologies that would benefit from using the MDDT in product development and evidence generation for a regulatory submission, thus stimulating and supporting translation of innovative devices to the clinic. Given these similar areas of interest, FDA CDRH and NCI SBIR have developed this joint contract topic to support innovation across our overlapping communities.

Project Goals

The goal of this contract topic is to stimulate the participation of small businesses in CDRH's MDDT program to develop datasets that can be used to assess medical devices in oncology settings. An MDDT can be a method, material, or measurement used to assess the safety, effectiveness, or performance of a medical device. MDDTs can accelerate the device development process by providing developers with qualified tools that do not need to be re-evaluated within each regulatory submission, thus streamlining device development and FDA regulatory decision-making. 

Several examples of datasets considered responsive under this solicitation include, but are not limited to: 

  • Imaging (radiology and pathology) with ground truth for algorithm validation of tumor/nuclei segmentation, prediction, classification, etc.
  • Cancer genomics and proteomics for evaluation of analytic tools for genome variation, integrative analysis of gene expression, protein expression, survival analysis, etc.
  • Structured data extracted from unstructured EHR.
  • Treatment outcome data (prospective or retrospective) for evaluation of clinical utility tools and methods.  

The following technical characteristics should be considered:

  • Focused on a specific cancer, clinical application (e.g., diagnosis, therapy), and modality (e.g., radiologic imaging systems, microscopy, spectroscopy, genomics, proteomics, laboratory testing, therapeutic or surgical devices, etc.).
  • Structured and well-characterized data, to include the best available ground truth or reference standard and the relevant metadata and data model to help with device development and evaluation. 
  •  Inclusion of a diverse patient population and multicenter data (prospective or retrospective). 
  • Anonymized with respect to protected health information and patient-identifying information.

Offerors are expected to follow the above requirements and conform to the two phases of the MDDT process. Please note that the MDDT process phases are separate from the SBIR phases. 

  • Proposal Phase: The goal is to determine if the MDDT is suitable for qualification consideration through the MDDT Program by submitting an MDDT Proposal that includes MDDT description, context of use, and an appropriate Qualification Plan for collecting evidence to support qualification of the tool for the defined context of use. The FDA makes a decision on whether to advance the tool to the qualification phase. 
  • Qualification Phase: The goal is to determine whether, for a specific context of use, the tool is qualified based on the evidence and justifications provided. The data collected according to the MDDT Proposal is submitted as the Full Qualification Package and reviewed by FDA for qualification decision.¬

During the NCI Phase I contract period, companies will engage with FDA in the proposal phase and develop their proposal for the MDDT. By the end of the Phase I contract, companies will submit their MDDT Proposal to FDA, and FDA review will determine if the tool is accepted into the MDDT Program. During the NCI Phase II contract period, companies will complete activities in the qualification phase. 

Examples of technologies considered responsive to this solicitation include, datasets intended for evaluation of cancer diagnostics (e.g., laboratory in vitro or imaging in vivo) and therapeutics (e.g., chemo, radiation, surgery, or immunotherapy). Offerors are expected to include in their project scope a demonstration that the dataset can be used for medical device evaluation. For example, FDA qualified a modeling software MDDT that can be used for medical device evaluation. 

Activities that would not be responsive under this announcement include datasets solely for the purpose of algorithm training and acquired without proper statistical considerations, or datasets that are applicable to assessing performance of only a single manufacturer’s device design. 

Phase I Activities and Deliverables:

  • Develop a pilot dataset that demonstrates how the data will be collected and what it will look like. In addition to truth data (from the clinician, an alternate modality, or patient outcome), include important patient sub-group information (demographics, disease type and stage, therapies) and information about the source of the data (site, date, sample prep, imaging device make and model, imaging protocol, and post-acquisition image processing, like reconstruction methods).
  • Develop an algorithm-assessment plan and corresponding software. Use the pilot dataset to demonstrate the algorithm-assessment plan: performance metric, uncertainty estimation, hypothesis test. This may require simulation or modeling of the dataset and a hypothetical algorithm. This should explore different levels of hypothetical algorithm performance, sources of variability from the algorithm, sources of variability from the dataset, and expected missing data.
  • If truth data is from a clinician or alternate modality, characterize the related uncertainty and account for it in all analyses. Multiple clinicians or multiple replicates are needed. Inclusion of a diverse patient population and multicenter data (prospective or retrospective) is ideal. 
  • Identify precision and performance-level parameters necessary for the dataset to become a clinically relevant tool that can be used for testing and evaluation of novel medical devices. This includes a sizing analysis to determine the size of a pivotal dataset following the algorithm-assessment plan. Develop a dataset and a statistical analysis plan for algorithm assessment. The plan should estimate the expected uncertainty of the algorithm assessment results for a range of algorithm performance levels using modeling and simulation.
  • Prepare an MDDT Proposal using the MDDT Qualification Plan Submission Template which includes specific requirements and activities with respect to the proposed MDDT. For additional details review ‘Qualification of Medical Device Development Tools - Guidance for Industry, Tool Developers, and Food and Drug Administration Staff.’
  • Demonstrate suitability of the dataset for the targeted test population and planned reference standard(s). 
  • Submit a complete MDDT Proposal to CDRH’s MDDT Program. The plan to collect evidence for qualification of the dataset should include details on the data source and planned patient population for the specified context of use.
  • Specify the quantitative technical and commercially relevant milestones that will be used to evaluate the success of the dataset.

Phase II Activities and Deliverables:

  • Collect the pivotal dataset and prepare it for sharing with end users: plan, establish, and demonstrate the sharing platform and methods. Fully document the data.
  • Characterize the precision and performance-level parameters of the dataset. If truth data is from a clinician or alternate modality, characterize the related uncertainty and account for it in all analyses. Multiple clinicians or multiple replicates are needed.
  • Compare and contrast the pivotal dataset against the simulated and modeled results related to the algorithm-assessment plan and sizing analysis from Phase I.
  • Demonstrate clinical utility and value of the dataset for use in testing and assessing novel medical devices.
  • Validate the dataset according to the specifications and feedback in the MDDT Proposal decision letter.
  • Prepare an MDDT Qualification Package based on the feedback in the MDDT Proposal decision letter.  
  • Submit a Full Qualification Package to CDRH’s MDDT Program including the data collected according to the FDA-approved Qualification Plan from the MDDT Proposal.
     

Receipt date: November 14, 2023, 5:00 p.m. Eastern Standard Time

Apply for this topic on the Contract Proposal Submission (eCPS) website.

View the full PHS2024-1 Contract Solicitation.

If you would like to reproduce some or all of this content, see Reuse of NCI Information for guidance about copyright and permissions. In the case of permitted digital reproduction, please credit the National Cancer Institute as the source and link to the original NCI product using the original product's title; e.g., “NIH/NCI 460 – Evaluation Datasets as Medical Device Development Tools for Testing Cancer Technologies was originally published by the National Cancer Institute.”

Email