In partnership with the American Association of Physicists in Medicine (AAPM), the Center for Virtual Imaging Trials (CVIT) is offering the TrueCT reconstruction challenge. The Center will provide participants with simulated sinogram data of various anatomies and disease types for them to reconstruct into high quality CT slices using any type of algebraic, statistical, iterative, and ML-based algorithms as trained on clinical data. Reconstructions will be quantitatively evaluated versus the known truth inherent in the Center’s simulated data.

Overview

Increased use of “approximating” reconstruction methods in CT, including iterative reconstruction (IR) and deep learning (DL) methods, creates uncertainty in how much of the native information might be distorted in the reconstruction process. If one had access to ground truth, i.e., the true underlying anatomy and physiology, the precise limitation of the reconstruction process could be objectively quantified. However, such information is not available in patient images. Ground truth is known in physical phantoms, but these phantoms have limited utility for this purpose as they do not model patient complex anatomy, motion, abnormalities, or variability, factors that are known to impact the performance of non-linear, scene-dependent reconstruction methods. Virtual imaging toolsets consisting of realistic computational phantoms and accurate imaging simulation techniques can provide sets of imaging data with a digitally-defined ground truth with which CT reconstruction methods can be objectively and quantitatively evaluated and compared. This challenge will take advantage of the resources of the CVIT to create a dataset of realistic CT images of virtual patients with known ground truth to provide an objective evaluation of CT reconstruction methods.

Data

The data set will include 200 unique computational phantoms from varied virtual patients sampling the ranges of weight, age, and sex of adult patients. Each phantom will include a particular pathology or disease: 67 COPD, 67 lung nodule, and 66 abdominal cases. The data will include a mixture of subtle pathologies of undisclosed details. All cases will be virtually imaged with a representative CT system with ranges of radiation dose level to represent clinical variability in techniques. The nature of this challenge precludes access to a training dataset.

Data Access

All cases will be made available from a publicly available web site in sinogram format following the standardized format established by the Mayo Clinic. Participants, once registered, will download the dataset and supplemental data needed for reconstruction (namely the scan geometry and associated data needed for the reconstruction). They can then perform their reconstruction and upload the resulting CT image data in DICOM format. Detailed instructions will be provided on how to submit results, including required formats and file organization.

Evaluation

For analysis, the reconstructed data will be registered to the ground truth. Ground truth will be established based on the mono-energetic representation of the phantoms. Data will be evaluated based on the similarity to the ground truth of the phantoms. For ranking purposes, we will only use task-generic (e.g., structural similarity index, root mean squared error) metrics. In addition to task-generic metrics, we will also evaluate the images based on task-specific (e.g., detectability index, radiomics, density quantification) metrics, per case, per pathology type, and across the entire dataset. We will provide all the metrics to the participants and devise a collective score and report that in the coming publication. Detailed information on the acquired data and evaluation metrics will be shared along with the sinogram data.

Results

Results of the TrueCT challenge will be presented at the AAPM Grand Challenges Symposium at the 2022 AAPM Annual Meeting. An individual from each of the two top-performing teams will receive a waiver of the meeting registration fee to present their methods during this session. Top performing participants will also be offered the opportunity for their algorithm to be integrated with Center for Virtual Imaging Trials resources with licensing option.

Publication

We will seek the publication of challenge results in the Medical Physics journal.

Get Started

1. Visit the TrueCT challenge site to register and get access to the Challenge data.
2. Download the sinogram data.
3. Apply your reconstruction techniques.
4. Submit your reconstructed images.

Important Dates

• Feb 24, 2022: Grand challenge website launch
• March 1, 2022: Sinogram data made available
• May 17, 2022: Final submission of results (midnight EST)
• June 10, 2022: Top two teams invited to present at challenge symposium
• July 10-14, 2022: Grand Challenge Symposium, AAPM 2022 Annual Meeting
• August 2022: Top five teams invited to participate in the challenge report and datasets made public

Organizers

• Ehsan Samei
• Ehsan Abadi
• Paul Segars
• Joseph Lo
• Samuel Armato and the AAPM Working Group on Grand Challenges

Contact

For further information, please contact:

cvit-inquire@duke.edu