AzTrials
AzTrials

Prostate CancerNCT05489341

Artificial Intelligence and Radiologists at Prostate Cancer Detection in MRI: The PI-CAI Challenge

An observational study of Prostate Cancer, run by Radboud University Medical Center at 1 site in Netherlands.

Completed

Finished. The study has ended and is not enrolling. Results may be posted on the official record.

Official record
Search all trials

Trial summary

The PI-CAI challenge aims to validate the diagnostic performance of artificial intelligence (AI) and radiologists at clinically significant prostate cancer (csPCa) detection/diagnosis in MRI, with respect to histopathology and follow-up (≥ 3 years) as reference. Read more about this study

Who is it for?
Ages 18 years and older, men onlyHealthy volunteers can take part
Conditions the study is for
Prostate Cancer
What is being tested?
Histopathology and Magnetic Resonance Imaging with Follow-Up and Histopathology and Magnetic Resonance Imaging
How many people
10,207 participants
Where
1 siteNetherlands
Study dates
1 Feb 2022to 1 Nov 2023
Phase
Observational
Are there results?
None posted in the registry

Key requirements

Who can take part, as listed by the sponsor. Only the study team can confirm whether you are eligible, so take this list to your doctor.

  • Ages 18 years and older
  • Men only
  • Healthy volunteers accepted
  • Adult, Older adult

Study population: All patient exams are of men suspected of harboring csPCa, with elevated levels of prostate-specific antigen (≥ 3 ng/mL) and/or abnormal findings on digital rectal exam, and without a history of treatment or any prior positive histopathology (ISUP ≥ 2) findings. Patients underwent prostate MRI, and were primarily examined at one of three Dutch centers (Radboud University Medical Center, Ziekenhuisgroep Twente, University Medical Center Groningen) or one Norwegian center (Norwegian University of Science and Technology) during regular clinical routine, between 2012-2021.

Who may be able to join

  • Men suspected of harboring csPCa, with elevated levels of prostate-specific antigen (≥ 3 ng/mL) and/or abnormal findings on digital rectal exam, who subsequently underwent prostate MRI.

Who cannot join

  • Patients who opted-out or did not give permission to reuse clinical data.
  • Patients with a history of prior prostate treatment.
  • Patients with a history of prior positive csPCa findings in histopathology (ISUP ≥ 2).
  • Patients whose prostate MRI exhibit severe artifacts (e.g. heavy warping due to rectal air, metal artifacts from hip prostheses, heavy motion blur), thereby impeding their usage.
  • Patients, whose positive histopathology findings (ISUP ≥ 2) cannot be reliably localized on MRI (e.g. MRI-invisible lesions, systematic biopsy diagnostic reports with ambiguous, "random" or missing location information).

Trial locations and contacts

1 site in 1 country.

Netherlands1 site
  • RadboudUMC

    Nijmegen, Gelderland, 6525 GA

Other trials in Netherlands

About this study

The PI-CAI challenge aims to validate the diagnostic performance of artificial intelligence (AI) and radiologists at clinically significant prostate cancer (csPCa) detection/diagnosis in MRI, with respect to histopathology and follow-up (≥ 3 years) as reference. The study hypothesizes that state-of-the-art AI algorithms, trained using thousands of patient exams, are non-inferior to radiologists reading bpMRI. As secondary end-points, it investigates the optimal AI model for csPCa detection/diagnosis, and the effects of dynamic contrast-enhanced imaging and reader experience on diagnostic accuracy and inter-reader variability.

Read the detailed description

Prostate cancer (PCa) is one of the most prevalent cancers in men worldwide. One million men receive a diagnosis and 300,000 die from clinically significant PCa (csPCa) (defined as ISUP≥2), each year, worldwide. Multiparametric magnetic resonance imaging (mpMRI) is playing an increasingly important role in the early diagnosis of prostate cancer, and has been recommended by the European Association of Urology (EAU), prior to biopsies. However, current guidelines for reading prostate mpMRI (i.e. PI-RADS v2.1) follow a semi-quantitative assessment that mandates substantial expertise for proper usage. This can lead to low inter-reader agreement (<50%), sub-optimal interpretation and overdiagnosis.

Modern artificial intelligence (AI) algorithms have paved the way for powerful computer-aided detection and diagnosis (CAD) systems that rival human performance in medical image analysis. Clinical trials are the gold standard for assessing new medications and interventions in a controlled and comparative manner, and the equivalent for developing AI algorithms are international competitions or "grand challenges", where increasingly large datasets are released to public to solve clinically relevant tasks with AI. Grand challenges can address the lack of trust, scientific evidence and adequate validation among AI solutions, by providing the means to compare algorithms against each other using common datasets and a unified experimental setup.

PI-CAI (Prostate Imaging: Cancer AI) is an all-new grand challenge, with over 10,000 carefully-curated prostate MRI exams to validate modern AI algorithms and estimate radiologists' performance at csPCa detection and diagnosis. Key aspects of the study design have been established in conjunction with an international, multi-disciplinary scientific advisory board (16 experts in prostate AI, radiology and urology) -to unify and standardize present-day guidelines, and to ensure meaningful validation of prostate-AI towards clinical translation.

The 2022 edition of PI-CAI will focus on validating AI at automated 3D detection and diagnosis of csPCa in bpMRI. PI-CAI primarily consists of two sub-studies:

  • AI Study (Grand Challenge): An annotated multi-center, multi-vendor dataset of 1500 bpMRI exams (including their clinical and acquisition variables) is made publicly available for all participating teams and the research community at large. Teams can use this dataset to develop AI models, and submit their trained algorithms (in Docker containers) for evaluation. At the end of this open development phase, all algorithms are ranked, based on their performance on a hidden testing cohort of 1000 unseen scans. In the closed testing phase, organizers retrain the top-ranking 5 AI algorithms using a larger dataset of 7500-9500 bpMRI scans (including additional training scans from a private dataset). Finally, their performance is re-evaluated on the hidden testing cohort (with rigorous statistical analyses), to determine the top 3 AI algorithms for automated 3D detection and diagnosis of csPCa in bpMRI (i.e. the winners of the grand challenge).
  • Reader Study: 50+ international prostate radiologists perform a reader study using a subset of 400 scans from the hidden testing cohort. For each case, radiologists complete their assessments in two rounds. At first, using clinical and acquisition variables plus bpMRI sequences only, enabling head-to-head comparisons against AI trained on the same. And then, using clinical and acquisition variables plus full mpMRI sequences, enabling comparisons between AI and current clinical practice (PI-RADS v2.1). Overall, the goal of this study is to estimate the performance of the average radiologist at detection and diagnosis of csPCa in MRI.

In the end, PI-CAI aims to benchmark state-of-the-art AI algorithms developed in the grand challenge, against prostate radiologists participating in the reader study -to evaluate the clinical viability of modern prostate-AI solutions at csPCa detection and diagnosis in MRI.

Conditions
Prostate Cancer
Being tested
Histopathology and Magnetic Resonance Imaging with Follow-Up, Histopathology and Magnetic Resonance Imaging
Keywords
Magnetic Resonance Imaging, Prostate Cancer, Artificial Intelligence, Computer-Aided Detection and Diagnosis
Medical subject headings
Prostatic Neoplasms, Disease

What participants receive

What participants receive. In a randomised study each person is assigned to one group by chance.

  • Histopathology and Magnetic Resonance Imaging with Follow-UpDiagnostic test

    Reference standard establishes histologically-confirmed (ISUP ≥ 2) cases of csPCa as positives, and histopathology- (ISUP ≤ 1) or MRI- (PI-RADS ≤ 2) with follow-up (≥ 3 years) confirmed cases of indolent PCa or benign tissue as negatives.

  • Histopathology and Magnetic Resonance ImagingDiagnostic test

    Reference standard establishes histologically-confirmed (ISUP ≥ 2) cases of csPCa as positives, and histopathology- (ISUP ≤ 1) or MRI- (PI-RADS ≤ 2) confirmed cases of indolent PCa or benign tissue as negatives.

Study groups in NCT05489341
Study groupRoleWhat they receive
Public Training and Development Set (1500 cases)Available for all participants and researchers, to train and develop AI models. Includes multi-vendor (Siemens Healthineers, Philips Medical Systems) prostate bpMRI cases from three Dutch centers (Radboud University Medical Center, Ziekenhuisgroep Twente, University Medical Center Groningen), acquired between 2012-2021. All data is fully anonymized and made available under a non-commercial CC BY-NC 4.0 license. Includes 328 cases from the PROSTATEx challenge (prostatex.grand-challenge.org). Imaging data has been released via: zenodo.org/record/6624726 (DOI: 10.5281/zenodo.6624726). Lesion annotations of csPCa have been released and are maintained via: github.com/DIAGNijmegen/picai_labels.
Receives: Diagnostic Test: Histopathology and Magnetic Resonance Imaging
Private Training Set (7500-9500 cases)Used exclusively by the organizers to retrain the top-ranking 5 AI algorithms, with large-scale data. Includes multi-vendor (Siemens Healthineers, Philips Medical Systems) prostate bpMRI cases from three Dutch centers (Radboud University Medical Center, Ziekenhuisgroep Twente, University Medical Center Groningen), acquired between 2012-2021.
Receives: Diagnostic Test: Histopathology and Magnetic Resonance Imaging
Hidden Validation and Tuning Cohort (100 cases)Used for a live, public leaderboard that enables AI model selection and tuning throughout the open development phase of the challenge. Includes multi-vendor (Siemens Healthineers, Philips Medical Systems) prostate bpMRI cases from three Dutch centers (Radboud University Medical Center, Ziekenhuisgroep Twente, University Medical Center Groningen), acquired between 2012-2021, that remain fully hidden throughout the course of the challenge.
Receives: Diagnostic Test: Histopathology and Magnetic Resonance Imaging with Follow-Up
Hidden Testing Cohort (1000 cases)Used to benchmark AI, radiologists, and test all hypotheses at the end of the PI-CAI challenge. A subset of 400 cases from this cohort is used to facilitate the PI-CAI: Reader Study. Includes multi-vendor (Siemens Healthineers, Philips Medical Systems) internal testing data (unseen prostate bpMRI cases from three seen Dutch centers {Radboud University Medical Center, Ziekenhuisgroep Twente, University Medical Center Groningen}) and external testing data (unseen prostate bpMRI cases from one unseen Norwegian center {Norwegian University of Science and Technology}), acquired between 2012-2021.
Receives: Diagnostic Test: Histopathology and Magnetic Resonance Imaging with Follow-Up

What is being measured

The primary outcome is the main question the study is designed to answer.

  1. AI vs Radiologists from Reader Study

    Diagnostic performance of the top 5 AI models from the grand challenge and 50+ radiologists from the reader study, at csPCa detection/diagnosis in prostate bpMRI, with respect to histopathology and MRI with follow-up (≥ 3 years) as reference, to assess the clinical viability of present-day AI solutions.

    Measured: 6 months

  2. AI vs Radiologists from Clinical Routine

    Diagnostic performance of the top 5 AI models from the grand challenge and the historical reads of radiologists from clinical routine, at csPCa detection/diagnosis in prostate bpMRI, with respect to histopathology and MRI with follow-up (≥ 3 years) as reference, to assess the clinical viability of present-day AI solutions.

    Measured: 6 months

Secondary outcomes (2)
  1. AI vs AI

    Diagnostic performance of the top 5 AI models from the grand challenge, at csPCa detection/diagnosis in prostate bpMRI, with respect to histopathology and MRI with follow-up (≥ 3 years) as reference, to deduce the optimal AI model architecture for this given task.

    Measured: 6 months

  2. Radiologists vs Radiologists from Reader Study

    Diagnostic performance and inter-reader variability of 50+ radiologists from the reader study, at csPCa detection/diagnosis in prostate mpMRI, with respect to histopathology and MRI with follow-up (≥ 3 years) as reference, to deduce the effects of dynamic contrast-enhanced imaging and reader experience.

    Measured: 6 months

Timeline

From the start of the study to its completion; the mark is today. Estimated dates are the sponsor’s plan.

  1. First posted5 Aug 2022
  2. Study start1 Feb 2022
  3. Primary completion1 Jun 2023
  4. Study completion1 Nov 2023

Resources

More details

For doctors and researchers: how the study is designed, who runs it, and its registry IDs.

Study design
Study type
Observational
Observational model
Cohort
Time perspective
Retrospective
Enrolment
10,207 participants (actual)
IDs, dates and publications
ClinicalTrials.gov ID
NCT05489341
Sponsor's protocol ID
CMO2016-3045-Project-20011
Status last verified
Jul 2022
First submitted
3 Aug 2022
Last updated
18 Nov 2023
Results posted
No
Data monitoring committee
No
FDA-regulated drug
No
FDA-regulated device
No
Individual data sharing
Yes: To facilitate open and transparent science, our end-to-end study protocol and our source code for preprocessing prostate MRI data archives, training baseline diagnostic AI models, evaluating lesion detection/diagnosis performance, and implementing statistical tests for AI/radiologists vs AI/radiologists comparisons, have been publicly released. Furthermore, a fully-anonymized dataset of 1500 prostate bpMRI scans from the PI-CAI challenge, and their outcomes, have been released to promote further research.

Publications

  • Twilt JJ, Saha A, Bosma JS, Giannarini G, Padhani AR, Yakar D, Elschot M, Veltman J, Futterer J, Huisman H, de Rooij M; PI-CAI Consortium. Evaluating an AI-driven Triaging Workflow for MRI-based Clinically Significant Prostate Cancer Diagnosis: A Simulation Study. Radiol Imaging Cancer. 2026 May;8(3):e250461. doi: 10.1148/rycan.250461.

    PubMed 41960994
  • Saha A, Bosma JS, Twilt JJ, van Ginneken B, Bjartell A, Padhani AR, Bonekamp D, Villeirs G, Salomon G, Giannarini G, Kalpathy-Cramer J, Barentsz J, Maier-Hein KH, Rusu M, Rouviere O, van den Bergh R, Panebianco V, Kasivisvanathan V, Obuchowski NA, Yakar D, Elschot M, Veltman J, Futterer JJ, de Rooij M, Huisman H; PI-CAI consortium. Artificial intelligence and radiologists in prostate cancer detection on MRI (PI-CAI): an international, paired, non-inferiority, confirmatory study. Lancet Oncol. 2024 Jul;25(7):879-887. doi: 10.1016/S1470-2045(24)00220-1. Epub 2024 Jun 11.

    PubMed 38876123

Questions about this trial

What is this clinical trial studying?

An observational study of Prostate Cancer, run by Radboud University Medical Center at 1 site in Netherlands.

The full registered description is in the About this study section and the official record.

Who may be eligible to take part?

The sponsor lists 1 inclusion and 5 exclusion criteria. Eligibility is decided by the study team after screening, so read the full criteria and contact a listed site.

Is this clinical trial currently recruiting?

The record says: finished, as of the sponsor's last update on 18 Nov 2023. Status can change between updates, so confirm with the study contact before making plans.

Where is the study taking place?

The record lists 1 site in Netherlands. Whether a site is taking part right now should be confirmed with that site. See every site.

What are the possible benefits and risks?

They depend on the study and the treatment, and no study can promise a benefit. The study team explains the known risks during informed consent, before anyone joins; the record itself does not list them.

Are results available for this study?

The study is completed, but results are not available in the source record. That does not mean the treatment did or did not work.

How can a doctor refer a patient or contact the research team?

The record lists no contact. The official record names the sponsor, who can be asked directly. AzTrials does not enrol participants or take referrals.

What does the primary outcome measure?

AI vs Radiologists from Reader Study (time frame: 6 months). The study lists 2 primary outcomes in all. It is what the study was designed to measure, not a finding.

Where can I read the original study information?

On the ClinicalTrials.gov record for NCT05489341, which also lists 2 linked publications.

Does a listing mean the treatment is approved or proven to work?

No. A listing means the study is registered. It does not show that the treatment is safe, works, or is approved by a regulator.

How often is this information updated?

AzTrials copies changed records from ClinicalTrials.gov every night and reloads the whole registry every week. The sponsor last updated this record on 18 Nov 2023.

Free account

Log in to keep searching

You’ve used your 3 free searches. AzTrials is free: log in or create a free account to keep searching clinical trials and save the ones you want to follow.

Create a free accountLog in