Colorectal CancerNCT07045207
Evaluating AI and Human Expert Decisions in Colorectal Cancer
An observational study of Colorectal Cancer, run by Peking University Cancer Hospital & Institute at 1 site in China.
Trial summary
The goal of this observational study is to evaluate the decision-making consistency between large language models (LLMs) and expert multidisciplinary teams (MDTs) in adult patients diagnosed with colorectal cancer who underwent MDT consultation between January 2023 and December 2024. Read more about this study
- Who is it for?
- Any age
- Conditions the study is for
- Colorectal Cancer
- What is being tested?
- LLM-MDT
- How many people
- 1,500 participants
- Where
- 1 siteChina
- Study dates
- 1 Jul 2025to 1 Jun 2026
- Phase
- Observational
- Are there results?
- Not yet: the study is under way
Key requirements
Who can take part, as listed by the sponsor. Only the study team can confirm whether you are eligible, so take this list to your doctor.
- Ages Any age
- All sexes
- No healthy volunteers
- Child, Adult, Older adult
Study population: This study includes adult patients with histologically confirmed colorectal cancer who received multidisciplinary team (MDT) consultation at Peking University Cancer Hospital between January 1, 2023 and December 31, 2024. All clinical data were retrospectively collected from electronic medical records, including imaging, pathology, and MDT treatment recommendations. Patients represent a real-world tertiary cancer center population and were not selected based on treatment response or prognosis.
Who may be able to join
- Patients with a histologically confirmed diagnosis of colorectal cancer
- Patients who received multidisciplinary team (MDT) consultation at Peking University Cancer Hospital between January 1, 2023 and December 31, 2024
- Availability of complete clinical records, including(MDT consultation notes, CT or MRI imaging reports, Pathology reports, Outpatient or inpatient medical summaries)
Who cannot join
- Incomplete or missing medical records related to MDT decision-making
- MDT consultations conducted for non-oncologic purposes (e.g., hernia evaluation, stoma planning)
- Missing critical clinical data such as imaging or pathology reports
- Duplicate or conflicting records that prevent reliable data analysis
Trial locations and contacts
1 site in 1 country. Contact the study team or a site near you to ask about taking part.
China1 site
Peking University Cancer Hospital
Beijing
Yongjiu Chen, PhD, +86 18813041827, yjchen@bjmu.edu.cn
About this study
Official title: Comparison of Large Language Models and Expert Multidisciplinary Team Decisions in Colorectal Cancer
The goal of this observational study is to evaluate the decision-making consistency between large language models (LLMs) and expert multidisciplinary teams (MDTs) in adult patients diagnosed with colorectal cancer who underwent MDT consultation between January 2023 and December 2024.
The main questions it aims to answer are:
Read more
How consistent are the treatment decisions generated by LLMs compared to actual MDT decisions? Do different LLMs (e.g., ChatGPT, DeepSeek) show varying levels of agreement with expert recommendations? What clinical factors contribute to differences between AI-generated and human expert decisions? Researchers will compare the AI-generated treatment recommendations with real-world MDT decisions using anonymized patient records to see if LLMs can reliably support clinical decision-making in oncology.
Participants will:
Have their de-identified clinical data (e.g., imaging, pathology, MDT notes) processed through several LLMs Not be contacted or receive any interventions, as this is a retrospective study using existing clinical records only.
Read the detailed description
This is a retrospective, non-interventional observational study aiming to evaluate the consistency between treatment decisions made by large language models (LLMs) and multidisciplinary team (MDT) experts in the management of colorectal cancer (CRC).
Colorectal cancer is a highly heterogeneous malignancy requiring personalized treatment strategies, often developed through MDT discussions that integrate input from surgery, oncology, radiology, pathology, and other specialties. While MDTs improve treatment planning and outcomes, they are time- and resource-intensive, and subject to variability in expert judgment. With the rise of artificial intelligence, especially LLMs such as ChatGPT and DeepSeek, there is growing interest in their potential role in assisting or standardizing clinical decision-making.
In this study, researchers will retrospectively analyze de-identified clinical records of approximately 1,500 patients with histologically confirmed colorectal cancer who underwent MDT consultation at a tertiary cancer center between January 2023 and December 2024. Key clinical data-including demographic information, imaging reports (CT, MRI), endoscopy results, pathology findings, and MDT recommendations-will be extracted and anonymized.
These de-identified records will be input into several LLMs (ChatGPT, DeepSeek, Baichuan, and Qwen) running on secure offline servers. The models will be asked to generate treatment recommendations, which will be categorized into predefined decision codes (e.g., surgery, systemic therapy, chemoradiotherapy, further diagnostics). Each case will be input three times to assess the consistency of the model output.
The primary outcome is the agreement between AI-generated recommendations and original MDT decisions, quantified using Cohen's Kappa. Secondary analyses include comparison among LLMs using chi-squared tests, evaluation of output consistency via Fleiss' Kappa, and identification of clinical factors associated with discordant decisions.
This study does not involve any direct patient contact, intervention, or new clinical procedures. All data are historical and anonymized in accordance with ethical and legal requirements. The results are expected to inform the potential value, limitations, and appropriate use of AI in supporting multidisciplinary decision-making in oncology.
- Conditions
- Colorectal Cancer
- Being tested
- LLM-MDT
- Keywords
- Colorectal cancer, Multidisciplinary Team, Artificial Intelligence, Large Language Model
- Medical subject headings
- Colorectal Neoplasms
What participants receive
What participants receive. In a randomised study each person is assigned to one group by chance.
LLM-MDTOther
Leveraging large language models (LLMs) to Generate Multidisciplinary Team (MDT) Treatment Recommendations
What is being measured
The primary outcome is the main question the study is designed to answer.
Agreement Between AI-Generated and MDT Treatment Decisions
Description: The primary outcome is the consistency between treatment recommendations generated by large language models (LLMs) and those made by expert multidisciplinary teams (MDTs) for colorectal cancer cases. Consistency will be quantified using Cohen's Kappa coefficient. Higher Kappa values indicate stronger agreement
Measured: January 1, 2023 to December 31, 2024 (based on MDT consultation date)
Secondary outcomes (3)
Comparison of Agreement Across Different AI Models
Description: To compare the consistency of treatment decisions generated by different large language models (e.g., ChatGPT, DeepSeek, Baichuan, Qwen) with expert MDT decisions using Cohen's Kappa and chi-squared tests. This outcome assesses whether performance varies across AI models.
Measured: January 1, 2023 to December 31, 2024
Output Stability of AI Models on Repeated InputsDescription
To evaluate the reproducibility of treatment decisions generated by each AI model when the same clinical case is input multiple times. Stability will be assessed using Fleiss' Kappa to measure consistency across repeated outputs.
Measured: January 1, 2023 to December 31, 2024
Identification of Clinical Factors Associated With Decision Discordance
To identify key clinical features (e.g., disease stage, metastasis status, treatment history) that are associated with discordant treatment decisions between AI models and MDT experts. Statistical analysis will be conducted to explore which case characteristics lead to lower agreement.
Measured: January 1, 2023 to December 31, 2024
Timeline
From the start of the study to its completion; the mark is today. Estimated dates are the sponsor’s plan.
- First posted1 Jul 2025
- Study start1 Jul 2025 (estimated)
- Primary completion1 Jun 2026 (estimated)
- Study completion1 Jun 2026 (estimated)
Resources
- Official record on ClinicalTrials.govThe registry entry this page is built from, with its full change history.
- Other trials for Colorectal CancerStudies of the same condition, open and under way.
- More trials run by Peking University Cancer Hospital & InstituteThe sponsor's other registered studies.
- What the trial phases meanHow a treatment moves from first tests in people to approval.
More details
For doctors and researchers: how the study is designed, who runs it, and its registry IDs.
Study design
- Study type
- Observational
- Observational model
- Case only
- Time perspective
- Retrospective
- Enrolment
- 1,500 participants (estimated)
Sponsor and investigators
- Lead sponsor
- Peking University Cancer Hospital & Institute
- Sponsor type
- University, hospital or other
- Responsible party
- Sponsor
IDs, dates and publications
- ClinicalTrials.gov ID
- NCT07045207
- Sponsor's protocol ID
- PekingUCHI2025YJZ57
- Status last verified
- Jun 2025
- First submitted
- 22 Jun 2025
- Last updated
- 1 Jul 2025
- Results posted
- No
- Data monitoring committee
- No
- FDA-regulated drug
- No
- FDA-regulated device
- No
- Individual data sharing
- No
Study at a glance
From the registry record, last updated by the sponsor on 1 Jul 2025. Recruitment status is as the sponsor reported it: confirm with the study team.
- Official title
- Comparison of Large Language Models and Expert Multidisciplinary Team Decisions in Colorectal Cancer
- ClinicalTrials.gov ID
- NCT07045207
- Acronym / sponsor ID
- PekingUCHI2025YJZ57
- Conditions
- Colorectal Cancer
- Keywords
- Colorectal cancer, Multidisciplinary Team, Artificial Intelligence, Large Language Model
- Study type
- Observational
- Recruitment status
- Not yet recruiting
- Enrolment
- 1,500 participants (estimated)
- Lead sponsor
- Peking University Cancer Hospital & InstituteUniversity, hospital or other
- Study start
- 1 Jul 2025 (estimated)
- Primary completion
- 1 Jun 2026 (estimated)
- Study completion
- 1 Jun 2026 (estimated)
- First posted
- 1 Jul 2025 (actual)
- Results
- Not posted in the source record
Clinical summary
The research question as the sponsor registered it. This describes what the study plans to test, not what it found.
The goal of this observational study is to evaluate the decision-making consistency between large language models (LLMs) and expert multidisciplinary teams (MDTs) in adult patients diagnosed with colorectal cancer who underwent MDT consultation between January 2023 and December 2024.
The main questions it aims to answer are:
How consistent are the treatment decisions generated by LLMs compared to actual MDT decisions? Do different LLMs (e.g., ChatGPT, DeepSeek) show varying levels of agreement with expert recommendations? What clinical factors contribute to differences between AI-generated and human expert decisions? Researchers will compare the AI-generated treatment recommendations with real-world MDT decisions using anonymized patient records to see if LLMs can reliably support clinical decision-making in oncology.
Participants will:
Have their de-identified clinical data (e.g., imaging, pathology, MDT notes) processed through several LLMs Not be contacted or receive any interventions, as this is a retrospective study using existing clinical records only.
Detailed description
This is a retrospective, non-interventional observational study aiming to evaluate the consistency between treatment decisions made by large language models (LLMs) and multidisciplinary team (MDT) experts in the management of colorectal cancer (CRC).
Colorectal cancer is a highly heterogeneous malignancy requiring personalized treatment strategies, often developed through MDT discussions that integrate input from surgery, oncology, radiology, pathology, and other specialties. While MDTs improve treatment planning and outcomes, they are time- and resource-intensive, and subject to variability in expert judgment. With the rise of artificial intelligence, especially LLMs such as ChatGPT and DeepSeek, there is growing interest in their potential role in assisting or standardizing clinical decision-making.
In this study, researchers will retrospectively analyze de-identified clinical records of approximately 1,500 patients with histologically confirmed colorectal cancer who underwent MDT consultation at a tertiary cancer center between January 2023 and December 2024. Key clinical data-including demographic information, imaging reports (CT, MRI), endoscopy results, pathology findings, and MDT recommendations-will be extracted and anonymized.
These de-identified records will be input into several LLMs (ChatGPT, DeepSeek, Baichuan, and Qwen) running on secure offline servers. The models will be asked to generate treatment recommendations, which will be categorized into predefined decision codes (e.g., surgery, systemic therapy, chemoradiotherapy, further diagnostics). Each case will be input three times to assess the consistency of the model output.
The primary outcome is the agreement between AI-generated recommendations and original MDT decisions, quantified using Cohen's Kappa. Secondary analyses include comparison among LLMs using chi-squared tests, evaluation of output consistency via Fleiss' Kappa, and identification of clinical factors associated with discordant decisions.
This study does not involve any direct patient contact, intervention, or new clinical procedures. All data are historical and anonymized in accordance with ethical and legal requirements. The results are expected to inform the potential value, limitations, and appropriate use of AI in supporting multidisciplinary decision-making in oncology.
Study design
- Observational model
- Case only
- Time perspective
- Retrospective
Who may qualify
- Sex
- All
- Healthy volunteers
- Not accepted
- Age groups
- Child, Adult, Older adult
- Study population
- This study includes adult patients with histologically confirmed colorectal cancer who received multidisciplinary team (MDT) consultation at Peking University Cancer Hospital between January 1, 2023 and December 31, 2024. All clinical data were retrospectively collected from electronic medical records, including imaging, pathology, and MDT treatment recommendations. Patients represent a real-world tertiary cancer center population and were not selected based on treatment response or prognosis.
Key inclusion criteria (3 listed)
- Patients with a histologically confirmed diagnosis of colorectal cancer
- Patients who received multidisciplinary team (MDT) consultation at Peking University Cancer Hospital between January 1, 2023 and December 31, 2024
- Availability of complete clinical records, including(MDT consultation notes, CT or MRI imaging reports, Pathology reports, Outpatient or inpatient medical summaries)
Key exclusion criteria (4 listed)
- Incomplete or missing medical records related to MDT decision-making
- MDT consultations conducted for non-oncologic purposes (e.g., hernia evaluation, stoma planning)
- Missing critical clinical data such as imaging or pathology reports
- Duplicate or conflicting records that prevent reliable data analysis
Eligibility is decided by the study team after screening; this overview does not establish that a patient qualifies. Full inclusion and exclusion criteria.
Interventions
| Intervention | Type | Arms |
|---|---|---|
| LLM-MDTLeveraging large language models (LLMs) to Generate Multidisciplinary Team (MDT) Treatment Recommendations | Other | – |
Primary outcome
Agreement Between AI-Generated and MDT Treatment Decisions
Time frame: January 1, 2023 to December 31, 2024 (based on MDT consultation date)
Description: The primary outcome is the consistency between treatment recommendations generated by large language models (LLMs) and those made by expert multidisciplinary teams (MDTs) for colorectal cancer cases. Consistency will be quantified using Cohen's Kappa coefficient. Higher Kappa values indicate stronger agreement
3 secondary outcomes
Comparison of Agreement Across Different AI Models
Time frame: January 1, 2023 to December 31, 2024
Output Stability of AI Models on Repeated InputsDescription
Time frame: January 1, 2023 to December 31, 2024
Identification of Clinical Factors Associated With Decision Discordance
Time frame: January 1, 2023 to December 31, 2024
Results and safety
This study is still under way. Results are not available in the source record yet.
Research links
- Official record on ClinicalTrials.govLast updated by the sponsor 1 Jul 2025.
- Individual participant dataSharing planned: No
Related trials for Colorectal Cancer
Not yet recruitingPhase 1
Hepatic Artery Infusion of Carfilzomib in Participants With Liver Metastatic Disease Previously Treated With Hepatic Artery Infusion Pump Therapy
Colorectal Neoplasms, Neoplasms
Active, not recruiting
Streamlined Geriatric and Oncological Evaluation Based On IC Technology
Breast Cancer, Comorbidities and Coexisting Conditions
RecruitingObservational study
Pre- and On-treatment Multi-omics Profiling of Locally Advanced Rectal Cancer
Rectal Neoplasms
Enrolling by invitationObservational study
Natural History Study of Early Life Exposures in Agriculture (ELEA)
Prostate Cancer, Leukemia
RecruitingPhase 2
Tislelizumab in People With Colorectal Cancer
Colorectal Cancer
Referral and contact
AzTrials does not enrol participants or take referrals. Contact the study team or a site listed below; they decide who can take part.
1 site in 1 country: see every site and its status.
Questions about this trial
What is this clinical trial studying?
An observational study of Colorectal Cancer, run by Peking University Cancer Hospital & Institute at 1 site in China.
The full registered description is in the About this study section and the official record.
Who may be eligible to take part?
The sponsor lists 3 inclusion and 4 exclusion criteria. Eligibility is decided by the study team after screening, so read the full criteria and contact a listed site.
Is this clinical trial currently recruiting?
The record says: opening soon, as of the sponsor's last update on 1 Jul 2025. Status can change between updates, so confirm with the study contact before making plans.
Where is the study taking place?
The record lists 1 site in China. Whether a site is taking part right now should be confirmed with that site. See every site.
What are the possible benefits and risks?
They depend on the study and the treatment, and no study can promise a benefit. The study team explains the known risks during informed consent, before anyone joins; the record itself does not list them.
Are results available for this study?
This study is still under way. Results are not available in the source record yet.
How can a doctor refer a patient or contact the research team?
Contact Yongjiu Chen, PhD (+86 18813041827, yjchen@bjmu.edu.cn), or a site near the patient. AzTrials does not enrol participants or take referrals.
What does the primary outcome measure?
Agreement Between AI-Generated and MDT Treatment Decisions (time frame: January 1, 2023 to December 31, 2024 (based on MDT consultation date)). It is what the study was designed to measure, not a finding.
Where can I read the original study information?
Does a listing mean the treatment is approved or proven to work?
No. A listing means the study is registered. It does not show that the treatment is safe, works, or is approved by a regulator.
How often is this information updated?
AzTrials copies changed records from ClinicalTrials.gov every night and reloads the whole registry every week. The sponsor last updated this record on 1 Jul 2025.