Urologic DiseasesNCT07378358
Evaluation of AI Large Models for Diagnosis and Treatment in Real-World Cases: Multicenter Retrospective Study
An observational study of Urologic Diseases, run by First Affiliated Hospital of Fujian Medical University at 1 site in China.
Trial summary
This multicenter retrospective study aims to evaluate the diagnostic and therapeutic performance of three large language models-ChatGPT, Gemini and Deepseek-using 800 archived inpatient medical records from urology departments across four tertiary hospitals. Read more about this study
- Who is it for?
- Ages 18 years and older
- Conditions the study is for
- Urologic Diseases
- What is being tested?
- Large Language Model Assessment (ChatGPT, Gemini, DeepSeek)
- How many people
- 800 participants
- Where
- 1 siteChina
- Study dates
- 1 Jan 2026to 1 Jun 2026
- Phase
- Observational
- Are there results?
- Not yet: the study is under way
Key requirements
Who can take part, as listed by the sponsor. Only the study team can confirm whether you are eligible, so take this list to your doctor.
- Ages 18 years and older
- All sexes
- No healthy volunteers
- Adult, Older adult
Study population: The study population was drawn from the following institutions: The First Affiliated Hospital of Fujian Medical University, The Second Affiliated Hospital of Fujian Medical University,Shishi City Hospital and Shaowu City Hospital
Who may be able to join
- The case data is sourced from the four hospitals involved in the study, with complete and authentic diagnosis and treatment records.
- Patients must be 18 years or older, with no gender restrictions.
- Complete medical records, including the following core information: patient' s basic information, present illness history, past medical history, physical examination, and auxiliary examinations (including laboratory and imaging tests).
- A clear discharge diagnosis and treatment plan (including therapeutic measures and follow-up arrangements).
- Medical records have been archived, with objective and accurate information that has not been altered.
- The patient or their legal representative has provided informed consent, agreeing to the use of their anonymized medical data for research analysis.
Who cannot join
- Medical records with significant missing information, such as key clinical details (present illness history, diagnostic or treatment records, etc.).
- Cases where the diagnosis or treatment plan is unclear, or where treatment has not been fully completed for an initial diagnosis.
- Cases where the primary diagnosis is not urological.
- Cases with major errors or inconsistencies in the records that could affect further assessment.
- Medical records in special formats or images that are not readable (e.g., handwritten notes, non-standard documentation).
- Patients who have not signed the informed consent form or who refuse to allow their medical data to be used for research.
Trial locations and contacts
1 site in 1 country. Contact the study team or a site near you to ask about taking part.
China1 site, 1 recruiting
- Recruiting
The First Affiliated Hospital of Fujian Medical University
Fuzhou
About this study
This multicenter retrospective study aims to evaluate the diagnostic and therapeutic performance of three large language models-ChatGPT, Gemini and Deepseek-using 800 archived inpatient medical records from urology departments across four tertiary hospitals. The study will focus on the accuracy and applicability of these models in disease recognition, preliminary diagnosis and treatment recommendation generation, in order to explore their potential value and limitations in supporting clinical decision-making in real-world settings.
- Conditions
- Urologic Diseases
- Being tested
- Large Language Model Assessment (ChatGPT, Gemini, DeepSeek)
- Keywords
- Large Language Models, Urologic Diseases, Clinical Decision Support, Retrospective Study
- Medical subject headings
- Urologic Diseases
What participants receive
What participants receive. In a randomised study each person is assigned to one group by chance.
Large Language Model Assessment (ChatGPT, Gemini, DeepSeek)Other
De-identified inpatient medical records were retrospectively collected from the urology departments of four tertiary hospitals (200 cases per site, 800 in total). Each case included standardized clinical information such as demographics, chief complaint, history of present illness, past medical history, physical examination, laboratory and imaging findings, discharge diagnosis and treatment plan.
To simulate the role of an AI system in a "first-visit physician" scenario, all diagnostic conclusions, differential diagnoses and treatment plans were removed before being input into the models. Three large language models (ChatGPT, Gemini and DeepSeek) were prompted with a standardized instruction: "Based on the above clinical information, provide your preliminary diagnosis, differential diagnoses and treatment recommendations." Each model generated outputs including (i) primary and secondary diagnoses, (ii) differential diagnosis lists with reasoning and (iii) preliminary treatment suggesti
What is being measured
The primary outcome is the main question the study is designed to answer.
Diagnostic Accuracy: Assessed by Top-1 accuracy
Top-1: Proportion of cases where the model's first diagnosis matches the true primary diagnosis.
Measured: Through study completion, an average of 3 months
Diagnostic Accuracy: Assessed by Top-3 accuracy
Top-3: Proportion of cases where the true diagnosis appears in the model's top 3.
Measured: Through study completion, an average of 3 months
Diagnostic Completeness
Proportion of the model's diagnoses that overlap with all diagnoses (primary and secondary) in the case.
Measured: Through study completion, an average of 3 months
Differential Diagnosis Quality
Evaluated by experts using a Likert 5-point scale, considering factors like common disease coverage, logical clarity, and specificity
Measured: Through study completion, an average of 3 months
Treatment Plan Quality
Assesses whether the model's treatment suggestions align with clinical guidelines, scored by experts on completeness, appropriateness, and safety.
Measured: Through study completion, an average of 3 months
Analysis Time
5.Time taken by the AI model to provide diagnoses and treatment suggestions (in seconds), reflecting real-time capability.
Measured: Through study completion, an average of 3 months
Timeline
From the start of the study to its completion; the mark is today. Estimated dates are the sponsor’s plan.
- First posted30 Jan 2026
- Study start1 Jan 2026 (estimated)
- Primary completion1 Apr 2026 (estimated)
- Study completion1 Jun 2026 (estimated)
Resources
- Official record on ClinicalTrials.govThe registry entry this page is built from, with its full change history.
- Other trials for Urologic DiseasesStudies of the same condition, open and under way.
- More trials run by First Affiliated Hospital of Fujian Medical UniversityThe sponsor's other registered studies.
- What the trial phases meanHow a treatment moves from first tests in people to approval.
More details
For doctors and researchers: how the study is designed, who runs it, and its registry IDs.
Study design
- Study type
- Observational
- Observational model
- Cohort
- Time perspective
- Retrospective
- Enrolment
- 800 participants (estimated)
Sponsor and investigators
- Sponsor type
- University, hospital or other
- Responsible party
- Sponsor
IDs, dates and publications
- ClinicalTrials.gov ID
- NCT07378358
- Sponsor's protocol ID
- MRCTA,ECFAH of FMU[2025]902
- Status last verified
- Jan 2026
- First submitted
- 9 Dec 2025
- Last updated
- 30 Jan 2026
- Results posted
- No
- FDA-regulated drug
- No
- FDA-regulated device
- No
- Individual data sharing
- No
Study at a glance
From the registry record, last updated by the sponsor on 30 Jan 2026. Recruitment status is as the sponsor reported it: confirm with the study team.
- Official title
- Evaluation of AI Large Models for Diagnosis and Treatment in Real-World Cases: Multicenter Retrospective Study
- ClinicalTrials.gov ID
- NCT07378358
- Acronym / sponsor ID
- MRCTA,ECFAH of FMU[2025]902
- Conditions
- Urologic Diseases
- Keywords
- Large Language Models, Urologic Diseases, Clinical Decision Support, Retrospective Study
- Study type
- Observational
- Recruitment status
- Recruiting
- Enrolment
- 800 participants (estimated)
- Lead sponsor
- First Affiliated Hospital of Fujian Medical UniversityUniversity, hospital or other
- Study start
- 1 Jan 2026 (estimated)
- Primary completion
- 1 Apr 2026 (estimated)
- Study completion
- 1 Jun 2026 (estimated)
- First posted
- 30 Jan 2026 (actual)
- Results
- Not posted in the source record
Clinical summary
The research question as the sponsor registered it. This describes what the study plans to test, not what it found.
This multicenter retrospective study aims to evaluate the diagnostic and therapeutic performance of three large language models-ChatGPT, Gemini and Deepseek-using 800 archived inpatient medical records from urology departments across four tertiary hospitals. The study will focus on the accuracy and applicability of these models in disease recognition, preliminary diagnosis and treatment recommendation generation, in order to explore their potential value and limitations in supporting clinical decision-making in real-world settings.
Study design
- Observational model
- Cohort
- Time perspective
- Retrospective
Who may qualify
- Ages
- 18 years and older
- Sex
- All
- Healthy volunteers
- Not accepted
- Age groups
- Adult, Older adult
- Study population
- The study population was drawn from the following institutions: The First Affiliated Hospital of Fujian Medical University, The Second Affiliated Hospital of Fujian Medical University,Shishi City Hospital and Shaowu City Hospital
Key inclusion criteria (6 listed)
- The case data is sourced from the four hospitals involved in the study, with complete and authentic diagnosis and treatment records.
- Patients must be 18 years or older, with no gender restrictions.
- Complete medical records, including the following core information: patient' s basic information, present illness history, past medical history, physical examination, and auxiliary examinations (including laboratory and imaging tests).
- A clear discharge diagnosis and treatment plan (including therapeutic measures and follow-up arrangements).
- Medical records have been archived, with objective and accurate information that has not been altered.
Key exclusion criteria (6 listed)
- Medical records with significant missing information, such as key clinical details (present illness history, diagnostic or treatment records, etc.).
- Cases where the diagnosis or treatment plan is unclear, or where treatment has not been fully completed for an initial diagnosis.
- Cases where the primary diagnosis is not urological.
- Cases with major errors or inconsistencies in the records that could affect further assessment.
- Medical records in special formats or images that are not readable (e.g., handwritten notes, non-standard documentation).
Eligibility is decided by the study team after screening; this overview does not establish that a patient qualifies. Full inclusion and exclusion criteria.
Interventions
| Intervention | Type | Arms |
|---|---|---|
| Large Language Model Assessment (ChatGPT, Gemini, DeepSeek)De-identified inpatient medical records were retrospectively collected from the urology departments of four tertiary hospitals (200 cases per site, 800 in total). Each case included standardized clinical information such as demographics, chief complaint, history of present illness, past medical history, physical examination, laboratory and imaging findings, discharge diagnosis and treatment plan. To simulate the role of an AI system in a "first-visit physician" scenario, all diagnostic conclusions, differential diagnoses and treatment plans were removed before being input into the models. Three large language models (ChatGPT, Gemini and DeepSeek) were prompted with a standardized instruction: "Based on the above clinical information, provide your preliminary diagnosis, differential diagnoses and treatment recommendations." Each model generated outputs including (i) primary and secondary diagnoses, (ii) differential diagnosis lists with reasoning and (iii) preliminary treatment suggesti | Other | – |
Primary outcome
Diagnostic Accuracy: Assessed by Top-1 accuracy
Time frame: Through study completion, an average of 3 months
Top-1: Proportion of cases where the model's first diagnosis matches the true primary diagnosis.
Diagnostic Accuracy: Assessed by Top-3 accuracy
Time frame: Through study completion, an average of 3 months
Top-3: Proportion of cases where the true diagnosis appears in the model's top 3.
Diagnostic Completeness
Time frame: Through study completion, an average of 3 months
Proportion of the model's diagnoses that overlap with all diagnoses (primary and secondary) in the case.
Differential Diagnosis Quality
Time frame: Through study completion, an average of 3 months
Evaluated by experts using a Likert 5-point scale, considering factors like common disease coverage, logical clarity, and specificity
Treatment Plan Quality
Time frame: Through study completion, an average of 3 months
Assesses whether the model's treatment suggestions align with clinical guidelines, scored by experts on completeness, appropriateness, and safety.
Analysis Time
Time frame: Through study completion, an average of 3 months
5.Time taken by the AI model to provide diagnoses and treatment suggestions (in seconds), reflecting real-time capability.
Results and safety
This study is still under way. Results are not available in the source record yet.
Research links
- Official record on ClinicalTrials.govLast updated by the sponsor 30 Jan 2026.
- Individual participant dataSharing planned: No
Related trials for Urologic Diseases
RecruitingPhase 1/2
Study of INBRX-106 and INBRX-106 in Combination With Pembrolizumab (Keytruda®) in Subjects With Locally Advanced or Metastatic Solid Tumors (Hexavalent OX40 Agonist)
Solid Tumor, Non-Small Cell Lung Cancer
RecruitingObservational study
Population Pharmacokinetics of Temocillin in Acute Enterobacterial Pyelonephritis in Children
Pyelonephritis
Active, not recruitingPhase 3
A Study of Belzutifan (MK-6482) in Combination With Lenvatinib Versus Cabozantinib for Treatment of Renal Cell Carcinoma (MK-6482-011)
Carcinoma, Renal Cell
Recruiting
Safety and Performance of UCon Patch Electrode
Urinary Incontinence, Urinary Incontinence, Urge
Active, not recruitingPhase 2
Safety and Efficacy of Empagliflozin in Hemodialysis
End Stage Renal Disease
Referral and contact
AzTrials does not enrol participants or take referrals. Contact the study team or a site listed below; they decide who can take part.
1 site in 1 country: see every site and its status.
Questions about this trial
What is this clinical trial studying?
An observational study of Urologic Diseases, run by First Affiliated Hospital of Fujian Medical University at 1 site in China.
The full registered description is in the About this study section and the official record.
Who may be eligible to take part?
The sponsor lists 6 inclusion and 6 exclusion criteria. Eligibility is decided by the study team after screening, so read the full criteria and contact a listed site.
Is this clinical trial currently recruiting?
The record says: looking for participants now, as of the sponsor's last update on 30 Jan 2026. Status can change between updates, so confirm with the study contact before making plans.
Where is the study taking place?
The record lists 1 site in China. Whether a site is taking part right now should be confirmed with that site. See every site.
What are the possible benefits and risks?
They depend on the study and the treatment, and no study can promise a benefit. The study team explains the known risks during informed consent, before anyone joins; the record itself does not list them.
Are results available for this study?
This study is still under way. Results are not available in the source record yet.
How can a doctor refer a patient or contact the research team?
Contact Ning Xu (+86-13235907575, drxun@fjmu.edu.cn), or a site near the patient. AzTrials does not enrol participants or take referrals.
What does the primary outcome measure?
Diagnostic Accuracy: Assessed by Top-1 accuracy (time frame: Through study completion, an average of 3 months). The study lists 6 primary outcomes in all. It is what the study was designed to measure, not a finding.
Where can I read the original study information?
Does a listing mean the treatment is approved or proven to work?
No. A listing means the study is registered. It does not show that the treatment is safe, works, or is approved by a regulator.
How often is this information updated?
AzTrials copies changed records from ClinicalTrials.gov every night and reloads the whole registry every week. The sponsor last updated this record on 30 Jan 2026.