DiagnosisNCT07374926
The Big Unknown: A Journey Into Generative AI's Transformative Effect on Medical Professions
A study testing GPT-4o in people with Diagnosis and Vignette of Fictional Patients, run by Maastricht University at 3 sites in 3 countries.
Trial summary
A parallel group randomized controlled trial using a superiority framework. Clinical vignettes will be used to assess the impact of a large language model on the clinical reasoning of physicians. Quantitative analyses will be performed on graded vignette responses. Read more about this study
- Who is it for?
- Any ageHealthy volunteers can take part
- Conditions the study is for
- Diagnosis, Vignette of Fictional Patients
- What is being tested?
- GPT-4o
- Could I receive a placebo?
- No, everyone receives an active treatment
- How many people
- 249 participants
- Where
- 3 sitesin 3 countries
- Study dates
- 1 Aug 2024to 17 Jan 2025
- Phase
- Interventional
- Are there results?
- None posted in the registry
Key requirements
Who can take part, as listed by the sponsor. Only the study team can confirm whether you are eligible, so take this list to your doctor.
- Ages Any age
- All sexes
- Healthy volunteers accepted
- Child, Adult, Older adult
Who may be able to join
- Registered medical physicians
- Training in internal or family medicine
Who cannot join
- Not currently practicing clinically
Trial locations and contacts
3 sites in 3 countries.
3 sites in 3 countries
About this study
Official title: The Big Unknown: A Journey Into Generative AI's Transformative Effect on Meical Professions
A parallel group randomized controlled trial using a superiority framework. Clinical vignettes will be used to assess the impact of a large language model on the clinical reasoning of physicians. Quantitative analyses will be performed on graded vignette responses.
Read the detailed description
This study is a multi-country, parallel-group randomized controlled trial designed to evaluate whether access to a large language model (LLM) improves physician clinical decision-making. The trial uses a superiority framework and compares physicians randomized to either complete standardized clinical vignettes with access to GPT-4o or without any AI assistance.
Clinical vignettes simulate common primary care conditions such as cardiovascular, respiratory, musculoskeletal, fatigue-related, and infectious diseases. Each vignette includes multiple steps in the clinical reasoning process, from initial history-taking to diagnosis, treatment, and follow-up. Physician responses are graded using rubrics developed from evidence-based, context-specific best-practice guidelines.
The study is conducted across three countries-Indonesia, Kenya, and the Netherlands-representing different income levels and health system contexts. The primary outcome is performance on clinical vignettes, defined as adherence to best-practice guidelines. Secondary objectives include examining cross-country variation in physician performance, variation in performance distributions, and the role of engagement with the LLM in shaping outcomes.
- Conditions
- Diagnosis, Vignette of Fictional Patients
- Being tested
- GPT-4o
- Keywords
- LLM, Quality of Care, Clinical Reasoning, Primary Care
- Medical subject headings
- Disease
What participants receive
What participants receive. In a randomised study each person is assigned to one group by chance.
GPT-4oOther
GPT-4o provided via an iFrame in the online Qualtrics environment
| Study group | Role | What they receive |
|---|---|---|
| Own Knowlege | No intervention | Group will not be given access to GPT-4 or other online resources |
| GPT-4o | Active comparator | Group given GPT-4o access Receives: Other: GPT-4o |
What is being measured
The primary outcome is the main question the study is designed to answer.
Percentage Correct Score
Following Peabody et al (2000), the primary outcome is a percentage correct score across all steps in a vignette. This is generated by dividing the weighted total sum of rubric items assessed as present by the total number of rubric items possible in a vignette. Rubric items will be weighted with regards to their relevance by our expert panel.
Measured: During Evaluation
Secondary outcomes (3)
Quality Per Answer
This outcome is generated as the average weight of rubric items assessed as present across vignettes. As each item is provided a weight (0.33,0.5,1), the average weight is the sum of weights divided by the number of answers marked as present.
Measured: During Evaluation
Number of Answers
This outcome is generated as the count of the total number of answers assessed as present by reviewers per vignette
Measured: During evaluation
Less obvious answers
This outcome is generated as the number of answers given that are less obvious, i.e. mentioned less frequently by the control group. If the answer is mentioned by 25% or less of the control group, it is considered less obvious.
Measured: During evaluation
Timeline
From the start of the study to its completion; the mark is today. Estimated dates are the sponsor’s plan.
- First posted29 Jan 2026
- Study start1 Aug 2024
- Primary completion17 Jan 2025
- Study completion17 Jan 2025
Resources
- Official record on ClinicalTrials.govThe registry entry this page is built from, with its full change history.
- Other trials for DiagnosisStudies of the same condition, open and under way.
- What the trial phases meanHow a treatment moves from first tests in people to approval.
More details
For doctors and researchers: how the study is designed, who runs it, and its registry IDs.
Study design
- Study type
- Interventional
- Phase
- Phase N/A
- Primary purpose
- Diagnosis
- Allocation
- Randomised
- Intervention model
- Parallel groups
- Model description
- We designed clinical vignettes to simulate patient-physician consultations. We follow the structure of clinical performance and value vignettes, a vignette design validated to perform well against standard patient designs. Our vignette design follows 9 stages: (1) Presenting Problem \& Initial Differential Diagnosis, (2) Asking about patient history, (3) Additional Differential Diagnosis, (4) Listing Physical Exams, (5) Differential Diagnosis, (6) Additional diagnostic (Lab) Tests, (7) Final Differential Diagnosis, (8) Medication, (9) Follow-Up/Advice. Participants were randomly assigned either to the control or intervention group using simple randomization and asked to complete the vignettes in an online environment. Intervention group participants were given access to an LLM (GPT-4o)
- Masking
- Single blind
- Who is masked
- Outcomes assessor
- Enrolment
- 249 participants (actual)
Sponsor and investigators
- Lead sponsor
- Maastricht University
- Sponsor type
- University, hospital or other
- Collaborators
- Aga Khan University, Universitas Indonesia
- Responsible party
- Sponsor
- Investigators
- Mark Levels, PhD, Principal investigator, Maastricht University
IDs, dates and publications
- ClinicalTrials.gov ID
- NCT07374926
- Sponsor's protocol ID
- ERCIC_572_25_04_2024
- Status last verified
- Sep 2025
- First submitted
- 21 Jan 2026
- Last updated
- 29 Jan 2026
- Results posted
- No
- Data monitoring committee
- No
- FDA-regulated drug
- No
- FDA-regulated device
- No
Study at a glance
From the registry record, last updated by the sponsor on 29 Jan 2026. Recruitment status is as the sponsor reported it: confirm with the study team.
- Official title
- The Big Unknown: A Journey Into Generative AI's Transformative Effect on Meical Professions
- ClinicalTrials.gov ID
- NCT07374926
- Acronym / sponsor ID
- ERCIC_572_25_04_2024
- Conditions
- Diagnosis, Vignette of Fictional Patients
- Keywords
- LLM, Quality of Care, Clinical Reasoning, Primary Care
- Study type
- Interventional
- Recruitment status
- Completed
- Enrolment
- 249 participants (actual)
- Lead sponsor
- Maastricht UniversityUniversity, hospital or other
- Collaborators
- Aga Khan University, Universitas Indonesia
- Study start
- 1 Aug 2024 (actual)
- Primary completion
- 17 Jan 2025 (actual)
- Study completion
- 17 Jan 2025 (actual)
- First posted
- 29 Jan 2026 (actual)
- Results
- Not posted in the source record
Clinical summary
The research question as the sponsor registered it. This describes what the study plans to test, not what it found.
A parallel group randomized controlled trial using a superiority framework. Clinical vignettes will be used to assess the impact of a large language model on the clinical reasoning of physicians. Quantitative analyses will be performed on graded vignette responses.
Detailed description
This study is a multi-country, parallel-group randomized controlled trial designed to evaluate whether access to a large language model (LLM) improves physician clinical decision-making. The trial uses a superiority framework and compares physicians randomized to either complete standardized clinical vignettes with access to GPT-4o or without any AI assistance.
Clinical vignettes simulate common primary care conditions such as cardiovascular, respiratory, musculoskeletal, fatigue-related, and infectious diseases. Each vignette includes multiple steps in the clinical reasoning process, from initial history-taking to diagnosis, treatment, and follow-up. Physician responses are graded using rubrics developed from evidence-based, context-specific best-practice guidelines.
The study is conducted across three countries-Indonesia, Kenya, and the Netherlands-representing different income levels and health system contexts. The primary outcome is performance on clinical vignettes, defined as adherence to best-practice guidelines. Secondary objectives include examining cross-country variation in physician performance, variation in performance distributions, and the role of engagement with the LLM in shaping outcomes.
Study design
- Primary purpose
- Diagnosis
- Allocation
- Randomised
- Intervention model
- Parallel groups
- Masking
- Single blind (outcomes assessor)
- Number of arms
- 2
- Model description
- We designed clinical vignettes to simulate patient-physician consultations. We follow the structure of clinical performance and value vignettes, a vignette design validated to perform well against standard patient designs. Our vignette design follows 9 stages: (1) Presenting Problem \& Initial Differential Diagnosis, (2) Asking about patient history, (3) Additional Differential Diagnosis, (4) Listing Physical Exams, (5) Differential Diagnosis, (6) Additional diagnostic (Lab) Tests, (7) Final Differential Diagnosis, (8) Medication, (9) Follow-Up/Advice. Participants were randomly assigned either to the control or intervention group using simple randomization and asked to complete the vignettes in an online environment. Intervention group participants were given access to an LLM (GPT-4o)
| Arm | Type | Receives |
|---|---|---|
| Own Knowlege | No intervention | –Group will not be given access to GPT-4 or other online resources |
| GPT-4o | Active comparator | Other: GPT-4oGroup given GPT-4o access |
Who may qualify
- Sex
- All
- Healthy volunteers
- Accepted
- Age groups
- Child, Adult, Older adult
Key inclusion criteria (2 listed)
- Registered medical physicians
- Training in internal or family medicine
Key exclusion criteria (1 listed)
- Not currently practicing clinically
Eligibility is decided by the study team after screening; this overview does not establish that a patient qualifies. Full inclusion and exclusion criteria.
Interventions
| Intervention | Type | Arms |
|---|---|---|
| GPT-4oGPT-4o provided via an iFrame in the online Qualtrics environment | Other | GPT-4o |
Primary outcome
Percentage Correct Score
Time frame: During Evaluation
Following Peabody et al (2000), the primary outcome is a percentage correct score across all steps in a vignette. This is generated by dividing the weighted total sum of rubric items assessed as present by the total number of rubric items possible in a vignette. Rubric items will be weighted with regards to their relevance by our expert panel.
3 secondary outcomes
Quality Per Answer
Time frame: During Evaluation
Number of Answers
Time frame: During evaluation
Less obvious answers
Time frame: During evaluation
Results and safety
The study is completed, but results are not available in the source record. That does not mean the treatment did or did not work.
Research links
- Official record on ClinicalTrials.govLast updated by the sponsor 29 Jan 2026.
Related trials for Diagnosis
Not yet recruitingPhase 1/2
177Lu-DOTA-EB-TATE in Adult Patients With Metastatic, Radioactive Iodine Non-Responsive Oncocytic (Hurthle-Cell) Thyroid Cancer
Thyroid Cancer
Recruiting
Tissue Modeling in Systemic Sclerosis Using Induced Pluripotent Stem Cells (iPSCs)
Systemic Sclerosis (SSc)
Not yet recruiting
GOT VOICE Feasibility With In-Person Group Singing
Elderly, Coronary Arterial Disease (CAD)
RecruitingObservational study
Investigating Complex Neurodegenerative Disorders Related to Amyotrophic Lateral Sclerosis and Frontotemporal Dementia
Frontotemporal Dementia, Amyotrophic Lateral Sclerosis
RecruitingObservational study
A Longitudinal Study of Familial Hypereosinophilia (FE): Natural History and Markers of Disease Progression
Eosinophilia, Hypereosinophilic Syndrome
Referral and contact
AzTrials does not enrol participants or take referrals. Contact the study team or a site listed below; they decide who can take part. This study is not recruiting at the moment.
Mark Levels, PhD principal investigator
Maastricht University
3 sites in 3 countries: see every site and its status.
Questions about this trial
What is this clinical trial studying?
A study testing GPT-4o in people with Diagnosis and Vignette of Fictional Patients, run by Maastricht University at 3 sites in 3 countries.
The full registered description is in the About this study section and the official record.
Who may be eligible to take part?
The sponsor lists 2 inclusion and 1 exclusion criteria. Eligibility is decided by the study team after screening, so read the full criteria and contact a listed site.
Is this clinical trial currently recruiting?
The record says: finished, as of the sponsor's last update on 29 Jan 2026. Status can change between updates, so confirm with the study contact before making plans.
Where is the study taking place?
The record lists 3 sites in Indonesia, Kenya, Netherlands. Whether a site is taking part right now should be confirmed with that site. See every site.
What are the possible benefits and risks?
They depend on the study and the treatment, and no study can promise a benefit. The study team explains the known risks during informed consent, before anyone joins; the record itself does not list them.
Are results available for this study?
The study is completed, but results are not available in the source record. That does not mean the treatment did or did not work.
How can a doctor refer a patient or contact the research team?
The record lists no contact. The official record names the sponsor, who can be asked directly. AzTrials does not enrol participants or take referrals.
What does the primary outcome measure?
Percentage Correct Score (time frame: During Evaluation). It is what the study was designed to measure, not a finding.
Where can I read the original study information?
Does a listing mean the treatment is approved or proven to work?
No. A listing means the study is registered. It does not show that the treatment is safe, works, or is approved by a regulator.
How often is this information updated?
AzTrials copies changed records from ClinicalTrials.gov every night and reloads the whole registry every week. The sponsor last updated this record on 29 Jan 2026.