AzTrials
AzTrials

Diagnostic Decision MakingNCT07457840

Integrating AI Predictions With Clinician Expertise

A study testing Bayesian-Updated Post-Test Probability, Standard AI Predicted Probability and Uncertainty Quantification (95% Confidence Interval) in people with Diagnostic Decision Making, run by University of California, San Francisco at 1 site in United States.

Enrolling by invitation

Enrolling by invitation. The study team selects and invites participants, usually from an existing group of patients.

Sites and contacts
Official record
Search all trials

Trial summary

Optimizing the interaction between the human and the machine is a major topic when deploying artificial intelligence (AI) at the bedside. The goal of this randomized clinical vignette study is to learn if presenting AI model outputs via continuous Bayesian updates and/or uncertainty quantification can improve… Read more about this study

Who is it for?
Ages 18 years and olderHealthy volunteers can take part
Conditions the study is for
Diagnostic Decision Making
What is being tested?
Bayesian-Updated Post-Test Probability, Standard AI Predicted Probability and Uncertainty Quantification (95% Confidence Interval)
Could I receive a placebo?
No, everyone receives an active treatment
How many people
100 participants
Where
1 siteUnited States
Study dates
15 Feb 2026to Dec 2026
Phase
Interventional
Are there results?
Not yet: the study is under way

Key requirements

Who can take part, as listed by the sponsor. Only the study team can confirm whether you are eligible, so take this list to your doctor.

  • Ages 18 years and older
  • All sexes
  • Healthy volunteers accepted
  • Adult, Older adult

Who may be able to join

  • Must hold one of the following clinical roles: Nurse Practitioner (NP), Physician Assistant/Physician Associate (PA), Resident Physician, Physician Fellow, or Attending Physician
  • Able to complete the survey in English
  • Access to a computer or tablet (mobile phones are not recommended due to the visual nature of the survey)

Who cannot join

  • Does not hold an eligible clinical role as defined above
  • Completes fewer than 2 of 8 clinical vignettes (less than 25% of the survey)
  • Has previously participated in this study
  • Unable to complete the survey in English

Trial locations and contacts

1 site in 1 country. Contact the study team or a site near you to ask about taking part.

United States1 site
  • ZSFG

    San Francisco, California, 94110

Other trials in United States

About this study

Official title: Transforming Clinical Decision Support Systems: Using Continuous Bayesian Updates to Integrate AI Predictions With Clinician Expertise

Optimizing the interaction between the human and the machine is a major topic when deploying artificial intelligence (AI) at the bedside. The goal of this randomized clinical vignette study is to learn if presenting AI model outputs via continuous Bayesian updates and/or uncertainty quantification can improve diagnostic accuracy and clinician trust in healthcare professionals (physicians, residents, fellows, physician assistants (PAs), and nurse practitioners (NPs)) from US academic institutions evaluating patients with chest pain or dyspnea.

The main questions it aims to answer are:

Read more
  • Does presenting AI predictions as Bayesian-updated post-test probabilities improve diagnostic accuracy compared to standard predicted probabilities?
  • Does the addition of uncertainty quantification (95% confidence intervals) to AI predictions improve diagnostic accuracy?
  • Do these interventions (Bayesian updating and/or uncertainty quantification) help clinicians recover from the negative effects of intentionally misleading AI predictions?

Comparison: Researchers will compare standard AI predicted probabilities (presented without uncertainty) to Bayesian-updated post-test probabilities and/or outputs containing 95% confidence intervals to see if the interventions improve diagnostic accuracy, clinician confidence, and resilience against misleading AI.

Participants will:

  • Review 8 clinical vignettes (simulated patient cases) focusing on chest pain or dyspnea.
  • Provide an initial "pre-test" diagnostic probability for 5 possible diagnoses based on the clinical history alone.
  • View AI model outputs that vary by experimental condition (standard probability vs. Bayesian update, with or without uncertainty intervals, and accurate vs. misleading).
  • Provide an updated "post-test" diagnostic probability for the diagnoses after viewing the AI output.
  • Select and rank diagnostic tests and therapeutic steps for each vignette. Complete a post-survey regarding their trust in the AI, comfort with the data presentation, and demographics.
Read the detailed description

Study Design: This is a 2x2 factorial within-subjects design. The two factors are (1) Bayesian updating via continuous likelihood ratios (CLR) vs. standard predicted probability, and (2) uncertainty quantification (95% confidence intervals) vs. point estimate only. AI prediction accuracy (accurate vs. intentionally misleading) is varied as a within-subjects stratification factor balanced across all 4 conditions, with half of each participant's vignettes receiving accurate predictions and half receiving misleading predictions. AI predictions are simulated (pre-programmed) for experimental control. Vignette order and condition assignment are independently randomized per participant.

Primary Analysis: Diagnostic accuracy is analyzed using a generalized linear mixed model (GLMM) with fixed effects for CLR, Uncertainty, Misleading, and vignette, and a participant random intercept. Pre-specified secondary analyses examine interactions of presentation format with misleading AI.

Sample Size: Simulation-based power analysis (1,000 Monte Carlo iterations per scenario) was conducted using the planned GLMM. Assuming 70% baseline diagnostic accuracy and within-participant ICC of 0.25, the study achieves 85.8% power for the CLR main effect and 85.7% for the Uncertainty main effect with N=100 at alpha=0.05 (two-tailed).

Being tested
Bayesian-Updated Post-Test Probability, Standard AI Predicted Probability, Uncertainty Quantification (95% Confidence Interval)

What participants receive

What participants receive. In a randomised study each person is assigned to one group by chance.

  • Bayesian-Updated Post-Test ProbabilityBehavioural

    Rather than presenting the AI model's raw predicted probability, the system takes the clinician's pre-test probability (entered before seeing AI output) and applies a continuous likelihood ratio (CLR) derived from the AI model to calculate a Bayesian-updated post-test probability. The output is displayed as a shift from the clinician's own assessment (e.g., "Your assessment: 45% -> Updated assessment: 72%"). The raw AI prediction is not shown. This approach mirrors how clinicians use diagnostic test results such as D-dimer to update pre-test probability of pulmonary embolism.

  • Standard AI Predicted ProbabilityBehavioural

    AI model prediction is presented as a simple predicted probability (0-100%) for each of the possible diagnoses, together with the top 3 clinical features driving the prediction (e.g., "Acute Myocardial Infarction: 68% - Key factors: elevated troponin, ST-segment changes on ECG, chest pain radiation to left arm"). This represents the most common current approach to presenting AI-based diagnostic predictions in clinical settings.

  • Uncertainty Quantification (95% Confidence Interval)Behavioural

    The AI output (whether Bayesian-updated post-test probability or standard predicted probability) is presented together with a 95% confidence band displayed as error bars on probability bars. For accurate AI predictions, confidence interval width is approximately +/-12-15 percentage points. For misleading AI predictions, confidence intervals are widened by a factor of 1.5x (approximately +/-18-23 percentage points) to simulate reduced model confidence in unfamiliar or edge-case scenarios. Confidence intervals are constrained to the 0-100% range.

Study groups in NCT07457840
Study groupRoleWhat they receive
Standard Probability + No Uncertainty (Control)Active comparatorAI model prediction is presented as a standard predicted probability for each possible diagnosis (point estimate only), together with the top 3 clinical features driving the prediction. No confidence interval is shown. This is the control condition representing the most common current approach to presenting AI predictions in clinical settings. Within this arm, half of vignettes contain accurate AI predictions and half contain intentionally misleading predictions (balanced by design).
Receives: Behavioral: Standard AI Predicted Probability
Bayesian Updating (CLR) + No UncertaintyExperimentalAI model prediction is used to perform Bayesian updating of the clinician's pre-test probability into a post-test probability using continuous likelihood ratios (CLR). The post-test probability is presented as a point estimate only, without a confidence interval. The raw AI predicted probability is not shown to the participant. Within this arm, half of vignettes contain accurate AI predictions and half contain intentionally misleading predictions (balanced by design).
Receives: Behavioral: Bayesian-Updated Post-Test Probability
Standard Probability + Uncertainty (95% CI)ExperimentalAI model prediction is presented as a standard predicted probability for each possible diagnosis, together with the top 3 clinical features driving the prediction. The predicted probability is accompanied by a 95% confidence interval. Within this arm, half of vignettes contain accurate AI predictions and half contain intentionally misleading predictions (balanced by design). When AI predictions are misleading, confidence intervals are widened by a factor of 1.5x to simulate greater model uncertainty.
Receives: Behavioral: Standard AI Predicted Probability; Behavioral: Uncertainty Quantification (95% Confidence Interval)
Bayesian Updating (CLR) + Uncertainty (95% CI)ExperimentalAI model prediction is used to perform Bayesian updating of the clinician's pre-test probability into a post-test probability using continuous likelihood ratios (CLR). The post-test probability is presented with a 95% confidence interval. The raw AI predicted probability is not shown to the participant. This arm represents the full intervention combining both candidate approaches. Within this arm, half of vignettes contain accurate AI predictions and half contain intentionally misleading predictions (balanced by design). When AI predictions are misleading, confidence intervals are widened by a factor of 1.5x to simulate greater model uncertainty.
Receives: Behavioral: Bayesian-Updated Post-Test Probability; Behavioral: Uncertainty Quantification (95% Confidence Interval)

What is being measured

The primary outcome is the main question the study is designed to answer.

  1. Clinician Diagnostic Accuracy

    Proportion of correct diagnostic assessments across all vignettes and experimental conditions. For each vignette, participants rate 5 possible diagnoses on a 0-100% probability scale. The diagnosis assigned the highest probability is considered the participant's final diagnosis. Accuracy is determined by comparing the final diagnosis to the ground truth diagnosis established by expert panel consensus (minimum 4 of 5 board-certified physicians in agreement). Analyzed using a generalized linear mixed model (GLMM) with binary outcome (correct vs. incorrect), fixed effects for CLR, uncertainty quantification, misleading AI, and vignette, and a random intercept for participant.

    Measured: Day 1 during survey completion

Secondary outcomes (3)
  1. Change in Diagnostic Probability Estimates

    Magnitude and direction of change in clinician-provided probability estimates from pre-test assessment (before AI output) to post-test assessment (after AI output) for each of 5 possible diagnoses per vignette. Measured on a 0-100% scale.

    Measured: Day 1 during survey completion

  2. Diagnostic Accuracy Under Misleading AI Predictions

    Proportion of correct final diagnoses when AI predictions are intentionally misleading vs. accurate, and whether the interventions (Bayesian updating, uncertainty quantification) mitigate the negative effect of misleading AI. Assessed via interaction terms (CLR x Misleading, Uncertainty x Misleading) in the primary GLMM.

    Measured: Day 1 during survey completion

  3. Clinician Satisfaction With AI Decision Support (Exploratory)

    Self-reported satisfaction with the AI-based clinical decision support, measured via question(s) in the post-survey questionnaire.

    Measured: Day 1 during survey completion

Timeline

From the start of the study to its completion; the mark is today. Estimated dates are the sponsor’s plan.

  1. First posted9 Mar 2026
  2. Study start15 Feb 2026
  3. Primary completionDec 2026 (estimated)
  4. Study completionDec 2026 (estimated)

Resources

More details

For doctors and researchers: how the study is designed, who runs it, and its registry IDs.

Study design
Study type
Interventional
Phase
Phase N/A
Primary purpose
Other
Allocation
Randomised
Intervention model
Factorial
Model description
2x2 factorial within-subjects design. Two interventions are crossed: (1) Bayesian updating via continuous likelihood ratios (CLR) vs. standard predicted probability, and (2) uncertainty quantification (95% CI) vs. point estimate only. Each participant reviews 8 clinical vignettes, with each vignette assigned to one of the 4 factorial conditions. AI prediction accuracy is varied as a stratification factor: half of each participant's vignettes receive accurate AI predictions and half receive intentionally misleading predictions, balanced across all 4 conditions. This tests whether the presentation formats help clinicians resist incorrect AI. Vignette order and condition assignment are randomized per participant. AI predictions are simulated (pre-programmed) for experimental control. Primary analysis: main effects of CLR and uncertainty on diagnostic accuracy via generalized linear mixed model. Secondary analyses: interaction of presentation format with misleading AI.
Masking
Single blind
Who is masked
Participant
Enrolment
100 participants (estimated)
IDs, dates and publications
ClinicalTrials.gov ID
NCT07457840
Sponsor's protocol ID
26-45852
Status last verified
Jul 2026
First submitted
24 Feb 2026
Last updated
14 Jul 2026
Results posted
No
Data monitoring committee
No
FDA-regulated drug
No
FDA-regulated device
No
Individual data sharing
No

Questions about this trial

What is this clinical trial studying?

A study testing Bayesian-Updated Post-Test Probability, Standard AI Predicted Probability and Uncertainty Quantification (95% Confidence Interval) in people with Diagnostic Decision Making, run by University of California, San Francisco at 1 site in United States.

The full registered description is in the About this study section and the official record.

Who may be eligible to take part?

The sponsor lists 3 inclusion and 4 exclusion criteria. Eligibility is decided by the study team after screening, so read the full criteria and contact a listed site.

Is this clinical trial currently recruiting?

The record says: enrolling by invitation, as of the sponsor's last update on 14 Jul 2026. Status can change between updates, so confirm with the study contact before making plans.

Where is the study taking place?

The record lists 1 site in United States. Whether a site is taking part right now should be confirmed with that site. See every site.

What are the possible benefits and risks?

They depend on the study and the treatment, and no study can promise a benefit. The study team explains the known risks during informed consent, before anyone joins; the record itself does not list them.

Are results available for this study?

This study is still under way. Results are not available in the source record yet.

How can a doctor refer a patient or contact the research team?

The record lists no contact. The official record names the sponsor, who can be asked directly. AzTrials does not enrol participants or take referrals.

What does the primary outcome measure?

Clinician Diagnostic Accuracy (time frame: Day 1 during survey completion). It is what the study was designed to measure, not a finding.

Where can I read the original study information?
Does a listing mean the treatment is approved or proven to work?

No. A listing means the study is registered. It does not show that the treatment is safe, works, or is approved by a regulator.

How often is this information updated?

AzTrials copies changed records from ClinicalTrials.gov every night and reloads the whole registry every week. The sponsor last updated this record on 14 Jul 2026.

Free account

Log in to keep searching

You’ve used your 3 free searches. AzTrials is free: log in or create a free account to keep searching clinical trials and save the ones you want to follow.

Create a free accountLog in