AzTrials
AzTrials

InfluenzaNCT05009251

Using Explainable AI Risk Predictions to Nudge Influenza Vaccine Uptake

A study testing Reminder, Risk reduction and Medical records-based recommendation in people with Influenza and Vaccination, run by National Bureau of Economic Research, Inc. at 1 site in United States.

Completed

Finished. The study has ended and is not enrolling. Results may be posted on the official record.

Official record
Search all trials

Trial summary

The study team previously demonstrated that patients are more likely to receive flu vaccine after learning that they are at high risk for flu complications. Building on this past work, the present study will explore whether providing reasons that patients are considered high risk for flu complications (a) further… Read more about this study

Who is it for?
Ages 18 years and older
Conditions the study is for
Influenza, Vaccination, Health Promotion
What is being tested?
Reminder, Risk reduction and Medical records-based recommendation
Could I receive a placebo?
No, everyone receives an active treatment
How many people
45,061 participants
Where
1 siteUnited States
Study dates
9 Sep 2021to 31 Jul 2022
Phase
Interventional
Are there results?
Yes, posted in the registrySummary in the doctors' view

Key requirements

Who can take part, as listed by the sponsor. Only the study team can confirm whether you are eligible, so take this list to your doctor.

  • Ages 18 years and older
  • All sexes
  • No healthy volunteers
  • Adult, Older adult

Who may be able to join

  • Aged 18 or older
  • Current Geisinger patient at the time of study
  • Falls in the top 10% of patients at highest risk, as identified by the flu-complication risk scores of machine learning algorithm (which operates on coded EHR data)

Who cannot join

  • Has contraindications for flu vaccination
  • Has opted out of receiving communications from Geisinger via all of the modalities being tested

Trial locations and contacts

1 site in 1 country.

United States1 site

  • Geisinger Clinic

    Danville, Pennsylvania, 17822

Other trials in United States

About this study

The study team previously demonstrated that patients are more likely to receive flu vaccine after learning that they are at high risk for flu complications. Building on this past work, the present study will explore whether providing reasons that patients are considered high risk for flu complications (a) further increases the likelihood they will receive flu vaccine and (b) decreases the likelihood that they receive diagnoses of flu and/or flu-like symptoms in the ensuing flu season. It will also examine whether informing patients that their high-risk status was determined by analyzing their medical records or by an artificial intelligence (AI) / machine-learning (ML) algorithm analyzing their medical records will affect the likelihood of receiving the flu vaccine or diagnoses of flu and/or flu-like symptoms.

Read the detailed description

Geisinger has partnered with Medial EarlySign and developed an ML algorithm to identify patients at risk for serious (moderate to severe) flu-associated complications on the basis of their existing electronic health record (EHR) data. Geisinger will apply this algorithm to current patients during the 2021-22 flu season.

This study will evaluate the effect of contacting patients identified as high risk with special messages to encourage vaccination. These communications will inform patients they are at high risk with either (a) no additional explanation, (b) an explanation that this determination comes from an analysis of their medical records, along with a short list of the top factors from their medical record that explain their risk, and (c) the additional explanation that an AI or ML algorithm made this determination, along with a short list of the top factors from their medical record that explain their risk.

Included in the study will be current Geisinger patients 18+ years of age with no contraindications for flu vaccine and who have been assessed by the Medial algorithm and assigned a risk score. The primary study outcomes will be the rates of flu vaccination and flu diagnosis during the 2020-21 season by targeted patients.

Being tested
Reminder, Risk reduction, Medical records-based recommendation, Algorithm-based recommendation
Keywords
Flu Vaccine, Choice Architecture, Machine Learning, Perceived Credibility, Personalized Risk Factors, Flu Complications
Medical subject headings
Influenza, Human, Health Behavior, Risk Reduction Behavior

What participants receive

What participants receive. In a randomised study each person is assigned to one group by chance.

  • ReminderBehavioural

    Mailed letter, short message service (SMS) text, and/or patient portal message

  • Risk reductionBehavioural

    Mailed letter, SMS, and/or patient portal message

  • Medical records-based recommendationBehavioural

    Also called Credibility

    Mailed letter, SMS, and/or patient portal message

  • Algorithm-based recommendationBehavioural

    Also called Credibility

    Mailed letter, SMS, and/or patient portal message

Study groups in NCT05009251
Study groupRoleWhat they receive
No-Contact ControlNo interventionSubjects in the no-contact control arm will receive no additional pro-vaccination intervention beyond the health system's normal efforts. Although some patients are currently targeted for flu vaccination encouragement due to a conventional non-ML assessment that they are at high risk for complications, these patients are not told that they are at high risk or that they have been targeted.
Reminder ControlExperimentalSubjects in the reminder control arm will receive messages reminding them to get the flu shot without being advised of their risk status.
Receives: Behavioral: Reminder
High Risk OnlyExperimentalSubjects in this treatment arm will receive messages telling them they have been identified to be at high risk for flu complications, without specifying how or why the health system believes this to be the case.
Receives: Behavioral: Reminder; Behavioral: Risk reduction
High Risk with Explanation Based on Medical RecordsExperimentalSubjects in this treatment arm will receive messages telling them they have been identified to be at high risk for flu complications via review of their medical records and will be provided a human-understandable short list of the top factors from their medical record that explain their risk.
Receives: Behavioral: Reminder; Behavioral: Risk reduction; Behavioral: Medical records-based recommendation
High Risk with Explanation Based on AlgorithmExperimentalSubjects in this treatment arm will receive messages telling them they have been identified to be at high risk for flu complications via analysis of their medical records by a computer algorithm and will be provided a human-understandable short list of the top factors from their medical record that explain their risk.
Receives: Behavioral: Reminder; Behavioral: Risk reduction; Behavioral: Medical records-based recommendation; Behavioral: Algorithm-based recommendation

What is being measured

The primary outcome is the main question the study is designed to answer.

  1. Flu Vaccination at 2 Weeks After Final Outreach Date

    Received flu vaccination

    Measured: Within 2 weeks of the final outreach date, 57 days (8.14 weeks) after the study start

Secondary outcomes (4)

  1. Flu Vaccination at 9 Weeks After Final Outreach Date

    Received a flu vaccination

    Measured: Within 9 weeks of the final outreach date, 106 days (15.14 weeks) after the study start

  2. Flu Diagnosis

    Received a "high confidence flu" diagnosis (with positive polymerase chain reaction [PCR]/antigen/molecular test) and/or "likely flu" diagnosis (as assessed via International Classification of Disease [ICD] codes or Tamiflu administration or positive PCR/antigen/molecular test)

    Note that "likely flu" is a superset of the "high confidence flu" diagnoses.

    Measured: 8 months (between September 9, 2021 and April 30, 2022)

  3. Flu Complications

    Diagnosed with flu-related complications

    Measured: 11 months (between September 9, 2021 and July 31, 2022)

  4. Healthcare Utilization

    Visited ER or was hospitalized

    NOTE: Our prespecified outcome was description was "Visited ER, was hospitalized, or had flu-related insurance claims." We did not receive claims data, so this outcomes includes only ER visits and hospitalizations.

    Measured: 11 months (between September 9, 2021 and July 31, 2022)

Timeline

From the start of the study to its completion; the mark is today. Estimated dates are the sponsor’s plan.

  1. First posted17 Aug 2021
  2. Study start9 Sep 2021
  3. Primary completion5 Nov 2021
  4. Study completion31 Jul 2022

Resources

More details

For doctors and researchers: how the study is designed, who runs it, and its registry IDs.

Study design
Study type
Interventional
Phase
Phase N/A
Primary purpose
Prevention
Allocation
Randomised
Intervention model
Parallel groups
Model description
Patients from the high-risk sample will be randomly assigned to one of five arms: * No-Contact Control Arm * Reminder Control Arm * High Risk Only Arm * High Risk with Explanation Based on Medical Records Arm * High Risk with Explanation Based on Algorithm Arm
Masking
Single blind
Who is masked
Care provider
Enrolment
45,061 participants (actual)
IDs, dates and publications
ClinicalTrials.gov ID
NCT05009251
Sponsor's protocol ID
2021-0483
Other IDs
P30AG034532
Status last verified
Dec 2024
First submitted
4 Aug 2021
Last updated
3 Jan 2025
Data monitoring committee
Yes
FDA-regulated drug
No
FDA-regulated device
No
Individual data sharing
Yes: Data with no personally identifiable information will be made available to other researchers on the Open Science Framework for transparency. This will include the essential data and code needed to replicate the analysis that yielded reported findings.

Publications

  • Rosenbaum GM, Goren A, Shermohammed M, Wolk DM, Tice AM, Doyle JJ Jr, Meyer MN, Chabris CF. Algorithm-assisted personalized risk communication to encourage flu vaccination in the USA: three randomized field trials. Nat Hum Behav. 2026 Oct 5. doi: 10.1038/s41562-026-02603-4. Online ahead of print.

    PubMed 42834081

Questions about this trial

What is this clinical trial studying?

A study testing Reminder, Risk reduction and Medical records-based recommendation in people with Influenza and Vaccination, run by National Bureau of Economic Research, Inc. at 1 site in United States.

The full registered description is in the About this study section and the official record.

Who may be eligible to take part?

The sponsor lists 3 inclusion and 2 exclusion criteria. Eligibility is decided by the study team after screening, so read the full criteria and contact a listed site.

Is this clinical trial currently recruiting?

The record says: finished, as of the sponsor's last update on 3 Jan 2025. Status can change between updates, so confirm with the study contact before making plans.

Where is the study taking place?

The record lists 1 site in United States. Whether a site is taking part right now should be confirmed with that site. See every site.

What are the possible benefits and risks?

They depend on the study and the treatment, and no study can promise a benefit. The study team explains the known risks during informed consent, before anyone joins; the record itself does not list them.

Are results available for this study?

Yes. Results are posted in the registry (first posted 12 Jan 2023). A summary is in the For doctors view, and the full results are on ClinicalTrials.gov.

How can a doctor refer a patient or contact the research team?

The record lists no contact. The official record names the sponsor, who can be asked directly. AzTrials does not enrol participants or take referrals.

What does the primary outcome measure?

Flu Vaccination at 2 Weeks After Final Outreach Date (time frame: Within 2 weeks of the final outreach date, 57 days (8.14 weeks) after the study start). It is what the study was designed to measure, not a finding.

Where can I read the original study information?

On the ClinicalTrials.gov record for NCT05009251, which also lists 1 linked publication.

Does a listing mean the treatment is approved or proven to work?

No. A listing means the study is registered. It does not show that the treatment is safe, works, or is approved by a regulator.

How often is this information updated?

AzTrials copies changed records from ClinicalTrials.gov every night and reloads the whole registry every week. The sponsor last updated this record on 3 Jan 2025.

AzTrials uses cookies for analytics and advertising. You can accept or reject them; the site works either way. Privacy Policy.

Free account

Log in to keep searching

You’ve used your 3 free searches. AzTrials is free: log in or create a free account to keep searching clinical trials and save the ones you want to follow.

Create a free accountLog in