CommunicationNCT06108037
DHL Survey on Generative AI for MyChart Messaging
A study testing Generative AI for electronic communication and disclosure in people with Communication, run by Duke University at 1 site in United States.
Trial summary
The purpose of this study is to understand how patients feel about the use of computer programs to create responses when they send electronic messages to their doctors. Read more about this study
- Who is it for?
- Ages 18 years and olderHealthy volunteers can take part
- Conditions the study is for
- Communication
- What is being tested?
- Generative AI for electronic communication and disclosure
- Type of treatment
- Lifestyle or therapy programme
- Could I receive a placebo?
- No, everyone receives an active treatment
- How many people
- 1,454 participants
- Where
- 1 siteUnited States
- Study dates
- 31 Oct 2023to 11 Dec 2023
- Phase
- Interventional
- Are there results?
- None posted in the registry
Key requirements
Who can take part, as listed by the sponsor. Only the study team can confirm whether you are eligible, so take this list to your doctor.
- Ages 18 years and older
- All sexes
- Healthy volunteers accepted
- Adult, Older adult
Who may be able to join
- Member of the Duke Health Listens patient advocacy community
Who cannot join
- Age < 18
Trial locations and contacts
1 site in 1 country.
United States1 site
Duke University Health System
Durham, North Carolina, 27710
About this study
Official title: Duke Health Listens Survey on Generative Artificial Intelligence (AI) for MyChart Messaging
The purpose of this study is to understand how patients feel about the use of computer programs to create responses when they send electronic messages to their doctors.
Read the detailed description
- The investigators will create short surveys online to ask patients how they feel about using computer programs that create messages in their medical records.
- The surveys will show fictional situations where patients ask questions and get answers from either real people or computer programs, with or without a disclosure about how the response was written.
- The investigators will ask the people taking the survey to share what they think about these situations using tools like rating scales, comparison scales, or written responses.
- If patients want to, they can provide their contact information to be part of future discussion groups. Participants do not have to give any personal information to complete the survey.
- Conditions
- Communication
- Being tested
- Generative AI for electronic communication and disclosure
- Keywords
- Artificial Intelligence, Generative Artificial Intelligence
- Medical subject headings
- Communication
What participants receive
What participants receive. In a randomised study each person is assigned to one group by chance.
Generative AI for electronic communication and disclosureBehavioural
We will use a large language model such as GPT 3.5 to automatically generate responses to fictional messages to a physician. We will disclose whether the message was generated using this technology or not. There are 3 clinical scenarios and 6 pairs of human/AI response and human disclosure/AI disclosure/not disclosed that will test patient attitudes toward this technology.
| Study group | Role | What they receive |
|---|---|---|
| Arm A | Other | Each arm will receive 3 clinical scenarios spaced over time across 3 Sends. The 6 groups (Arms A-F) will be arranged with naming conventions as such:
* First letter = A, B, or C where A = scenario 1, B = scenario 2, and C = scenario 3.
* Second letter(s) = H or AI, where H = human response and AI = AI-written response to the patient question posed.
* Third letter(s) = N, C, or H, where N = no disclosure, C = computer disclosure, and H = human disclosure. This refers to the disclosure at the bottom of the response message whereby the author is or is not disclosed.
Arm A receives AHN in Send 1, BAIC in Send 2, and CHH in Send 3 Receives: Behavioral: Generative AI for electronic communication and disclosure |
| Arm B | Other | Each arm will receive 3 clinical scenarios spaced over time across 3 Sends. The 6 groups (Arms A-F) will be arranged with naming conventions as such:
* First letter = A, B, or C where A = scenario 1, B = scenario 2, and C = scenario 3.
* Second letter(s) = H or AI, where H = human response and AI = AI-written response to the patient question posed.
* Third letter(s) = N, C, or H, where N = no disclosure, C = computer disclosure, and H = human disclosure. This refers to the disclosure at the bottom of the response message whereby the author is or is not disclosed.
Arm B receives BHC in Send 1, CAIH in Send 2, and AAIN in Send 3 Receives: Behavioral: Generative AI for electronic communication and disclosure |
| Arm C | Other | Each arm will receive 3 clinical scenarios spaced over time across 3 Sends. The 6 groups (Arms A-F) will be arranged with naming conventions as such:
* First letter = A, B, or C where A = scenario 1, B = scenario 2, and C = scenario 3.
* Second letter(s) = H or AI, where H = human response and AI = AI-written response to the patient question posed.
* Third letter(s) = N, C, or H, where N = no disclosure, C = computer disclosure, and H = human disclosure. This refers to the disclosure at the bottom of the response message whereby the author is or is not disclosed.
Arm C receives CHC in Send 1, AHH in Send 2, and BAIN in Send 3 Receives: Behavioral: Generative AI for electronic communication and disclosure |
| Arm D | Other | Each arm will receive 3 clinical scenarios spaced over time across 3 Sends. The 6 groups (Arms A-F) will be arranged with naming conventions as such:
* First letter = A, B, or C where A = scenario 1, B = scenario 2, and C = scenario 3.
* Second letter(s) = H or AI, where H = human response and AI = AI-written response to the patient question posed.
* Third letter(s) = N, C, or H, where N = no disclosure, C = computer disclosure, and H = human disclosure. This refers to the disclosure at the bottom of the response message whereby the author is or is not disclosed.
Arm D receives AAIH in Send 1, BHN in Send 2, and CAIC in Send 3 Receives: Behavioral: Generative AI for electronic communication and disclosure |
| Arm E | Other | Each arm will receive 3 clinical scenarios spaced over time across 3 Sends. The 6 groups (Arms A-F) will be arranged with naming conventions as such:
* First letter = A, B, or C where A = scenario 1, B = scenario 2, and C = scenario 3.
* Second letter(s) = H or AI, where H = human response and AI = AI-written response to the patient question posed.
* Third letter(s) = N, C, or H, where N = no disclosure, C = computer disclosure, and H = human disclosure. This refers to the disclosure at the bottom of the response message whereby the author is or is not disclosed.
Arm E receives BAIH in Send 1, CHN in Send 2, and AHC in Send 3 Receives: Behavioral: Generative AI for electronic communication and disclosure |
| Arm F | Other | Each arm will receive 3 clinical scenarios spaced over time across 3 Sends. The 6 groups (Arms A-F) will be arranged with naming conventions as such:
* First letter = A, B, or C where A = scenario 1, B = scenario 2, and C = scenario 3.
* Second letter(s) = H or AI, where H = human response and AI = AI-written response to the patient question posed.
* Third letter(s) = N, C, or H, where N = no disclosure, C = computer disclosure, and H = human disclosure. This refers to the disclosure at the bottom of the response message whereby the author is or is not disclosed.
Arm F receives CAIN in Send 1, AAIC in Send 2, and BHH in Send 3 Receives: Behavioral: Generative AI for electronic communication and disclosure |
What is being measured
The primary outcome is the main question the study is designed to answer.
Patient satisfaction, as measured by survey
Likert-scale responses to satisfaction question: "I am satisfied with this interaction", on a scale from 1-5 with answer options of Strongly Disagree (1), Disagree (2), Neither agree nor disagree (3), Agree (4), and Strongly agree (5).
Measured: Up to 2 weeks
Patient attitudes towards utility, as measured by survey
Likert-scale responses to utility question: "The information is useful", on a scale from 1-5 with answer options of Strongly Disagree (1), Disagree (2), Neither agree nor disagree (3), Agree (4), and Strongly agree (5).
Measured: Up to 2 weeks
Patient empathy, as measured by survey
Likert-scale responses to empathy question: "I feel cared for during this interaction", on a scale from 1-5 with answer options of Strongly Disagree (1), Disagree (2), Neither agree nor disagree (3), Agree (4), and Strongly agree (5).
Measured: Up to 2 weeks
Timeline
From the start of the study to its completion; the mark is today. Estimated dates are the sponsor’s plan.
- First posted30 Oct 2023
- Study start31 Oct 2023
- Primary completion11 Dec 2023
- Study completion11 Dec 2023
Resources
- Official record on ClinicalTrials.govThe registry entry this page is built from, with its full change history.
- Publications (1)Papers the sponsor has linked to this study.
- Other trials for CommunicationStudies of the same condition, open and under way.
- More trials run by Duke UniversityThe sponsor's other registered studies.
- What the trial phases meanHow a treatment moves from first tests in people to approval.
More details
For doctors and researchers: how the study is designed, who runs it, and its registry IDs.
Study design
- Study type
- Interventional
- Phase
- Phase N/A
- Primary purpose
- Other
- Allocation
- Randomised
- Intervention model
- Parallel groups
- Model description
- * Participants will be randomly allocated to one of six arms. * Over the course of the study period, there will be multiple rounds of surveys, which will consist of a clinical scenario, a response (human or AI generated), and a disclosure statement (none, human, or AI). * All respondents within an arm will be assigned to a sequence of response-disclosure pairs prior to the first round of surveys.
- Masking
- Single blind
- Who is masked
- Participant
- Enrolment
- 1,454 participants (actual)
Sponsor and investigators
- Lead sponsor
- Duke University
- Sponsor type
- University, hospital or other
- Responsible party
- Sponsor
- Investigators
- Anand Chowdhury, MD, MMCi, Principal investigator, Duke University
IDs, dates and publications
- ClinicalTrials.gov ID
- NCT06108037
- Sponsor's protocol ID
- Pro00113587
- Status last verified
- Oct 2023
- First submitted
- 23 Oct 2023
- Last updated
- 13 Dec 2023
- Results posted
- No
- Data monitoring committee
- Yes
- FDA-regulated drug
- No
- FDA-regulated device
- No
- Individual data sharing
- No: The investigators will not collect individual patient identifiers, and aggregate data will be reported
Publications
Ayers JW, Poliak A, Dredze M, Leas EC, Zhu Z, Kelley JB, Faix DJ, Goodman AM, Longhurst CA, Hogarth M, Smith DM. Comparing Physician and Artificial Intelligence Chatbot Responses to Patient Questions Posted to a Public Social Media Forum. JAMA Intern Med. 2023 Jun 1;183(6):589-596. doi: 10.1001/jamainternmed.2023.1838.
PubMed 37115527
Study at a glance
From the registry record, last updated by the sponsor on 13 Dec 2023. Recruitment status is as the sponsor reported it: confirm with the study team.
- Official title
- Duke Health Listens Survey on Generative Artificial Intelligence (AI) for MyChart Messaging
- ClinicalTrials.gov ID
- NCT06108037
- Acronym / sponsor ID
- Pro00113587
- Conditions
- Communication
- Keywords
- Artificial Intelligence, Generative Artificial Intelligence
- Study type
- Interventional
- Recruitment status
- Completed
- Enrolment
- 1,454 participants (actual)
- Lead sponsor
- Duke UniversityUniversity, hospital or other
- Study start
- 31 Oct 2023 (actual)
- Primary completion
- 11 Dec 2023 (actual)
- Study completion
- 11 Dec 2023 (actual)
- First posted
- 30 Oct 2023 (actual)
- Results
- Not posted in the source record
Clinical summary
The research question as the sponsor registered it. This describes what the study plans to test, not what it found.
The purpose of this study is to understand how patients feel about the use of computer programs to create responses when they send electronic messages to their doctors.
Detailed description
- The investigators will create short surveys online to ask patients how they feel about using computer programs that create messages in their medical records.
- The surveys will show fictional situations where patients ask questions and get answers from either real people or computer programs, with or without a disclosure about how the response was written.
- The investigators will ask the people taking the survey to share what they think about these situations using tools like rating scales, comparison scales, or written responses.
- If patients want to, they can provide their contact information to be part of future discussion groups. Participants do not have to give any personal information to complete the survey.
Study design
- Primary purpose
- Other
- Allocation
- Randomised
- Intervention model
- Parallel groups
- Masking
- Single blind (participant)
- Number of arms
- 6
- Model description
- * Participants will be randomly allocated to one of six arms. * Over the course of the study period, there will be multiple rounds of surveys, which will consist of a clinical scenario, a response (human or AI generated), and a disclosure statement (none, human, or AI). * All respondents within an arm will be assigned to a sequence of response-disclosure pairs prior to the first round of surveys.
| Arm | Type | Receives |
|---|---|---|
| Arm A | Other | Behavioral: Generative AI for electronic communication and disclosureEach arm will receive 3 clinical scenarios spaced over time across 3 Sends. The 6 groups (Arms A-F) will be arranged with naming conventions as such: * First letter = A, B, or C where A = scenario 1, B = scenario 2, and C = scenario 3. * Second letter(s) = H or AI, where H = human response and AI = AI-written response to the patient question posed. * Third letter(s) = N, C, or H, where N = no disclosure, C = computer disclosure, and H = human disclosure. This refers to the disclosure at the bottom of the response message whereby the author is or is not disclosed. Arm A receives AHN in Send 1, BAIC in Send 2, and CHH in Send 3 |
| Arm B | Other | Behavioral: Generative AI for electronic communication and disclosureEach arm will receive 3 clinical scenarios spaced over time across 3 Sends. The 6 groups (Arms A-F) will be arranged with naming conventions as such: * First letter = A, B, or C where A = scenario 1, B = scenario 2, and C = scenario 3. * Second letter(s) = H or AI, where H = human response and AI = AI-written response to the patient question posed. * Third letter(s) = N, C, or H, where N = no disclosure, C = computer disclosure, and H = human disclosure. This refers to the disclosure at the bottom of the response message whereby the author is or is not disclosed. Arm B receives BHC in Send 1, CAIH in Send 2, and AAIN in Send 3 |
| Arm C | Other | Behavioral: Generative AI for electronic communication and disclosureEach arm will receive 3 clinical scenarios spaced over time across 3 Sends. The 6 groups (Arms A-F) will be arranged with naming conventions as such: * First letter = A, B, or C where A = scenario 1, B = scenario 2, and C = scenario 3. * Second letter(s) = H or AI, where H = human response and AI = AI-written response to the patient question posed. * Third letter(s) = N, C, or H, where N = no disclosure, C = computer disclosure, and H = human disclosure. This refers to the disclosure at the bottom of the response message whereby the author is or is not disclosed. Arm C receives CHC in Send 1, AHH in Send 2, and BAIN in Send 3 |
| Arm D | Other | Behavioral: Generative AI for electronic communication and disclosureEach arm will receive 3 clinical scenarios spaced over time across 3 Sends. The 6 groups (Arms A-F) will be arranged with naming conventions as such: * First letter = A, B, or C where A = scenario 1, B = scenario 2, and C = scenario 3. * Second letter(s) = H or AI, where H = human response and AI = AI-written response to the patient question posed. * Third letter(s) = N, C, or H, where N = no disclosure, C = computer disclosure, and H = human disclosure. This refers to the disclosure at the bottom of the response message whereby the author is or is not disclosed. Arm D receives AAIH in Send 1, BHN in Send 2, and CAIC in Send 3 |
| Arm E | Other | Behavioral: Generative AI for electronic communication and disclosureEach arm will receive 3 clinical scenarios spaced over time across 3 Sends. The 6 groups (Arms A-F) will be arranged with naming conventions as such: * First letter = A, B, or C where A = scenario 1, B = scenario 2, and C = scenario 3. * Second letter(s) = H or AI, where H = human response and AI = AI-written response to the patient question posed. * Third letter(s) = N, C, or H, where N = no disclosure, C = computer disclosure, and H = human disclosure. This refers to the disclosure at the bottom of the response message whereby the author is or is not disclosed. Arm E receives BAIH in Send 1, CHN in Send 2, and AHC in Send 3 |
| Arm F | Other | Behavioral: Generative AI for electronic communication and disclosureEach arm will receive 3 clinical scenarios spaced over time across 3 Sends. The 6 groups (Arms A-F) will be arranged with naming conventions as such: * First letter = A, B, or C where A = scenario 1, B = scenario 2, and C = scenario 3. * Second letter(s) = H or AI, where H = human response and AI = AI-written response to the patient question posed. * Third letter(s) = N, C, or H, where N = no disclosure, C = computer disclosure, and H = human disclosure. This refers to the disclosure at the bottom of the response message whereby the author is or is not disclosed. Arm F receives CAIN in Send 1, AAIC in Send 2, and BHH in Send 3 |
Who may qualify
- Ages
- 18 years and older
- Sex
- All
- Healthy volunteers
- Accepted
- Age groups
- Adult, Older adult
Key inclusion criteria (1 listed)
- Member of the Duke Health Listens patient advocacy community
Key exclusion criteria (1 listed)
- Age < 18
Eligibility is decided by the study team after screening; this overview does not establish that a patient qualifies. Full inclusion and exclusion criteria.
Interventions
| Intervention | Type | Arms |
|---|---|---|
| Generative AI for electronic communication and disclosureWe will use a large language model such as GPT 3.5 to automatically generate responses to fictional messages to a physician. We will disclose whether the message was generated using this technology or not. There are 3 clinical scenarios and 6 pairs of human/AI response and human disclosure/AI disclosure/not disclosed that will test patient attitudes toward this technology. | Behavioural | Arm A, Arm B, Arm C, Arm D, Arm E, Arm F |
Primary outcome
Patient satisfaction, as measured by survey
Time frame: Up to 2 weeks
Likert-scale responses to satisfaction question: "I am satisfied with this interaction", on a scale from 1-5 with answer options of Strongly Disagree (1), Disagree (2), Neither agree nor disagree (3), Agree (4), and Strongly agree (5).
Patient attitudes towards utility, as measured by survey
Time frame: Up to 2 weeks
Likert-scale responses to utility question: "The information is useful", on a scale from 1-5 with answer options of Strongly Disagree (1), Disagree (2), Neither agree nor disagree (3), Agree (4), and Strongly agree (5).
Patient empathy, as measured by survey
Time frame: Up to 2 weeks
Likert-scale responses to empathy question: "I feel cared for during this interaction", on a scale from 1-5 with answer options of Strongly Disagree (1), Disagree (2), Neither agree nor disagree (3), Agree (4), and Strongly agree (5).
Results and safety
The study is completed, but results are not available in the source record. That does not mean the treatment did or did not work.
Research links
- Official record on ClinicalTrials.govLast updated by the sponsor 13 Dec 2023.
- Background: PubMed 37115527Ayers JW, Poliak A, Dredze M, Leas EC, Zhu Z, Kelley JB, Faix DJ, Goodman AM, Longhurst CA, Hogarth M, Smith DM. Comparing Physician and Artificial Intelligence Chatbot Responses to Patient Questions Posted to a Public Social Media Forum. JAMA Intern Med. 2023 Jun 1;183(6):589-596. doi: 10.1001/jamainternmed.2023.1838.
- Individual participant dataSharing planned: No. The investigators will not collect individual patient identifiers, and aggregate data will be reported
Related trials for Communication
Recruiting
Keep Social Randomized Controlled Trial for Young Adults With Cancer
Cancer
Active, not recruitingPhase 3
PREVENT HPV -Related Cancers Trial
HPV Vaccination, Uptake Vaccination
Not yet recruiting
Culturally-Tailored AI Chatbot Simulation for Therapeutic Communication and Empathy in Psychiatric Nursing Students
Therapeutic Communication
Enrolling by invitation
Effects of Semantic Context on Criterion-learning Naming Treatment in Aphasia
Aphasia
Recruiting
Feasibility and Preliminary Effects of a Therapeutic Approach Using Ultrasound Visual Biofeedback in Children With AFI: a Pilot Randomized Clinical Trial.
Childhood Apraxia of Speech, Speech Sound Disorders
Referral and contact
AzTrials does not enrol participants or take referrals. Contact the study team or a site listed below; they decide who can take part. This study is not recruiting at the moment.
Anand Chowdhury, MD, MMCi principal investigator
Duke University
1 site in 1 country: see every site and its status.
Questions about this trial
What is this clinical trial studying?
A study testing Generative AI for electronic communication and disclosure in people with Communication, run by Duke University at 1 site in United States.
The full registered description is in the About this study section and the official record.
Who may be eligible to take part?
The sponsor lists 1 inclusion and 1 exclusion criteria. Eligibility is decided by the study team after screening, so read the full criteria and contact a listed site.
Is this clinical trial currently recruiting?
The record says: finished, as of the sponsor's last update on 13 Dec 2023. Status can change between updates, so confirm with the study contact before making plans.
Where is the study taking place?
The record lists 1 site in United States. Whether a site is taking part right now should be confirmed with that site. See every site.
What are the possible benefits and risks?
They depend on the study and the treatment, and no study can promise a benefit. The study team explains the known risks during informed consent, before anyone joins; the record itself does not list them.
Are results available for this study?
The study is completed, but results are not available in the source record. That does not mean the treatment did or did not work.
How can a doctor refer a patient or contact the research team?
The record lists no contact. The official record names the sponsor, who can be asked directly. AzTrials does not enrol participants or take referrals.
What does the primary outcome measure?
Patient satisfaction, as measured by survey (time frame: Up to 2 weeks). The study lists 3 primary outcomes in all. It is what the study was designed to measure, not a finding.
Where can I read the original study information?
On the ClinicalTrials.gov record for NCT06108037, which also lists 1 linked publication.
Does a listing mean the treatment is approved or proven to work?
No. A listing means the study is registered. It does not show that the treatment is safe, works, or is approved by a regulator.
How often is this information updated?
AzTrials copies changed records from ClinicalTrials.gov every night and reloads the whole registry every week. The sponsor last updated this record on 13 Dec 2023; AzTrials last refreshed its copy on 31 Oct 2026.