Medical EducationSARANCT07844746
Teaching Medical Students to Verify AI-Generated Citation Hallucinations
A study testing SARA Citation-Verification Training and Responsible GenAI Education Intervention Description in people with Medical Education and Generative Artificial Intelligence, run by Quaid-e-Azam Medical College.
Trial summary
Generative artificial intelligence (GenAI) tools are increasingly used by medical students for literature searching, scientific writing, and research. However, AI-generated citations may contain errors. Read more about this study
- Who is it for?
- Any ageHealthy volunteers can take part
- Conditions the study is for
- Medical Education, Generative Artificial Intelligence
- What is being tested?
- SARA Citation-Verification Training and Responsible GenAI Education Intervention Description
- Type of treatment
- Lifestyle or therapy programme
- Could I receive a placebo?
- No, everyone receives an active treatment
- How many people
- 320 participants
- Where
- No sites listed yet
- Study dates
- 26 Sep 2026to 1 Feb 2027
- Phase
- Interventional
- Are there results?
- Not yet: the study is under way
Key requirements
Who can take part, as listed by the sponsor. Only the study team can confirm whether you are eligible, so take this list to your doctor.
- Ages Any age
- All sexes
- Healthy volunteers accepted
- Child, Adult, Older adult
Who may be able to join
- Enrolled as an MBBS student in Year 3, Year 4, or Year 5 at Quaid-e-Azam Medical College, Bahawalpur.
- Has received basic curricular exposure to research methodology or scientific literature searching.
- Has not previously received formal structured training specifically in verification of AI-generated citations.
- Willing and able to provide written informed consent.
- Meets the age eligibility specified in the approved study protocol.
Who cannot join
- Previous participation in development, cognitive interviewing, piloting, or validation testing of the SARA assessment materials.
- Previous formal training in the SARA citation-verification framework before enrolment.
- Unwilling or unable to provide informed consent.
Trial locations and contacts
No sites are listed yet. Contact the study team or a site near you to ask about taking part.
About this study
Official title: Effect of the SARA Citation-Verification Framework on Medical Students' Verification of AI-Generated Citation Hallucinations: A Randomized Controlled Trial With an Explanatory Sequential Mixed-Methods Component
Generative artificial intelligence (GenAI) tools are increasingly used by medical students for literature searching, scientific writing, and research. However, AI-generated citations may contain errors. Some references may not exist or may contain incorrect bibliographic details, while other references may be genuine but may not actually support the claim for which they are cited.
This randomized educational study will evaluate whether a structured citation-verification framework called SARA can improve medical students' ability to identify and correct these problems. SARA teaches students to assess four elements of a citation: Source authenticity, Accuracy, Relevance, and Alignment between the cited evidence and the claim being made.
Read more
Year 3-5 medical students will be randomly assigned to either a 75-minute SARA training session or an attention-matched session on responsible use of GenAI in academic work. Participants will complete citation-verification assessments before training, immediately after training, and 12 weeks later. The main outcome will assess whether students can independently apply the verification approach to an unstructured literature-review task 12 weeks after training, without being prompted to use SARA.
The study will also examine which types of citation errors remain difficult, whether students initiate appropriate verification, the quality of their verification, handling of valid citations, and confidence in their judgments. A subgroup of participants will take part in think-aloud and follow-up interviews to explore how they approach citation verification and transfer the strategy to new tasks.
Read the detailed description
AI-generated citation errors can occur at different levels. Some are source-level errors, such as fabricated references or incorrect bibliographic details. Others are more difficult to detect because the cited publication is genuine, but the evidence may be irrelevant, may not support the direction, magnitude, population, or conclusion of the claim, or may be presented without important qualifiers or limitations. In this study, these problems are examined across a five-level citation-error taxonomy ranging from non-existent sources to contextual omission or overgeneralization.
The SARA framework was developed as an educational scaffold to make citation-verification reasoning explicit. It prompts learners to ask whether the source exists, whether the citation details are accurate, whether the source is relevant to the specific claim, and whether the evidence is aligned with the claim as written. The intervention uses modelling, coached practice, feedback, articulation, reflection, and progressive removal of support so that learners move from guided verification toward independent judgment.
A key feature of the study is assessment of delayed transfer rather than immediate post-training performance alone. At 12 weeks, participants will review a realistic literature-review passage containing both valid and problematic citation-claim relationships without being told how many errors are present or being prompted to use SARA. This is intended to evaluate whether the verification strategy has become sufficiently internalized to be applied independently in a new scholarly context.
The explanatory qualitative component will explore why some learners successfully transfer verification skills while others continue to miss deeper evidence-claim problems. Think-aloud and stimulated-recall interviews will examine what triggers verification, how learners decide that evidence is sufficient, when they stop checking, and how they distinguish source existence from evidentiary support.
- Being tested
- SARA Citation-Verification Training, Responsible GenAI Education Intervention Description
- Keywords
- Generative Artificial Intelligence, Citation Hallucination, Citation Verification, Medical Students, Research Integrity, Evidence Appraisal
What participants receive
What participants receive. In a randomised study each person is assigned to one group by chance.
SARA Citation-Verification TrainingBehavioural
A standardized 75-minute educational intervention using the SARA framework: Source authenticity, Accuracy, Relevance, and Alignment. The session includes facilitator modelling and think-aloud demonstration, coached practice with feedback, a one-page verification scaffold, learner articulation and reflection, and progressive fading of prompts before independent practice. The intervention is designed to teach medical students to identify both source-level citation errors and deeper evidence-claim errors and to apply verification independently.
Responsible GenAI Education Intervention DescriptionBehavioural
A standardized 75-minute attention-matched educational session on responsible use of generative artificial intelligence in academic work. The session covers privacy, disclosure, authorship, academic integrity, and general awareness that GenAI may produce errors. It does not teach the SARA framework or another structured citation-verification process. The session is matched to the SARA intervention for duration, group size, facilitator interaction, and delivery format.
| Study group | Role | What they receive |
|---|---|---|
| SARA Citation-Verification Training | Experimental | Participants assigned to this arm will receive a standardized 75-minute educational intervention using the SARA citation-verification framework: Source authenticity, Accuracy, Relevance, and Alignment. Training includes facilitator modelling and think-aloud demonstration, coached practice with feedback, use of a one-page verification scaffold, articulation and reflection, and progressive fading of prompts before independent practice. The intervention is designed to teach students to distinguish source-level citation errors from deeper evidence-claim errors and to apply structured verification independently. Receives: Behavioral: SARA Citation-Verification Training |
| Responsible GenAI Education Control | Active comparator | Participants assigned to this arm will receive an attention-matched 75-minute educational session on responsible use of generative artificial intelligence in academic work. The session covers privacy, disclosure, authorship, academic integrity, and general awareness that GenAI may produce errors. It does not teach the SARA framework or another structured citation-verification process. The session is matched to the intervention for duration, group size, facilitator interaction, and delivery format. Receives: Behavioral: Responsible GenAI Education Intervention Description |
What is being measured
The primary outcome is the main question the study is designed to answer.
Independent Citation-Verification Performance on an Unstructured Scholarly Transfer Task
Participants will review a novel approximately 650-word AI-generated literature-review passage containing eight embedded citation-claim relationships. The passage includes both valid and erroneous citations spanning the study's five-level citation-error taxonomy. Participants will not be prompted to use the SARA framework and will not be told how many or what types of errors are present. Each citation-claim relationship will be scored against a prespecified gold standard for the appropriateness of the participant's final disposition. One point will be awarded for each appropriate disposition, producing a total score from 0 to 8, with higher scores indicating better independent citation-verification performance. Responses will be scored by outcome assessors blinded to intervention allocation.
Measured: 12 weeks after the intervention
Secondary outcomes (7)
Structured Citation-Verification Performance
Citation-verification performance will be measured using parallel 24-item structured assessment forms. Each item requires an appropriate final disposition of a citation-claim pair against a prespecified gold standard. One point will be awarded for each correct disposition, giving a total score from 0 to 24. Higher scores indicate better structured citation-verification performance.
Measured: Immediately after the intervention and 12 weeks after the intervention
Citation-Verification Accuracy by Citation-Error Condition
Accuracy will be assessed separately for valid/aligned citations and for Levels 1-5 of the prespecified citation-error taxonomy: non-existent citations, altered bibliographic details, irrelevant sources, evidence-claim misalignment, and contextual omission or overgeneralization. The outcome will be expressed as the proportion of citation-claim decisions with an appropriate final disposition within each condition.
Measured: Immediately after the intervention and 12 weeks after the intervention
Verification Initiation
Verification initiation will be measured as the proportion of citation-claim decisions for which the participant initiates an external verification action, such as consulting a bibliographic database, search engine, publisher source, abstract, or full text. Higher proportions indicate more frequent initiation of verification.
Measured: Immediately after the intervention and 12 weeks after the intervention
High-Quality Citation Verification
High-quality verification will be defined as examining the source element necessary to resolve the specific citation problem. Merely locating an article will not be considered adequate for evidence-claim errors if the participant does not assess whether the evidence supports the claim. The outcome will be expressed as the proportion of citation-claim decisions meeting the prespecified high-quality verification criterion.
Measured: 12 weeks after the intervention
Unnecessary Alteration or Rejection of Valid Citations
Participants will encounter valid citation-claim relationships within the transfer task. The number of valid citations that are unnecessarily altered, rejected, replaced, or deleted will be recorded. The score ranges from 0 to 2, with lower values indicating more appropriate retention of valid evidence.
Measured: 12 weeks after the intervention
Confidence Calibration in Citation-Verification Judgments
Participants will rate confidence in each final citation-verification judgment from 0% to 100%. Calibration error will be calculated by comparing confidence with correctness of the corresponding judgment. Lower absolute calibration error indicates closer correspondence between confidence and actual performance.
Measured: 12 weeks after the intervention
Retention of Structured Citation-Verification Performance
Retention will be assessed using the 24-item structured verification score at 12 weeks and compared with baseline and immediate post-intervention performance. Scores range from 0 to 24, with higher scores indicating better retained structured citation-verification performance.
Measured: Baseline, immediately after the intervention, and 12 weeks after the intervention
Timeline
From the start of the study to its completion; the mark is today. Estimated dates are the sponsor’s plan.
- First posted28 Sep 2026
- Study start26 Sep 2026 (estimated)
- Primary completion16 Jan 2027 (estimated)
- Study completion1 Feb 2027 (estimated)
Resources
- Official record on ClinicalTrials.govThe registry entry this page is built from, with its full change history.
- Other trials for Medical EducationStudies of the same condition, open and under way.
- What the trial phases meanHow a treatment moves from first tests in people to approval.
More details
For doctors and researchers: how the study is designed, who runs it, and its registry IDs.
Study design
- Study type
- Interventional
- Phase
- Phase N/A
- Primary purpose
- Other
- Allocation
- Randomised
- Intervention model
- Parallel groups
- Model description
- Participants are individually randomized 1:1 to either the SARA citation-verification intervention or an attention-matched responsible GenAI education control group. Both groups are assessed at baseline, immediately after the intervention, and at 12 weeks.
- Masking
- Single blind
- Who is masked
- Outcomes assessor
- Enrolment
- 320 participants (estimated)
Sponsor and investigators
- Lead sponsor
- Quaid-e-Azam Medical College
- Sponsor type
- University, hospital or other
- Responsible party
- Principal investigator, Sara Reza, Quaid-e-Azam Medical College
IDs, dates and publications
- ClinicalTrials.gov ID
- NCT07844746
- Sponsor's protocol ID
- QAMC-SARA-2026-01
- Acronym
- SARA
- Status last verified
- Sep 2026
- First submitted
- 21 Sep 2026
- Last updated
- 28 Sep 2026
- Results posted
- No
- Data monitoring committee
- No
- FDA-regulated drug
- No
- FDA-regulated device
- No
- Individual data sharing
- Yes: De-identified individual participant quantitative data underlying the results reported in the primary publication will be made available. A data dictionary and relevant supporting documentation will also be provided. Qualitative interview transcripts will not be shared publicly because of the risk of participant re-identification.
Study at a glance
From the registry record, last updated by the sponsor on 28 Sep 2026. Recruitment status is as the sponsor reported it: confirm with the study team.
- Official title
- Effect of the SARA Citation-Verification Framework on Medical Students' Verification of AI-Generated Citation Hallucinations: A Randomized Controlled Trial With an Explanatory Sequential Mixed-Methods Component
- ClinicalTrials.gov ID
- NCT07844746
- Acronym / sponsor ID
- SARA / QAMC-SARA-2026-01
- Keywords
- Generative Artificial Intelligence, Citation Hallucination, Citation Verification, Medical Students, Research Integrity, Evidence Appraisal
- Study type
- Interventional
- Recruitment status
- Not yet recruiting
- Enrolment
- 320 participants (estimated)
- Lead sponsor
- Quaid-e-Azam Medical CollegeUniversity, hospital or other
- Study start
- 26 Sep 2026 (estimated)
- Primary completion
- 16 Jan 2027 (estimated)
- Study completion
- 1 Feb 2027 (estimated)
- First posted
- 28 Sep 2026 (actual)
- Results
- Not posted in the source record
Clinical summary
The research question as the sponsor registered it. This describes what the study plans to test, not what it found.
Generative artificial intelligence (GenAI) tools are increasingly used by medical students for literature searching, scientific writing, and research. However, AI-generated citations may contain errors. Some references may not exist or may contain incorrect bibliographic details, while other references may be genuine but may not actually support the claim for which they are cited.
This randomized educational study will evaluate whether a structured citation-verification framework called SARA can improve medical students' ability to identify and correct these problems. SARA teaches students to assess four elements of a citation: Source authenticity, Accuracy, Relevance, and Alignment between the cited evidence and the claim being made.
Year 3-5 medical students will be randomly assigned to either a 75-minute SARA training session or an attention-matched session on responsible use of GenAI in academic work. Participants will complete citation-verification assessments before training, immediately after training, and 12 weeks later. The main outcome will assess whether students can independently apply the verification approach to an unstructured literature-review task 12 weeks after training, without being prompted to use SARA.
The study will also examine which types of citation errors remain difficult, whether students initiate appropriate verification, the quality of their verification, handling of valid citations, and confidence in their judgments. A subgroup of participants will take part in think-aloud and follow-up interviews to explore how they approach citation verification and transfer the strategy to new tasks.
Detailed description
AI-generated citation errors can occur at different levels. Some are source-level errors, such as fabricated references or incorrect bibliographic details. Others are more difficult to detect because the cited publication is genuine, but the evidence may be irrelevant, may not support the direction, magnitude, population, or conclusion of the claim, or may be presented without important qualifiers or limitations. In this study, these problems are examined across a five-level citation-error taxonomy ranging from non-existent sources to contextual omission or overgeneralization.
The SARA framework was developed as an educational scaffold to make citation-verification reasoning explicit. It prompts learners to ask whether the source exists, whether the citation details are accurate, whether the source is relevant to the specific claim, and whether the evidence is aligned with the claim as written. The intervention uses modelling, coached practice, feedback, articulation, reflection, and progressive removal of support so that learners move from guided verification toward independent judgment.
A key feature of the study is assessment of delayed transfer rather than immediate post-training performance alone. At 12 weeks, participants will review a realistic literature-review passage containing both valid and problematic citation-claim relationships without being told how many errors are present or being prompted to use SARA. This is intended to evaluate whether the verification strategy has become sufficiently internalized to be applied independently in a new scholarly context.
The explanatory qualitative component will explore why some learners successfully transfer verification skills while others continue to miss deeper evidence-claim problems. Think-aloud and stimulated-recall interviews will examine what triggers verification, how learners decide that evidence is sufficient, when they stop checking, and how they distinguish source existence from evidentiary support.
Study design
- Primary purpose
- Other
- Allocation
- Randomised
- Intervention model
- Parallel groups
- Masking
- Single blind (outcomes assessor)
- Number of arms
- 2
- Model description
- Participants are individually randomized 1:1 to either the SARA citation-verification intervention or an attention-matched responsible GenAI education control group. Both groups are assessed at baseline, immediately after the intervention, and at 12 weeks.
| Arm | Type | Receives |
|---|---|---|
| SARA Citation-Verification Training | Experimental | Behavioral: SARA Citation-Verification TrainingParticipants assigned to this arm will receive a standardized 75-minute educational intervention using the SARA citation-verification framework: Source authenticity, Accuracy, Relevance, and Alignment. Training includes facilitator modelling and think-aloud demonstration, coached practice with feedback, use of a one-page verification scaffold, articulation and reflection, and progressive fading of prompts before independent practice. The intervention is designed to teach students to distinguish source-level citation errors from deeper evidence-claim errors and to apply structured verification independently. |
| Responsible GenAI Education Control | Active comparator | Behavioral: Responsible GenAI Education Intervention DescriptionParticipants assigned to this arm will receive an attention-matched 75-minute educational session on responsible use of generative artificial intelligence in academic work. The session covers privacy, disclosure, authorship, academic integrity, and general awareness that GenAI may produce errors. It does not teach the SARA framework or another structured citation-verification process. The session is matched to the intervention for duration, group size, facilitator interaction, and delivery format. |
Who may qualify
- Sex
- All
- Healthy volunteers
- Accepted
- Age groups
- Child, Adult, Older adult
Key inclusion criteria (5 listed)
- Enrolled as an MBBS student in Year 3, Year 4, or Year 5 at Quaid-e-Azam Medical College, Bahawalpur.
- Has received basic curricular exposure to research methodology or scientific literature searching.
- Has not previously received formal structured training specifically in verification of AI-generated citations.
- Willing and able to provide written informed consent.
- Meets the age eligibility specified in the approved study protocol.
Key exclusion criteria (3 listed)
- Previous participation in development, cognitive interviewing, piloting, or validation testing of the SARA assessment materials.
- Previous formal training in the SARA citation-verification framework before enrolment.
- Unwilling or unable to provide informed consent.
Eligibility is decided by the study team after screening; this overview does not establish that a patient qualifies. Full inclusion and exclusion criteria.
Interventions
| Intervention | Type | Arms |
|---|---|---|
| SARA Citation-Verification TrainingA standardized 75-minute educational intervention using the SARA framework: Source authenticity, Accuracy, Relevance, and Alignment. The session includes facilitator modelling and think-aloud demonstration, coached practice with feedback, a one-page verification scaffold, learner articulation and reflection, and progressive fading of prompts before independent practice. The intervention is designed to teach medical students to identify both source-level citation errors and deeper evidence-claim errors and to apply verification independently. | Behavioural | SARA Citation-Verification Training |
| Responsible GenAI Education Intervention DescriptionA standardized 75-minute attention-matched educational session on responsible use of generative artificial intelligence in academic work. The session covers privacy, disclosure, authorship, academic integrity, and general awareness that GenAI may produce errors. It does not teach the SARA framework or another structured citation-verification process. The session is matched to the SARA intervention for duration, group size, facilitator interaction, and delivery format. | Behavioural | Responsible GenAI Education Control |
Primary outcome
Independent Citation-Verification Performance on an Unstructured Scholarly Transfer Task
Time frame: 12 weeks after the intervention
Participants will review a novel approximately 650-word AI-generated literature-review passage containing eight embedded citation-claim relationships. The passage includes both valid and erroneous citations spanning the study's five-level citation-error taxonomy. Participants will not be prompted to use the SARA framework and will not be told how many or what types of errors are present. Each citation-claim relationship will be scored against a prespecified gold standard for the appropriateness of the participant's final disposition. One point will be awarded for each appropriate disposition, producing a total score from 0 to 8, with higher scores indicating better independent citation-verification performance. Responses will be scored by outcome assessors blinded to intervention allocation.
7 secondary outcomes
Structured Citation-Verification Performance
Time frame: Immediately after the intervention and 12 weeks after the intervention
Citation-Verification Accuracy by Citation-Error Condition
Time frame: Immediately after the intervention and 12 weeks after the intervention
Verification Initiation
Time frame: Immediately after the intervention and 12 weeks after the intervention
High-Quality Citation Verification
Time frame: 12 weeks after the intervention
Unnecessary Alteration or Rejection of Valid Citations
Time frame: 12 weeks after the intervention
Confidence Calibration in Citation-Verification Judgments
Time frame: 12 weeks after the intervention
Retention of Structured Citation-Verification Performance
Time frame: Baseline, immediately after the intervention, and 12 weeks after the intervention
Results and safety
This study is still under way. Results are not available in the source record yet.
Research links
- Official record on ClinicalTrials.govLast updated by the sponsor 28 Sep 2026.
- Individual participant dataSharing planned: Yes. De-identified individual participant quantitative data underlying the results reported in the primary publication will be made available. A data dictionary and relevant supporting documentation will also be provided. Qualitative interview transcripts will not be shared publicly because of the risk of participant re-identification.
Related trials for Medical Education
Not yet recruiting
AI Smart Glasses Feedback for Intubation Training in Medical Students
Intubation, Intratracheal, Medical Education
Recruiting
Linking Education, Produce Provision, and Community Referrals to Improve Diabetes Care (LINK)
Diabetes Mellitus, Type 2, Nutrition Poor
Not yet recruiting
Title of Study: Anesthesiologists' Learning Curves for Ultrasound Assessment of Difficult Airway
Airway Ultrasound, Difficult Airway Management
Not yet recruiting
Comparing the Efficacy of a Conversational Agent (QuitBot) Versus Standard Health Education on Smoking Cessation Among Medical Students in Cairo University
Conversational Agent, Smoking Cessation
Not yet recruiting
Generative AI-Assisted Personalized Feedback for Laparoscopic Skill Learning in Medical Undergraduates
Medical Education
Referral and contact
AzTrials does not enrol participants or take referrals. Contact the study team or a site listed below; they decide who can take part.
Questions about this trial
What is this clinical trial studying?
A study testing SARA Citation-Verification Training and Responsible GenAI Education Intervention Description in people with Medical Education and Generative Artificial Intelligence, run by Quaid-e-Azam Medical College.
The full registered description is in the About this study section and the official record.
Who may be eligible to take part?
The sponsor lists 5 inclusion and 3 exclusion criteria. Eligibility is decided by the study team after screening, so read the full criteria and contact a listed site.
Is this clinical trial currently recruiting?
The record says: opening soon, as of the sponsor's last update on 28 Sep 2026. Status can change between updates, so confirm with the study contact before making plans.
What are the possible benefits and risks?
They depend on the study and the treatment, and no study can promise a benefit. The study team explains the known risks during informed consent, before anyone joins; the record itself does not list them.
Are results available for this study?
This study is still under way. Results are not available in the source record yet.
How can a doctor refer a patient or contact the research team?
Contact Sara Reza (+92-333-4603388, sara.qmc@qamc.edu.pk), or a site near the patient. AzTrials does not enrol participants or take referrals.
What does the primary outcome measure?
Independent Citation-Verification Performance on an Unstructured Scholarly Transfer Task (time frame: 12 weeks after the intervention). It is what the study was designed to measure, not a finding.
Where can I read the original study information?
Does a listing mean the treatment is approved or proven to work?
No. A listing means the study is registered. It does not show that the treatment is safe, works, or is approved by a regulator.
How often is this information updated?
AzTrials copies changed records from ClinicalTrials.gov every night and reloads the whole registry every week. The sponsor last updated this record on 28 Sep 2026.