Medical EducationNCT07522658
Artificial Intelligence-Generated vs Academician-Developed Multiple True/False Questions in Anesthesiology Education
An observational study of Medical Education and Artificial Intelligence, run by Kutahya Health Sciences University at 1 site in Turkey (Türkiye).
Trial summary
This prospective observational study aims to evaluate the effectiveness and educational value of artificial intelligence (AI)-generated multiple true/false questions compared to those developed by experienced academicians in anesthesiology training. A total of 27 anesthesiology residents will be included in the study. Read more about this study
- Who is it for?
- Ages 18 years and olderHealthy volunteers can take part
- Conditions the study is for
- Medical Education, Artificial Intelligence, Assessment Methods in Medical Training
- How many people
- 26 participants
- Where
- 1 siteTurkey (Türkiye)
- Study dates
- 14 Apr 2026to 30 Apr 2026
- Phase
- Observational
- Are there results?
- None posted in the registry
Key requirements
Who can take part, as listed by the sponsor. Only the study team can confirm whether you are eligible, so take this list to your doctor.
- Ages 18 years and older
- All sexes
- Healthy volunteers accepted
- Adult, Older adult
Study population: Anesthesiology residents undergoing training at Kütahya Health Sciences University.
Who may be able to join
- Being an anesthesiology resident
- Voluntary participation in the study
- Providing informed consent
Who cannot join
- Refusal to participate
- Incomplete test responses
Trial locations and contacts
1 site in 1 country.
About this study
Official title: Comparison of Artificial Intelligence-Generated and Academician-Developed Multiple True/False Questions in Anesthesiology Education: A Prospective Cohort Study
This prospective observational study aims to evaluate the effectiveness and educational value of artificial intelligence (AI)-generated multiple true/false questions compared to those developed by experienced academicians in anesthesiology training.
A total of 27 anesthesiology residents will be included in the study. Question sets consisting of 200 multiple true/false items will be created, with half generated by academicians and the other half generated using an artificial intelligence model (ChatGPT-based system). The questions will be based on standardized educational materials from the anesthesiology training curriculum.
Read more
Participants will complete the test in a single session. Each correct answer will be scored as one point, and total scores will be calculated. In addition to test performance, item difficulty, discrimination indices, and test reliability will be analyzed. Furthermore, participants' perceptions regarding question quality will be evaluated.
The study aims to determine whether AI-generated questions can provide a reliable and effective alternative to traditional question development methods in medical education and contribute to more objective and standardized assessment processes.
Read the detailed description
This single-center, prospective observational cohort study is designed to evaluate the effectiveness, reliability, and educational value of artificial intelligence (AI)-generated multiple true/false (MTF) questions compared to those developed by experienced academicians in anesthesiology training.
The study will be conducted at the Department of Anesthesiology and Reanimation, Kütahya Health Sciences University. A total of 27 anesthesiology residents will be included.
A total of 200 MTF questions will be developed based on standardized anesthesiology educational materials. Half of the questions (n=100) will be prepared by experienced academicians, while the remaining half (n=100) will be generated using an artificial intelligence model (ChatGPT-based system). All questions will be structured according to predefined criteria, including difficulty level (easy, moderate, difficult), clinical relevance, and educational appropriateness.
Participants will complete the question sets in a single session under standardized conditions. Each correct answer will be scored as 1 point, and incorrect answers will be scored as 0. Total test scores will be calculated for each participant.
Item analysis will be performed to evaluate the psychometric properties of the questions. Item difficulty index, item discrimination index, and overall test reliability will be calculated. Additionally, perceived question quality will be assessed using participant feedback.
Statistical analysis will be conducted using SPSS software. The distribution of variables will be assessed, and appropriate parametric or non-parametric tests will be used accordingly. Comparisons between groups (junior vs senior residents) and between question sources (AI-generated vs academician-developed) will be performed. A p-value of <0.05 will be considered statistically significant.
The study does not involve any clinical intervention, drug administration, or invasive procedure. Participation is voluntary, and written informed consent will be obtained from all participants. All data will be collected anonymously and used solely for research purposes.
The results of this study are expected to provide insight into the potential role of artificial intelligence in medical education, particularly in the development of assessment tools, and may contribute to more objective, standardized, and efficient evaluation methods in anesthesiology training.
- Keywords
- Artificial Intelligence, Anesthesiology Education, Multiple True False Questions, Medical Education Assessment, Item Analysis, Question Quality
What participants receive
What participants receive. In a randomised study each person is assigned to one group by chance.
| Study group | Role | What they receive |
|---|---|---|
| AI-Generated Questions | Multiple true/false questions generated using an artificial intelligence model | |
| Academician-Developed Question | Multiple true/false questions prepared by experienced academicians in anesthesiology. |
What is being measured
The primary outcome is the main question the study is designed to answer.
Item Difficulty Index of AI-generated and expert-authored questions
For each question, the item difficulty index will be calculated as the proportion of participants who answer the item correctly. Item difficulty indices will be compared between AI-generated and expert-authored questions.
Measured: Assessed once after completion of each participant's single 60-minute examination session; final item analysis performed after all participants complete the examination, up to 1 month.
Timeline
From the start of the study to its completion; the mark is today. Estimated dates are the sponsor’s plan.
- First posted13 Apr 2026
- Study start14 Apr 2026
- Primary completion20 Apr 2026
- Study completion30 Apr 2026
Resources
- Official record on ClinicalTrials.govThe registry entry this page is built from, with its full change history.
- Publications (3)Papers the sponsor has linked to this study.
- Other trials for Medical EducationStudies of the same condition, open and under way.
- What the trial phases meanHow a treatment moves from first tests in people to approval.
More details
For doctors and researchers: how the study is designed, who runs it, and its registry IDs.
Study design
- Study type
- Observational
- Observational model
- Cohort
- Time perspective
- Prospective
- Enrolment
- 26 participants (actual)
Sponsor and investigators
- Lead sponsor
- Kutahya Health Sciences University
- Sponsor type
- University, hospital or other
- Responsible party
- Principal investigator, Serkan TELLİ, Kutahya Health Sciences University
IDs, dates and publications
- ClinicalTrials.gov ID
- NCT07522658
- Sponsor's protocol ID
- E-41997688-050.99-212840
- Status last verified
- Apr 2026
- First submitted
- 28 Mar 2026
- Last updated
- 6 May 2026
- Results posted
- No
- Data monitoring committee
- No
- FDA-regulated drug
- No
- FDA-regulated device
- No
- Individual data sharing
- No
Publications
Kaya M, Sonmez E, Halici A, Yildirim H, Coskun A. Comparison of AI-generated and clinician-designed multiple-choice questions in emergency medicine exam: a psychometric analysis. BMC Med Educ. 2025 Jul 1;25(1):949. doi: 10.1186/s12909-025-07528-6.
PubMed 40597998Reid M, French M, Andreopoulos S, Wong C, Kee N. AI-generated multiple-choice questions in health science education: Stakeholder perspectives and implementation considerations. Curr Res Physiol. 2025 Aug 1;8:100160. doi: 10.1016/j.crphys.2025.100160. eCollection 2025.
PubMed 40799313Kocer Tulgar Y, Tulgar S, Guven Kose S, Kose HC, Cevik Nasirlier G, Dogan M, Thomas DT. Anesthesiologists' Perspective on the Use of Artificial Intelligence in Ultrasound-Guided Regional Anaesthesia in Terms of Medical Ethics and Medical Education: A Survey Study. Eurasian J Med. 2023 Jun;55(2):146-151. doi: 10.5152/eurasianjmed.2023.22254.
PubMed 37161553
Study at a glance
From the registry record, last updated by the sponsor on 6 May 2026. Recruitment status is as the sponsor reported it: confirm with the study team.
- Official title
- Comparison of Artificial Intelligence-Generated and Academician-Developed Multiple True/False Questions in Anesthesiology Education: A Prospective Cohort Study
- ClinicalTrials.gov ID
- NCT07522658
- Acronym / sponsor ID
- E-41997688-050.99-212840
- Keywords
- Artificial Intelligence, Anesthesiology Education, Multiple True False Questions, Medical Education Assessment, Item Analysis, Question Quality
- Study type
- Observational
- Recruitment status
- Completed
- Enrolment
- 26 participants (actual)
- Lead sponsor
- Kutahya Health Sciences UniversityUniversity, hospital or other
- Study start
- 14 Apr 2026 (actual)
- Primary completion
- 20 Apr 2026 (actual)
- Study completion
- 30 Apr 2026 (actual)
- First posted
- 13 Apr 2026 (actual)
- Results
- Not posted in the source record
Clinical summary
The research question as the sponsor registered it. This describes what the study plans to test, not what it found.
This prospective observational study aims to evaluate the effectiveness and educational value of artificial intelligence (AI)-generated multiple true/false questions compared to those developed by experienced academicians in anesthesiology training.
A total of 27 anesthesiology residents will be included in the study. Question sets consisting of 200 multiple true/false items will be created, with half generated by academicians and the other half generated using an artificial intelligence model (ChatGPT-based system). The questions will be based on standardized educational materials from the anesthesiology training curriculum.
Participants will complete the test in a single session. Each correct answer will be scored as one point, and total scores will be calculated. In addition to test performance, item difficulty, discrimination indices, and test reliability will be analyzed. Furthermore, participants' perceptions regarding question quality will be evaluated.
The study aims to determine whether AI-generated questions can provide a reliable and effective alternative to traditional question development methods in medical education and contribute to more objective and standardized assessment processes.
Detailed description
This single-center, prospective observational cohort study is designed to evaluate the effectiveness, reliability, and educational value of artificial intelligence (AI)-generated multiple true/false (MTF) questions compared to those developed by experienced academicians in anesthesiology training.
The study will be conducted at the Department of Anesthesiology and Reanimation, Kütahya Health Sciences University. A total of 27 anesthesiology residents will be included.
A total of 200 MTF questions will be developed based on standardized anesthesiology educational materials. Half of the questions (n=100) will be prepared by experienced academicians, while the remaining half (n=100) will be generated using an artificial intelligence model (ChatGPT-based system). All questions will be structured according to predefined criteria, including difficulty level (easy, moderate, difficult), clinical relevance, and educational appropriateness.
Participants will complete the question sets in a single session under standardized conditions. Each correct answer will be scored as 1 point, and incorrect answers will be scored as 0. Total test scores will be calculated for each participant.
Item analysis will be performed to evaluate the psychometric properties of the questions. Item difficulty index, item discrimination index, and overall test reliability will be calculated. Additionally, perceived question quality will be assessed using participant feedback.
Statistical analysis will be conducted using SPSS software. The distribution of variables will be assessed, and appropriate parametric or non-parametric tests will be used accordingly. Comparisons between groups (junior vs senior residents) and between question sources (AI-generated vs academician-developed) will be performed. A p-value of <0.05 will be considered statistically significant.
The study does not involve any clinical intervention, drug administration, or invasive procedure. Participation is voluntary, and written informed consent will be obtained from all participants. All data will be collected anonymously and used solely for research purposes.
The results of this study are expected to provide insight into the potential role of artificial intelligence in medical education, particularly in the development of assessment tools, and may contribute to more objective, standardized, and efficient evaluation methods in anesthesiology training.
Study design
- Observational model
- Cohort
- Time perspective
- Prospective
- Number of arms
- 2
| Arm | Type | Receives |
|---|---|---|
| AI-Generated Questions | – | –Multiple true/false questions generated using an artificial intelligence model |
| Academician-Developed Question | – | –Multiple true/false questions prepared by experienced academicians in anesthesiology. |
Who may qualify
- Ages
- 18 years and older
- Sex
- All
- Healthy volunteers
- Accepted
- Age groups
- Adult, Older adult
- Study population
- Anesthesiology residents undergoing training at Kütahya Health Sciences University.
Key inclusion criteria (3 listed)
- Being an anesthesiology resident
- Voluntary participation in the study
- Providing informed consent
Key exclusion criteria (2 listed)
- Refusal to participate
- Incomplete test responses
Eligibility is decided by the study team after screening; this overview does not establish that a patient qualifies. Full inclusion and exclusion criteria.
Primary outcome
Item Difficulty Index of AI-generated and expert-authored questions
Time frame: Assessed once after completion of each participant's single 60-minute examination session; final item analysis performed after all participants complete the examination, up to 1 month.
For each question, the item difficulty index will be calculated as the proportion of participants who answer the item correctly. Item difficulty indices will be compared between AI-generated and expert-authored questions.
Results and safety
The study is completed, but results are not available in the source record. That does not mean the treatment did or did not work.
Research links
- Official record on ClinicalTrials.govLast updated by the sponsor 6 May 2026.
- Background: PubMed 40597998Kaya M, Sonmez E, Halici A, Yildirim H, Coskun A. Comparison of AI-generated and clinician-designed multiple-choice questions in emergency medicine exam: a psychometric analysis. BMC Med Educ. 2025 Jul 1;25(1):949. doi: 10.1186/s12909-025-07528-6.
- Background: PubMed 40799313Reid M, French M, Andreopoulos S, Wong C, Kee N. AI-generated multiple-choice questions in health science education: Stakeholder perspectives and implementation considerations. Curr Res Physiol. 2025 Aug 1;8:100160. doi: 10.1016/j.crphys.2025.100160. eCollection 2025.
- Background: PubMed 37161553Kocer Tulgar Y, Tulgar S, Guven Kose S, Kose HC, Cevik Nasirlier G, Dogan M, Thomas DT. Anesthesiologists' Perspective on the Use of Artificial Intelligence in Ultrasound-Guided Regional Anaesthesia in Terms of Medical Ethics and Medical Education: A Survey Study. Eurasian J Med. 2023 Jun;55(2):146-151. doi: 10.5152/eurasianjmed.2023.22254.
- Individual participant dataSharing planned: No
Related trials for Medical Education
RecruitingObservational study
Artificial Intelligence for Automated Clinical Data Exploration From Electronic Medical Records (CardioMining-AI)
Artificial Intelligence, Machine Learning
Not yet recruiting
AI Smart Glasses Feedback for Intubation Training in Medical Students
Intubation, Intratracheal, Medical Education
Active, not recruiting
Comparing the Effect of Scenario-Based Teaching Method to Conventional Method on Medical Students' Knowledge and Satisfaction in Psychiatry Interns
Knowledge, Satisfaction
Recruiting
Surgical Ergonomics in OBGYN
Current OBGYN Attending, Resident, or Fellow at University Hospitals
Not yet recruiting
AI-Generated and Rule-Based Individualized Feedback for Medical Students' Nephrology Learning
Medical Education
Referral and contact
AzTrials does not enrol participants or take referrals. Contact the study team or a site listed below; they decide who can take part. This study is not recruiting at the moment.
The record lists no contact. Use the official record to reach the sponsor.
1 site in 1 country: see every site and its status.
Questions about this trial
What is this clinical trial studying?
An observational study of Medical Education and Artificial Intelligence, run by Kutahya Health Sciences University at 1 site in Turkey (Türkiye).
The full registered description is in the About this study section and the official record.
Who may be eligible to take part?
The sponsor lists 3 inclusion and 2 exclusion criteria. Eligibility is decided by the study team after screening, so read the full criteria and contact a listed site.
Is this clinical trial currently recruiting?
The record says: finished, as of the sponsor's last update on 6 May 2026. Status can change between updates, so confirm with the study contact before making plans.
Where is the study taking place?
The record lists 1 site in Turkey (Türkiye). Whether a site is taking part right now should be confirmed with that site. See every site.
What are the possible benefits and risks?
They depend on the study and the treatment, and no study can promise a benefit. The study team explains the known risks during informed consent, before anyone joins; the record itself does not list them.
Are results available for this study?
The study is completed, but results are not available in the source record. That does not mean the treatment did or did not work.
How can a doctor refer a patient or contact the research team?
The record lists no contact. The official record names the sponsor, who can be asked directly. AzTrials does not enrol participants or take referrals.
What does the primary outcome measure?
Item Difficulty Index of AI-generated and expert-authored questions (time frame: Assessed once after completion of each participant's single 60-minute examination session; final item analysis performed after all participants complete the examination, up to 1 month.). It is what the study was designed to measure, not a finding.
Where can I read the original study information?
On the ClinicalTrials.gov record for NCT07522658, which also lists 3 linked publications.
Does a listing mean the treatment is approved or proven to work?
No. A listing means the study is registered. It does not show that the treatment is safe, works, or is approved by a regulator.
How often is this information updated?
AzTrials copies changed records from ClinicalTrials.gov every night and reloads the whole registry every week. The sponsor last updated this record on 6 May 2026.