CataractNCT06229379
The Effects of a Large Language Model on Clinical Questioning Skills
A study testing "Digital twin patient" and Interaction with real patients in people with Cataract and Glaucoma, run by Sun Yat-sen University at 1 site in China.
Trial summary
The researchers have used the ophthalmology textbook, clinical guideline consensus, the Internet conversation data and knowledge base of Zhongshan Ophthalmology Center in the early stage, combined with artificial feedback reinforcement learning and other techniques to fine-tune and train the LLM, and developed… Read more about this study
- Who is it for?
- Ages 18 to 25 yearsHealthy volunteers can take part
- Conditions the study is for
- Cataract, Glaucoma, Diabetic Retinopathy
- What is being tested?
- "Digital twin patient" and Interaction with real patients
- Type of treatment
- Medical device, Lifestyle or therapy programme
- Could I receive a placebo?
- No, everyone receives an active treatment
- How many people
- 84 participants
- Where
- 1 siteChina
- Study dates
- 13 Nov 2023to 7 Aug 2024
- Phase
- Interventional
- Are there results?
- None posted in the registry
Key requirements
Who can take part, as listed by the sponsor. Only the study team can confirm whether you are eligible, so take this list to your doctor.
- Ages 18 to 25 years
- All sexes
- Healthy volunteers accepted
- Adult
Who may be able to join
- All undergraduate students from Sun Yat-sen University who participate in the ophthalmological internship.
Who cannot join
- Students who refuse to sign informed consent.
Trial locations and contacts
1 site in 1 country.
China1 site
Zhongshan Ophthalmic Center, Sun Yat-sen Univerisity
Guangzhou, Guangdong, 510060
About this study
Official title: A Randomized Controlled Trial of the Effects of a Large Language Model on Medical Students' Clinical Questioning Skills
The researchers have used the ophthalmology textbook, clinical guideline consensus, the Internet conversation data and knowledge base of Zhongshan Ophthalmology Center in the early stage, combined with artificial feedback reinforcement learning and other techniques to fine-tune and train the LLM, and developed "Digital Twin Patient", a localized large language model that has the ability to answer ophthalmology-related medical questions, and also constructed a combination of automated model evaluation and manual evaluation by medical experts. The evaluation system combining automated model evaluation and manual evaluation by medical experts was constructed at the same time.
This project intends to integrate "Digital Twin Patient" into undergraduate ophthalmology apprenticeship, simulate the consultation process of real patients through the online interaction between students and "Digital Twin Patient", explore the effect of "Digital Twin Patient" consultation teaching, provide emerging technology tools for guiding medical students to actively learn a variety of ophthalmology cases, cultivate clinical thinking, and provide the possibility of creating a new mode of intelligent teaching.
Read the detailed description
At present, the main form of clinical questioning skills teaching is to let undergraduates who participate in the apprenticeship first learn the characteristics and diagnostic points of cases, and then practice questioning on real patients in the wards. However, due to the large number of trainee students, it is difficult to meet the teaching demand in terms of the number of cases available for questioning and the richness of disease types under the current teaching mode. Therefore, it is necessary to utilize new intelligent technologies and create a new model of questioning skills teaching to improve teaching efficiency and enhance students' clinical thinking.
Large-scale language modeling (LLM) is a deep learning technology that can learn knowledge from a large amount of text, and AI chatbots such as ChatGPT are a typical example of its application. AI chatbots are characterized by anthropomorphic comprehension and diversified natural language generation abilities in different contexts, and have been initially applied in the medical field, such as passing the U.S. Medical Licensing Examination, assisting in ophthalmic history documentation and answering ophthalmic questions. However, it has been found that although LLM has fair modeling performance in general medical knowledge, it still needs to be improved in the area of specialty diseases. Based on this, the researcher's team has used the ophthalmology textbook, clinical guideline consensus, the Internet conversation data and knowledge base of Zhongshan Ophthalmology Center in the early stage, combined with artificial feedback reinforcement learning and other techniques to fine-tune and train the LLM, and developed "Digital Twin Patient", a localized large language model that has the ability to answer ophthalmology-related medical questions, and also constructed a combination of automated model evaluation and manual evaluation by medical experts. The evaluation system combining automated model evaluation and manual evaluation by medical experts was constructed at the same time.
This project intends to integrate "Digital Twin Patient" into undergraduate ophthalmology apprenticeship, simulate the consultation process of real patients through the online interaction between students and "Digital Twin Patient", explore the effect of "Digital Twin Patient" consultation teaching, provide emerging technology tools for guiding medical students to actively learn a variety of ophthalmology cases, cultivate clinical thinking, and provide the possibility of creating a new mode of intelligent teaching.
- Conditions
- Cataract, Glaucoma, Diabetic Retinopathy, Keratitis, Conjunctivitis
- Being tested
- "Digital twin patient", Interaction with real patients
- Keywords
- Large language model, Clinical education, Ophthalmology
- Medical subject headings
- Cataract, Glaucoma, Diabetic Retinopathy, Keratitis, Conjunctivitis
What participants receive
What participants receive. In a randomised study each person is assigned to one group by chance.
"Digital twin patient"Device
"Digital twin patient" can serve as patients with specific diseases for medical students to acquire disease history and thus practice clinical questioning skills.
Interaction with real patientsBehavioural
As in traditional medical education, medical students need to interact with real patients to practice history collection skills.
| Study group | Role | What they receive |
|---|---|---|
| "Digital twin patient" | Experimental | The students in the "Digital twin patient" group were trained in history taking using a "digital twin patient" on the first day of a training program, and then took a 15-minute clinical questioning exam using the "digital twin patient" on the second day. Receives: Device: "Digital twin patient" |
| Real patient | Other | The students in the Real patient group were trained in history taking using a real patient on the first day of a training program, and then took a 15-minute clinical questioning exam using the "digital twin patient" on the second day. Receives: Device: "Digital twin patient"; Behavioral: Interaction with real patients |
What is being measured
The primary outcome is the main question the study is designed to answer.
Students' scores in the medical history acquisition exam
Measured: Weekly during this study (up to 10 months)
Timeline
From the start of the study to its completion; the mark is today. Estimated dates are the sponsor’s plan.
- First posted29 Jan 2024
- Study start13 Nov 2023
- Primary completion10 May 2024
- Study completion7 Aug 2024
Resources
- Official record on ClinicalTrials.govThe registry entry this page is built from, with its full change history.
- Publications (1)Papers the sponsor has linked to this study.
- Other trials for CataractStudies of the same condition, open and under way.
- More trials run by Sun Yat-sen UniversityThe sponsor's other registered studies.
- What the trial phases meanHow a treatment moves from first tests in people to approval.
More details
For doctors and researchers: how the study is designed, who runs it, and its registry IDs.
Study design
- Study type
- Interventional
- Phase
- Phase N/A
- Primary purpose
- Other
- Allocation
- Randomised
- Intervention model
- Parallel groups
- Masking
- None (open label)
- Enrolment
- 84 participants (actual)
Sponsor and investigators
- Lead sponsor
- Sun Yat-sen University
- Sponsor type
- University, hospital or other
- Responsible party
- Principal investigator, Haotian Lin, Sun Yat-sen University
IDs, dates and publications
- ClinicalTrials.gov ID
- NCT06229379
- Sponsor's protocol ID
- 2023KYPJ283
- Status last verified
- Feb 2024
- First submitted
- 3 Jan 2024
- Last updated
- 22 Nov 2024
- Results posted
- No
- Data monitoring committee
- Yes
- FDA-regulated drug
- No
- FDA-regulated device
- No
- Individual data sharing
- No
Publications
Luo MJ, Bi S, Pang J, Liu L, Tsui CK, Lai Y, Chen W, Yang Y, Xu K, Zhao L, Jin L, Lin D, Wu X, Chen J, Chen R, Liu Z, Zou Y, Yang Y, Li Y, Lin H. A large language model digital patient system enhances ophthalmology history taking skills. NPJ Digit Med. 2025 Aug 4;8(1):502. doi: 10.1038/s41746-025-01841-6.
PubMed 40760042
Study at a glance
From the registry record, last updated by the sponsor on 22 Nov 2024. Recruitment status is as the sponsor reported it: confirm with the study team.
- Official title
- A Randomized Controlled Trial of the Effects of a Large Language Model on Medical Students' Clinical Questioning Skills
- ClinicalTrials.gov ID
- NCT06229379
- Acronym / sponsor ID
- 2023KYPJ283
- Conditions
- Cataract, Glaucoma, Diabetic Retinopathy, Keratitis, Conjunctivitis
- Keywords
- Large language model, Clinical education, Ophthalmology
- Study type
- Interventional
- Recruitment status
- Completed
- Enrolment
- 84 participants (actual)
- Lead sponsor
- Sun Yat-sen UniversityUniversity, hospital or other
- Study start
- 13 Nov 2023 (actual)
- Primary completion
- 10 May 2024 (actual)
- Study completion
- 7 Aug 2024 (actual)
- First posted
- 29 Jan 2024 (actual)
- Results
- Not posted in the source record
Clinical summary
The research question as the sponsor registered it. This describes what the study plans to test, not what it found.
The researchers have used the ophthalmology textbook, clinical guideline consensus, the Internet conversation data and knowledge base of Zhongshan Ophthalmology Center in the early stage, combined with artificial feedback reinforcement learning and other techniques to fine-tune and train the LLM, and developed "Digital Twin Patient", a localized large language model that has the ability to answer ophthalmology-related medical questions, and also constructed a combination of automated model evaluation and manual evaluation by medical experts. The evaluation system combining automated model evaluation and manual evaluation by medical experts was constructed at the same time.
This project intends to integrate "Digital Twin Patient" into undergraduate ophthalmology apprenticeship, simulate the consultation process of real patients through the online interaction between students and "Digital Twin Patient", explore the effect of "Digital Twin Patient" consultation teaching, provide emerging technology tools for guiding medical students to actively learn a variety of ophthalmology cases, cultivate clinical thinking, and provide the possibility of creating a new mode of intelligent teaching.
Detailed description
At present, the main form of clinical questioning skills teaching is to let undergraduates who participate in the apprenticeship first learn the characteristics and diagnostic points of cases, and then practice questioning on real patients in the wards. However, due to the large number of trainee students, it is difficult to meet the teaching demand in terms of the number of cases available for questioning and the richness of disease types under the current teaching mode. Therefore, it is necessary to utilize new intelligent technologies and create a new model of questioning skills teaching to improve teaching efficiency and enhance students' clinical thinking.
Large-scale language modeling (LLM) is a deep learning technology that can learn knowledge from a large amount of text, and AI chatbots such as ChatGPT are a typical example of its application. AI chatbots are characterized by anthropomorphic comprehension and diversified natural language generation abilities in different contexts, and have been initially applied in the medical field, such as passing the U.S. Medical Licensing Examination, assisting in ophthalmic history documentation and answering ophthalmic questions. However, it has been found that although LLM has fair modeling performance in general medical knowledge, it still needs to be improved in the area of specialty diseases. Based on this, the researcher's team has used the ophthalmology textbook, clinical guideline consensus, the Internet conversation data and knowledge base of Zhongshan Ophthalmology Center in the early stage, combined with artificial feedback reinforcement learning and other techniques to fine-tune and train the LLM, and developed "Digital Twin Patient", a localized large language model that has the ability to answer ophthalmology-related medical questions, and also constructed a combination of automated model evaluation and manual evaluation by medical experts. The evaluation system combining automated model evaluation and manual evaluation by medical experts was constructed at the same time.
This project intends to integrate "Digital Twin Patient" into undergraduate ophthalmology apprenticeship, simulate the consultation process of real patients through the online interaction between students and "Digital Twin Patient", explore the effect of "Digital Twin Patient" consultation teaching, provide emerging technology tools for guiding medical students to actively learn a variety of ophthalmology cases, cultivate clinical thinking, and provide the possibility of creating a new mode of intelligent teaching.
Study design
- Primary purpose
- Other
- Allocation
- Randomised
- Intervention model
- Parallel groups
- Masking
- None (open label)
- Number of arms
- 2
| Arm | Type | Receives |
|---|---|---|
| "Digital twin patient" | Experimental | Device: "Digital twin patient"The students in the "Digital twin patient" group were trained in history taking using a "digital twin patient" on the first day of a training program, and then took a 15-minute clinical questioning exam using the "digital twin patient" on the second day. |
| Real patient | Other | Device: "Digital twin patient", Behavioral: Interaction with real patientsThe students in the Real patient group were trained in history taking using a real patient on the first day of a training program, and then took a 15-minute clinical questioning exam using the "digital twin patient" on the second day. |
Who may qualify
- Ages
- 18 to 25 years
- Sex
- All
- Healthy volunteers
- Accepted
- Age groups
- Adult
Key inclusion criteria (1 listed)
- All undergraduate students from Sun Yat-sen University who participate in the ophthalmological internship.
Key exclusion criteria (1 listed)
- Students who refuse to sign informed consent.
Eligibility is decided by the study team after screening; this overview does not establish that a patient qualifies. Full inclusion and exclusion criteria.
Interventions
| Intervention | Type | Arms |
|---|---|---|
| "Digital twin patient""Digital twin patient" can serve as patients with specific diseases for medical students to acquire disease history and thus practice clinical questioning skills. | Device | "Digital twin patient", Real patient |
| Interaction with real patientsAs in traditional medical education, medical students need to interact with real patients to practice history collection skills. | Behavioural | Real patient |
Primary outcome
Students' scores in the medical history acquisition exam
Time frame: Weekly during this study (up to 10 months)
Results and safety
The study is completed, but results are not available in the source record. That does not mean the treatment did or did not work.
Research links
- Official record on ClinicalTrials.govLast updated by the sponsor 22 Nov 2024.
- Derived from this study: PubMed 40760042Luo MJ, Bi S, Pang J, Liu L, Tsui CK, Lai Y, Chen W, Yang Y, Xu K, Zhao L, Jin L, Lin D, Wu X, Chen J, Chen R, Liu Z, Zou Y, Yang Y, Li Y, Lin H. A large language model digital patient system enhances ophthalmology history taking skills. NPJ Digit Med. 2025 Aug 4;8(1):502. doi: 10.1038/s41746-025-01841-6.
- Individual participant dataSharing planned: No
Related trials for Cataract
Recruiting
Implementing AI-based Glaucoma Screening Within Federally Qualified Health Centers
Glaucoma, Ocular Hypertension
Not yet recruiting
Trial to Investigate the Safety and Effectiveness of the KI-362-US1 (YP2.2VM) Intraocular Lens (IOL) in Adult Participants for Correction of Aphakia Following Cataract Extraction
Aphakia, Postcataract
Not yet recruiting
Effects of Oxygen Cage Support During Cataract Surgery
Cataract
Active, not recruitingObservational study
Age-Related Eye Disease Study (AREDS) Follow-Up
Age-Related Macular Degeneration, Cataract
RecruitingObservational study
Objective Depth-of-Focus Changes During Light Adjustable Lens UV Adjustments
Light Adjustable Lens, Objective Depth of Focus
Referral and contact
AzTrials does not enrol participants or take referrals. Contact the study team or a site listed below; they decide who can take part. This study is not recruiting at the moment.
The record lists no contact. Use the official record to reach the sponsor.
1 site in 1 country: see every site and its status.
Questions about this trial
What is this clinical trial studying?
A study testing "Digital twin patient" and Interaction with real patients in people with Cataract and Glaucoma, run by Sun Yat-sen University at 1 site in China.
The full registered description is in the About this study section and the official record.
Who may be eligible to take part?
The sponsor lists 1 inclusion and 1 exclusion criteria. Eligibility is decided by the study team after screening, so read the full criteria and contact a listed site.
Is this clinical trial currently recruiting?
The record says: finished, as of the sponsor's last update on 22 Nov 2024. Status can change between updates, so confirm with the study contact before making plans.
Where is the study taking place?
The record lists 1 site in China. Whether a site is taking part right now should be confirmed with that site. See every site.
What are the possible benefits and risks?
They depend on the study and the treatment, and no study can promise a benefit. The study team explains the known risks during informed consent, before anyone joins; the record itself does not list them.
Are results available for this study?
The study is completed, but results are not available in the source record. That does not mean the treatment did or did not work.
How can a doctor refer a patient or contact the research team?
The record lists no contact. The official record names the sponsor, who can be asked directly. AzTrials does not enrol participants or take referrals.
What does the primary outcome measure?
Students' scores in the medical history acquisition exam (time frame: Weekly during this study (up to 10 months)). It is what the study was designed to measure, not a finding.
Where can I read the original study information?
On the ClinicalTrials.gov record for NCT06229379, which also lists 1 linked publication.
Does a listing mean the treatment is approved or proven to work?
No. A listing means the study is registered. It does not show that the treatment is safe, works, or is approved by a regulator.
How often is this information updated?
AzTrials copies changed records from ClinicalTrials.gov every night and reloads the whole registry every week. The sponsor last updated this record on 22 Nov 2024.