Surgical EducationNCT05168150
Testing the Efficacy of an Artificial Intelligence Real-Time Coaching SystemSystemSimulatioTraining of Medical Students
A study testing Experimental: Experimental Group - Intelligent Continuous Expertise Monitoring System group and Experimental Group In-person expert-mediated instruction group in people with Surgical Education, run by McGill University at 1 site in Canada.
Trial summary
Background: Trainees learn surgical technical skills through apprenticeship model working closely with surgeons and given increased responsibility in patient cases under expert supervision. Read more about this study
- Who is it for?
- Ages 18 years and olderHealthy volunteers can take part
- Conditions the study is for
- Surgical Education
- What is being tested?
- Experimental: Experimental Group - Intelligent Continuous Expertise Monitoring System group and Experimental Group In-person expert-mediated instruction group
- Type of treatment
- Lifestyle or therapy programme
- Could I receive a placebo?
- No, everyone receives an active treatment
- How many people
- 98 participants
- Where
- 1 siteCanada
- Study dates
- 5 Jan 2022to 3 May 2022
- Phase
- Interventional
- Are there results?
- None posted in the registry
Key requirements
Who can take part, as listed by the sponsor. Only the study team can confirm whether you are eligible, so take this list to your doctor.
- Ages 18 years and older
- All sexes
- Healthy volunteers accepted
- Adult, Older adult
Who may be able to join
- Medical students who are actively enrolled in medical school in any Canadian institution who do not meet the exclusion criteria.
- Premedical students who are actively enrolled in medical school in any Canadian institution who do not meet the exclusion criteria.
Who cannot join
- Participation in previous trials involving the NeuroVR (CAE Healthcare) simulator.
Trial locations and contacts
1 site in 1 country.
Canada1 site
Montreal Neurological Institute and Hospital
Montreal, Quebec, H3A 2B4
About this study
Official title: Testing the Efficacy of an Artificial Intelligence Real-Time Coaching System in Comparison to In Person Expert Instruction in Surgical Simulation Training of Medical Students - A Randomized Controlled Trial
Background:
Trainees learn surgical technical skills through apprenticeship model working closely with surgeons and given increased responsibility in patient cases under expert supervision. However, factors such as surgeons' busy schedule, number of available patient cases, patient safety and lack of objectivity and standardization in training pose strong limitations. Virtual reality surgical simulators integrated with artificial intelligence (AI) systems provide a standardized realistic simulation environment and detailed performance data that allows accurate quantitation of surgical skills and tailored feedback. These platforms make repetitive practice of surgical skills possible in a risk-free environment.
Read more
The Intelligent Continuous Monitoring System (ICEMS), a deep learning application integrated in NeuroVR simulation platform, was developed to assess surgical performance continuously in 0.2 second intervals and provide coaching and risk detection. Although a predictive validity for assessment module was provided previously, the effectiveness of real-time coaching and risk detection ability with this AI system remains to be explored.
The objective of this study is to compare the error-oriented intelligent feedback provided by the ICEMS to in-person expert instruction in surgical simulation training by monitoring the improvement of medical student technical skills on a series of virtual reality tumor resection tasks.
Read the detailed description
Background Advancements in the fields of surgical simulation and artificial intelligence have resulted in the development of intelligent technical skill assessment and tutoring systems to meet the needs of the competency-based approach in medical education. These novel educational tools offer solutions for standardization and objectivity of the learner's technical competence assessment and training, in risk-free simulated operative environments. These systems can supplement the gold-standard apprenticeship model in training in surgery and other procedural based medicine by providing detailed information about technical skills, action-oriented procedural guidance, and risk assessment. Such applications can help educators in developing objective formative and summative student assessment and teaching methodologies.
Rationale The Intelligent Continuous Monitoring System (ICEMS), an artificial intelligence (AI) deep learning application was developed to assess surgical performance continuously in 0.2 second intervals with the ability of coaching trainees to expert level and risk detection. This system involved in a patent pending entitled 'Methods and systems for continuous monitoring of task performance on virtual simulators' (2020; patent No. 05001770-883USPR). Previously, a predictive validity for assessment module was provided however, the effectiveness of this system in real-time coaching trainees and risk detection needs to be explored.
Research Objectives To compare the effectiveness of an intelligent real-time coaching system to the gold standard expert mediated feedback. To test the transfer of learning during the practice tasks to the more complex realistic scenario.
Hypotheses
- The ICEMS feedback group will significantly improve in the composite-score between the first attempt to the last attempt in practice scenario.
- The control Group with no-expert mediated post hoc benchmark feedback will not statistically improve in the composite-score between first attempt to the last attempt in practice scenario.
- The ICEMS feedback group will have a statistically higher composite-score compared to the control group, in the fifth attempt in the practice scenario.
- The ICEMS feedback group will have a composite-score statistically non-inferior compared to the expert mediated instruction group in the fifth attempt on the practice scenario.
- The global OSATS score of the ICEMS feedback group will be non-inferior to that of the expert mediated instruction group in the complex realistic scenario.
- That ICEMS feedback will not elect a difference in emotion or cognitive load compared to the expert mediated instruction group.
Specific Aims: 1) To assess if the efficacy of AI mediated real-time tailored feedback is statistically non-inferior to expert mediated in-person feedback in improving medical students' bimanual surgical skills on two virtually simulated surgery tasks.
2)To outline if different emotions and cognitive load are elicited by the ICEMS feedback system as compared to human instruction
Design: A three-arm single blinded randomized controlled trial of AI training versus expert mediated training versus no-expert mediated training.
Setting: Neurosurgical Simulation and Artificial Intelligence Learning Centre, Montreal Neurological Institute.
Participants: Students who are enrolled in their premed year or first to fourth year in medical schools across the province of Quebec.
Task: Complete removal of a simulated practice tumor five times and a complex simulated brain tumor once with minimal bleeding and damage to surrounding simulated healthy brain structures using two surgical instruments (Ultrasonic Aspirator and Bipolar pincers) using the NeuroVR (CAE Healthcare) surgical simulator.
Intervention: A single 90-minute training session, including six virtual subpial tumor resection attempts (five simple practice scenarios and one complex realistic scenario) with assessment and feedback from either:
- the Intelligent Continuous Expertise Monitoring System (ICEMS) (Group 1)
- an in-person expert instruction feedback (Group 2)
- no-expert mediated post hoc benchmark feedback (Group 3)
Main Outcomes and Measures:
Primary outcome is surgical performance improvement in the composite-score assessed by the Intelligent Continuous Expertise Monitoring System's (ICEMS) previously validated assessment module. Performance improvement is measured as the composite-score difference between participant's attempts during the five practice attempts. Learning transfer will be assessed by the composite-score obtained in the complex realistic scenario.
Secondary outcome is participants' performance score on the realistic task which will be scored by blinded experts using the Objective Structured Assessment of Technical Skills (OSATS) global rating scale. Differences in strength of emotions elicited will be measured by Duffy's Medical Emotional Scale (MES), before, and after the intervention and difference in cognitive load will be measured using Leppink's Cognitive Load Index (CLI) after the intervention
This study, to the knowledge of the investigators, is the first application comparing the effectiveness of a real-time intelligent feedback system to the gold standard human instruction. This study aims to identify efficient training methodologies using artificial intelligence powered surgical simulator training platforms, hence trainees are provided objective, tailored technical skill assessment and coaching using state of art technologies.
Statistical Analysis Plan Participant data will be anonymized and stored. The Intelligent Continuous Expertise Monitoring System (ICEMS) will be used to score surgical performance using raw performance data and provide a performance score at 0.2 second intervals. An average composite-score will be calculated for each task. Within and between groups comparisons will be made to assess the learning from first attempt (baseline performance) to the fifth attempt and compare improvement between groups. The composite score on the realistic scenario will be calculated by the ICEMS to assess transfer of learning. Participant score improvement will be examined by repeated measures ANOVA. Considering a potential effect size of 25%, a statistical power of 0.95 and a significance of 0.05 this study needs to recruit 29 participants in each group. Participants' performance score in the fifth practice attempt will be compared between groups using ANOVA to compare learning. The same analysis will be done for the realistic attempt to compare learning transfer between groups.
Performance in the realistic attempt will be rated by blind experts using the OSATS global rating scale, using participant performance video. The global OSATS score will be compared between groups to compare efficacy in learning using ANOVA.
- Conditions
- Surgical Education
- Being tested
- Experimental: Experimental Group - Intelligent Continuous Expertise Monitoring System group, Experimental Group In-person expert-mediated instruction group
- Keywords
- Virtual reality, Artificial Intelligence, Coaching
What participants receive
What participants receive. In a randomised study each person is assigned to one group by chance.
Experimental: Experimental Group - Intelligent Continuous Expertise Monitoring System groupBehavioural
During each of the practice task they will receive real-time auditory feedback instructed by the intelligent system. After each attempt, a student takes a 5-minute break. They will be shown the errors they made during the task regarding five performance metrics. After seeing each error, they will be shown video demonstration to learn how to expertly perform at each performance metrics. On their 6th attempt they will perform on the realistic scenario without any feedback given.
Experimental Group In-person expert-mediated instruction groupBehavioural
During each of the practice task, students will receive verbal feedback from the expert instructor present in the room. After each task, experts will summarize their performance and outline the errors the student made. Based on the student's performance, expert will demonstrate how to expertly perform the task in the simulation, and how to improve their performance in the next attempt. Students will perform the 6th attempt on the realistic scenario without any instruction given.
| Study group | Role | What they receive |
|---|---|---|
| Control Group No-expert mediated post hoc benchmark group | No intervention | 30 participants. Individuals receive identical introductory information,same time, to perform, same scenarios as other groups. Students receive their scores on 5 performance metrics compared to expert performance benchmarks. Scores are presented in the 5 minute breaks between tasks. Student goal is to be within the benchmark in all five metrics. |
| Experimental Group - Intelligent Continuous Expertise Monitoring System group | Experimental | 30 Participants. Introductory information provided on simulator and scenario. They perform 5 simple practice subpial tumor resections with 5 minutes per trial. On 6th attempt 13 minutes to perform a complex realistic scenario.
During first practice task, participants receive no feedback. For the subsequent 4 practice tasks participants will receive real-time auditory feedback instruction by the intelligent system. After each of the 5 attempts, a student takes a 5-minute break. During each of the 5 breaks the participants will be shown the errors they made during the task by the intelligent system regarding five performance metrics monitored. After seeing each error outline, the participant will be shown a video demonstration to learn how to expertly perform on each performance metric. On their 6th attempt they will perform on the realistic scenario without any feedback given. Receives: Behavioral: Experimental: Experimental Group - Intelligent Continuous Expertise Monitoring System group |
| Experimental Group In-person expert-mediated instruction group | Experimental | 30 Participants. Introductory information provided on simulator and scenario. They perform 5 simple practice subpial tumor resections with 5 minutes per trial. On 6th attempt 13 minutes to perform a complex realistic scenario.
During first practice task participants receive no feedback. For the subsequent 4 practice tasks participants receive real-time auditory feedback instruction by in-person expert during the task. After each of the 5 tasks, students takes a 5-minute break. During each of the 5 breaks the in-person expert provides feedback to the participant based on their OSATS score assessment during the previous trial. If the expert feels it is appropriate the expert will demonstrate how to do the specific procedure which has been found to be a concern on the simulator themselves so the participant can understand how to improve their performance. On their 6th attempt they will perform on the realistic scenario without any feedback given. Receives: Behavioral: Experimental Group In-person expert-mediated instruction group |
What is being measured
The primary outcome is the main question the study is designed to answer.
Change in performance
Performance will be measured using the composite-score assessed by the ICEMS system
Measured: 1 Day of Study
Transfer of learning
Performance on the complex realistic scenario will be evaluated by the composite-score assessed by the ICEMS system.
Measured: 1 Day of Study
Secondary outcomes (3)
Objective Structured Assessment of Technical Skills (OSATS) global rating scale
Performance on the realistic attempt will be rated by blinded experts using the Objective Structured Assessment of Technical Skills (OSATS) global rating scale. Efficacy in learning with real-time intelligent feedback system will be compared to expert-mediated instruction and to no-expert mediated post hoc feedback.
Measured: 1 Day of Study
Differences in strength of emotions elicited
Measured by Duffy's Medical Emotional Scale (MES)
Measured: 1 Day of Study
Difference in Cognitive Load
Measured using Leppink's Cognitive Load Index (CLI) after the intervention
Measured: 1 Day of study
Timeline
From the start of the study to its completion; the mark is today. Estimated dates are the sponsor’s plan.
- First posted23 Dec 2021
- Study start5 Jan 2022
- Primary completion3 May 2022
- Study completion3 May 2022
Resources
- Official record on ClinicalTrials.govThe registry entry this page is built from, with its full change history.
- Publications (4)Papers the sponsor has linked to this study.
- Other trials for Surgical EducationStudies of the same condition, open and under way.
- What the trial phases meanHow a treatment moves from first tests in people to approval.
More details
For doctors and researchers: how the study is designed, who runs it, and its registry IDs.
Study design
- Study type
- Interventional
- Phase
- Phase N/A
- Primary purpose
- Health services research
- Allocation
- Randomised
- Intervention model
- Parallel groups
- Model description
- Randomized Control trial
- Masking
- Double blind
- Who is masked
- Participant, Outcomes assessor
- Enrolment
- 98 participants (actual)
Sponsor and investigators
- Lead sponsor
- McGill University
- Sponsor type
- University, hospital or other
- Responsible party
- Principal investigator, Rolando Del Maestro, McGill University
- Investigators
- Rolando Del Maestro, MD, Principal investigator, McGill University
IDs, dates and publications
- ClinicalTrials.gov ID
- NCT05168150
- Sponsor's protocol ID
- 2010-270, NEU-09-042-Trial 3
- Status last verified
- Aug 2022
- First submitted
- 20 Dec 2021
- Last updated
- 8 Aug 2022
- Results posted
- No
- Data monitoring committee
- No
- FDA-regulated drug
- No
- FDA-regulated device
- No
- Individual data sharing
- Yes: Data obtained from primary and secondary outcomes may be shared if other researchers have an interest in this data.
Publications
Delorme S, Laroche D, DiRaddo R, Del Maestro RF. NeuroTouch: a physics-based virtual simulator for cranial microneurosurgery training. Neurosurgery. 2012 Sep;71(1 Suppl Operative):32-42. doi: 10.1227/NEU.0b013e318249c744.
PubMed 22233921Brightwell A, Grant J. Competency-based training: who benefits? Postgrad Med J. 2013 Feb;89(1048):107-10. doi: 10.1136/postgradmedj-2012-130881. Epub 2012 Sep 27.
PubMed 23019588Chan J, Pangal DJ, Cardinal T, Kugener G, Zhu Y, Roshannai A, Markarian N, Sinha A, Anandkumar A, Hung A, Zada G, Donoho DA. A systematic review of virtual reality for the assessment of technical skills in neurosurgery. Neurosurg Focus. 2021 Aug;51(2):E15. doi: 10.3171/2021.5.FOCUS21210.
PubMed 34333472Winkler-Schwartz A, Yilmaz R, Mirchi N, Bissonnette V, Ledwos N, Siyar S, Azarnoush H, Karlik B, Del Maestro R. Machine Learning Identification of Surgical and Operative Factors Associated With Surgical Expertise in Virtual Reality Simulation. JAMA Netw Open. 2019 Aug 2;2(8):e198363. doi: 10.1001/jamanetworkopen.2019.8363.
PubMed 31373651
Study at a glance
From the registry record, last updated by the sponsor on 8 Aug 2022. Recruitment status is as the sponsor reported it: confirm with the study team.
- Official title
- Testing the Efficacy of an Artificial Intelligence Real-Time Coaching System in Comparison to In Person Expert Instruction in Surgical Simulation Training of Medical Students - A Randomized Controlled Trial
- ClinicalTrials.gov ID
- NCT05168150
- Acronym / sponsor ID
- 2010-270, NEU-09-042-Trial 3
- Conditions
- Surgical Education
- Keywords
- Virtual reality, Artificial Intelligence, Coaching
- Study type
- Interventional
- Recruitment status
- Completed
- Enrolment
- 98 participants (actual)
- Lead sponsor
- McGill UniversityUniversity, hospital or other
- Study start
- 5 Jan 2022 (actual)
- Primary completion
- 3 May 2022 (actual)
- Study completion
- 3 May 2022 (actual)
- First posted
- 23 Dec 2021 (actual)
- Results
- Not posted in the source record
Clinical summary
The research question as the sponsor registered it. This describes what the study plans to test, not what it found.
Background:
Trainees learn surgical technical skills through apprenticeship model working closely with surgeons and given increased responsibility in patient cases under expert supervision. However, factors such as surgeons' busy schedule, number of available patient cases, patient safety and lack of objectivity and standardization in training pose strong limitations. Virtual reality surgical simulators integrated with artificial intelligence (AI) systems provide a standardized realistic simulation environment and detailed performance data that allows accurate quantitation of surgical skills and tailored feedback. These platforms make repetitive practice of surgical skills possible in a risk-free environment.
The Intelligent Continuous Monitoring System (ICEMS), a deep learning application integrated in NeuroVR simulation platform, was developed to assess surgical performance continuously in 0.2 second intervals and provide coaching and risk detection. Although a predictive validity for assessment module was provided previously, the effectiveness of real-time coaching and risk detection ability with this AI system remains to be explored.
The objective of this study is to compare the error-oriented intelligent feedback provided by the ICEMS to in-person expert instruction in surgical simulation training by monitoring the improvement of medical student technical skills on a series of virtual reality tumor resection tasks.
Detailed description
Background Advancements in the fields of surgical simulation and artificial intelligence have resulted in the development of intelligent technical skill assessment and tutoring systems to meet the needs of the competency-based approach in medical education. These novel educational tools offer solutions for standardization and objectivity of the learner's technical competence assessment and training, in risk-free simulated operative environments. These systems can supplement the gold-standard apprenticeship model in training in surgery and other procedural based medicine by providing detailed information about technical skills, action-oriented procedural guidance, and risk assessment. Such applications can help educators in developing objective formative and summative student assessment and teaching methodologies.
Rationale The Intelligent Continuous Monitoring System (ICEMS), an artificial intelligence (AI) deep learning application was developed to assess surgical performance continuously in 0.2 second intervals with the ability of coaching trainees to expert level and risk detection. This system involved in a patent pending entitled 'Methods and systems for continuous monitoring of task performance on virtual simulators' (2020; patent No. 05001770-883USPR). Previously, a predictive validity for assessment module was provided however, the effectiveness of this system in real-time coaching trainees and risk detection needs to be explored.
Research Objectives To compare the effectiveness of an intelligent real-time coaching system to the gold standard expert mediated feedback. To test the transfer of learning during the practice tasks to the more complex realistic scenario.
Hypotheses
- The ICEMS feedback group will significantly improve in the composite-score between the first attempt to the last attempt in practice scenario.
- The control Group with no-expert mediated post hoc benchmark feedback will not statistically improve in the composite-score between first attempt to the last attempt in practice scenario.
- The ICEMS feedback group will have a statistically higher composite-score compared to the control group, in the fifth attempt in the practice scenario.
- The ICEMS feedback group will have a composite-score statistically non-inferior compared to the expert mediated instruction group in the fifth attempt on the practice scenario.
- The global OSATS score of the ICEMS feedback group will be non-inferior to that of the expert mediated instruction group in the complex realistic scenario.
- That ICEMS feedback will not elect a difference in emotion or cognitive load compared to the expert mediated instruction group.
Specific Aims: 1) To assess if the efficacy of AI mediated real-time tailored feedback is statistically non-inferior to expert mediated in-person feedback in improving medical students' bimanual surgical skills on two virtually simulated surgery tasks.
2)To outline if different emotions and cognitive load are elicited by the ICEMS feedback system as compared to human instruction
Design: A three-arm single blinded randomized controlled trial of AI training versus expert mediated training versus no-expert mediated training.
Setting: Neurosurgical Simulation and Artificial Intelligence Learning Centre, Montreal Neurological Institute.
Participants: Students who are enrolled in their premed year or first to fourth year in medical schools across the province of Quebec.
Task: Complete removal of a simulated practice tumor five times and a complex simulated brain tumor once with minimal bleeding and damage to surrounding simulated healthy brain structures using two surgical instruments (Ultrasonic Aspirator and Bipolar pincers) using the NeuroVR (CAE Healthcare) surgical simulator.
Intervention: A single 90-minute training session, including six virtual subpial tumor resection attempts (five simple practice scenarios and one complex realistic scenario) with assessment and feedback from either:
- the Intelligent Continuous Expertise Monitoring System (ICEMS) (Group 1)
- an in-person expert instruction feedback (Group 2)
- no-expert mediated post hoc benchmark feedback (Group 3)
Main Outcomes and Measures:
Primary outcome is surgical performance improvement in the composite-score assessed by the Intelligent Continuous Expertise Monitoring System's (ICEMS) previously validated assessment module. Performance improvement is measured as the composite-score difference between participant's attempts during the five practice attempts. Learning transfer will be assessed by the composite-score obtained in the complex realistic scenario.
Secondary outcome is participants' performance score on the realistic task which will be scored by blinded experts using the Objective Structured Assessment of Technical Skills (OSATS) global rating scale. Differences in strength of emotions elicited will be measured by Duffy's Medical Emotional Scale (MES), before, and after the intervention and difference in cognitive load will be measured using Leppink's Cognitive Load Index (CLI) after the intervention
This study, to the knowledge of the investigators, is the first application comparing the effectiveness of a real-time intelligent feedback system to the gold standard human instruction. This study aims to identify efficient training methodologies using artificial intelligence powered surgical simulator training platforms, hence trainees are provided objective, tailored technical skill assessment and coaching using state of art technologies.
Statistical Analysis Plan Participant data will be anonymized and stored. The Intelligent Continuous Expertise Monitoring System (ICEMS) will be used to score surgical performance using raw performance data and provide a performance score at 0.2 second intervals. An average composite-score will be calculated for each task. Within and between groups comparisons will be made to assess the learning from first attempt (baseline performance) to the fifth attempt and compare improvement between groups. The composite score on the realistic scenario will be calculated by the ICEMS to assess transfer of learning. Participant score improvement will be examined by repeated measures ANOVA. Considering a potential effect size of 25%, a statistical power of 0.95 and a significance of 0.05 this study needs to recruit 29 participants in each group. Participants' performance score in the fifth practice attempt will be compared between groups using ANOVA to compare learning. The same analysis will be done for the realistic attempt to compare learning transfer between groups.
Performance in the realistic attempt will be rated by blind experts using the OSATS global rating scale, using participant performance video. The global OSATS score will be compared between groups to compare efficacy in learning using ANOVA.
Study design
- Primary purpose
- Health services research
- Allocation
- Randomised
- Intervention model
- Parallel groups
- Masking
- Double blind (participant, outcomes assessor)
- Number of arms
- 3
- Model description
- Randomized Control trial
| Arm | Type | Receives |
|---|---|---|
| Control Group No-expert mediated post hoc benchmark group | No intervention | –30 participants. Individuals receive identical introductory information,same time, to perform, same scenarios as other groups. Students receive their scores on 5 performance metrics compared to expert performance benchmarks. Scores are presented in the 5 minute breaks between tasks. Student goal is to be within the benchmark in all five metrics. |
| Experimental Group - Intelligent Continuous Expertise Monitoring System group | Experimental | Behavioral: Experimental: Experimental Group - Intelligent Continuous Expertise Monitoring System group30 Participants. Introductory information provided on simulator and scenario. They perform 5 simple practice subpial tumor resections with 5 minutes per trial. On 6th attempt 13 minutes to perform a complex realistic scenario. During first practice task, participants receive no feedback. For the subsequent 4 practice tasks participants will receive real-time auditory feedback instruction by the intelligent system. After each of the 5 attempts, a student takes a 5-minute break. During each of the 5 breaks the participants will be shown the errors they made during the task by the intelligent system regarding five performance metrics monitored. After seeing each error outline, the participant will be shown a video demonstration to learn how to expertly perform on each performance metric. On their 6th attempt they will perform on the realistic scenario without any feedback given. |
| Experimental Group In-person expert-mediated instruction group | Experimental | Behavioral: Experimental Group In-person expert-mediated instruction group30 Participants. Introductory information provided on simulator and scenario. They perform 5 simple practice subpial tumor resections with 5 minutes per trial. On 6th attempt 13 minutes to perform a complex realistic scenario. During first practice task participants receive no feedback. For the subsequent 4 practice tasks participants receive real-time auditory feedback instruction by in-person expert during the task. After each of the 5 tasks, students takes a 5-minute break. During each of the 5 breaks the in-person expert provides feedback to the participant based on their OSATS score assessment during the previous trial. If the expert feels it is appropriate the expert will demonstrate how to do the specific procedure which has been found to be a concern on the simulator themselves so the participant can understand how to improve their performance. On their 6th attempt they will perform on the realistic scenario without any feedback given. |
Who may qualify
- Ages
- 18 years and older
- Sex
- All
- Healthy volunteers
- Accepted
- Age groups
- Adult, Older adult
Key inclusion criteria (2 listed)
- Medical students who are actively enrolled in medical school in any Canadian institution who do not meet the exclusion criteria.
- Premedical students who are actively enrolled in medical school in any Canadian institution who do not meet the exclusion criteria.
Key exclusion criteria (1 listed)
- Participation in previous trials involving the NeuroVR (CAE Healthcare) simulator.
Eligibility is decided by the study team after screening; this overview does not establish that a patient qualifies. Full inclusion and exclusion criteria.
Interventions
| Intervention | Type | Arms |
|---|---|---|
| Experimental: Experimental Group - Intelligent Continuous Expertise Monitoring System groupDuring each of the practice task they will receive real-time auditory feedback instructed by the intelligent system. After each attempt, a student takes a 5-minute break. They will be shown the errors they made during the task regarding five performance metrics. After seeing each error, they will be shown video demonstration to learn how to expertly perform at each performance metrics. On their 6th attempt they will perform on the realistic scenario without any feedback given. | Behavioural | Experimental Group - Intelligent Continuous Expertise Monitoring System group |
| Experimental Group In-person expert-mediated instruction groupDuring each of the practice task, students will receive verbal feedback from the expert instructor present in the room. After each task, experts will summarize their performance and outline the errors the student made. Based on the student's performance, expert will demonstrate how to expertly perform the task in the simulation, and how to improve their performance in the next attempt. Students will perform the 6th attempt on the realistic scenario without any instruction given. | Behavioural | Experimental Group In-person expert-mediated instruction group |
Primary outcome
Change in performance
Time frame: 1 Day of Study
Performance will be measured using the composite-score assessed by the ICEMS system
Transfer of learning
Time frame: 1 Day of Study
Performance on the complex realistic scenario will be evaluated by the composite-score assessed by the ICEMS system.
3 secondary outcomes
Objective Structured Assessment of Technical Skills (OSATS) global rating scale
Time frame: 1 Day of Study
Differences in strength of emotions elicited
Time frame: 1 Day of Study
Difference in Cognitive Load
Time frame: 1 Day of study
Results and safety
The study is completed, but results are not available in the source record. That does not mean the treatment did or did not work.
Research links
- Official record on ClinicalTrials.govLast updated by the sponsor 8 Aug 2022.
- Background: PubMed 22233921Delorme S, Laroche D, DiRaddo R, Del Maestro RF. NeuroTouch: a physics-based virtual simulator for cranial microneurosurgery training. Neurosurgery. 2012 Sep;71(1 Suppl Operative):32-42. doi: 10.1227/NEU.0b013e318249c744.
- Background: PubMed 23019588Brightwell A, Grant J. Competency-based training: who benefits? Postgrad Med J. 2013 Feb;89(1048):107-10. doi: 10.1136/postgradmedj-2012-130881. Epub 2012 Sep 27.
- Background: PubMed 34333472Chan J, Pangal DJ, Cardinal T, Kugener G, Zhu Y, Roshannai A, Markarian N, Sinha A, Anandkumar A, Hung A, Zada G, Donoho DA. A systematic review of virtual reality for the assessment of technical skills in neurosurgery. Neurosurg Focus. 2021 Aug;51(2):E15. doi: 10.3171/2021.5.FOCUS21210.
- Background: PubMed 31373651Winkler-Schwartz A, Yilmaz R, Mirchi N, Bissonnette V, Ledwos N, Siyar S, Azarnoush H, Karlik B, Del Maestro R. Machine Learning Identification of Surgical and Operative Factors Associated With Surgical Expertise in Virtual Reality Simulation. JAMA Netw Open. 2019 Aug 2;2(8):e198363. doi: 10.1001/jamanetworkopen.2019.8363.
- Individual participant dataSharing planned: Yes. Data obtained from primary and secondary outcomes may be shared if other researchers have an interest in this data.
Related trials for Surgical Education
Not yet recruiting
Generative AI-Assisted Personalized Feedback for Laparoscopic Skill Learning in Medical Undergraduates
Medical Education
RecruitingObservational study
Robotic Inflammatory Bowel Disease Development, Learning and Efficacy Study
Inflammatory Bowel Diseases, Crohn Disease
RecruitingObservational study
Surgical Intuition in Anastomotic Leakage After Colorectal Surgery
Anastomotic Leaks
RecruitingObservational study
Electrosurgical Patterns and Learning Curve in Elective Laparoscopic Cholecystectomy
Cholelithiasis
Not yet recruiting
AI-Assisted Education Before Surgery to Prevent Lung Complications in Older Surgical Patients
Postoperative Pulmonary Complications
Referral and contact
AzTrials does not enrol participants or take referrals. Contact the study team or a site listed below; they decide who can take part. This study is not recruiting at the moment.
Rolando Del Maestro, MD principal investigator
McGill University
1 site in 1 country: see every site and its status.
Questions about this trial
What is this clinical trial studying?
A study testing Experimental: Experimental Group - Intelligent Continuous Expertise Monitoring System group and Experimental Group In-person expert-mediated instruction group in people with Surgical Education, run by McGill University at 1 site in Canada.
The full registered description is in the About this study section and the official record.
Who may be eligible to take part?
The sponsor lists 2 inclusion and 1 exclusion criteria. Eligibility is decided by the study team after screening, so read the full criteria and contact a listed site.
Is this clinical trial currently recruiting?
The record says: finished, as of the sponsor's last update on 8 Aug 2022. Status can change between updates, so confirm with the study contact before making plans.
Where is the study taking place?
The record lists 1 site in Canada. Whether a site is taking part right now should be confirmed with that site. See every site.
What are the possible benefits and risks?
They depend on the study and the treatment, and no study can promise a benefit. The study team explains the known risks during informed consent, before anyone joins; the record itself does not list them.
Are results available for this study?
The study is completed, but results are not available in the source record. That does not mean the treatment did or did not work.
How can a doctor refer a patient or contact the research team?
The record lists no contact. The official record names the sponsor, who can be asked directly. AzTrials does not enrol participants or take referrals.
What does the primary outcome measure?
Change in performance (time frame: 1 Day of Study). The study lists 2 primary outcomes in all. It is what the study was designed to measure, not a finding.
Where can I read the original study information?
On the ClinicalTrials.gov record for NCT05168150, which also lists 4 linked publications.
Does a listing mean the treatment is approved or proven to work?
No. A listing means the study is registered. It does not show that the treatment is safe, works, or is approved by a regulator.
How often is this information updated?
AzTrials copies changed records from ClinicalTrials.gov every night and reloads the whole registry every week. The sponsor last updated this record on 8 Aug 2022.