Performance AssessmentNCT01052168
Do Motion Metrics Lead to Improved Skill Acquisition on Simulators?
A study testing skills training in people with Performance Assessment and Motion Metrics, run by Wake Forest University Health Sciences at 1 site in United States.
Trial summary
Emphasizing the growing popularity of motion metrics are the majority of available virtual reality simulators and some newer hybrid models that offer motion tracking for performance assessment. Read more about this study
- Who is it for?
- Any ageHealthy volunteers can take part
- Conditions the study is for
- Performance Assessment, Motion Metrics
- What is being tested?
- skills training
- Could I receive a placebo?
- No, everyone receives an active treatment
- How many people
- 42 participants
- Where
- 1 siteUnited States
- Study dates
- Nov 2009to Dec 2011
- Phase
- Interventional
- Are there results?
- None posted in the registry
Key requirements
Who can take part, as listed by the sponsor. Only the study team can confirm whether you are eligible, so take this list to your doctor.
- Ages Any age
- All sexes
- Healthy volunteers accepted
- Child, Adult, Older adult
Who may be able to join
- novices with no previous laparoscopic or simulation experience
- voluntary participation
Who cannot join
- expert in or familiarity with laparoscopy or simulation
- physical condition that prevents the performance of laparoscopic suturing
Trial locations and contacts
1 site in 1 country.
United States1 site
Carolinas Simulation Center
Charlotte, North Carolina, 28205
About this study
Emphasizing the growing popularity of motion metrics are the majority of available virtual reality simulators and some newer hybrid models that offer motion tracking for performance assessment. A popular hybrid model (PROMIS) allows training with regular laparoscopic instruments in a box-trainer while automatically recording task duration and movement efficiency (pathlength and smoothness) that are immediately offered as feedback to trainees.
Despite the increasing availability of simulators that track motion, our knowledge of the impact those metrics have on trainee learning is severely limited. We do not know if it is more important to use speed, accuracy, motion efficiency or a combination thereof for performance assessment and how these metrics impact skill transfer to the OR.
Read more
Based on sound educational principles we have developed a proficiency-based laparoscopic suturing simulator curriculum. This curriculum focuses on deliberate and distributed practice, provides trainees with augmented feedback and sets expert-derived performance goals based on time and errors. We have previously demonstrated that this curriculum leads to improved operative performance of trainees compared to controls.
To measure operative performance and determine transferability, we will use a live porcine Nissen fundoplication model. Instead of placing actual patients at risk, the porcine model is preferable for this purpose as it offers objective metrics (targets are established, distances measured, knots are disrupted for slippage scoring), complete standardization, and allows multiple individuals to be tested on the same day.
We hypothesize that proficiency-based simulator training in laparoscopic suturing to expert-derived levels of speed and motion will result in better operative performance compared to participants training to levels of speed or motion alone. The study is powered to detect an at least 10% performance difference between the groups.
Specific Aims
- Compare whether any performance differences between the groups persist long-term
- Assess whether the groups demonstrate differences in safety in the operating room by comparing the inadvertent injuries in the animal OR between the groups
- Identify the training duration required by novices to reach proficiency in laparoscopic suturing based on speed, motion efficiency, or a combination of these metrics
- Identify any baseline participant characteristics that may predict individual metric-specific performance
Read the detailed description
OBJECTIVE:: We hypothesized that training to expert-derived levels of speed and motion will lead to improved learning and will translate to better operating room (OR) performance of novices than training to goals of speed or motion alone. BACKGROUND:: Motion tracking has been suggested to be a more sensitive performance metric than time and errors for the assessment of surgical performance. METHODS:: An institutional review board-approved, single blinded, randomized controlled trial was conducted at our level-I American College of Surgeons accredited Education Institute. Forty-two novices trained to proficiency in laparoscopic suturing after being randomized into 3 groups: The speed group (n = 14) had to achieve expert levels of speed, the motion group (n = 15) expert levels of motion (path length and smoothness), and the speed and motion group (n = 13) both levels. To achieve proficiency, all groups also had to demonstrate error-free performance. The FLS suture module (task 5) was used for training inside the ProMIS simulator that tracks instrument motion. All groups participated in transfer and retention tests in the OR. OR performance was assessed by a blinded expert rater using Global Operative Assessment of Laparoscopic Skills, speed, accuracy, and inadvertent injuries.
- Conditions
- Performance Assessment, Motion Metrics
- Being tested
- skills training
- Keywords
- simulation, simulators, motion metrics, performance metrics, skills training, proficiency, laparoscopic
What participants receive
What participants receive. In a randomised study each person is assigned to one group by chance.
skills trainingOther
participants will train using different performance goals (based on different metrics)
| Study group | Role | What they receive |
|---|---|---|
| Speed Group | Active comparator | The Speed Group, (n=20) will train in laparoscopic suturing on the validated FLS suturing model until the expert level of speed (i.e. task duration < 70 seconds) has been achieved on two consecutive attempts. Receives: Other: skills training |
| Motion Group | Experimental | The Motion Group, (n=20) will train in laparoscopic suturing until expert levels of motion (pathlength 6700 and smoothness 560) have been achieved. Receives: Other: skills training |
| Speed and Motion Group | Experimental | The Speed and Motion Group (n=20) will train in laparoscopic suturing until expert levels of speed AND motion have been achieved. Receives: Other: skills training |
What is being measured
The primary outcome is the main question the study is designed to answer.
Laparoscopic suturing performance in the animal operating room
Measured: end of training and retention test after 3 months
Secondary outcomes (2)
inadvertent injuries in the animal OR
Measured: end of training test and 3 month retention test
training duration required by novices to reach proficiency in laparoscopic suturing based on speed, motion efficiency, or a combination of these metrics
Measured: end of study (within one year)
Timeline
From the start of the study to its completion; the mark is today. Estimated dates are the sponsor’s plan.
- First posted20 Jan 2010 (estimated)
- Study startNov 2009
- Primary completionDec 2011
- Study completionDec 2011
Resources
- Official record on ClinicalTrials.govThe registry entry this page is built from, with its full change history.
- Publications (35)Papers the sponsor has linked to this study.
- Other trials for Performance AssessmentStudies of the same condition, open and under way.
- What the trial phases meanHow a treatment moves from first tests in people to approval.
More details
For doctors and researchers: how the study is designed, who runs it, and its registry IDs.
Study design
- Study type
- Interventional
- Phase
- Phase N/A
- Primary purpose
- Basic science
- Allocation
- Randomised
- Intervention model
- Parallel groups
- Masking
- Single blind
- Who is masked
- Outcomes assessor
- Enrolment
- 42 participants (actual)
Sponsor and investigators
- Lead sponsor
- Wake Forest University Health Sciences
- Sponsor type
- University, hospital or other
- Collaborators
- Tulane University, Ethicon Endo-Surgery
- Responsible party
- Sponsor
- Investigators
- Dimitrios Stefanidis, MD, PhD, Principal investigator, Carolinas Simulation Center
IDs, dates and publications
- ClinicalTrials.gov ID
- NCT01052168
- Sponsor's protocol ID
- 11-06-20E
- Status last verified
- Mar 2013
- First submitted
- 18 Jan 2010
- Last updated
- 27 Apr 2022
- Results posted
- No
- Data monitoring committee
- Yes
Publications
Southern Surgeons Club. A prospective analysis of 1518 laparoscopic cholecystectomies. N Engl J Med. 1991 Apr 18;324(16):1073-8. doi: 10.1056/NEJM199104183241601.
PubMed 1826143Moore MJ, Bennett CL. The learning curve for laparoscopic cholecystectomy. The Southern Surgeons Club. Am J Surg. 1995 Jul;170(1):55-9. doi: 10.1016/s0002-9610(99)80252-9.
PubMed 7793496Institute of Medicine (US) Committee on Quality of Health Care in America; Kohn LT, Corrigan JM, Donaldson MS, editors. To Err is Human: Building a Safer Health System. Washington (DC): National Academies Press (US); 2000. Available from http://www.ncbi.nlm.nih.gov/books/NBK225182/
PubMed 25077248Hasan A, Pozzi M, Hamilton JR. New surgical procedures: can we minimise the learning curve? BMJ. 2000 Jan 15;320(7228):171-3. doi: 10.1136/bmj.320.7228.171. No abstract available.
PubMed 10634741Bridges M, Diamond DL. The financial impact of teaching surgical residents in the operating room. Am J Surg. 1999 Jan;177(1):28-32. doi: 10.1016/s0002-9610(98)00289-x.
PubMed 10037304Callery MP. Expansion beyond compression. Surg Endosc. 2003 May;17(5):677-8. doi: 10.1007/s00464-003-0017-6. Epub 2003 Apr 3. No abstract available.
PubMed 12669229Korndorffer JR Jr, Dunne JB, Sierra R, Stefanidis D, Touchard CL, Scott DJ. Simulator training for laparoscopic suturing using performance goals translates to the operating room. J Am Coll Surg. 2005 Jul;201(1):23-9. doi: 10.1016/j.jamcollsurg.2005.02.021.
PubMed 15978440Scott DJ, Bergen PC, Rege RV, Laycock R, Tesfay ST, Valentine RJ, Euhus DM, Jeyarajah DR, Thompson WM, Jones DB. Laparoscopic training on bench models: better and more cost effective than operating room experience? J Am Coll Surg. 2000 Sep;191(3):272-83. doi: 10.1016/s1072-7515(00)00339-2.
PubMed 10989902Hamilton EC, Scott DJ, Kapoor A, Nwariaku F, Bergen PC, Rege RV, Tesfay ST, Jones DB. Improving operative performance using a laparoscopic hernia simulator. Am J Surg. 2001 Dec;182(6):725-8. doi: 10.1016/s0002-9610(01)00800-5.
PubMed 11839347Seymour NE, Gallagher AG, Roman SA, O'Brien MK, Bansal VK, Andersen DK, Satava RM. Virtual reality training improves operating room performance: results of a randomized, double-blinded study. Ann Surg. 2002 Oct;236(4):458-63; discussion 463-4. doi: 10.1097/00000658-200210000-00008.
PubMed 12368674Grantcharov TP, Kristiansen VB, Bendix J, Bardram L, Rosenberg J, Funch-Jensen P. Randomized clinical trial of virtual reality simulation for laparoscopic skills training. Br J Surg. 2004 Feb;91(2):146-50. doi: 10.1002/bjs.4407.
PubMed 14760660Peters JH, Fried GM, Swanstrom LL, Soper NJ, Sillin LF, Schirmer B, Hoffman K; SAGES FLS Committee. Development and validation of a comprehensive program of education and assessment of the basic fundamentals of laparoscopic surgery. Surgery. 2004 Jan;135(1):21-7. doi: 10.1016/s0039-6060(03)00156-9. No abstract available.
PubMed 14694297Fried GM, Feldman LS, Vassiliou MC, Fraser SA, Stanbridge D, Ghitulescu G, Andrew CG. Proving the value of simulation in laparoscopic surgery. Ann Surg. 2004 Sep;240(3):518-25; discussion 525-8. doi: 10.1097/01.sla.0000136941.46529.56.
PubMed 15319723Datta V, Mackay S, Mandalia M, Darzi A. The use of electromagnetic motion tracking analysis to objectively measure open surgical skill in the laboratory-based model. J Am Coll Surg. 2001 Nov;193(5):479-85. doi: 10.1016/s1072-7515(01)01041-9.
PubMed 11708503Van Sickle KR, McClusky DA 3rd, Gallagher AG, Smith CD. Construct validation of the ProMIS simulator using a novel laparoscopic suturing task. Surg Endosc. 2005 Sep;19(9):1227-31. doi: 10.1007/s00464-004-8274-6. Epub 2005 Jul 21.
PubMed 16025195
Study at a glance
From the registry record, last updated by the sponsor on 27 Apr 2022. Recruitment status is as the sponsor reported it: confirm with the study team.
- Official title
- Do Motion Metrics Lead to Improved Skill Acquisition on Simulators?
- ClinicalTrials.gov ID
- NCT01052168
- Acronym / sponsor ID
- 11-06-20E
- Conditions
- Performance Assessment, Motion Metrics
- Keywords
- simulation, simulators, motion metrics, performance metrics, skills training, proficiency, laparoscopic
- Study type
- Interventional
- Recruitment status
- Completed
- Enrolment
- 42 participants (actual)
- Lead sponsor
- Wake Forest University Health SciencesUniversity, hospital or other
- Collaborators
- Tulane University, Ethicon Endo-Surgery
- Study start
- Nov 2009
- Primary completion
- Dec 2011 (actual)
- Study completion
- Dec 2011 (actual)
- First posted
- 20 Jan 2010 (estimated)
- Results
- Not posted in the source record
Clinical summary
The research question as the sponsor registered it. This describes what the study plans to test, not what it found.
Emphasizing the growing popularity of motion metrics are the majority of available virtual reality simulators and some newer hybrid models that offer motion tracking for performance assessment. A popular hybrid model (PROMIS) allows training with regular laparoscopic instruments in a box-trainer while automatically recording task duration and movement efficiency (pathlength and smoothness) that are immediately offered as feedback to trainees.
Despite the increasing availability of simulators that track motion, our knowledge of the impact those metrics have on trainee learning is severely limited. We do not know if it is more important to use speed, accuracy, motion efficiency or a combination thereof for performance assessment and how these metrics impact skill transfer to the OR.
Based on sound educational principles we have developed a proficiency-based laparoscopic suturing simulator curriculum. This curriculum focuses on deliberate and distributed practice, provides trainees with augmented feedback and sets expert-derived performance goals based on time and errors. We have previously demonstrated that this curriculum leads to improved operative performance of trainees compared to controls.
To measure operative performance and determine transferability, we will use a live porcine Nissen fundoplication model. Instead of placing actual patients at risk, the porcine model is preferable for this purpose as it offers objective metrics (targets are established, distances measured, knots are disrupted for slippage scoring), complete standardization, and allows multiple individuals to be tested on the same day.
We hypothesize that proficiency-based simulator training in laparoscopic suturing to expert-derived levels of speed and motion will result in better operative performance compared to participants training to levels of speed or motion alone. The study is powered to detect an at least 10% performance difference between the groups.
Specific Aims
- Compare whether any performance differences between the groups persist long-term
- Assess whether the groups demonstrate differences in safety in the operating room by comparing the inadvertent injuries in the animal OR between the groups
- Identify the training duration required by novices to reach proficiency in laparoscopic suturing based on speed, motion efficiency, or a combination of these metrics
- Identify any baseline participant characteristics that may predict individual metric-specific performance
Detailed description
OBJECTIVE:: We hypothesized that training to expert-derived levels of speed and motion will lead to improved learning and will translate to better operating room (OR) performance of novices than training to goals of speed or motion alone. BACKGROUND:: Motion tracking has been suggested to be a more sensitive performance metric than time and errors for the assessment of surgical performance. METHODS:: An institutional review board-approved, single blinded, randomized controlled trial was conducted at our level-I American College of Surgeons accredited Education Institute. Forty-two novices trained to proficiency in laparoscopic suturing after being randomized into 3 groups: The speed group (n = 14) had to achieve expert levels of speed, the motion group (n = 15) expert levels of motion (path length and smoothness), and the speed and motion group (n = 13) both levels. To achieve proficiency, all groups also had to demonstrate error-free performance. The FLS suture module (task 5) was used for training inside the ProMIS simulator that tracks instrument motion. All groups participated in transfer and retention tests in the OR. OR performance was assessed by a blinded expert rater using Global Operative Assessment of Laparoscopic Skills, speed, accuracy, and inadvertent injuries.
Study design
- Primary purpose
- Basic science
- Allocation
- Randomised
- Intervention model
- Parallel groups
- Masking
- Single blind (outcomes assessor)
- Number of arms
- 3
| Arm | Type | Receives |
|---|---|---|
| Speed Group | Active comparator | Other: skills trainingThe Speed Group, (n=20) will train in laparoscopic suturing on the validated FLS suturing model until the expert level of speed (i.e. task duration < 70 seconds) has been achieved on two consecutive attempts. |
| Motion Group | Experimental | Other: skills trainingThe Motion Group, (n=20) will train in laparoscopic suturing until expert levels of motion (pathlength 6700 and smoothness 560) have been achieved. |
| Speed and Motion Group | Experimental | Other: skills trainingThe Speed and Motion Group (n=20) will train in laparoscopic suturing until expert levels of speed AND motion have been achieved. |
Who may qualify
- Sex
- All
- Healthy volunteers
- Accepted
- Age groups
- Child, Adult, Older adult
Key inclusion criteria (2 listed)
- novices with no previous laparoscopic or simulation experience
- voluntary participation
Key exclusion criteria (2 listed)
- expert in or familiarity with laparoscopy or simulation
- physical condition that prevents the performance of laparoscopic suturing
Eligibility is decided by the study team after screening; this overview does not establish that a patient qualifies. Full inclusion and exclusion criteria.
Interventions
| Intervention | Type | Arms |
|---|---|---|
| skills trainingparticipants will train using different performance goals (based on different metrics) | Other | Motion Group, Speed Group, Speed and Motion Group |
Primary outcome
Laparoscopic suturing performance in the animal operating room
Time frame: end of training and retention test after 3 months
2 secondary outcomes
inadvertent injuries in the animal OR
Time frame: end of training test and 3 month retention test
training duration required by novices to reach proficiency in laparoscopic suturing based on speed, motion efficiency, or a combination of these metrics
Time frame: end of study (within one year)
Results and safety
The study is completed, but results are not available in the source record. That does not mean the treatment did or did not work.
Research links
- Official record on ClinicalTrials.govLast updated by the sponsor 27 Apr 2022.
- Background: PubMed 1826143Southern Surgeons Club. A prospective analysis of 1518 laparoscopic cholecystectomies. N Engl J Med. 1991 Apr 18;324(16):1073-8. doi: 10.1056/NEJM199104183241601.
- Background: PubMed 7793496Moore MJ, Bennett CL. The learning curve for laparoscopic cholecystectomy. The Southern Surgeons Club. Am J Surg. 1995 Jul;170(1):55-9. doi: 10.1016/s0002-9610(99)80252-9.
- Background: PubMed 25077248Institute of Medicine (US) Committee on Quality of Health Care in America; Kohn LT, Corrigan JM, Donaldson MS, editors. To Err is Human: Building a Safer Health System. Washington (DC): National Academies Press (US); 2000. Available from http://www.ncbi.nlm.nih.gov/books/NBK225182/
- Background: PubMed 10634741Hasan A, Pozzi M, Hamilton JR. New surgical procedures: can we minimise the learning curve? BMJ. 2000 Jan 15;320(7228):171-3. doi: 10.1136/bmj.320.7228.171. No abstract available.
- Background: PubMed 10037304Bridges M, Diamond DL. The financial impact of teaching surgical residents in the operating room. Am J Surg. 1999 Jan;177(1):28-32. doi: 10.1016/s0002-9610(98)00289-x.
- Background: PubMed 12669229Callery MP. Expansion beyond compression. Surg Endosc. 2003 May;17(5):677-8. doi: 10.1007/s00464-003-0017-6. Epub 2003 Apr 3. No abstract available.
- Background: PubMed 15978440Korndorffer JR Jr, Dunne JB, Sierra R, Stefanidis D, Touchard CL, Scott DJ. Simulator training for laparoscopic suturing using performance goals translates to the operating room. J Am Coll Surg. 2005 Jul;201(1):23-9. doi: 10.1016/j.jamcollsurg.2005.02.021.
- Background: PubMed 10989902Scott DJ, Bergen PC, Rege RV, Laycock R, Tesfay ST, Valentine RJ, Euhus DM, Jeyarajah DR, Thompson WM, Jones DB. Laparoscopic training on bench models: better and more cost effective than operating room experience? J Am Coll Surg. 2000 Sep;191(3):272-83. doi: 10.1016/s1072-7515(00)00339-2.
- Background: PubMed 11839347Hamilton EC, Scott DJ, Kapoor A, Nwariaku F, Bergen PC, Rege RV, Tesfay ST, Jones DB. Improving operative performance using a laparoscopic hernia simulator. Am J Surg. 2001 Dec;182(6):725-8. doi: 10.1016/s0002-9610(01)00800-5.
- Background: PubMed 12368674Seymour NE, Gallagher AG, Roman SA, O'Brien MK, Bansal VK, Andersen DK, Satava RM. Virtual reality training improves operating room performance: results of a randomized, double-blinded study. Ann Surg. 2002 Oct;236(4):458-63; discussion 463-4. doi: 10.1097/00000658-200210000-00008.
- Background: PubMed 14760660Grantcharov TP, Kristiansen VB, Bendix J, Bardram L, Rosenberg J, Funch-Jensen P. Randomized clinical trial of virtual reality simulation for laparoscopic skills training. Br J Surg. 2004 Feb;91(2):146-50. doi: 10.1002/bjs.4407.
- Background: PubMed 14694297Peters JH, Fried GM, Swanstrom LL, Soper NJ, Sillin LF, Schirmer B, Hoffman K; SAGES FLS Committee. Development and validation of a comprehensive program of education and assessment of the basic fundamentals of laparoscopic surgery. Surgery. 2004 Jan;135(1):21-7. doi: 10.1016/s0039-6060(03)00156-9. No abstract available.
Related trials for Performance Assessment
Active, not recruitingObservational study
Evaluation of the Performance and Safety of the Monobloc® - Silicone SoftOne® Breast Implants at 10 Years
Breast Augmentation, Breast Reconstruction
Not yet recruiting
Study to Evaluate the Performance of the INSTI® HCV Antibody Test for Point-of-Care Use in Individuals at Risk for HCV Infection or Presenting With Signs or Symptoms of HCV Infection
HCV Infection
Not yet recruiting
Clinical Study to Evaluate the Performance of the INSTI® HCV Antibody Test for CLIA-Waived Use
HCV Infection
RecruitingObservational study
Physical Function and Fall Risk in Older Adults in the Lalín Frailty Prevention Program
Frailty, Fall Risk, Fall Prevention
Not yet recruiting
Effect of HMB Supplementation and Vivifrail Exercise on Body Composition, Handgrip Strength and Physical Performance in Women Over 60 Years of Age
Aging, Muscle Strength
Referral and contact
AzTrials does not enrol participants or take referrals. Contact the study team or a site listed below; they decide who can take part. This study is not recruiting at the moment.
Dimitrios Stefanidis, MD, PhD principal investigator
Carolinas Simulation Center
1 site in 1 country: see every site and its status.
Questions about this trial
What is this clinical trial studying?
A study testing skills training in people with Performance Assessment and Motion Metrics, run by Wake Forest University Health Sciences at 1 site in United States.
The full registered description is in the About this study section and the official record.
Who may be eligible to take part?
The sponsor lists 2 inclusion and 2 exclusion criteria. Eligibility is decided by the study team after screening, so read the full criteria and contact a listed site.
Is this clinical trial currently recruiting?
The record says: finished, as of the sponsor's last update on 27 Apr 2022. Status can change between updates, so confirm with the study contact before making plans.
Where is the study taking place?
The record lists 1 site in United States. Whether a site is taking part right now should be confirmed with that site. See every site.
What are the possible benefits and risks?
They depend on the study and the treatment, and no study can promise a benefit. The study team explains the known risks during informed consent, before anyone joins; the record itself does not list them.
Are results available for this study?
The study is completed, but results are not available in the source record. That does not mean the treatment did or did not work.
How can a doctor refer a patient or contact the research team?
The record lists no contact. The official record names the sponsor, who can be asked directly. AzTrials does not enrol participants or take referrals.
What does the primary outcome measure?
Laparoscopic suturing performance in the animal operating room (time frame: end of training and retention test after 3 months). It is what the study was designed to measure, not a finding.
Where can I read the original study information?
On the ClinicalTrials.gov record for NCT01052168, which also lists 35 linked publications.
Does a listing mean the treatment is approved or proven to work?
No. A listing means the study is registered. It does not show that the treatment is safe, works, or is approved by a regulator.
How often is this information updated?
AzTrials copies changed records from ClinicalTrials.gov every night and reloads the whole registry every week. The sponsor last updated this record on 27 Apr 2022.