A Randomized Controlled Trial of a Robotic Simulation Curriculum to Teach Robotic Suturing

BACKGROUND AND RATIONALE

A number of pressures make it challenging to learn robotic suturing in the operating room including budgetary constraints necessitating near maximum operating room efficiency, concerns about patient safety, and decreased trainee working hours. Robotic surgery is a novel surgical technique which has significantly decreased surgical morbidity compared to open surgery.

Robotic simulation is a technique that enables skill acquisition in robotic surgery without the constraints of the operating room. The Mimic® dv-Trainer TM is a novel virtual reality robotic surgery simulator which first became available in 2007. Further refinements and developments led to incorporation of the Mimic software into the da Vinci® Skills Simulator. This simulator has been shown to have face validity (appears realistic to experienced surgeons) and construct validity (surgeons with more experience in robotic surgery score higher than less experienced surgeons when performing the simulation exercises). The impact of training using the da Vinci® Skills Simulator on performance using the actual surgical robot (da Vinci® Surgical System) has started to be studied. One study assessed the impact of training using the da Vinci® Skills Simulator on performance of three animal tissue based tasks using the da Vinci® Si HD TM Surgical System (Hung, A.J., et al., Concurrent and predictive validation of a novel robotic surgery simulator: a prospective, randomized study. Journal of Urology, 2012. 187(2): p. 630-7). One of three tasks assessed was suturing closed a hole in an injured animal bladder. This study showed statistically significant improvement in the animal tissue exercises for the subgroup of participants who started with scores in the lower 50th percent for those exercises.

Our study will differ from this study in several ways. First, we will be focusing on a more select group of simulation exercises focusing on robotic suturing (six exercises rather than seventeen). Second, we will be developing a proficiency curriculum rather than one based on time spent training. Third, our primary outcome will be performance of suturing, using the actual surgical robot (da Vinci® Surgical System), of a vaginal cuff model made out of inanimate materials rather than an animal tissue model. Due to logistic reasons, the suturing of the inanimate model will be performed with the da Vinci® S Surgical System at the March 23rd testing date (pretest for group 1) and with the da Vinci® Si Surgical System at the April 27 testing date (pretest for group 2 and post-test 1 for group 1) and at the June 1 testing date (post-test for group 2 and post-test 2 for group 1 controls who then cross over to training in May). Our study will be open to trainees and surgeons from obstetrics and gynecology, general surgery, urology, and cardiac surgery rather than just urology.

SAMPLE SIZE CALCULATIONS

Our sample size calculations suggest the need for 38 participants, 19 in each group. This was calculated using PASS 11. The plan is to compare the following: (a) control group scores at final evaluation minus control group scores at initial evaluation versus (b) training group scores at final evaluation minus training group scores at initial evaluation. Thus, we assume two independent groups (because the differences in the control group should be independent of the differences in the training group). For our sample size calculation, we select the Mann-Whitney test and assume a mean difference between the improvement in scores in the training group versus the improvement of scores in the control group of 7 (in the context of a maximum possible score of 70 on any possible evaluation). We will also assume a standard deviation of the differences of 7, an alpha of 0.05, and a power 0.8. The non-parametric Mann-Whitney test (equivalent to the Wilcoxon Rank Sum Test) for comparing the ranks of independent groups forms the basis for the estimates.

If we look at a sample size calculation for paired means (for example training group at initial assessment versus post-training), we calculate the need for a sample size of 16 in the training group (32 total with controls). The same assumptions for difference in means, standard deviation, alpha and beta error are made as above. The Wilcoxon (signed rank) test for comparing ranks of two paired groups forms the basis of the estimate.

RANDOMIZATION AND ALLOCATION CONCEALMENT

Randomization was achieved with an online random number generator (www.randomization.com) to create a random sequence of train versus control for 40 participants using randomly permuted blocks of 4 or 6 with a ratio of participants of 1:1 train:control in each block. The randomization sequence was generated by an associate not otherwise involved with the study. The group assignment (train or control) was written by this associate on an index card and then a piece of carbon paper was stapled onto the front of the index card and it was placed in an opaque, sequentially numbered, sealed envelope for opening by the participants in the order of arrival on the pre-test day. Participants write their name on the front of the envelope before opening so that this is transmitted by the carbon paper to the index card. Two participants initially gave consent to participate on the condition that they would train in a given 5 week period but then changed their mind on the day of the pre-test and agreed to be randomized.

DESIGN AND DESCRIPTION OF METHODOLOGY

After obtaining informed consent, participants will undergo baseline testing on two domains: (1) virtual reality robotic simulation using a da Vinci® Skills Simulator and (2) robotic suturing of an inanimate model using the da Vinci® Surgical System. For testing using the da Vinci® Skills Simulator, participants will perform the following task: Suture sponge 1. Scoring will be done automatically by the built-in algorithm of the da Vinci® Skills Simulator. Baseline scores for the other da Vinci® Skills Simulator tasks will be obtained during the first training session. It is due to time constraints on the testing days that we limit assessment with the da Vinci® Skills Simulator to the Suture Sponge 1 task alone.

For baseline physical (as opposed to virtual reality) robotic simulation testing, participants will suture a vaginal cuff model using the actual surgical robot, the da Vinci® Surgical System. Due to logistic reasons, the da Vinci® S Surgical System will be used for the testing on March 23 (pre-test for group 1) and the da Vinci® Si Surgical System will be used for the testing on April 27 (post-test 1 for group 1 and pre-test for group 2) and June 1 (post-test 2 for group 1 controls who do cross-over training in the period April 27 to May 31 and post-test for group 2). The vaginal cuff model is built from inexpensive inanimate materials based on a prototype from the Journal of Robotic Surgery[Finan MA, C.M., Rocconi, A novel method for training residents in robotic hysterectomy. Journal of Robotic Surgery, 2010. 4(1): p. 33-39.]. Their performance of this task will be recorded using the built-in video camera of the da Vinci® Surgical System. Participants will be given a ten minute time limit to try to suture as many figure of 8 knots in the model of the vaginal cuff as possible. They are instructed to place a double throw and then three single throws for each knot. At study conclusion, one expert robotic surgeon and one gynecologic oncology fellow, blinded to participant identity and group randomization status, will grade the performance of this task. In order to ensure blinding, the videorecordings will be arranged in random order by a member of the study team (who is not involved in grading performance) in such a way that no content is lost but so that the sequence of participants suturing and whether this was a pre- or post-intervention recording is randomly arranged using a random number generator. The original study codes will also be removed from this random sequence of videos and a new study code will be attached to each video. A master sheet will be kept to re-assign the original codes but will not be available to the evaluators until after grading is completed. To score the videotaped performance, grading will performed using the GOALS score for laparoscopy [Vassiliou, M.C., et al., A global assessment tool for evaluation of intraoperative laparoscopic skills. American Journal of Surgery, 2005. 190(1): p. 107-13.] plus two additional metrics described by AJ Hung [Hung, A.J., et al., Concurrent and predictive validation of a novel robotic surgery simulator: a prospective, randomized study. Journal of Urology, 2012. 187(2): p. 630-7.] specifically for robotics: 1) precision and 2) instrument and camera awareness.

Following initial testing, participants will be randomized to two groups: 1)participation in the da Vinci® Skills Simulator-based robotic suturing simulation proficiency curriculum in addition to usual clinical/residency practice or 2) usual clinical/residency practice alone. The robotic suturing simulation curriculum is a "proficiency" curriculum in that it is focused on the goal of achieving proficiency targets for five of the six tasks included in the curriculum. Those five tasks are : Camera targeting 1, Camera targeting 2, Suture sponge 1, Suture sponge 2, and Suture sponge 3. For these five exercises, we will use the latest version of the MScore TM assessment. MScore TM is part of the software accompanying the da Vinci® Skills Simulator which provides scoring of task performance. We will ask participants in the training group to aim to complete these exercises to the target score (overall score green checkmark, equivalent to a "completed score") on two nonconsecutive attempts. For the "Suturing skills (Symbionix TM): Horizontal suturing defect" task, the scoring is different and adjusts to the participant's prior attempts, indicating whether improvement has been made. Thus, we have elected to ask participants to aim to complete this task ten times, rather than aiming for a particular score. The curriculum is designated "proficiency-based" (with the exception of the Horizontal suturing defect task) in that the goal is to achieve the target scores rather than to complete a set amount of time training or a set number of repetitions. However, to provide some structure, we will recommend at least five hours total of training sessions over the five week study period. We will also facilitate the organization and scheduling of additional training time during the study period, if desired by individual participants in the training group. In terms of simulator access, participants in the training group will be using a da Vinci® Skills Simulator attached to the mentoring console of a da Vinci Si robot. This da Vinci® Si robot is used for actual surgery during regular working hours. In general this is Monday-Friday during the regular, elective surgery schedule. Thus, the da Vinci® Skills Simulator will, in general (unless elective robotic cases are cancelled or not scheduled that day), be available to participants only after the elective robotic surgeries of the day are completed on week days. On Saturday and Sunday, the da Vinci® Skills Simulator will generally be available to participants all day.

The control group will carry on with regular clinical work or residency training. After the intervention period, we will then re-evaluate all participants in the same da Vinci® Skills Simulator task (Suture Sponge 1) and in the task of suturing the vaginal cuff model using the da Vinci® S Surgical System. Testing of the task of suturing the vaginal cuff model using the da Vinci® Surgical System will be graded in the same manner as the pretesting: one expert robotic surgeon, one surgeon with intermediate robotic experience, and one gynecologic oncology fellow (blinded to participant group and to whether the task was performed pre- or post-intervention) will review the videotaped recordings. {*Note that we initially planned to have the videos graded by one expert robotic surgeon and one gynecologic fellow. We decided to add grading by one surgeon with intermediate robotic experience. This decision was made to ensure at least two sets of grading were completed by our group's deadline of Sept 11th, to strengthen the validity of our results, and to allow better assessment of inter-observer agreement. The decision to add a third rater was made when only the first set of data was in (expert robotic surgeon) and before data was decoded or analyzed (such that it was not known whether any score applied to a participant in the pre- or post- or in the training or control group). The first complete set of video grades was received on Aug 4 (expert robotic surgeon), the second set on Sept 8 (gynecologic oncology fellow), and the third set on Sept 10 (surgeon with intermediate robotic experience). On Sept 11, the process of decoding the randomized order of the data began so that the link between scores and whether the participant was pre- or post-intervention and in the control group or training group was revealed in order to begin analysis.} Following this, the control group that started March 23rd will be allowed to crossover and perform the proficiency-based curriculum. Finally, this group (originally controls, then crossed over to training) will be tested one last time in the simulation task "Suture Sponge 1" using the da Vinci® Skills Simulator and in the task of robotic suturing of the vaginal cuff model using the da Vinci® Surgical System. The robotic suturing of the vaginal cuff model will again be assessed by review of videotapes by blinded experts. In order to organize the training sessions participants will be given the investigator's contact information (DK). Access to the da Vinci® Skills Simulator will be organized by a proctor familiar with the da Vinci® Skills Simulator (DK), who will be present to facilitate access to the simulator (given that it is located in the operating room) and to supervise.

There are two separate dates on which participants can enter the study and be randomized. They are: March 23 or April 27, 2013. Participants entering the study March 23 will have pretesting that day and be randomized to training March 24 to April 26 or April 28 - May 31, 2013. All participants entering the study March 23 will be asked to participate in a posttest on April 27, 2013. Those participants randomized to training April 29 to May 31 (that is, the initial control group participants which elect to crossover to training in the next period) will be asked to participate in a second post-test on June 1, 2013.

Participants who enter the study on April 27, 2013 will have a pretest that day and then be randomized to training April 29 to May 31 or control with optional training June 2 to June 28. All participants entering the study April 27 will be asked to participate in a post-test on June 1, 2013. Those participants entering the study on April 27 and randomized to the control group April 28 to May 31 will have the option to crossover and complete training June 2-28. However, at present, no post-test is planned for after this training period (June 2 - 28) due to logistic reasons.

DATA ANALYSIS PLAN

There are three time periods for each of the two entry points (March 23 and April 27) in this study: time 0 = initial testing; time 1 = post-intervention period; time 2 = post-cross-over intervention period. To clarify, time 1 will occur at the time of retesting following the initial study period of training or no training. Time 2 will occur after the control group crosses over, completes training, and is then retested. A subgroup analysis of the participants starting with pretest scores in the lower 50% is planned.

Independent group comparisons will also be conducted using the Wilcoxon rank sum test. Independent group comparisons include (a) the differences from time 0 to time 1 for training group versus (b) the differences from time 0 to time 1 for the control group. Paired group comparisons will be conducted with the Wilcoxon signed rank test. Paired group comparisons include: training group at time 0 versus at time 1 and control group with cross-over training at time 2 versus at time 1 and at time 1 versus time 0.