Mixed-reality REBOA Simulator With Virtual Coaching

Methods:

Study design: This is a multi-center randomized simulation study. Setting, location, participants: This multi-center study will be held at University of Florida (UF) Health and Madigan Health centers. Planned time is between June 01 to October 31, 2021. The study population will be trainees (students, residents, fellows, and physicians) working at UF Health Center and Madigan Health Center. Exclusion criteria are: i. not obtaining informed consent, ii. inability to complete all the procedures.

Timeframe of training and data collection: June 01 to October 31, 2021 Learner or trainee population: Learners will be healthcare staff in two hosting institutions. Trainers will be the academic staff from UF Health and Madigan Medical Center.

Measurements:

Development and pilot testing of assessment tool: REBOA simulator will be developed at the Center for Safety, Simulation, and Advanced Learning Technologies (CSSALT). Pilot testing will be on primary team members. The trainer will play confederate nurse role and assist the participant.

Psychometric properties of previously developed and pilot tested tools: The hosting center (CSSALT) has developed a central venous access (CVA) simulator with a virtual coach. The same simulator development technology (SMMARTS) will be used in the proposed project. Study team has prior data regarding psychometric properties of previously developed simulators.

Rater training to use assessment tools and scores consistently: The simulator has a limit for the number of instruments (4) to be tracked. Therefore, the it will track the procedures that occur under the skin out of a human instructor's sight by the simulator's software, whereas the procedures that are visible by the trainers are not automatically tracked by the software. There will therefore be two rating systems: One is by trainers or assistants (T) and the other is by the simulator's software (SS). Rating by SS will be electronically and automatically scored. The study team will train the in-person trainers prior to study to score the learners consistently. For this purpose, the study will plan an online training for trainers prior to the data collection phase to obtain consistent rating of the participants. Based on the current literature, ratings will be based on these items and rating systems, which are given in parentheses for each item:

1. Sheath insertion with Seldinger technique i. Sheath priming (T) ii. Ultrasound-guided puncture (SS) iii. Sheath guidewire insertion (T) iv. Sheath insertion: Participants receive "fail" grade if they did not complete this item successfully to common femoral artery (CFA) or inserted to the common femoral vein (CFV) (SS) b. Balloon insertion i. Balloon priming (T and SS) ii. Balloon test: Participants receive "fail" grade if they inflated the balloon during preparation (T and SS) iii. Balloon position measurement using body landmarks (SS) iv. Balloon inflation (SS) v. Final balloon position (SS)

In addition to the reported items above, we will track several items to build a better training system. The virtual coaching system of the SMMARTS simulator will automatically capture and store the data required for the study including:

1. Balloon inflation volume: We developed a syringe simulator system. (SS) b. Inflation of the balloon with saline [Zone 1: 8 mL (cc) Zone 3: 2 mL (cc)] (SS) c. Observe the simulated blood pressure monitor to see the increase of the blood pressure. (SS) d. Timeframes (SS) e. Back walling (SS) f. Precise discrimination of the Zone where the balloon was inflated. (SS) g. Puncturing: i. Needle tip remains in femoral artery from time of arterial access to guidewire placement (SS) ii. Needle tip completely in US plane at time of arterial puncture. (SS) iii. Needle angle to the skin surface at vascular access. (SS) iv. Advancing needle while the tip is outside the ultrasound insonation plane. (SS)

Standard setting methodology

Clear definition of mastery: Mastery is defined as two consecutive successful applications of Prytime REBOA catheter in two different Zones; one is for Zone 1 and the other is to Zone 3. Using right or left groin does not matter. The participant can apply on the same side, or do the second try using the other side.

Description of scoring rules: Scoring will be based on the measurements by the simulator software and ratings by trainer. These items include but are not limited to:

Clear definition of how minimum passing standards were set: Minimum passing standards are set as:

Insert two REBOA catheters Without complications (venous puncture, inflating balloon in wrong zone or volume) Within 240 seconds from hitting the start button to secure the REBOA Interventions

Detailed description of curriculum development including:

Teaching tools: An experienced clinician will be the instructor with online training. The instructor will explain the study and simulator through an online presentation to the participants. Volunteers will sign the informed consent if they agree to participate. The instructor will present an online didactic course to participants. Online training will be recorded and available to the participants at any stage of the study for their reference. Then participants will take the "Guay Visualization of Views Test" (limited to 8 minutes) to assess their spatial abilities. Afterwards, participants will start hands-on training. One simulator will be set up in a specific room for the study with the required equipment. Participants will practice one-by-one, and they will be able to schedule hands-on training time based on mutual availability of the instructor and participant. The training intervention is expected to be completed in 4 months. Every participant will practice until mastery is achieved. (see Figure 1). There will be no control group.

Participants will be randomized into two groups (see Figure 1) with a simple randomization using a random number table. One group will be trained by an average human instructor and the other group will be trained by the virtual coach. In some institutions like UF, there are scoring systems for the assessment of instructors, e.g., by residents. Some instructors are excellent, but some are not. The instructor's performance may affect the results of the simulation studies. Therefore the study proposes to include instructors within the normal ranges of assessment (not the worst or the best) and term them as 'average instructor.' Baseline assessment: First try of the participants at placing a REBOA on the simulator will be the baseline measurement.

Nature of practice (e.g. duration, intensity, individuals or team): The hands-on training session to mastery is expected to be under 3 hours for each participant.

Feedback/debriefing (source, type, frequency): After each try, the instructor or virtual coach will provide feedback and a 3-minute debriefing based on the simulator's software measurements.

Post-training assessment with minimum passing standards (MPS): After every try, the trainer and the simulator's software will report the participant's score based on MPS Stakes for participants: During the COVID-19 pandemic, participants run the risk of infection during training. We will conduct online orientation and didactic training via cloud-based video communications apps. Participants will undergo hands-on training sessions individually. If the pandemic is not resolved, both instructor and trainee will stay 6 feet apart and use face covers. Hands-on training may also be risky for COVID-19 infection. Participants will take the training one-by-one and clean the simulator with disinfectant after completing the training. The simulator and the handheld tools will be cleaned and wiped down with 55% Isopropyl Alcohol between participants. This study will be voluntary. If a trainee or a staff member opts not to participate, they will not be penalized in any way.

Outcome measures

Primary outcome is to evaluate the non-inferiority of the REBOA simulator with a virtual coach.

Secondary outcomes are:

1. Define the time required for a complete REBOA insertion on a simulator ii. Define the time required the time to mastery iii. Define the complication rates in the simulator iv. Association of success rates and complications on the simulator to spatial ability

Sample size: The study conducted a power analysis/sample size calculation for a non-inferiority analysis on the difference in two independent proportions, assuming α=.05, power=.95, a high success rate expected for both groups (99%) and a non-inferiority margin of 15% (https://www.sealedenvelope.com/power/binary-noninferior/ [Accessed March 26, 2021]). Sample size required per group is 10. The investigators decided to include a total of 28 participants to the study. The non-inferiority margin is set so that a difference bigger than this limit would matter in clinical practice. It is usually assumed that the success proportion is the same in both groups unless there is good reason to believe that one treatment is, in fact, superior.

Statistical methods

Descriptive analyses: The investigators will use SPSS for statistical analysis. Yahya A. Acar, MD, will perform the analysis. Demographic data will be presented as frequencies and percentages. Continuous variables will be presented as mean and standard deviation.

Analyses used to assess: The chi-square test will be used for the comparison of two groups concerning categorical variables. Wilcoxon signed-rank test and the dependent t-test will be used for pairwise comparisons (first and last measurements) of continuous data that do not conform to a normal distribution. Logistic regression analysis will be used to identify factors independently predictive of success. Those factors remaining in the model will be used in a set of linear scores. Receiver operating characteristic curves (ROC) will be constructed for each model, and the areas under the curve will be calculated.