CompetenceOPPE16NCT03229226
Development and Assessment of a Simulation Model of Focused and Ongoing Professional Practice Evaluations
An observational study of Competence and Emergencies, run by Rush University Medical Center.
Trial summary
Since 2007, the Joint Commission on Accreditation of Healthcare Organizations (JCAHO) has required evaluation of practicing attending physicians by management to ensure that procedural skills and medical care are meeting the accepted standards of care as assessed by each individual institution. Read more about this study
- Who is it for?
- Any ageHealthy volunteers can take part
- Conditions the study is for
- Competence, Emergencies
- What is being tested?
- Verbal versus procedural assessment
- How many people
- 17 participants
- Where
- No sites listed yet
- Study dates
- 1 Oct 2015to 1 May 2016
- Phase
- Observational
- Are there results?
- None posted in the registry
Key requirements
Who can take part, as listed by the sponsor. Only the study team can confirm whether you are eligible, so take this list to your doctor.
- Ages Any age
- All sexes
- Healthy volunteers accepted
- Child, Adult, Older adult
Study population: All faculty practicing in an urban emergency department were eligible for enrollment
Who may be able to join
- Emergency Medicine physician
Who cannot join
- Not Emergency Medicine physician
About this study
Since 2007, the Joint Commission on Accreditation of Healthcare Organizations (JCAHO) has required evaluation of practicing attending physicians by management to ensure that procedural skills and medical care are meeting the accepted standards of care as assessed by each individual institution. This assessment is known as the ongoing professional practice evaluation (OPPE). There are several methods by which this can be accomplished, including through chart review of clinician practice and by verbally assessing knowledge of the steps required to do a particular procedure. However, for infrequently performed or complicated procedures, these methods may not allow objective evaluation on a regular basis. Simulation using task trainers or manikin models offers an alternative method of objective evaluation in a standardized setting.
The goal of this study is to develop two simulated scenarios to assess physician skill in relatively uncommon procedures and compare the data obtained against the verbal assessment and chart review model previously used at our institution. Two raters will assess each practitioner's verbal and simulated procedures. They will also assess globally their confidence that the physician is capable of performing the procedure safely and correctly based on the verbal or simulated trial.
The investigators anticipate that using a simulated experience for assessment will increase the ability of raters to assess proficiency for the purposes of an OPPE, specifically by increasing the number of critical procedural components that can be objectively evaluated. The investigators hypothesize that there will not be a strong relationship between the scores obtained on the verbal assessment and the scores obtained on the task trainer assessment. The investigators anticipate that there will be a portion of study participants that do not meet a minimum passing standard and may require additional deliberate practice and further testing.
Read the detailed description
Physician training has developed over recent years to include simulation-based learning as a method of both formative and summative assessment. Simulation based medical education (SBME) with deliberate practice has been shown to be superior to traditional clinical education with respect to procedural skills in physicians in training (McGaghie,W.C. 2011), and a similar concept with Mastery Learning has shown to increase resident skill in lumbar puncture placement. In addition, it has been shown that Simulation Based Mastery Learning (SBML) in central line placement reduces hospital wide rates of central line associated bacterial infections.
When used in continuing medical education (CME), simulation has been shown to be feasible, and to increase clinician confidence in patient management. A recent study of nephrology fellows and attendings in practice showed that both groups scored similarly on a pre-test examination, and that all fellows who completed mastery learning subsequently passed the checklist assessment at the competency level and rated the training session highly.
In other work, simulation has been used for ongoing professional practice assessments (OPPE), for example in psychiatry using online simulated patients and assessments as a way of identifying a smaller number of attendings requiring focused assessment in person). Using electronically collected clinical data, anesthesia cases were reviewed for deviation from the standard of care on several key pieces of documentation common to many anesthesia procedures. This method of direct chart assessment for OPPE may be possible in anesthesia, a field where many cases have similarities in procedure, but would not work for a field such as Emergency Medicine, where the type and acuity of patient encounters varies greatly from day to day.
The ability of Simulation to present a standardized, specific procedural need offers an opportunity to assess Attending Emergency Physicians on a variety of critical procedures in a concise way. The investigators' research aims to present one possible approach to this assessment.
- Conditions
- Competence, Emergencies
- Being tested
- Verbal versus procedural assessment
- Keywords
- Simulation, Task Trainer
- Medical subject headings
- Emergencies
What participants receive
What participants receive. In a randomised study each person is assigned to one group by chance.
Verbal versus procedural assessmentOther
Faculty were assessed on both verbal and simulated performance of procedures.
| Study group | Role | What they receive |
|---|---|---|
| 2016 OPPE | All faculty participating in yearly ongoing professional Practice evaluation were eligible for enrollment Receives: Other: Verbal versus procedural assessment |
What is being measured
The primary outcome is the main question the study is designed to answer.
Performance on Procedural Checklist
A 29-item procedural checklist for Central Line and a 21-item procedural checklist for Lumbar puncture will be used to assess performance by clinicians on Central Venous Catheter insertion and Lumbar Puncture performance.
Measured: Immediate
Secondary outcomes (1)
Acceptance of simulated training and assessment format
A survey tool will be used to query participants on their preferred way of both practicing and being assessed on procedural skill. It will also assess whether participants feel inclined to complete a future simulated procedural training session.
Measured: Immediate
Timeline
From the start of the study to its completion; the mark is today. Estimated dates are the sponsor’s plan.
- First posted25 Jul 2017
- Study start1 Oct 2015
- Primary completion1 May 2016
- Study completion1 May 2016
Resources
- Official record on ClinicalTrials.govThe registry entry this page is built from, with its full change history.
- Publications (8)Papers the sponsor has linked to this study.
- Other trials for CompetenceStudies of the same condition, open and under way.
- More trials run by Rush University Medical CenterThe sponsor's other registered studies.
- What the trial phases meanHow a treatment moves from first tests in people to approval.
More details
For doctors and researchers: how the study is designed, who runs it, and its registry IDs.
Study design
- Study type
- Observational
- Observational model
- Case only
- Time perspective
- Other
- Enrolment
- 17 participants (actual)
Sponsor and investigators
- Lead sponsor
- Rush University Medical Center
- Sponsor type
- University, hospital or other
- Responsible party
- Sponsor
- Investigators
- Sara M Hock, MD, Principal investigator, Rush University Medical Center
IDs, dates and publications
- ClinicalTrials.gov ID
- NCT03229226
- Sponsor's protocol ID
- 15100805-IRB01-AM01
- Acronym
- OPPE16
- Status last verified
- Jul 2017
- First submitted
- 17 Jul 2017
- Last updated
- 28 Jul 2017
- Results posted
- No
- Data monitoring committee
- Yes
- FDA-regulated drug
- No
- FDA-regulated device
- No
- Individual data sharing
- Undecided: If requested, data may be shared. Limited generalizability of data may limit its usefulness to other investigators
Publications
OPPE and FPPE: Tools to help make privileging decisions 2013. Wise RA. (Accessed Jul 19, 2017, at http://www.jointcommission.org/jc_physician_blog/oppe_fppe_tools_privileging_decisions/).
McGaghie WC, Issenberg SB, Cohen ER, Barsuk JH, Wayne DB. Does simulation-based medical education with deliberate practice yield better results than traditional clinical education? A meta-analytic comparative review of the evidence. Acad Med. 2011 Jun;86(6):706-11. doi: 10.1097/ACM.0b013e318217e119.
PubMed 21512370Barsuk JH, Cohen ER, Caprio T, McGaghie WC, Simuni T, Wayne DB. Simulation-based education with mastery learning improves residents' lumbar puncture skills. Neurology. 2012 Jul 10;79(2):132-7. doi: 10.1212/WNL.0b013e31825dd39d. Epub 2012 Jun 6.
PubMed 22675080Barsuk JH, McGaghie WC, Cohen ER, O'Leary KJ, Wayne DB. Simulation-based mastery learning reduces complications during central venous catheter insertion in a medical intensive care unit. Crit Care Med. 2009 Oct;37(10):2697-701.
PubMed 19885989Barsuk JH, Cohen ER, Potts S, Demo H, Gupta S, Feinglass J, McGaghie WC, Wayne DB. Dissemination of a simulation-based mastery learning intervention reduces central line-associated bloodstream infections. BMJ Qual Saf. 2014 Sep;23(9):749-56. doi: 10.1136/bmjqs-2013-002665. Epub 2014 Mar 14.
PubMed 24632995Kerr B, Hawkins TL, Herman R, Barnes S, Kaufmann S, Fraser K, Ma IW. Feasibility of scenario-based simulation training versus traditional workshops in continuing medical education: a randomized controlled trial. Med Educ Online. 2013 Jul 18;18:21312. doi: 10.3402/meo.v18i0.21312.
PubMed 23870304McQuillan RF, Clark E, Zahirieh A, Cohen ER, Paparello JJ, Wayne DB, Barsuk JH. Performance of Temporary Hemodialysis Catheter Insertion by Nephrology Fellows and Attending Nephrologists. Clin J Am Soc Nephrol. 2015 Oct 7;10(10):1767-72. doi: 10.2215/CJN.01720215. Epub 2015 Sep 25.
PubMed 26408550Barsuk JH, Cohen ER, Nguyen D, Mitra D, O'Hara K, Okuda Y, Feinglass J, Cameron KA, McGaghie WC, Wayne DB. Attending Physician Adherence to a 29-Component Central Venous Catheter Bundle Checklist During Simulated Procedures. Crit Care Med. 2016 Oct;44(10):1871-81. doi: 10.1097/CCM.0000000000001831.
PubMed 27336437
Study at a glance
From the registry record, last updated by the sponsor on 28 Jul 2017. Recruitment status is as the sponsor reported it: confirm with the study team.
- Official title
- Development and Assessment of a Simulation Model of Focused and Ongoing Professional Practice Evaluations
- ClinicalTrials.gov ID
- NCT03229226
- Acronym / sponsor ID
- OPPE16 / 15100805-IRB01-AM01
- Conditions
- Competence, Emergencies
- Keywords
- Simulation, Task Trainer
- Study type
- Observational
- Recruitment status
- Completed
- Enrolment
- 17 participants (actual)
- Lead sponsor
- Rush University Medical CenterUniversity, hospital or other
- Study start
- 1 Oct 2015 (actual)
- Primary completion
- 1 May 2016 (actual)
- Study completion
- 1 May 2016 (actual)
- First posted
- 25 Jul 2017 (actual)
- Results
- Not posted in the source record
Clinical summary
The research question as the sponsor registered it. This describes what the study plans to test, not what it found.
Since 2007, the Joint Commission on Accreditation of Healthcare Organizations (JCAHO) has required evaluation of practicing attending physicians by management to ensure that procedural skills and medical care are meeting the accepted standards of care as assessed by each individual institution. This assessment is known as the ongoing professional practice evaluation (OPPE). There are several methods by which this can be accomplished, including through chart review of clinician practice and by verbally assessing knowledge of the steps required to do a particular procedure. However, for infrequently performed or complicated procedures, these methods may not allow objective evaluation on a regular basis. Simulation using task trainers or manikin models offers an alternative method of objective evaluation in a standardized setting.
The goal of this study is to develop two simulated scenarios to assess physician skill in relatively uncommon procedures and compare the data obtained against the verbal assessment and chart review model previously used at our institution. Two raters will assess each practitioner's verbal and simulated procedures. They will also assess globally their confidence that the physician is capable of performing the procedure safely and correctly based on the verbal or simulated trial.
The investigators anticipate that using a simulated experience for assessment will increase the ability of raters to assess proficiency for the purposes of an OPPE, specifically by increasing the number of critical procedural components that can be objectively evaluated. The investigators hypothesize that there will not be a strong relationship between the scores obtained on the verbal assessment and the scores obtained on the task trainer assessment. The investigators anticipate that there will be a portion of study participants that do not meet a minimum passing standard and may require additional deliberate practice and further testing.
Detailed description
Physician training has developed over recent years to include simulation-based learning as a method of both formative and summative assessment. Simulation based medical education (SBME) with deliberate practice has been shown to be superior to traditional clinical education with respect to procedural skills in physicians in training (McGaghie,W.C. 2011), and a similar concept with Mastery Learning has shown to increase resident skill in lumbar puncture placement. In addition, it has been shown that Simulation Based Mastery Learning (SBML) in central line placement reduces hospital wide rates of central line associated bacterial infections.
When used in continuing medical education (CME), simulation has been shown to be feasible, and to increase clinician confidence in patient management. A recent study of nephrology fellows and attendings in practice showed that both groups scored similarly on a pre-test examination, and that all fellows who completed mastery learning subsequently passed the checklist assessment at the competency level and rated the training session highly.
In other work, simulation has been used for ongoing professional practice assessments (OPPE), for example in psychiatry using online simulated patients and assessments as a way of identifying a smaller number of attendings requiring focused assessment in person). Using electronically collected clinical data, anesthesia cases were reviewed for deviation from the standard of care on several key pieces of documentation common to many anesthesia procedures. This method of direct chart assessment for OPPE may be possible in anesthesia, a field where many cases have similarities in procedure, but would not work for a field such as Emergency Medicine, where the type and acuity of patient encounters varies greatly from day to day.
The ability of Simulation to present a standardized, specific procedural need offers an opportunity to assess Attending Emergency Physicians on a variety of critical procedures in a concise way. The investigators' research aims to present one possible approach to this assessment.
Study design
- Observational model
- Case only
- Time perspective
- Other
- Number of arms
- 1
| Arm | Type | Receives |
|---|---|---|
| 2016 OPPE | – | Other: Verbal versus procedural assessmentAll faculty participating in yearly ongoing professional Practice evaluation were eligible for enrollment |
Who may qualify
- Sex
- All
- Healthy volunteers
- Accepted
- Age groups
- Child, Adult, Older adult
- Study population
- All faculty practicing in an urban emergency department were eligible for enrollment
Key inclusion criteria (1 listed)
- Emergency Medicine physician
Key exclusion criteria (1 listed)
- Not Emergency Medicine physician
Eligibility is decided by the study team after screening; this overview does not establish that a patient qualifies. Full inclusion and exclusion criteria.
Interventions
| Intervention | Type | Arms |
|---|---|---|
| Verbal versus procedural assessmentFaculty were assessed on both verbal and simulated performance of procedures. | Other | 2016 OPPE |
Primary outcome
Performance on Procedural Checklist
Time frame: Immediate
A 29-item procedural checklist for Central Line and a 21-item procedural checklist for Lumbar puncture will be used to assess performance by clinicians on Central Venous Catheter insertion and Lumbar Puncture performance.
1 secondary outcome
Acceptance of simulated training and assessment format
Time frame: Immediate
Results and safety
The study is completed, but results are not available in the source record. That does not mean the treatment did or did not work.
Research links
- Official record on ClinicalTrials.govLast updated by the sponsor 28 Jul 2017.
- BackgroundOPPE and FPPE: Tools to help make privileging decisions 2013. Wise RA. (Accessed Jul 19, 2017, at http://www.jointcommission.org/jc_physician_blog/oppe_fppe_tools_privileging_decisions/).
- Background: PubMed 21512370McGaghie WC, Issenberg SB, Cohen ER, Barsuk JH, Wayne DB. Does simulation-based medical education with deliberate practice yield better results than traditional clinical education? A meta-analytic comparative review of the evidence. Acad Med. 2011 Jun;86(6):706-11. doi: 10.1097/ACM.0b013e318217e119.
- Background: PubMed 22675080Barsuk JH, Cohen ER, Caprio T, McGaghie WC, Simuni T, Wayne DB. Simulation-based education with mastery learning improves residents' lumbar puncture skills. Neurology. 2012 Jul 10;79(2):132-7. doi: 10.1212/WNL.0b013e31825dd39d. Epub 2012 Jun 6.
- Background: PubMed 19885989Barsuk JH, McGaghie WC, Cohen ER, O'Leary KJ, Wayne DB. Simulation-based mastery learning reduces complications during central venous catheter insertion in a medical intensive care unit. Crit Care Med. 2009 Oct;37(10):2697-701.
- Background: PubMed 24632995Barsuk JH, Cohen ER, Potts S, Demo H, Gupta S, Feinglass J, McGaghie WC, Wayne DB. Dissemination of a simulation-based mastery learning intervention reduces central line-associated bloodstream infections. BMJ Qual Saf. 2014 Sep;23(9):749-56. doi: 10.1136/bmjqs-2013-002665. Epub 2014 Mar 14.
- Background: PubMed 23870304Kerr B, Hawkins TL, Herman R, Barnes S, Kaufmann S, Fraser K, Ma IW. Feasibility of scenario-based simulation training versus traditional workshops in continuing medical education: a randomized controlled trial. Med Educ Online. 2013 Jul 18;18:21312. doi: 10.3402/meo.v18i0.21312.
- Background: PubMed 26408550McQuillan RF, Clark E, Zahirieh A, Cohen ER, Paparello JJ, Wayne DB, Barsuk JH. Performance of Temporary Hemodialysis Catheter Insertion by Nephrology Fellows and Attending Nephrologists. Clin J Am Soc Nephrol. 2015 Oct 7;10(10):1767-72. doi: 10.2215/CJN.01720215. Epub 2015 Sep 25.
- Background: PubMed 27336437Barsuk JH, Cohen ER, Nguyen D, Mitra D, O'Hara K, Okuda Y, Feinglass J, Cameron KA, McGaghie WC, Wayne DB. Attending Physician Adherence to a 29-Component Central Venous Catheter Bundle Checklist During Simulated Procedures. Crit Care Med. 2016 Oct;44(10):1871-81. doi: 10.1097/CCM.0000000000001831.
- Individual participant dataSharing planned: Undecided. If requested, data may be shared. Limited generalizability of data may limit its usefulness to other investigators
Related trials for Competence
Not yet recruiting
SELF-ASSESSMENT AND KNOWLEDGE LEVEL OF NURSING STUDENTS' PATIENT SAFETY COMPETENCY
Simulation Training, Nursing Education
Recruiting
IMPLEMENTING PATIENTS' COMPETENCE IN ORAL EBC THERAPY PERSISTENCE
HR+/HER2- Early Breast Cancer
Not yet recruiting
Compression-Only Versus Standard Basic Life Support Training in Nursing Students
Cardiopulmonary Resuscitation, Out-of-Hospital Cardiac Arrest
Enrolling by invitationObservational study
Follow-Up Evaluation for Gene-Therapy-Related Delayed Adverse Events After Participation in Pediatric Oncology Branch Clinical Trials
Pediatric Cancers, Hematologic Malignancies
Not yet recruiting
Culturally-Tailored AI Chatbot Simulation for Therapeutic Communication and Empathy in Psychiatric Nursing Students
Therapeutic Communication
Referral and contact
AzTrials does not enrol participants or take referrals. Contact the study team or a site listed below; they decide who can take part. This study is not recruiting at the moment.
Sara M Hock, MD principal investigator
Rush University Medical Center
Questions about this trial
What is this clinical trial studying?
An observational study of Competence and Emergencies, run by Rush University Medical Center.
The full registered description is in the About this study section and the official record.
Who may be eligible to take part?
The sponsor lists 1 inclusion and 1 exclusion criteria. Eligibility is decided by the study team after screening, so read the full criteria and contact a listed site.
Is this clinical trial currently recruiting?
The record says: finished, as of the sponsor's last update on 28 Jul 2017. Status can change between updates, so confirm with the study contact before making plans.
What are the possible benefits and risks?
They depend on the study and the treatment, and no study can promise a benefit. The study team explains the known risks during informed consent, before anyone joins; the record itself does not list them.
Are results available for this study?
The study is completed, but results are not available in the source record. That does not mean the treatment did or did not work.
How can a doctor refer a patient or contact the research team?
The record lists no contact. The official record names the sponsor, who can be asked directly. AzTrials does not enrol participants or take referrals.
What does the primary outcome measure?
Performance on Procedural Checklist (time frame: Immediate). It is what the study was designed to measure, not a finding.
Where can I read the original study information?
On the ClinicalTrials.gov record for NCT03229226, which also lists 8 linked publications.
Does a listing mean the treatment is approved or proven to work?
No. A listing means the study is registered. It does not show that the treatment is safe, works, or is approved by a regulator.
How often is this information updated?
AzTrials copies changed records from ClinicalTrials.gov every night and reloads the whole registry every week. The sponsor last updated this record on 28 Jul 2017.