Faculty DevelopmentNCT00667940
Faculty Development to Improve Mini-CEX Ratings
A study testing Rater training workshop in people with Faculty Development, run by Mayo Clinic at 1 site in United States.
Trial summary
We will conduct a faculty development workshop with intent to improve interrater reliability and rater accuracy. Outcomes will include interrater reliability, accuracy, and workshop evaluations. Read more about this study
- Who is it for?
- Any ageHealthy volunteers can take part
- Conditions the study is for
- Faculty Development
- What is being tested?
- Rater training workshop
- Type of treatment
- Lifestyle or therapy programme
- Could I receive a placebo?
- No, everyone receives an active treatment
- How many people
- 52 participants
- Where
- 1 siteUnited States
- Study dates
- Mar 2006to Jun 2007
- Phase
- Interventional
- Are there results?
- None posted in the registry
Key requirements
Who can take part, as listed by the sponsor. Only the study team can confirm whether you are eligible, so take this list to your doctor.
- Ages Any age
- All sexes
- Healthy volunteers accepted
- Child, Adult, Older adult
Who may be able to join
- Internal medicine residency continuity clinic preceptor at Mayo Clinic Rochester Minnesota
Who cannot join
- None
Trial locations and contacts
1 site in 1 country.
About this study
Official title: Effect of Faculty Development on Mini-CEX Score Reliability and Accuracy: A Randomized, Controlled Trial
We will conduct a faculty development workshop with intent to improve interrater reliability and rater accuracy. Outcomes will include interrater reliability, accuracy, and workshop evaluations.
Read the detailed description
Hypothesis: An intervention for training faculty preceptors on the use of the mini-CEX will improve inter-rater reliability and classification accuracy.
Methods. We will develop a faculty development workshop aimed at improving the inter-rater reliability and accuracy of mini-CEX scores. To evaluate the workshop we will conduct a randomized, controlled trial. Preceptors will be randomly assigned to immediate or delayed workshop. The primary outcome will be inter-rater reliability of mini-CEX scores as preceptors rate videotaped resident-patient encounters before and four weeks after the workshop. We will also evaluate the accuracy of performance classifications. We will also collect mini-CEX ratings of actual resident-patient interactions one year preceding and one year following the workshop, and calculate interrater reliability.
Statistical methods. Inter-rater reliability will be determined using the intraclass correlation coefficient (ICC) and corresponding confidence intervals. ICC will be used to determine chance-corrected agreement (accuracy). Discrimination (differences in mean scores among scripted levels of performance) will also be evaluated as a measure of accuracy.
Scientific basis. Clinical performance assessments constitute a key element of resident education. The effect of rater training on the inter-rater reliability of mini-CEX scores is unknown. This workshop will utilize methods (rater error and field of reference training) shown to be effective in improving rater accuracy in non-medical fields.
Inclusion criteria. All internal medicine residency continuity clinic preceptors are eligible.
- Conditions
- Faculty Development
- Being tested
- Rater training workshop
- Keywords
- Medical education, faculty development, rater training, clinical competence, assessment
What participants receive
What participants receive. In a randomised study each person is assigned to one group by chance.
Rater training workshopBehavioural
Rater training workshop: rater error training, performance dimension training, behavioral observation training, and frame of reference training using lecture, video, and facilitated discussion
| Study group | Role | What they receive |
|---|---|---|
| Workshop | Experimental | Rater training workshop: rater error training, performance dimension training, behavioral observation training, and frame of reference training using lecture, video, and facilitated discussion Receives: Behavioral: Rater training workshop |
| Delayed workshop | No intervention | No specific intervention until after follow-up assessment, then receives same workshop. |
What is being measured
The primary outcome is the main question the study is designed to answer.
Interrater reliability (videotaped resident-patient encounters)
Measured: April-June 2006
Secondary outcomes (2)
Rater accuracy (agreement with scripted levels of performance on videotaped resident-patient encounters): percent agreement, chance-corrected agreement, mean scores
Measured: April-June 2006
Interrater reliability (actual resident-patient encounters)
Measured: March 2005 (historical) to June 2007
Timeline
From the start of the study to its completion; the mark is today. Estimated dates are the sponsor’s plan.
- First posted28 Apr 2008 (estimated)
- Study startMar 2006
- Primary completionJun 2006
- Study completionJun 2007
Resources
- Official record on ClinicalTrials.govThe registry entry this page is built from, with its full change history.
- Publications (1)Papers the sponsor has linked to this study.
- Other trials for Faculty DevelopmentStudies of the same condition, open and under way.
- More trials run by Mayo ClinicThe sponsor's other registered studies.
- What the trial phases meanHow a treatment moves from first tests in people to approval.
More details
For doctors and researchers: how the study is designed, who runs it, and its registry IDs.
Study design
- Study type
- Interventional
- Phase
- Phase N/A
- Primary purpose
- Treatment
- Allocation
- Randomised
- Intervention model
- Parallel groups
- Masking
- None (open label)
- Enrolment
- 52 participants (actual)
Sponsor and investigators
- Lead sponsor
- Mayo Clinic
- Sponsor type
- University, hospital or other
- Investigators
- David A Cook, MD, MHPE, Principal investigator, Mayo Clinic
IDs, dates and publications
- ClinicalTrials.gov ID
- NCT00667940
- Sponsor's protocol ID
- 06-002730
- Status last verified
- Apr 2008
- First submitted
- 23 Apr 2008
- Last updated
- 28 Apr 2008
- Results posted
- No
- Data monitoring committee
- No
Publications
Cook DA, Dupras DM, Beckman TJ, Thomas KG, Pankratz VS. Effect of rater training on reliability and accuracy of mini-CEX scores: a randomized, controlled trial. J Gen Intern Med. 2009 Jan;24(1):74-9. doi: 10.1007/s11606-008-0842-3. Epub 2008 Nov 11.
PubMed 19002533
Study at a glance
From the registry record, last updated by the sponsor on 28 Apr 2008. Recruitment status is as the sponsor reported it: confirm with the study team.
- Official title
- Effect of Faculty Development on Mini-CEX Score Reliability and Accuracy: A Randomized, Controlled Trial
- ClinicalTrials.gov ID
- NCT00667940
- Acronym / sponsor ID
- 06-002730
- Conditions
- Faculty Development
- Keywords
- Medical education, faculty development, rater training, clinical competence, assessment
- Study type
- Interventional
- Recruitment status
- Completed
- Enrolment
- 52 participants (actual)
- Lead sponsor
- Mayo ClinicUniversity, hospital or other
- Study start
- Mar 2006
- Primary completion
- Jun 2006 (actual)
- Study completion
- Jun 2007 (actual)
- First posted
- 28 Apr 2008 (estimated)
- Results
- Not posted in the source record
Clinical summary
The research question as the sponsor registered it. This describes what the study plans to test, not what it found.
We will conduct a faculty development workshop with intent to improve interrater reliability and rater accuracy. Outcomes will include interrater reliability, accuracy, and workshop evaluations.
Detailed description
Hypothesis: An intervention for training faculty preceptors on the use of the mini-CEX will improve inter-rater reliability and classification accuracy.
Methods. We will develop a faculty development workshop aimed at improving the inter-rater reliability and accuracy of mini-CEX scores. To evaluate the workshop we will conduct a randomized, controlled trial. Preceptors will be randomly assigned to immediate or delayed workshop. The primary outcome will be inter-rater reliability of mini-CEX scores as preceptors rate videotaped resident-patient encounters before and four weeks after the workshop. We will also evaluate the accuracy of performance classifications. We will also collect mini-CEX ratings of actual resident-patient interactions one year preceding and one year following the workshop, and calculate interrater reliability.
Statistical methods. Inter-rater reliability will be determined using the intraclass correlation coefficient (ICC) and corresponding confidence intervals. ICC will be used to determine chance-corrected agreement (accuracy). Discrimination (differences in mean scores among scripted levels of performance) will also be evaluated as a measure of accuracy.
Scientific basis. Clinical performance assessments constitute a key element of resident education. The effect of rater training on the inter-rater reliability of mini-CEX scores is unknown. This workshop will utilize methods (rater error and field of reference training) shown to be effective in improving rater accuracy in non-medical fields.
Inclusion criteria. All internal medicine residency continuity clinic preceptors are eligible.
Study design
- Primary purpose
- Treatment
- Allocation
- Randomised
- Intervention model
- Parallel groups
- Masking
- None (open label)
- Number of arms
- 2
| Arm | Type | Receives |
|---|---|---|
| Workshop | Experimental | Behavioral: Rater training workshopRater training workshop: rater error training, performance dimension training, behavioral observation training, and frame of reference training using lecture, video, and facilitated discussion |
| Delayed workshop | No intervention | –No specific intervention until after follow-up assessment, then receives same workshop. |
Who may qualify
- Sex
- All
- Healthy volunteers
- Accepted
- Age groups
- Child, Adult, Older adult
Key inclusion criteria (1 listed)
- Internal medicine residency continuity clinic preceptor at Mayo Clinic Rochester Minnesota
Key exclusion criteria (1 listed)
- None
Eligibility is decided by the study team after screening; this overview does not establish that a patient qualifies. Full inclusion and exclusion criteria.
Interventions
| Intervention | Type | Arms |
|---|---|---|
| Rater training workshopRater training workshop: rater error training, performance dimension training, behavioral observation training, and frame of reference training using lecture, video, and facilitated discussion | Behavioural | Workshop |
Primary outcome
Interrater reliability (videotaped resident-patient encounters)
Time frame: April-June 2006
2 secondary outcomes
Rater accuracy (agreement with scripted levels of performance on videotaped resident-patient encounters): percent agreement, chance-corrected agreement, mean scores
Time frame: April-June 2006
Interrater reliability (actual resident-patient encounters)
Time frame: March 2005 (historical) to June 2007
Results and safety
The study is completed, but results are not available in the source record. That does not mean the treatment did or did not work.
Research links
- Official record on ClinicalTrials.govLast updated by the sponsor 28 Apr 2008.
- Derived from this study: PubMed 19002533Cook DA, Dupras DM, Beckman TJ, Thomas KG, Pankratz VS. Effect of rater training on reliability and accuracy of mini-CEX scores: a randomized, controlled trial. J Gen Intern Med. 2009 Jan;24(1):74-9. doi: 10.1007/s11606-008-0842-3. Epub 2008 Nov 11.
Related trials for Faculty Development
Active, not recruiting
Development of Human Motor Skills and the Promotion of Motivation for Physical Activity
Physical Inactivity, Obesity & Overweight
Referral and contact
AzTrials does not enrol participants or take referrals. Contact the study team or a site listed below; they decide who can take part. This study is not recruiting at the moment.
David A Cook, MD, MHPE principal investigator
Mayo Clinic
1 site in 1 country: see every site and its status.
Questions about this trial
What is this clinical trial studying?
A study testing Rater training workshop in people with Faculty Development, run by Mayo Clinic at 1 site in United States.
The full registered description is in the About this study section and the official record.
Who may be eligible to take part?
The sponsor lists 1 inclusion and 1 exclusion criteria. Eligibility is decided by the study team after screening, so read the full criteria and contact a listed site.
Is this clinical trial currently recruiting?
The record says: finished, as of the sponsor's last update on 28 Apr 2008. Status can change between updates, so confirm with the study contact before making plans.
Where is the study taking place?
The record lists 1 site in United States. Whether a site is taking part right now should be confirmed with that site. See every site.
What are the possible benefits and risks?
They depend on the study and the treatment, and no study can promise a benefit. The study team explains the known risks during informed consent, before anyone joins; the record itself does not list them.
Are results available for this study?
The study is completed, but results are not available in the source record. That does not mean the treatment did or did not work.
How can a doctor refer a patient or contact the research team?
The record lists no contact. The official record names the sponsor, who can be asked directly. AzTrials does not enrol participants or take referrals.
What does the primary outcome measure?
Interrater reliability (videotaped resident-patient encounters) (time frame: April-June 2006). It is what the study was designed to measure, not a finding.
Where can I read the original study information?
On the ClinicalTrials.gov record for NCT00667940, which also lists 1 linked publication.
Does a listing mean the treatment is approved or proven to work?
No. A listing means the study is registered. It does not show that the treatment is safe, works, or is approved by a regulator.
How often is this information updated?
AzTrials copies changed records from ClinicalTrials.gov every night and reloads the whole registry every week. The sponsor last updated this record on 28 Apr 2008.