Assess Fidelity to Cognitive-behavioral Therapy for YouthNCT02820623

Project FACTS (Fidelity Accuracy: Comparing Three Strategies)

A study testing Self-report, Chart-Stimulated Recall and Behavioral Rehearsal in people with Assess Fidelity to Cognitive-behavioral Therapy for Youth, run by Northwestern University.

Completed

Finished. The study has ended and is not enrolling. Results may be posted on the official record.

Official record

Search all trials

Trial summary

Who is it for?: Ages 3 years and olderHealthy volunteers can take part
Conditions the study is for: Assess Fidelity to Cognitive-behavioral Therapy for Youth
What is being tested?: Self-report, Chart-Stimulated Recall and Behavioral Rehearsal
Type of treatment: Lifestyle or therapy programme
Could I receive a placebo?: No, everyone receives an active treatment
How many people: 126 participants
Where: No sites listed yet
Study dates: Nov 2016to 30 Jun 2020
Phase: Interventional
Are there results?: Yes, posted in the registrySummary in the doctors' view

Key requirements

Who can take part, as listed by the sponsor. Only the study team can confirm whether you are eligible, so take this list to your doctor.

Ages 3 years and older
All sexes
Healthy volunteers accepted
Child, Adult, Older adult

Who may be able to join

they provide mental health treatment services in community mental health agencies in the City of Philadelphia participating in this study and
have been trained in cognitive-behavioral therapy (CBT) through the City-sponsored CBT evidence-based practice initiatives, and/or
respond to a brief screening survey that they use CBT; and/or
are nominated by a supervisor as a therapist who uses CBT.
they are supervisors or administrators (i.e., they hold leadership positions) in community mental health agencies in the City of Philadelphia participating in this study.
Youth and their legal guardians will be eligible to participate in this study if the youth
is 3-24 years old and
has completed at least one session with a therapist who has enrolled to participate in this study.

Who cannot join

n/a
n/a
if the child does not have a legal guardian who is able to consent (e.g., the Department of Human Services (DHS) is the guardian).

About this study

Official title: Increasing Accuracy and Efficiency of Fidelity Measurement in CBT

The objective is to compare the accuracy, costs, and cost-effectiveness of three fidelity measurement methods to assess fidelity to cognitive-behavioral therapy for youth. The investigators will randomize 135 therapists, implementing cognitive-behavioral therapy, to 3 conditions: self-report, chart stimulated recall, and behavioral rehearsal (N = 45 for each group). To calculate the outcomes of interest, each condition will be compared to the gold-standard fidelity measurement method, direct observation.

Read the detailed description

To achieve the primary objective, the investigators plan to (1) identify the most accurate fidelity measurement method; (2) estimate the economic costs and cost-effectiveness of the fidelity measurement methods; and (3) compare stakeholders' motivation to use each method, as well as identify their perceived barriers and facilitators to use of each method. This study will have a significant positive impact in two ways. First, it will validate fidelity measurement methods that can be used for research. Second, it will produce tools that can be used by community mental health clinics to monitor therapist fidelity, an indicator of therapy quality.

Conditions: Assess Fidelity to Cognitive-behavioral Therapy for Youth
Being tested: Self-report, Chart-Stimulated Recall, Behavioral Rehearsal
Keywords: implementation, fidelity, cognitive behavioral therapy

What participants receive

What participants receive. In a randomised study each person is assigned to one group by chance.

Self-reportBehavioural
Chart-Stimulated RecallBehavioural
Behavioral RehearsalBehavioural

Study groups in NCT02820623
Study group	Role	What they receive
Self-report	Experimental	Therapists randomized to this condition will complete a brief self-report measure, the Therapy Process Observational Coding System for Child Psychotherapy Strategies Scale-Self Report version (TPOCS-SR) for each of the recorded clinical encounters with enrolled youth. The TPOCS-SR will be a self-report version of the Therapy Process Observational Coding System for Child Psychotherapy-Strategies Scale (TPOCS-S) and will be created in collaboration with the instrument developer (McLeod). In this condition, the investigators will (a) provide an operational definition for each item on the TPOCS-SR (e.g., cognitive education: teaches client the cognitive model (e.g., thoughts influence behavior)/identifies how the cognitive model applies to a specific aspects of the client's life), and (b) provide therapists with a 30-minute training session that includes sample vignettes of particular behaviors and information about how those vignettes should be rated. Receives: Behavioral: Self-report
Chart Stimulated Recall	Experimental	Therapists randomized to this condition will be asked to bring the charts of three enrolled youth to the chart-stimulated recall interview. A trained interviewer will ask the therapists how well they recall the encounter (rating of memory quality) followed by an open-ended question ("Talk me through your last session with your client. Tell me what you did."). While the therapists are speaking, the interviewer will note any elements that represent a prescribed CBT strategy. The interviewer will go through a list of cognitive-behavioral strategies based upon the TPOCS-S and probe to determine if the therapists completed any of the strategies. Follow-up questions will be used to explore to what degree an element was used and how skillfully and responsively the strategies were used. Receives: Behavioral: Chart-Stimulated Recall
Behavioral Rehearsal	Experimental	Therapists randomized to this condition will be asked to engage in role-plays demonstrating the CBT strategies used with the three enrolled youth. The investigators will provide therapists with a list of the TPOCS-S CBT strategies and ask them to identify the CBT strategies used in their recorded encounter. The investigators will randomly select one of the strategies they report for each role-play. The investigators will then tell them, "Please role-play how you used this strategy in session with your client, with the trained actor in front of you." Later, an independent rater will rate therapists' adherence and skill based on established scoring criteria. Receives: Behavioral: Behavioral Rehearsal

What is being measured

The primary outcome is the main question the study is designed to answer.

Total Fidelity Scores for Each Study Condition Measured Via Direct Observation (TPOCS-RS) and Study Arm Fidelity Method Scores
Client sessions for all 3 conditions were scored for CBT adherence via direct observation using the Therapy Process Observational Coding System-Revised Strategies (TPOCS-RS) Scale (McLeod et al., 2015) on a 7-point Likert Scale (1='not at all,' 7='extensively'). We analyzed the TPOCS-RS' Maximum CBT score (highest coded intervention technique across all 12 possible interventions in a given session).
We also measured CBT adherence during client sessions using the strategies from each study condition (described in 'Arms/Groups' below). All 3 conditions used parallel 7-point scales to the TPOCS-RS.
Higher scores for the 3 study condition strategies indicate greater reported use, and higher scores on the TPOCS-RS indicate greater observed use of CBT interventions. Our goal was to compare the study condition methods of assessing CBT adherence to direct observation (TPOCS-RS), the gold standard.
Measured: through study completion, an average of 1 month

Timeline

From the start of the study to its completion; the mark is today. Estimated dates are the sponsor’s plan.

Nov 201630 Jun 2020

First posted1 Jul 2016 (estimated)
Study startNov 2016
Primary completion30 Jun 2020
Study completion30 Jun 2020

Resources

Official record on ClinicalTrials.govThe registry entry this page is built from, with its full change history.
Posted resultsParticipant flow, outcome measures and adverse events reported by the sponsor.
Publications (2)Papers the sponsor has linked to this study.
Other trials for Assess Fidelity to Cognitive-behavioral Therapy for YouthStudies of the same condition, open and under way.
More trials run by Northwestern UniversityThe sponsor's other registered studies.
What the trial phases meanHow a treatment moves from first tests in people to approval.

More details

For doctors and researchers: how the study is designed, who runs it, and its registry IDs.

Study design

Study type: Interventional
Phase: Phase N/A
Primary purpose: Health services research
Allocation: Randomised
Intervention model: Parallel groups
Masking: None (open label)
Enrolment: 126 participants (actual)

IDs, dates and publications

ClinicalTrials.gov ID: NCT02820623
Sponsor's protocol ID: R01MH108551
Status last verified: Apr 2023
First submitted: 3 Jun 2016
Last updated: 21 Apr 2023
Results posted: Yes, on ClinicalTrials.gov
Data monitoring committee: Yes
Individual data sharing: Yes

Publications

Becker-Haimes EM, Marcus SC, Klein MR, Schoenwald SK, Fugo PB, McLeod BD, Dorsey S, Williams NJ, Mandell DS, Beidas RS. A Randomized Trial to Identify Accurate Measurement Methods for Adherence to Cognitive-Behavioral Therapy. Behav Ther. 2022 Nov;53(6):1191-1204. doi: 10.1016/j.beth.2022.06.001. Epub 2022 Jun 10.
PubMed 36229116
Beidas RS, Maclean JC, Fishman J, Dorsey S, Schoenwald SK, Mandell DS, Shea JA, McLeod BD, French MT, Hogue A, Adams DR, Lieberman A, Becker-Haimes EM, Marcus SC. A randomized trial to identify accurate and cost-effective fidelity measurement methods for cognitive-behavioral therapy: project FACTS study protocol. BMC Psychiatry. 2016 Sep 15;16(1):323. doi: 10.1186/s12888-016-1034-z.
PubMed 27633780

Study at a glance

From the registry record, last updated by the sponsor on 21 Apr 2023. Recruitment status is as the sponsor reported it: confirm with the study team.

Official title: Increasing Accuracy and Efficiency of Fidelity Measurement in CBT
ClinicalTrials.gov ID: NCT02820623
Acronym / sponsor ID: R01MH108551
Conditions: Assess Fidelity to Cognitive-behavioral Therapy for Youth
Keywords: implementation, fidelity, cognitive behavioral therapy
Study type: Interventional
Recruitment status: Completed
Enrolment: 126 participants (actual)
Lead sponsor: Northwestern UniversityUniversity, hospital or other
Collaborators: University of Pennsylvania
Study start: Nov 2016
Primary completion: 30 Jun 2020 (actual)
Study completion: 30 Jun 2020 (actual)
First posted: 1 Jul 2016 (estimated)
Results: Posted 31 Jan 2023

Clinical summary

The research question as the sponsor registered it. This describes what the study plans to test, not what it found.

Detailed description

Study design

Primary purpose: Health services research
Allocation: Randomised
Intervention model: Parallel groups
Masking: None (open label)
Number of arms: 3

Arm	Type	Receives
Self-report	Experimental	Behavioral: Self-reportTherapists randomized to this condition will complete a brief self-report measure, the Therapy Process Observational Coding System for Child Psychotherapy Strategies Scale-Self Report version (TPOCS-SR) for each of the recorded clinical encounters with enrolled youth. The TPOCS-SR will be a self-report version of the Therapy Process Observational Coding System for Child Psychotherapy-Strategies Scale (TPOCS-S) and will be created in collaboration with the instrument developer (McLeod). In this condition, the investigators will (a) provide an operational definition for each item on the TPOCS-SR (e.g., cognitive education: teaches client the cognitive model (e.g., thoughts influence behavior)/identifies how the cognitive model applies to a specific aspects of the client's life), and (b) provide therapists with a 30-minute training session that includes sample vignettes of particular behaviors and information about how those vignettes should be rated.
Chart Stimulated Recall	Experimental	Behavioral: Chart-Stimulated RecallTherapists randomized to this condition will be asked to bring the charts of three enrolled youth to the chart-stimulated recall interview. A trained interviewer will ask the therapists how well they recall the encounter (rating of memory quality) followed by an open-ended question ("Talk me through your last session with your client. Tell me what you did."). While the therapists are speaking, the interviewer will note any elements that represent a prescribed CBT strategy. The interviewer will go through a list of cognitive-behavioral strategies based upon the TPOCS-S and probe to determine if the therapists completed any of the strategies. Follow-up questions will be used to explore to what degree an element was used and how skillfully and responsively the strategies were used.
Behavioral Rehearsal	Experimental	Behavioral: Behavioral RehearsalTherapists randomized to this condition will be asked to engage in role-plays demonstrating the CBT strategies used with the three enrolled youth. The investigators will provide therapists with a list of the TPOCS-S CBT strategies and ask them to identify the CBT strategies used in their recorded encounter. The investigators will randomly select one of the strategies they report for each role-play. The investigators will then tell them, "Please role-play how you used this strategy in session with your client, with the trained actor in front of you." Later, an independent rater will rate therapists' adherence and skill based on established scoring criteria.

Who may qualify

Ages: 3 years and older
Sex: All
Healthy volunteers: Accepted
Age groups: Child, Adult, Older adult

Key inclusion criteria (8 listed)

they provide mental health treatment services in community mental health agencies in the City of Philadelphia participating in this study and
they are supervisors or administrators (i.e., they hold leadership positions) in community mental health agencies in the City of Philadelphia participating in this study.
Youth and their legal guardians will be eligible to participate in this study if the youth

Key exclusion criteria (3 listed)

n/a
n/a
if the child does not have a legal guardian who is able to consent (e.g., the Department of Human Services (DHS) is the guardian).

Eligibility is decided by the study team after screening; this overview does not establish that a patient qualifies. Full inclusion and exclusion criteria.

Interventions

Intervention	Type	Arms
Self-report	Behavioural	Self-report
Chart-Stimulated Recall	Behavioural	Chart Stimulated Recall
Behavioral Rehearsal	Behavioural	Behavioral Rehearsal

Primary outcome

Total Fidelity Scores for Each Study Condition Measured Via Direct Observation (TPOCS-RS) and Study Arm Fidelity Method Scores
Time frame: through study completion, an average of 1 month
Client sessions for all 3 conditions were scored for CBT adherence via direct observation using the Therapy Process Observational Coding System-Revised Strategies (TPOCS-RS) Scale (McLeod et al., 2015) on a 7-point Likert Scale (1='not at all,' 7='extensively'). We analyzed the TPOCS-RS' Maximum CBT score (highest coded intervention technique across all 12 possible interventions in a given session). We also measured CBT adherence during client sessions using the strategies from each study condition (described in 'Arms/Groups' below). All 3 conditions used parallel 7-point scales to the TPOCS-RS. Higher scores for the 3 study condition strategies indicate greater reported use, and higher scores on the TPOCS-RS indicate greater observed use of CBT interventions. Our goal was to compare the study condition methods of assessing CBT adherence to direct observation (TPOCS-RS), the gold standard.

Results and safety

Results posted to the registry on 31 Jan 2023, as reported by the sponsor. Figures are copied from the record, not re-analysed. Full results on ClinicalTrials.gov.

Participant flow

Group	Started	Completed
Self-report	41	36
Chart Stimulated Recall	42	35
Behavioral Rehearsal	43	32

Primary outcome: Total Fidelity Scores for Each Study Condition Measured Via Direct Observation (TPOCS-RS) and Study Arm Fidelity Method Scores

Time frame: through study completion, an average of 1 month. Mean (Standard Deviation). Unit: Maximum CBT score on a scale

Measure	Self-report n = 41	Chart Stimulated Recall n = 42	Behavioral Rehearsal n = 43
Maximum Mean Condition Score	5.28 (1.17)	4.53 (1.18)	4.09 (1.48)
Maximum Mean direct observation (TPOCS-RS) Score	3.42 (1.55)	3.69 (1.36)	3.72 (1.59)

Population analysed: Participants (i.e., therapists) were assigned to one of 3 conditions: self-report, chart stimulated recall, or behavioral rehearsal. The unit of analysis for Maximum Mean condition score and Maximum Mean direct observation (TPOCS-RS) score was client session; therapists could include more than one client session for inclusion in analyses.

Adverse events

Time frame: Adverse events were monitored from study initiation through study completion, an average of 6 weeks for each participant.. Other adverse events reported at 0% frequency or more

Group	Serious adverse events	Other adverse events	Deaths
Self-report	0 of 41 (0%)	0 of 41 (0%)	0 of 41
Chart Stimulated Recall	0 of 42 (0%)	0 of 42 (0%)	0 of 42
Behavioral Rehearsal	0 of 43 (0%)	0 of 43 (0%)	0 of 43

Loading the posted results from ClinicalTrials.gov…

Research links

Official record on ClinicalTrials.govLast updated by the sponsor 21 Apr 2023.
Posted resultsParticipant flow, baseline, all outcome measures and adverse events.
Derived from this study: PubMed 36229116Becker-Haimes EM, Marcus SC, Klein MR, Schoenwald SK, Fugo PB, McLeod BD, Dorsey S, Williams NJ, Mandell DS, Beidas RS. A Randomized Trial to Identify Accurate Measurement Methods for Adherence to Cognitive-Behavioral Therapy. Behav Ther. 2022 Nov;53(6):1191-1204. doi: 10.1016/j.beth.2022.06.001. Epub 2022 Jun 10.
Derived from this study: PubMed 27633780Beidas RS, Maclean JC, Fishman J, Dorsey S, Schoenwald SK, Mandell DS, Shea JA, McLeod BD, French MT, Hogue A, Adams DR, Lieberman A, Becker-Haimes EM, Marcus SC. A randomized trial to identify accurate and cost-effective fidelity measurement methods for cognitive-behavioral therapy: project FACTS study protocol. BMC Psychiatry. 2016 Sep 15;16(1):323. doi: 10.1186/s12888-016-1034-z.
Individual participant dataSharing planned: Yes

Referral and contact

AzTrials does not enrol participants or take referrals. Contact the study team or a site listed below; they decide who can take part. This study is not recruiting at the moment.

Rinad S Beidas, PhD principal investigator

Northwestern University

Emily M Becker-Haimes, PhD principal investigator

University of Pennsylvania

Questions about this trial

What is this clinical trial studying?

A study testing Self-report, Chart-Stimulated Recall and Behavioral Rehearsal in people with Assess Fidelity to Cognitive-behavioral Therapy for Youth, run by Northwestern University.

The full registered description is in the About this study section and the official record.

Who may be eligible to take part?

The sponsor lists 8 inclusion and 3 exclusion criteria. Eligibility is decided by the study team after screening, so read the full criteria and contact a listed site.

Is this clinical trial currently recruiting?

The record says: finished, as of the sponsor's last update on 21 Apr 2023. Status can change between updates, so confirm with the study contact before making plans.

What are the possible benefits and risks?

They depend on the study and the treatment, and no study can promise a benefit. The study team explains the known risks during informed consent, before anyone joins; the record itself does not list them.

Are results available for this study?

Yes. Results are posted in the registry (first posted 31 Jan 2023). A summary is in the For doctors view, and the full results are on ClinicalTrials.gov.

How can a doctor refer a patient or contact the research team?

The record lists no contact. The official record names the sponsor, who can be asked directly. AzTrials does not enrol participants or take referrals.

What does the primary outcome measure?

Total Fidelity Scores for Each Study Condition Measured Via Direct Observation (TPOCS-RS) and Study Arm Fidelity Method Scores (time frame: through study completion, an average of 1 month). It is what the study was designed to measure, not a finding.

Where can I read the original study information?

On the ClinicalTrials.gov record for NCT02820623, which also lists 2 linked publications.

Does a listing mean the treatment is approved or proven to work?

No. A listing means the study is registered. It does not show that the treatment is safe, works, or is approved by a regulator.

How often is this information updated?

AzTrials copies changed records from ClinicalTrials.gov every night and reloads the whole registry every week. The sponsor last updated this record on 21 Apr 2023.