AzTrials
AzTrials

Mobile Application-Based CBT Worksheet CollectionNCT03479398

Innovative Methods to Assess Psychotherapy Practices (imAPP)

An observational study of Mobile Application-Based CBT Worksheet Collection and Paper-Based Worksheet Collection, run by Palo Alto Veterans Institute for Research at 4 sites in United States.

Unknown status

Status not confirmed. The sponsor has not verified this record for over two years, so it may be out of date.

Official record
Search all trials

Trial summary

This project compares two methods of assessing the quality of cognitive behavioral therapy (CBT) that do not involve directly observing sessions: 1) adherence checklists embedded in clinical notes, and 2) rating the quality of worksheets that are completed with therapist guidance during sessions. Read more about this study

Who is it for?
Ages 18 years and olderHealthy volunteers can take part
What is being tested?
Cognitive Behavioral Therapy
How many people
439 participants
Where
4 sitesUnited States
Study dates
1 Apr 2017to 15 Oct 2023
Phase
Observational
Are there results?
None posted in the registry

Key requirements

Who can take part, as listed by the sponsor. Only the study team can confirm whether you are eligible, so take this list to your doctor.

  • Ages 18 years and older
  • All sexes
  • Healthy volunteers accepted
  • Adult, Older adult

Study population: Therapists who provide Cognitive Behavioral Therapy, and their patients who receive CBT for depression, anxiety, or PTSD. Therapists and patients will be recruited from community-based, public mental health (e.g., Boston and Palo Alto VA) and private practice treatment settings.

Who may be able to join

  • Clinician eligibility:
  • Private practice or employment at an agency at which the administration agrees to allow the recruitment and participation of their providers in research related activities
  • English Speaking
  • No anticipated plans to leave their current agency for at least 18 months
  • Willingness to allow their CBT sessions, worksheets, surveys, and interview data, and clinical notes to be used for research purposes
  • Carry a caseload that typically includes patients who experience depression, anxiety, or PTSD, with whom they regularly conduct individual therapy sessions and/or capacity to increase the proportion of such patients (e.g., clinic sees a substantial number of individuals with PTSD or depression)
Show all 26Show fewer
  • Must be trained (worksheet or web-based training and consultation) or in training for Cognitive Processing Therapy (CPT) for PTSD or CBT for depression or anxiety that uses worksheets Does not include CBT-i Can include aspects (i.e., worksheets) of CBT for substance use as long as depression or anxiety is primary diagnoses
  • Must anticipate at least 3 eligible patients
  • Must be willing to record sessions and provide worksheets and symptom measures to the study
  • Must have computer and internet access
  • Must be willing to use a mobile app on a tablet or mobile device
  • Patient eligibility:
  • Must be 18 yrs. of age or older
  • Experience one or more of the following (both a diagnosis and cut off score):
  • Clinician diagnosis of primary PTSD (PTSD-Checklist-5 score of 33 or more) Depressive disorder (e.g., major depressive disorder, dysthymia; PHQ of 10 or above) Or an Anxiety Disorder (Beck Anxiety Disorder score of 22 or above)
  • Note that if a patient has a score close to the cut-off score, it's up to the therapist digression
  • Must be willing to allow the team to collect session recordings, measures, notes and worksheets
  • Must be able to read and write at a sixth-grade level or above
  • Able to participate in sessions conducted in English, or Spanish (if working with bilingual clinicians in community or private practice)
  • Must be willing to engage in CBT/CPT
  • Therapist considers the treatment with the individual patient to be "mostly" CBT or CPT
  • Cannot have:
  • Imminent risk of suicide or homicide (requiring hospitalization) that require immediate treatment In need of detoxification (can be enrolled when substance abuse treatment is not the primary treatment target) Active psychosis or manic episode unless well controlled by medication and not the primary focus of treatment Cognitive impairments that preclude any participation in therapy
  • Administrator eligibility:
  • Must have a support or managerial/supervisory position at the clinic(s) from which A and B are being recruited from
  • Must also be willing to complete a packet of study measures/partake in an interview

Who cannot join

None listed.

Trial locations and contacts

4 sites in 1 country.

United States4 sites
  • NCPTSD - VA Boston HCS

    Boston, Massachusetts, 02130

  • Massachusetts General Hospital

    Chelsea, Massachusetts, 02150

  • VA Palo Alto Health Care System

    Palo Alto, California, 94304

  • Aaron T. Beck Psychopathology Research Center

    Philadelphia, Pennsylvania, 19104

Other trials in United States

About this study

Official title: Leveraging Routine Clinical Materials and Mobile Technology to Assess CBT Quality

This project compares two methods of assessing the quality of cognitive behavioral therapy (CBT) that do not involve directly observing sessions: 1) adherence checklists embedded in clinical notes, and 2) rating the quality of worksheets that are completed with therapist guidance during sessions. It also examines whether ratings of worksheets completed on a mobile app are reliable and valid quality measures. This information can inform strategies to monitor and enhance CBT quality, which can ultimately improve the quality of care and clinical outcomes.

Read the detailed description

Cognitive Behavioral Therapy (CBT) has been demonstrated to be effective for numerous presenting problems, including depression, anxiety, and post-traumatic stress disorder (PTSD). Several large mental health systems have invested heavily in programs to train their clinicians in CBTs, but relatively little attention has been devoted to the monitoring or promotion of CBT quality after training is complete. Identifying strategies to do so can facilitate research and training, and is critical to ensuring consumer access to high quality, evidence-based treatments. The lack of a scalable, effective, and efficient method of monitoring quality is a key barrier to efforts to promote high-quality implementation. Self-report fidelity assessments increase clinician and consumer burden and may not accurately reflect clinician skill or the intensity with which CBT interventions are delivered. Observation and expert ratings are time and resource intensive and unlikely to be feasible or affordable in large systems. To maximize the likelihood of broad implementation once effective strategies to monitor quality are established, it is essential that these strategies are feasible and acceptable in routine care contexts, leveraging information collected during routine care. To date, few monitoring strategies that do not involve observation, client/caregiver reports, or clinician self-reports have been tested. To address this critical implementation challenge, we propose to refine and evaluate a method of monitoring quality that is based on an evaluation of CBT worksheets that are completed in session. Because the worksheets were developed to implement core cognitive and behavioral elements and are embedded in CBTs across diagnostic categories, they may be used to elucidate the clinician's ability to guide the client through CBT interventions in session. Preliminary research with this measure demonstrated high correlations between the measure and observer ratings of clinician competence, associations with subsequent symptom change, and high agreement between raters with differing levels of familiarity with CBT. Completion of the ratings based on worksheets requires only a small fraction of time required for session observation and ratings. This project will compare this novel strategy to observer ratings and adherence checklists that are embedded in clinical notes. Furthermore, it will compare the accuracy of worksheet data collected by mobile app to paper-form worksheets, and assess the feasibility and acceptability of these strategies. Because the core elements of CBT and its worksheets are common across many CBTs, this research has broad implications for monitoring fidelity to CBTs in a variety of mental health and healthcare systems and settings. This research will be conducted by a team of investigators with expertise in CBT, training, implementation, psychotherapy process and outcome research, psychometrics, longitudinal data analysis, mobile technologies and healthcare economics, with input from community partners and end-users. The resulting products have the potential to significantly improve efforts to monitor and ensure ongoing high quality implementation of CBT in routine care settings.

Keywords
Fidelity, Cognitive Behavioral Therapy, Mobile Application, Routine Practice

What participants receive

What participants receive. In a randomised study each person is assigned to one group by chance.

  • Cognitive Behavioral TherapyBehavioural

    Also called Cognitive Processing Therapy

    Cognitive Behavioral Therapy (CBT) is an evidence-based psychotherapy that treats a variety of disorders. In this study, clinicians will be enrolled who use types of CBT for anxiety disorders, depression and PTSD in their routine clinical practice. CBT is usually a 12-16 session treatment that focuses on intervening on disorders cognitively and behaviorally through the use of worksheets during session and outside of session.

Study groups in NCT03479398
Study groupRoleWhat they receive
App ConditionAlthough we will be mostly observing routine practice, we will randomize patients into either an App condition or paper condition. The App condition refers to patients completing routine Cognitive Behavioral Therapy (CBT) worksheets on a mobile application during session. Everything else that occurs in treatment sessions will be consistent with routine care practices.
Receives: Behavioral: Cognitive Behavioral Therapy
Paper ConditionAlthough we will be mostly observing routine practice, we do randomize patients into either an App condition or paper condition. The paper condition refers to patients completing routine CBT worksheets on paper (the current standard) during session.
Receives: Behavioral: Cognitive Behavioral Therapy

What is being measured

The primary outcome is the main question the study is designed to answer.

  1. Change in Patient Health Questionnaire (PHQ-9)

    9-item self-report measure of depression symptoms with good internal reliability (α = .89) and test-retest reliability (r = .84).

    Measured: Baseline (pre-treatment), Every session (weekly) if primary /target problem up to 16 weeks

  2. Change in Posttraumatic Stress Disorder Checklist (PCL-5)

    A 20-item self-report measure that assesses the 20 Diagnostic and Statistical Manual (DSM-5) symptoms of PTSD. Good internal consistency and concurrent validity with the Clinician-assessed PTSD Scale.

    Measured: Baseline (pre-treatment), Every session (weekly) if primary /target problem up to 16 weeks

  3. Change in Beck Anxiety Inventory

    21-item self-report instrument for measuring the severity of anxiety symptoms. Good internal consistency (α = .92) and high test-retest reliability (r = .75.).

    Measured: Baseline (pre-treatment), Every session (weekly) if primary /target problem up to 16 weeks

Secondary outcomes (4)
  1. Change in Inventory of Psycho social Functioning-Brief Version

    14 items, yields a grand mean and 7 subscale means for functioning in romance, family, friendships, parenting, education, work, and self-care activities (non-applicable categories can be skipped; total score reflects mean of applicable items). Excellent reliability and internal consistency ( α = 0.93); correlates with other measures of quality of life.

    Measured: Completed at baseline, One Month, and post-treatment (week 16)

  2. Evidence-Based Practice Attitudes Scale-50 (EBPAS)

    This measure looks at Evidence-based practice attitudes and motivation among clinicians, 1 (worse)-5 (best) total score range

    Measured: Completed at therapist baseline (prior to providing treatment in the study)

  3. Perceived Characteristics of Interventions (PCIS)

    Innovation characteristics hypothesized to influence adoption and sustainability

    Measured: Completed at therapist baseline (prior to providing treatment in the study)

  4. Implementation Climate Assessment (ICA)

    Brief validated measure of implementation climate ( manager support for EBPs, etc.)

    Measured: Completed at therapist baseline (prior to providing treatment in the study)

Timeline

From the start of the study to its completion; the mark is today. Estimated dates are the sponsor’s plan.

  1. First posted27 Mar 2018
  2. Study start1 Apr 2017
  3. Primary completion15 Dec 2021
  4. Study completion15 Oct 2023 (estimated)

Resources

More details

For doctors and researchers: how the study is designed, who runs it, and its registry IDs.

Study design
Study type
Observational
Observational model
Ecologic or community
Time perspective
Prospective
Enrolment
439 participants (actual)
IDs, dates and publications
ClinicalTrials.gov ID
NCT03479398
Sponsor's protocol ID
WIS0003AGG
Other IDs
R01MH112628
Status last verified
Oct 2022
First submitted
8 Mar 2018
Last updated
3 Nov 2022
Results posted
No
Data monitoring committee
Yes
FDA-regulated drug
No
FDA-regulated device
No

Publications

  • McHugh RK, Barlow DH. The dissemination and implementation of evidence-based psychological treatments. A review of current efforts. Am Psychol. 2010 Feb-Mar;65(2):73-84. doi: 10.1037/a0018121.

    PubMed 20141263
  • Schoenwald SK. It's a Bird, It's A Plane, It's ... Fidelity Measurement In the Real World. Clin Psychol (New York). 2011 Jun;18(2):142-147. doi: 10.1111/j.1468-2850.2011.01245.x.

    PubMed 21691439
  • Schoenwald SK, Garland AF, Chapman JE, Frazier SL, Sheidow AJ, Southam-Gerow MA. Toward the effective and efficient measurement of implementation fidelity. Adm Policy Ment Health. 2011 Jan;38(1):32-43. doi: 10.1007/s10488-010-0321-0.

    PubMed 20957425
  • Ruzek JI, Rosen RC. Disseminating evidence-based treatments for PTSD in organizational settings: A high priority focus area. Behav Res Ther. 2009 Nov;47(11):980-9. doi: 10.1016/j.brat.2009.07.008. Epub 2009 Jul 26.

    PubMed 19632668
  • Carroll KM, Martino S, Rounsaville BJ. No train, no gain? Clinical Psychology: Science and Practice. 2010;17(1):36-40.

  • Decker SE, Jameson MT, Naugle AE. Therapist training in empirically supported treatments: a review of evaluation methods for short- and long-term outcomes. Adm Policy Ment Health. 2011 Jul;38(4):254-86. doi: 10.1007/s10488-011-0360-1.

    PubMed 21656256
  • Resick PA, Galovski TE, Uhlmansiek MO, Scher CD, Clum GA, Young-Xu Y. A randomized clinical trial to dismantle components of cognitive processing therapy for posttraumatic stress disorder in female victims of interpersonal violence. J Consult Clin Psychol. 2008 Apr;76(2):243-258. doi: 10.1037/0022-006X.76.2.243.

    PubMed 18377121
  • Resick PA, Monson CM, Chard KM. Cognitive processing therapy veteran/military version: Therapist and patient materials manual. . Washington, DC: Department of Veterans' Affairs; 2008.

  • Beck AT. The current state of cognitive therapy: a 40-year retrospective. Arch Gen Psychiatry. 2005 Sep;62(9):953-9. doi: 10.1001/archpsyc.62.9.953.

    PubMed 16143727
  • Clark DM. Implementing NICE guidelines for the psychological treatment of depression and anxiety disorders: the IAPT experience. Int Rev Psychiatry. 2011 Aug;23(4):318-27. doi: 10.3109/09540261.2011.606803.

    PubMed 22026487
  • Karlin BE, Cross G. From the laboratory to the therapy room: National dissemination and implementation of evidence-based psychotherapies in the U.S. Department of Veterans Affairs Health Care System. Am Psychol. 2014 Jan;69(1):19-33. doi: 10.1037/a0033888. Epub 2013 Sep 2.

    PubMed 24001035
  • Karlin BE, Ruzek JI, Chard KM, Eftekhari A, Monson CM, Hembree EA, Resick PA, Foa EB. Dissemination of evidence-based psychological treatments for posttraumatic stress disorder in the Veterans Health Administration. J Trauma Stress. 2010 Dec;23(6):663-73. doi: 10.1002/jts.20588. Epub 2010 Nov 15.

    PubMed 21171126
  • Foa EB, Keane TM, Friedman MJ, Cohen JA. Effective treatments for PTSD: Practice guidelines from the International Society for Traumatic Stress Studies (2nd ed.). New York, NY US: Guilford Press; 2009.

  • Stirman SW, Buchhofer R, McLaulin JB, Evans AC, Beck AT. Public-academic partnerships: the Beck Initiative: a partnership to implement cognitive therapy in a community behavioral health system. Psychiatr Serv. 2009 Oct;60(10):1302-4. doi: 10.1176/ps.2009.60.10.1302.

    PubMed 19797367
  • Mendel P, Meredith LS, Schoenbaum M, Sherbourne CD, Wells KB. Interventions in organizational and community context: a framework for building evidence on dissemination and implementation in health services research. Adm Policy Ment Health. 2008 Mar;35(1-2):21-37. doi: 10.1007/s10488-007-0144-9. Epub 2007 Nov 8.

    PubMed 17990095

Questions about this trial

What is this clinical trial studying?

An observational study of Mobile Application-Based CBT Worksheet Collection and Paper-Based Worksheet Collection, run by Palo Alto Veterans Institute for Research at 4 sites in United States.

The full registered description is in the About this study section and the official record.

Who may be eligible to take part?

The sponsor lists 26 inclusion and 0 exclusion criteria. Eligibility is decided by the study team after screening, so read the full criteria and contact a listed site.

Is this clinical trial currently recruiting?

The record says: status not confirmed, as of the sponsor's last update on 3 Nov 2022. Status can change between updates, so confirm with the study contact before making plans.

Where is the study taking place?

The record lists 4 sites in United States. Whether a site is taking part right now should be confirmed with that site. See every site.

What are the possible benefits and risks?

They depend on the study and the treatment, and no study can promise a benefit. The study team explains the known risks during informed consent, before anyone joins; the record itself does not list them.

Are results available for this study?

Results are not available in the source record.

How can a doctor refer a patient or contact the research team?

The record lists no contact. The official record names the sponsor, who can be asked directly. AzTrials does not enrol participants or take referrals.

What does the primary outcome measure?

Change in Patient Health Questionnaire (PHQ-9) (time frame: Baseline (pre-treatment), Every session (weekly) if primary /target problem up to 16 weeks). The study lists 3 primary outcomes in all. It is what the study was designed to measure, not a finding.

Where can I read the original study information?

On the ClinicalTrials.gov record for NCT03479398, which also lists 22 linked publications.

Does a listing mean the treatment is approved or proven to work?

No. A listing means the study is registered. It does not show that the treatment is safe, works, or is approved by a regulator.

How often is this information updated?

AzTrials copies changed records from ClinicalTrials.gov every night and reloads the whole registry every week. The sponsor last updated this record on 3 Nov 2022.

Free account

Keep searching with a free account

You’ve used your 3 free searches for this session. Log in or create a free AzTrials account to keep searching clinical trials and save the ones you want to follow.

Create a free accountLog in