AzTrials
AzTrials

Cognitive Behavioral TherapyAFFECTNCT05340738

AI-Based Fidelity Feedback to Enhance CBT

A study testing LyssnCBT in people with Cognitive Behavioral Therapy and Therapy, run by University of Pennsylvania at 1 site in United States.

Completed

Finished. The study has ended and is not enrolling. Results may be posted on the official record.

Official record
Search all trials

Trial summary

This study is being conducted together by researchers at the University of Pennsylvania and Lyssn.io, Inc., ("Lyssn"), a technology start-up developing digital tools to support evidence-based psychotherapies (EBPs) for mental health disorders and addiction. Read more about this study

Who is it for?
Ages 18 years and olderHealthy volunteers can take part
Conditions the study is for
Cognitive Behavioral Therapy, Therapy
What is being tested?
LyssnCBT
Could I receive a placebo?
No, everyone receives an active treatment
How many people
448 participants
Where
1 siteUnited States
Study dates
9 Mar 2023to 31 Jan 2026
Phase
Interventional
Are there results?
None posted in the registry

Key requirements

Who can take part, as listed by the sponsor. Only the study team can confirm whether you are eligible, so take this list to your doctor.

  • Ages 18 years and older
  • All sexes
  • Healthy volunteers accepted
  • Adult, Older adult

Who may be able to join

  • Therapists
  • Able to participate in therapy sessions conducted in English
  • Employed at a Philadelphia CMH treatment center that allows the recruitment and participation of therapists in research-related activities
  • Willing to allow their session recordings to be used for research purposes
  • Computer and internet access
  • Supervisors
Show all 11Show fewer
  • Oversee participating therapists
  • Computer and internet access
  • Clients
  • Able to participate in therapy sessions conducted in English
  • Willing to allow the team to collect data and use their session recordings for research purposes

Who cannot join

  • Unwilling to allow the research team to access their therapy session recordings

Trial locations and contacts

1 site in 1 country.

United States1 site
  • The Penn Collaborative for CBT and Implementation Science

    Philadelphia, Pennsylvania, 19104

Other trials in United States

About this study

Official title: Enhancing the Quality of CBT in Community Mental Health Through AI-generated Fidelity Feedback

This study is being conducted together by researchers at the University of Pennsylvania and Lyssn.io, Inc., ("Lyssn"), a technology start-up developing digital tools to support evidence-based psychotherapies (EBPs) for mental health disorders and addiction. This study will implement a technology to assess and enhance the quality of EBPs like Cognitive Behavioral Therapy (CBT) that includes a user interface geared to clinical, supervision, and administrative workflows and needs, and then assess this technology for effectiveness in comparison to usual care.

There is a tremendous global burden of mental illness: Over 50 million American adults have a diagnosable mental health disorder, and major depression on its own is the leading cause of disability worldwide. In the face of this burden, clinical research has documented a variety of effective EBPs (e.g. CBT), and these psychotherapies are utilized on a massive scale. Systems have invested over $2 billion in training providers in specific EBPs. Once trained, however, therapists' adherence to the EBP, also called fidelity, is both crucial for effectiveness and difficult to assess. There is no scalable method to assess the fidelity and quality of EBPs in community practice settings. This is a foundational problem for healthcare systems.

Read more

Advances in speech processing and machine learning make technology a promising solution to this problem. The use of technology - instead of humans - to evaluate EBPs means that objective, performance-based feedback can be provided quickly, efficiently, cost-effectively, and without human error. If successful, the present research will be among the first examples of a method for building, monitoring, and assessing the quality of therapy that can scale up to large, real-world healthcare settings.

In this study, the investigators will implement an existing, fully-functional prototype (LyssnCBT) that includes a user interface geared to community mental health (CMH) clinical, supervision, and administrative workflows and needs, and then assess for effectiveness of psychotherapy supported by LyssnCBT in comparison to usual care.

This study will implement LyssnCBT in 5 community mental health agencies, beginning with a single-arm pilot field trial to identify and address any specific barriers to implementing the tool in a community mental health context. The study team will then conduct a larger study in community mental health agencies comparing LyssnCBT to services as usual.

Read the detailed description

The research team will first recruit 10 therapists and two supervisors from one CMH clinic in order to conduct a single-arm field trial of LyssnCBT. Each therapist will use the LyssnCBT platform with two clients over the course of two weeks (four total sessions per therapist). In addition, supervisors will conduct a supervision session with each therapist using the LyssnCBT tool's fidelity feedback. Next, participants will complete brief Likert assessments on technical reliability, functional reliability, and experiences integrating the system into the daily workflow. Usage data from LyssnCBT will be captured automatically by the system, including: which software features were used, time spent reviewing sessions and transcripts, and time spent reviewing artificial intelligence (AI) generated CBT fidelity feedback. This data and feedback will be used for a final refinement of the LyssnCBT software and related clinical and supervision protocols prior to the main trial.

Following the pilot study, the research team will recruit 4 additional CMH clinics to participate in a type 2 hybrid implementation-effectiveness, randomized stepped-wedge study comparing LyssnCBT to SAU. 50 therapists and their supervisors will be recruited from the participating clinics, and each therapist will be asked to have at least 5 clients participating in the study at any given time, from among their regular caseload. Across 18 months of planned data collection (~75 weeks), the investigators expect a minimum of 1,875 clients for 50 therapists (i.e., 50 therapists x 5 sessions per week x 75 weeks = 18,750 sessions, with an average of 10 sessions per client). All 5 clinics will start with SAU, and clinics will be randomized to begin LyssnCBT sequentially over time.

The primary data being collected throughout the project are recordings of therapy sessions, which are also collected as part of the typical operating procedures of the Penn Beck Community Initiative (BCI). CBT fidelity will be assessed by AI-generated Cognitive Therapy Rating Scale (CTRS) scores for every recorded therapy session, which will be recorded via the Lyssn platform during both SAU and LyssnCBT phases of the study. For client outcomes, the PHQ-9 and GAD-7 will be collected at each session and client drop-out will be assessed via a brief monthly survey sent out to participating therapists. Finally, after three months of engagement with the LyssnCBT tools, each participating therapist and supervisor will complete our battery of implementation measures, including the system usability scale, AIM, IAM, and FIM.

Being tested
LyssnCBT
Keywords
depression, cognitive behavioral therapy, therapy, evidence-based psychotherapy, training, community mental health, CBT fidelity, anxiety
Medical subject headings
Depression, Anxiety Disorders

What participants receive

What participants receive. In a randomised study each person is assigned to one group by chance.

  • LyssnCBTOther

    LyssnCBT is a technology that allows therapists and supervisors access to tools that assist with assessing CBT session fidelity, including speech-to-text transcription, annotation tools, and AI-generated metrics.

Study groups in NCT05340738
Study groupRoleWhat they receive
LyssnCBTActive comparatorTherapists will use the LyssnCBT tool with clients for recording and session-sharing functionalities. Therapists and supervisors will also have access to LyssnCBT features like speech-to-text transcription, annotation tools, and AI-generated metrics.
Receives: Other: LyssnCBT
SAU (services-as-usual)No interventionTherapists will use the LyssnCBT tool with clients for recording and session-sharing functionalities. No other LyssnCBT features will be available for therapist or supervisor review.

What is being measured

The primary outcome is the main question the study is designed to answer.

  1. Change in therapist CBT fidelity as assessed by CTRS scores

    CBT Fidelity will be assessed by AI-generated Cognitive Therapy Rating Scale (CTRS) scores for every recorded therapy session, which will be recorded via the Lyssn platform during both SAU and LyssnCBT phases of the study. CBT sessions are scored on different session components (agenda, feedback, understanding, etc.) with the CTRS on a scale of 0-6, from "poor" to "excellent", where higher scores indicate a greater degree of CBT fidelity. CTRS deviation scores will be created.Mean CTRS scores during SAU will be estimated for each therapist and for each CTRS item. These means will be subtracted from each corresponding CTRS score during the LyssnCBT phase.

    Measured: Phase 2; Immediately after every recorded therapy session

  2. Acceptability of LyssnCBT tools (Acceptability of Intervention Measure; AIM)

    A brief, 4-item measure of a mental health professional's perception of how acceptable an innovation would be for use in their context. Each statement about the acceptability of the innovation is rated on a 5-point ordinal scale ranging from "completely disagree" to "completely agree," where higher scores indicate higher levels of acceptability. This measure has demonstrated good test-retest reliability and sensitivity to change among community mental health providers.

    Measured: After three months of engagement with the LyssnCBT tools

  3. Appropriateness of LyssnCBT tools (Intervention Appropriateness Measure; IAM)

    A brief, 4-item measure of a mental health professional's perception of how appropriate an innovation would be for use in their context. Each statement about the appropriateness of the innovation is rated on a 5-point ordinal scale ranging from "completely disagree" to "completely agree," where higher scores indicate higher levels of appropriateness. This measure has demonstrated good test-retest reliability and sensitivity to change among community mental health providers.

    Measured: Phase 2; Three months after the site moves from SAU to LyssnCBT

  4. Feasibility of LyssnCBT tools (Feasibility of Intervention Measure; FIM)

    A brief, 4-item measure of a mental health professional's perception of how feasible an innovation would be for use in their context. Each statement about the feasibility of the innovation is rated on a 5-point ordinal scale ranging from "completely disagree" to "completely agree," where higher scores indicate higher levels of feasibility. This measure has demonstrated good test-retest reliability and sensitivity to change among community mental health providers.

    Measured: Phase 2; Three months after the site moves from SAU to LyssnCBT

  5. Usability of LyssnCBT tools (System Usability Scale; SUS)

    A brief, 10-item questionnaire to evaluate an end-user's perceptions of an innovation's usability. Each statement is rated on a 5-point ordinal scale ranging from "strongly disagree" to "strongly agree," with every other question reverse-coded. Scores are converted to a common scale, added together and then multiplied by 2.5 to convert the original scores to a scale of 0-100. "Average" usability is defined as a total score >68

    Measured: Phase 2; Three months after the site moves from SAU to LyssnCBT

  6. Change in client depression symptoms (Patient Health Questionnaire; PHQ-9)

    The PHQ-9 (Patient Health Questionnaire) is a brief, widely used depression inventory assessing frequency of symptoms with 9 total items rated on a likert scale of 0-3. Higher scores indicate a greater frequency of symptoms of depression and presence of a clinically significant depressive disorder. This measure has demonstrated good test-retest reliability and can be completed within 2-3 minutes. This measure will be collected at every therapy session.

    Measured: Immediately before weekly therapy sessions for the duration of treatment or through study completion, a maximum of 18 months

  7. Change in client anxiety symptoms (General Anxiety Disorder; GAD-7)

    The GAD-7 (General Anxiety Disorder) is a brief, widely used anxiety inventory assessing frequency of symptoms with 7 total items rated on a likert scale of 0-3. Higher scores indicate a greater frequency of symptoms of anxiety and presence of a clinically significant anxiety disorder. This measure has demonstrated good test-retest reliability and can be completed within 2-3 minutes. This measure will be collected at every therapy session.

    Measured: Immediately before weekly therapy sessions for the duration of treatment or through study completion, a maximum of 18 months

  8. Client drop-out/premature termination from therapeutic engagement

    Dropout is collected via a brief monthly survey sent out to participating therapists which will ask, for each participating client, whether the client has been discharged and if that discharge was planned.

    Measured: Immediately before weekly therapy sessions for the duration of treatment or through study completion, a maximum of 18 months

Timeline

From the start of the study to its completion; the mark is today. Estimated dates are the sponsor’s plan.

  1. First posted22 Apr 2022
  2. Study start9 Mar 2023
  3. Primary completion31 Oct 2025
  4. Study completion31 Jan 2026

Resources

More details

For doctors and researchers: how the study is designed, who runs it, and its registry IDs.

Study design
Study type
Interventional
Phase
Phase N/A
Primary purpose
Health services research
Allocation
Randomised
Intervention model
Crossover
Model description
Type 2 hybrid implementation-effectiveness, randomized stepped-wedge study comparing LyssnCBT-supported psychotherapy to services as usual (SAU)
Masking
None (open label)
Enrolment
448 participants (actual)
IDs, dates and publications
ClinicalTrials.gov ID
NCT05340738
Sponsor's protocol ID
R42MH128101
Other IDs
R42MH128101
Acronym
AFFECT
Status last verified
Mar 2026
First submitted
18 Feb 2022
Last updated
26 Oct 2026
Results posted
No
FDA-regulated drug
No
FDA-regulated device
No
Individual data sharing
No

Publications

  • Creed TA, Salama L, Slevin R, Tanana M, Imel Z, Narayanan S, Atkins DC. Enhancing the quality of cognitive behavioral therapy in community mental health through artificial intelligence generated fidelity feedback (Project AFFECT): a study protocol. BMC Health Serv Res. 2022 Sep 20;22(1):1177. doi: 10.1186/s12913-022-08519-9.

    PubMed 36127689

Questions about this trial

What is this clinical trial studying?

A study testing LyssnCBT in people with Cognitive Behavioral Therapy and Therapy, run by University of Pennsylvania at 1 site in United States.

The full registered description is in the About this study section and the official record.

Who may be eligible to take part?

The sponsor lists 11 inclusion and 1 exclusion criteria. Eligibility is decided by the study team after screening, so read the full criteria and contact a listed site.

Is this clinical trial currently recruiting?

The record says: finished, as of the sponsor's last update on 26 Oct 2026. Status can change between updates, so confirm with the study contact before making plans.

Where is the study taking place?

The record lists 1 site in United States. Whether a site is taking part right now should be confirmed with that site. See every site.

What are the possible benefits and risks?

They depend on the study and the treatment, and no study can promise a benefit. The study team explains the known risks during informed consent, before anyone joins; the record itself does not list them.

Are results available for this study?

The study is completed, but results are not available in the source record. That does not mean the treatment did or did not work.

How can a doctor refer a patient or contact the research team?

The record lists no contact. The official record names the sponsor, who can be asked directly. AzTrials does not enrol participants or take referrals.

What does the primary outcome measure?

Change in therapist CBT fidelity as assessed by CTRS scores (time frame: Phase 2; Immediately after every recorded therapy session). The study lists 8 primary outcomes in all. It is what the study was designed to measure, not a finding.

Where can I read the original study information?

On the ClinicalTrials.gov record for NCT05340738, which also lists 1 linked publication.

Does a listing mean the treatment is approved or proven to work?

No. A listing means the study is registered. It does not show that the treatment is safe, works, or is approved by a regulator.

How often is this information updated?

AzTrials copies changed records from ClinicalTrials.gov every night and reloads the whole registry every week. The sponsor last updated this record on 26 Oct 2026; AzTrials last refreshed its copy on 30 Oct 2026.

Free account

Keep searching with a free account

You’ve used your 3 free searches for this session. Log in or create a free AzTrials account to keep searching clinical trials and save the ones you want to follow.

Create a free accountLog in