ImplementationWISDOMNCT04096274
Working to Implement and Sustain Digital Outcome Measures
A study testing Leadership for Organization Change and Implementation (LOCI) and Training and Technical Assistance Only in people with Implementation and Behavioral Symptoms, run by Boise State University at 4 sites in United States.
Trial summary
This study will investigate the effects of an organizational implementation strategy called Leadership and Organizational Change for Implementation (LOCI), relative to training and technical assistance only, on fidelity to, and youth service outcomes of, a well-established digital measurement-based care intervention… Read more about this study
- Who is it for?
- Ages 4 years and olderHealthy volunteers can take part
- Conditions the study is for
- Implementation, Behavioral Symptoms
- What is being tested?
- Leadership for Organization Change and Implementation (LOCI) and Training and Technical Assistance Only
- Type of treatment
- Lifestyle or therapy programme
- Could I receive a placebo?
- No, everyone receives an active treatment
- How many people
- 686 participants
- Where
- 4 sitesUnited States
- Study dates
- 1 Oct 2019to 17 Sep 2022
- Phase
- Interventional
- Are there results?
- Yes, posted in the registrySummary in the doctors' view
Key requirements
Who can take part, as listed by the sponsor. Only the study team can confirm whether you are eligible, so take this list to your doctor.
- Ages 4 years and older
- All sexes
- Healthy volunteers accepted
- Child, Adult, Older adult
Who may be able to join
- Provide outpatient psychotherapy services to children ages 4 to 18 who have emotional and/ or behavioral disorders and their families
- Have at least 3 full time equivalent clinicians on staff
- Not currently implementing a digital measurement-based care system clinic wide
- 1. Identified as Chief Executive Officer, Executive Director, or high-level administrator at an enrolled clinic
- 1. Identified as a clinical supervisor or clinical work-group supervisor/ leader at an enrolled clinic
- Employed as staff at participating clinic
Show all 11Show fewer
- Provides outpatient mental health services (psychotherapy) to youth clients
- Inclusion Criteria for Caregivers of Youth Who Receive Services at Participating Clinics
- Child is ages 4 to 18 years at intake,
- Child has been diagnosed with an emotional or behavioral disorder by clinic staff
- Clinical staff at the site determined that the youth's treatment needs can be appropriately addressed by the clinic
Who cannot join
None listed.
Trial locations and contacts
4 sites in 1 country.
United States4 sites
Boise State University
Boise, Idaho, 83725
University of Central Florida
Orlando, Florida, 32816
University of Pennsylvania
Philadelphia, Pennsylvania, 19104
University of California, San Diego
San Diego, California, 92093
About this study
Official title: Randomized Trial of a Leadership and Organizational Change Strategy to Improve the Implementation and Sustainment of Digital Measurement-based Care in Youth Mental Health Services
This study will investigate the effects of an organizational implementation strategy called Leadership and Organizational Change for Implementation (LOCI), relative to training and technical assistance only, on fidelity to, and youth service outcomes of, a well-established digital measurement-based care intervention called the Outcomes Questionnaire-Analyst in outpatient community mental health clinics.
Read the detailed description
Using a cluster randomized, controlled, hybrid type III effectiveness-implementation design, this trial will investigate the effects of LOCI on the fidelity and and clinical outcomes of a digital measurement-based care (MBC) system called the Outcomes Questionnaire Analyst (OQ-A). The trial will enroll up to 22 outpatient mental health clinics that serve youth and randomly assign them using covariate constrained randomization to either LOCI or training and technical assistance only. Within each clinic, up to 2 first level leaders will be recruited (max N of 40 total) and a minimum of 3 clinicians will be recruited per site (60 total). Data on youth outpatients who receive services will be collected in two phases. In each phase, a unique cohort of 360 caregivers of youth who participate in services will be sampled from the participating clinics. Caregivers will report on the service outcomes and experiences of eligible youth who receive services. The total caregiver enrollment for two phases will be 720 (360*2).
- Conditions
- Implementation, Behavioral Symptoms
- Being tested
- Leadership for Organization Change and Implementation (LOCI), Training and Technical Assistance Only
- Keywords
- LOCI, measurement-based care, evidence-based practice, behavioral health, youth mental health, leadership, implementation science
- Medical subject headings
- Behavioral Symptoms
What participants receive
What participants receive. In a randomised study each person is assigned to one group by chance.
Leadership for Organization Change and Implementation (LOCI)Behavioural
LOCI is a multicomponent implementation strategy that engages organizational executives and first-level leaders (i.e., those who administratively supervise clinicians) to build an organizational climate to support the implementation of a focal evidence-based practice (EBP) with fidelity. In this study, the focal EBP is the OQ-A system. LOCI includes two overarching components: (1) monthly organizational strategy meetings between executives and LOCI consultants/trainers to develop and embed policies, procedures, and practices that support implementation of a focal EBP, and (2) training and coaching for first-level leaders, to develop their skills in leading implementation. The aim of these components is to develop an organizational implementation climate in which clinicians' perceive that use of the OQ-A with high fidelity is expected, supported, and rewarded.
Training and Technical Assistance OnlyBehavioural
All leaders and clinicians in participating clinics will receive standardized OQ-A training and technical assistance provided by the OQ-A purveyor organization. This includes an initial, 6-hr, in-person OQ-A training; two, live, virtual, 1-hr booster trainings, offered 3 and 5 months after the initial training; and, year-round technical assistance from the OQ-A purveyor organization. Technical assistance includes virtual training sessions, online library of training videos, and customer care representative for technical support.
In addition, to encourage participation in the study, a set of four 1-hr, web-based general leadership seminars will be offered to leaders in the control condition. These will cover topics ranging from effective leadership, to giving effective feedback.
| Study group | Role | What they receive |
|---|---|---|
| LOCI (Intervention) | Experimental | In clinics assigned to the LOCI condition, executives and first-level leaders will receive leadership training and coaching to support implementation of the OQ-A system. In addition, leaders and clinicians in this condition will receive training and technical assistance to implement the OQ-A measurement-based care system. Receives: Behavioral: Leadership for Organization Change and Implementation (LOCI); Behavioral: Training and Technical Assistance Only |
| Training and Technical Assistance only (Control) | Active comparator | In clinics assigned to the control group, leaders and clinicians will receive training and technical assistance to implement the OQ-A measurement-based care system. In addition, to support enrollment in this condition, leaders in this condition will be offered access to general web-based leadership seminars. Receives: Behavioral: Training and Technical Assistance Only |
What is being measured
The primary outcome is the main question the study is designed to answer.
Percentage Fidelity to the OQ-A System Experienced by the Youth (0-100%)
Fidelity to the OQ-A will be measured by using electronic meta-data from the OQ-A system combined with caregiver reported information on the number of sessions the youth attended. For each youth, a fidelity index will be generated that represents the product of two quantities: (a) the youth's completion rate (i.e., number of measures administered relative to the number of sessions attended within the 6-month observation period), and (b) the youth's viewing rate (i.e., the number of feedback reports viewed by the clinician relative to the number of measures administered). Note that this product is equivalent to the ratio of viewed feedback reports to total sessions; it represents an events/trials proportion. MBC fidelity index scores summarize the level of MBC fidelity experienced by each youth (range=0-1). Higher scores indicate the youth experienced greater fidelity to MBC.
Measured: 0-6 months after youth's baseline/ entry into treatment
Change From Baseline to 6-months in Youth Total Problems Score on the Short Form Assessment for Children (SAC) - Phase I Cohort
The SAC Total Problem Score is a 48-item measure of youth internalizing (e.g., anxious, depressed) and externalizing (e.g., aggressive, noncompliant, overactive) behaviors (range=0-96) completed by caregivers of youth. Total Problem Score was assessed at baseline (youth's entry into treatment) and monthly for 6 months, change from baseline to month 6 is reported.
Measured: 0-6 months after youth's baseline/ entry into treatment
Change From Baseline to 6-months in Youth Total Problems Score on the Short-form Assessment for Children (SAC) - Phase II Cohort
The SAC Total Problem Score is a 48-item measure of youth internalizing (e.g., anxious, depressed) and externalizing (e.g., aggressive, noncompliant, overactive) behaviors (range=0-96) completed by caregivers of youth. Total Problem Score was assessed at baseline (youth's entry into treatment) and monthly for 6 months, change from baseline to month 6 is reported.
Measured: 0-6 months after youth's baseline/ entry into treatment
Timeline
From the start of the study to its completion; the mark is today. Estimated dates are the sponsor’s plan.
- First posted19 Sep 2019
- Study start1 Oct 2019
- Primary completion17 Sep 2022
- Study completion17 Sep 2022
Resources
- Official record on ClinicalTrials.govThe registry entry this page is built from, with its full change history.
- Posted resultsParticipant flow, outcome measures and adverse events reported by the sponsor.
- Publications (5)Papers the sponsor has linked to this study.
- Other trials for ImplementationStudies of the same condition, open and under way.
- What the trial phases meanHow a treatment moves from first tests in people to approval.
More details
For doctors and researchers: how the study is designed, who runs it, and its registry IDs.
Study design
- Study type
- Interventional
- Phase
- Phase N/A
- Primary purpose
- Health services research
- Allocation
- Randomised
- Intervention model
- Parallel groups
- Masking
- Double blind
- Who is masked
- Participant, Care provider
- Enrolment
- 686 participants (actual)
Sponsor and investigators
- Lead sponsor
- Boise State University
- Sponsor type
- University, hospital or other
- Collaborators
- University of California, San Diego, University of Central Florida, University of Pennsylvania, National Institute of Mental Health (NIMH)
- Responsible party
- Sponsor
- Investigators
- Nate Williams, PhD, Principal investigator, Boise State University
IDs, dates and publications
- ClinicalTrials.gov ID
- NCT04096274
- Sponsor's protocol ID
- 8424
- Other IDs
- R01MH119127
- Acronym
- WISDOM
- Status last verified
- Jul 2022
- First submitted
- 9 Sep 2019
- Last updated
- 20 Feb 2024
- Results posted
- Yes, on ClinicalTrials.gov
- Data monitoring committee
- Yes
- FDA-regulated drug
- No
- FDA-regulated device
- No
- Individual data sharing
- No
Publications
Williams NJ, Choy-Brown M, Vega N, Aarons GA, Ehrhart MG, Marcus SC. Testing normalization process theory in a randomized trial of mental health clinics implementing digital measurement-based care. Implement Sci. 2026 Feb 9;21(1):16. doi: 10.1186/s13012-026-01485-4.
PubMed 41656239Williams NJ, Gomes AE, Vega NR, Esp S, Choy-Brown M, Beidas RS. A multilevel framework for recruitment and retention in implementation trials: An illustrative example. Clin Trials. 2025 Jun;22(3):325-341. doi: 10.1177/17407745241307948. Epub 2025 Jan 10.
PubMed 39797386Williams NJ, Ehrhart MG, Aarons GA, Esp S, Sklar M, Carandang K, Vega NR, Brookman-Frazee L, Marcus SC. Improving measurement-based care implementation in youth mental health through organizational leadership and climate: a mechanistic analysis within a randomized trial. Implement Sci. 2024 Mar 28;19(1):29. doi: 10.1186/s13012-024-01356-w.
PubMed 38549122Williams NJ, Marcus SC, Ehrhart MG, Sklar M, Esp SM, Carandang K, Vega N, Gomes AE, Brookman-Frazee L, Aarons GA. Randomized Trial of an Organizational Implementation Strategy to Improve Measurement-Based Care Fidelity and Youth Outcomes in Community Mental Health. J Am Acad Child Adolesc Psychiatry. 2024 Oct;63(10):991-1004. doi: 10.1016/j.jaac.2023.11.010. Epub 2023 Dec 7.
PubMed 38070868Choy-Brown M, Williams NJ, Ramirez N, Esp S. Psychometric evaluation of a pragmatic measure of clinical supervision as an implementation strategy. Implement Sci Commun. 2023 Apr 6;4(1):39. doi: 10.1186/s43058-023-00419-1.
PubMed 37024945
Study at a glance
From the registry record, last updated by the sponsor on 20 Feb 2024. Recruitment status is as the sponsor reported it: confirm with the study team.
- Official title
- Randomized Trial of a Leadership and Organizational Change Strategy to Improve the Implementation and Sustainment of Digital Measurement-based Care in Youth Mental Health Services
- ClinicalTrials.gov ID
- NCT04096274
- Acronym / sponsor ID
- WISDOM / 8424
- Conditions
- Implementation, Behavioral Symptoms
- Keywords
- LOCI, measurement-based care, evidence-based practice, behavioral health, youth mental health, leadership, implementation science
- Study type
- Interventional
- Recruitment status
- Completed
- Enrolment
- 686 participants (actual)
- Lead sponsor
- Boise State UniversityUniversity, hospital or other
- Collaborators
- University of California, San Diego, University of Central Florida, University of Pennsylvania, National Institute of Mental Health (NIMH)
- Study start
- 1 Oct 2019 (actual)
- Primary completion
- 17 Sep 2022 (actual)
- Study completion
- 17 Sep 2022 (actual)
- First posted
- 19 Sep 2019 (actual)
- Results
- Posted 20 Feb 2024
Clinical summary
The research question as the sponsor registered it. This describes what the study plans to test, not what it found.
This study will investigate the effects of an organizational implementation strategy called Leadership and Organizational Change for Implementation (LOCI), relative to training and technical assistance only, on fidelity to, and youth service outcomes of, a well-established digital measurement-based care intervention called the Outcomes Questionnaire-Analyst in outpatient community mental health clinics.
Detailed description
Using a cluster randomized, controlled, hybrid type III effectiveness-implementation design, this trial will investigate the effects of LOCI on the fidelity and and clinical outcomes of a digital measurement-based care (MBC) system called the Outcomes Questionnaire Analyst (OQ-A). The trial will enroll up to 22 outpatient mental health clinics that serve youth and randomly assign them using covariate constrained randomization to either LOCI or training and technical assistance only. Within each clinic, up to 2 first level leaders will be recruited (max N of 40 total) and a minimum of 3 clinicians will be recruited per site (60 total). Data on youth outpatients who receive services will be collected in two phases. In each phase, a unique cohort of 360 caregivers of youth who participate in services will be sampled from the participating clinics. Caregivers will report on the service outcomes and experiences of eligible youth who receive services. The total caregiver enrollment for two phases will be 720 (360*2).
Study design
- Primary purpose
- Health services research
- Allocation
- Randomised
- Intervention model
- Parallel groups
- Masking
- Double blind (participant, care provider)
- Number of arms
- 2
| Arm | Type | Receives |
|---|---|---|
| LOCI (Intervention) | Experimental | Behavioral: Leadership for Organization Change and Implementation (LOCI), Behavioral: Training and Technical Assistance OnlyIn clinics assigned to the LOCI condition, executives and first-level leaders will receive leadership training and coaching to support implementation of the OQ-A system. In addition, leaders and clinicians in this condition will receive training and technical assistance to implement the OQ-A measurement-based care system. |
| Training and Technical Assistance only (Control) | Active comparator | Behavioral: Training and Technical Assistance OnlyIn clinics assigned to the control group, leaders and clinicians will receive training and technical assistance to implement the OQ-A measurement-based care system. In addition, to support enrollment in this condition, leaders in this condition will be offered access to general web-based leadership seminars. |
Who may qualify
- Ages
- 4 years and older
- Sex
- All
- Healthy volunteers
- Accepted
- Age groups
- Child, Adult, Older adult
Key inclusion criteria (11 listed)
- Provide outpatient psychotherapy services to children ages 4 to 18 who have emotional and/ or behavioral disorders and their families
- Have at least 3 full time equivalent clinicians on staff
- Not currently implementing a digital measurement-based care system clinic wide
- 1. Identified as Chief Executive Officer, Executive Director, or high-level administrator at an enrolled clinic
- 1. Identified as a clinical supervisor or clinical work-group supervisor/ leader at an enrolled clinic
Key exclusion criteria (0 listed)
Eligibility is decided by the study team after screening; this overview does not establish that a patient qualifies. Full inclusion and exclusion criteria.
Interventions
| Intervention | Type | Arms |
|---|---|---|
| Leadership for Organization Change and Implementation (LOCI)LOCI is a multicomponent implementation strategy that engages organizational executives and first-level leaders (i.e., those who administratively supervise clinicians) to build an organizational climate to support the implementation of a focal evidence-based practice (EBP) with fidelity. In this study, the focal EBP is the OQ-A system. LOCI includes two overarching components: (1) monthly organizational strategy meetings between executives and LOCI consultants/trainers to develop and embed policies, procedures, and practices that support implementation of a focal EBP, and (2) training and coaching for first-level leaders, to develop their skills in leading implementation. The aim of these components is to develop an organizational implementation climate in which clinicians' perceive that use of the OQ-A with high fidelity is expected, supported, and rewarded. | Behavioural | LOCI (Intervention) |
| Training and Technical Assistance OnlyAll leaders and clinicians in participating clinics will receive standardized OQ-A training and technical assistance provided by the OQ-A purveyor organization. This includes an initial, 6-hr, in-person OQ-A training; two, live, virtual, 1-hr booster trainings, offered 3 and 5 months after the initial training; and, year-round technical assistance from the OQ-A purveyor organization. Technical assistance includes virtual training sessions, online library of training videos, and customer care representative for technical support. In addition, to encourage participation in the study, a set of four 1-hr, web-based general leadership seminars will be offered to leaders in the control condition. These will cover topics ranging from effective leadership, to giving effective feedback. | Behavioural | LOCI (Intervention), Training and Technical Assistance only (Control) |
Primary outcome
Percentage Fidelity to the OQ-A System Experienced by the Youth (0-100%)
Time frame: 0-6 months after youth's baseline/ entry into treatment
Fidelity to the OQ-A will be measured by using electronic meta-data from the OQ-A system combined with caregiver reported information on the number of sessions the youth attended. For each youth, a fidelity index will be generated that represents the product of two quantities: (a) the youth's completion rate (i.e., number of measures administered relative to the number of sessions attended within the 6-month observation period), and (b) the youth's viewing rate (i.e., the number of feedback reports viewed by the clinician relative to the number of measures administered). Note that this product is equivalent to the ratio of viewed feedback reports to total sessions; it represents an events/trials proportion. MBC fidelity index scores summarize the level of MBC fidelity experienced by each youth (range=0-1). Higher scores indicate the youth experienced greater fidelity to MBC.
Change From Baseline to 6-months in Youth Total Problems Score on the Short Form Assessment for Children (SAC) - Phase I Cohort
Time frame: 0-6 months after youth's baseline/ entry into treatment
The SAC Total Problem Score is a 48-item measure of youth internalizing (e.g., anxious, depressed) and externalizing (e.g., aggressive, noncompliant, overactive) behaviors (range=0-96) completed by caregivers of youth. Total Problem Score was assessed at baseline (youth's entry into treatment) and monthly for 6 months, change from baseline to month 6 is reported.
Change From Baseline to 6-months in Youth Total Problems Score on the Short-form Assessment for Children (SAC) - Phase II Cohort
Time frame: 0-6 months after youth's baseline/ entry into treatment
The SAC Total Problem Score is a 48-item measure of youth internalizing (e.g., anxious, depressed) and externalizing (e.g., aggressive, noncompliant, overactive) behaviors (range=0-96) completed by caregivers of youth. Total Problem Score was assessed at baseline (youth's entry into treatment) and monthly for 6 months, change from baseline to month 6 is reported.
Results and safety
Loading the posted results from ClinicalTrials.gov…
Research links
- Official record on ClinicalTrials.govLast updated by the sponsor 20 Feb 2024.
- Posted resultsParticipant flow, baseline, all outcome measures and adverse events.
- Derived from this study: PubMed 41656239Williams NJ, Choy-Brown M, Vega N, Aarons GA, Ehrhart MG, Marcus SC. Testing normalization process theory in a randomized trial of mental health clinics implementing digital measurement-based care. Implement Sci. 2026 Feb 9;21(1):16. doi: 10.1186/s13012-026-01485-4.
- Derived from this study: PubMed 39797386Williams NJ, Gomes AE, Vega NR, Esp S, Choy-Brown M, Beidas RS. A multilevel framework for recruitment and retention in implementation trials: An illustrative example. Clin Trials. 2025 Jun;22(3):325-341. doi: 10.1177/17407745241307948. Epub 2025 Jan 10.
- Derived from this study: PubMed 38549122Williams NJ, Ehrhart MG, Aarons GA, Esp S, Sklar M, Carandang K, Vega NR, Brookman-Frazee L, Marcus SC. Improving measurement-based care implementation in youth mental health through organizational leadership and climate: a mechanistic analysis within a randomized trial. Implement Sci. 2024 Mar 28;19(1):29. doi: 10.1186/s13012-024-01356-w.
- Derived from this study: PubMed 38070868Williams NJ, Marcus SC, Ehrhart MG, Sklar M, Esp SM, Carandang K, Vega N, Gomes AE, Brookman-Frazee L, Aarons GA. Randomized Trial of an Organizational Implementation Strategy to Improve Measurement-Based Care Fidelity and Youth Outcomes in Community Mental Health. J Am Acad Child Adolesc Psychiatry. 2024 Oct;63(10):991-1004. doi: 10.1016/j.jaac.2023.11.010. Epub 2023 Dec 7.
- Derived from this study: PubMed 37024945Choy-Brown M, Williams NJ, Ramirez N, Esp S. Psychometric evaluation of a pragmatic measure of clinical supervision as an implementation strategy. Implement Sci Commun. 2023 Apr 6;4(1):39. doi: 10.1186/s43058-023-00419-1.
- Information about Leadership for Organizational Change for Implementation (LOCI).Link provided by the sponsor.
- Information about the digital measurement-based care system, OQ-A.Link provided by the sponsor.
- Individual participant dataSharing planned: No
Related trials for Implementation
Recruiting
Implementation of a Pain Management Plan in Major Open Abdominal Surgeries.
Abdomen Cancer, Pancreas Adenocarcinoma (MSI-H)
RecruitingObservational study
Avera Cancer Sequencing and Analytics Protocol (ASAP)
Cancer, Cancer Diagnosis
Not yet recruiting
Diagnostic Accuracy, Clinical Utility and Implementation of Point-of-Care Absolute Neutrophil Count in Suspected Neutropenic Fever (DUCATIS) - Diagnostic Accuracy (S1)
Suspected Neutropenic Fever, Confirmed Neutropenic Fever
Not yet recruitingPhase 4
Starting HIV Prevention Medicine (PrEP) During an Emergency Department Visit With a Peer Navigator
HIV Prevention, Pre-Exposure Prophylaxis (PrEP)
Not yet recruiting
Short-Term Enteral Nutrition for Infants With Bronchiolitis in Pediatric Emergency Departments
Bronchiolitis, Clinical Phenotype
Referral and contact
AzTrials does not enrol participants or take referrals. Contact the study team or a site listed below; they decide who can take part. This study is not recruiting at the moment.
Nate Williams, PhD principal investigator
Boise State University
4 sites in 1 country: see every site and its status.
Questions about this trial
What is this clinical trial studying?
A study testing Leadership for Organization Change and Implementation (LOCI) and Training and Technical Assistance Only in people with Implementation and Behavioral Symptoms, run by Boise State University at 4 sites in United States.
The full registered description is in the About this study section and the official record.
Who may be eligible to take part?
The sponsor lists 11 inclusion and 0 exclusion criteria. Eligibility is decided by the study team after screening, so read the full criteria and contact a listed site.
Is this clinical trial currently recruiting?
The record says: finished, as of the sponsor's last update on 20 Feb 2024. Status can change between updates, so confirm with the study contact before making plans.
Where is the study taking place?
The record lists 4 sites in United States. Whether a site is taking part right now should be confirmed with that site. See every site.
What are the possible benefits and risks?
They depend on the study and the treatment, and no study can promise a benefit. The study team explains the known risks during informed consent, before anyone joins; the record itself does not list them.
Are results available for this study?
Yes. Results are posted in the registry (first posted 20 Feb 2024). A summary is in the For doctors view, and the full results are on ClinicalTrials.gov.
How can a doctor refer a patient or contact the research team?
The record lists no contact. The official record names the sponsor, who can be asked directly. AzTrials does not enrol participants or take referrals.
What does the primary outcome measure?
Percentage Fidelity to the OQ-A System Experienced by the Youth (0-100%) (time frame: 0-6 months after youth's baseline/ entry into treatment). The study lists 3 primary outcomes in all. It is what the study was designed to measure, not a finding.
Where can I read the original study information?
On the ClinicalTrials.gov record for NCT04096274, which also lists 5 linked publications.
Does a listing mean the treatment is approved or proven to work?
No. A listing means the study is registered. It does not show that the treatment is safe, works, or is approved by a regulator.
How often is this information updated?
AzTrials copies changed records from ClinicalTrials.gov every night and reloads the whole registry every week. The sponsor last updated this record on 20 Feb 2024.