Developing a Smartphone Application to Support Veteran Opioid Use Disorder Treatment

PHASE 1: Initial Development of the ACT to RECOVER System Phase 1: Development Phase (Months 0-15; 15 months): Veterans in MOUD (n=10) and VA MOUD clinical staff (n=10) will provide feedback and input on the intervention content drafts developed by the study team.

ACT to RECOVER Content. ACT to RECOVER will be designed to act as a flexible, semi- structured, user-friendly mobile application adjunct to MOUD. Content will include components appropriate for either self-management and/or use with providers. ACT to RECOVER will be designed to have major ACT-focused modules completable within 1 month, though Veterans will be able to continue using the app and its features after study completion, upon request. Core content for ACT to RECOVER's modules will be adapted from major ACT texts (e.g., Hayes and colleagues' Acceptance and Commitment Therapy: The Process and Practice of Mindful Change) and technology (e.g., VA ACT Coach). Core Content modules will cover: 1) Behavior change and addiction triggers, 2) values clarification, 3) acceptance, 4) cognitive defusing, 5) committed action, and 6) willingness. Additional "general" content modules will also be included (e.g., providing support for medication and appointment tracking) to support treatment engagement.

Phase 1 Qualitative Interviews Interview Guide. Interviews will include open-ended questions on ACT to RECOVER's acceptability, feasibility of engagement, potential concerns, and initial reactions. Interview questions are based on the Practical, Robust Implementation and Sustainability Model (PRISM).

Interview Analysis Qualitative data analysis will use a thematic approach with the goal of ascertaining intervention acceptability, feasibility of engagement, and referral issues. Two independent reviewers will code each interview transcription using open-ended coding. Each element will be summarized separately, and themes and key points will be generated to assess data saturation. Two reviewers will analyze the qualitative data using modified grounded coding to identify themes from audio recordings after each interview. Intervention areas will be modified as feedback is given. Each set of interviews will take place with 3-4 participants per group. The number of changes needed for each intervention area will be tracked; it is anticipated that less intervention refinement will be needed by the end of the interview cycle. If later interviews still suggest a need for significant refinement (e.g., increasing number of changes needed), further testing will be done.

PHASE 2: Iterative Usability Testing Phase 2: Pilot Evaluation of Intervention Components in Field-Based Iterative Usability Testing (Months 15-33; 18 months): Veterans will be recruited in MOUD (N=12-15) to conduct usability and content acceptability testing of ACT to RECOVER across 3 iterative waves (n=4-5 per wave).

Iterative Usability Assessment Procedures Eligible Veterans will come to the VACHS West Haven campus to complete baseline measures. Following, Veterans will be provided with a smartphone containing a beta-version of ACT to RECOVER. First, Veterans will be walked through portions of the mobile application and its user interface. Different images for options of the ACT to RECOVER app, as well as a PowerPoint with information about the user interface will be shown to simulate the intervention and its user interface. Second, Veterans will be provided with written instructions and asked to do a behavioral walk-through of a portion of the ACT to RECOVER system using the beta app version and provided PowerPoint slides. Third, Veterans will be asked to move through the ACT to RECOVER system unguided. Throughout, a combined usability and contextual interview approach will be used in which participant system use is recorded while they are asked questions about their engagement and preferences. Finally, participants will complete acceptability and usability measures, as well as a semi-structured qualitative exit interview. An average of 2 weeks between waves to allow time for Geisel Software Inc. to address identified functionality and interface concerns. Go/no-go limits to determine whether Phase 3 should commence are set a priori (see section 6b. Outcome Measures).

Data Collection and Analysis Quantitative statistics will be computed and reported to describe the sample and assess for pre-defined go/no-go limits for the next Phase. In accordance with an adapted Integrated Technology Acceptance Model (ITAM) framework, correlations will also be assessed between health factors and demographic information with intrinsic motivation, perceived ease of use/usefulness, and behavioral intent to use ACT to RECOVER. It is expected that positive correlations between all ITAM constructs will be present.

Qualitative Combined Usability and Contextual Interview Analysis. During usability sessions, participants will assess ACT to RECOVER using a "think-aloud" protocol, verbalizing their thoughts as they navigate the app. After a period of exploration with the app, the research staff will use a pre-created protocol to ask Veterans to perform a series of tasks (e.g., accessing different modules). Session and on-screen recording will occur alongside real-time observation. Additionally, Veterans will complete a series of brief usability questionnaires following user testing and the qualitative interview.

A modified consensual qualitative research (CQR) approach will be used to code the session transcriptions. Research team investigators initial codebook based on key activities and questions in the user-testing protocol will be developed and employed for identifying usability themes within screen-recorded videos, with additional research staff identified to act as a code auditor: (a) Two coders utilize the initial codebook throughout data analysis to foster multiple perspectives; (b) consensus is reached between coders about the meaning of the data; (c) at least one auditor must check the work of the primary team of judges to minimize potential bias; and (d) domain and theme frequency are reported.

Themes will then be categorized according to the following CQR groupings: general (include all or all but one of the cases), typical (include more than half of the cases up to the cut off for general), and variant (at least two cases up to the cutoff for typical). General comments will be classified regarding usability as necessary to include in iterative development, typical and variant will be discussed within the research group to decide upon potential ways to incorporate this feedback into intervention refinement. Feedback will then be triangulated per testing wave by 1) summarizing suggested changes, 2) obtaining study team input, and 3) meeting with the software development team to implement changes.

PHASE 3: Stage 1b Pilot RCT Phase 3: Stage 1b Pilot RCT (Months 33-60; 27 months): A Stage 1b RCT pilot will be conducted to assess the acceptability (treatment satisfaction, perceived ease of use and usefulness, behavioral intent to use, actual use) and feasibility (recruitment, treatment receptivity, sustained participation, and attrition in each condition) of comparing ACT to RECOVER (n=20) to a smartphone-based symptom monitoring control condition (n=20). Phase 3 will focus on (3a) assessing the acceptability and feasibility of the intervention and research procedures and (3b) exploring the data characteristics of, and describing changes in, functional (e.g., values-consistent living) and clinical outcomes (e.g., illicit opioid use).

Stage 1b Pilot RCT Procedures At the start of this visit, research staff will confirm participant eligibility and verify informed consent. Following, Veterans will complete baseline demographic and health measures, as well as relevant outcome measures. Veterans will have been randomized to receive either ACT to RECOVER (treatment arm) or smartphone-based symptom monitoring (control) condition prior to their baseline study visit, with instructions for the appropriate app download and use provided during the baseline session (please refer to section 7. Arms \& Interventions for details). Veterans will be asked to use the respective condition's app over the next 5 weeks and to attend midpoint (week 2), endpoint (week 4), and follow-up (week 5) study visits, during which indicated measures will be collected (see section 6b. Outcome Measures).

Semi-structured qualitative exit interview. A 30-minute, semi-structured qualitative interview will be conducted with Veterans in both arms during the endpoint assessment, to further assess acceptability, feasibility, usability, and dissemination issues. Questions will inquire about any problems with app access and use, perceived effects, if Veterans would recommend the app to a friend, and recommended changes.

Data Management and Statistical Analysis Power analysis. Consistent with the NIH Stage model for intervention development, this Stage 1b RCT will focus on determining the acceptability and feasibility of a given intervention and study procedures (vs. efficacy testing), and accordingly proposes a sample of N=40 (n=20 eligible participants per group).

Primary Analyses and Hypotheses, Stage 1b RCT - Aim 3a: Acceptability and feasibility of study intervention and procedures. Targets for aim 3a include descriptively assessing if the ACT to RECOVER and smartphone-based symptom monitoring control apps reach thresholds for intervention acceptability, feasibility, and maintenance of usability thresholds (see section 6b Study Outcomes for details on these measures and thresholds). If study thresholds are not met, the study team and Geisel Software's development team will convene to discuss further intervention and/or study procedure refinement, which may include additional data collection until thresholds are met. Threshold achievement would indicate the ACT to RECOVER system and trial procedures are suitable for a subsequent efficacy trial.

Aim 3a Hypotheses:

It is hypothesized that all constructs will positively correlate with each other (e.g., greater medical care satisfaction, info seeking, and health care need will positively correlate with intrinsic motivation for app use, which will positively correlate with perceived ease of use and usefulness, behavioral intention to use the app, and ultimately with actual app use). Analysis of post-intervention interviews will also be done using notes taken by study staff during the exit interview; interviews will also be audiotaped and transcribed for later analysis. Research staff will apply CRQ qualitative analytic techniques to these data (e.g., coding and extracting major themes) to assess barriers and facilitators to app use per condition.

Secondary Analyses, Stage 1b RCT - Aim 3b: Descriptive analyses to provide preliminary empirical evidence for a subsequent efficacy trial and inform its design.

Consistent with the Stage of Phase 3's proposed trial, Aim 3b will also assess if the proposed study procedures are suitable to collect the requisite data for future efficacy testing. For each condition, the study team will calculate descriptive statistics, assess univariate distributions, explore bivariate relationships, and conduct paired-samples t-tests to evaluate the potential changes in functional and clinical outcomes within-subjects. These analyses will examine baseline in comparison to each subsequent study interval (i.e., baseline vs. midpoint, baseline vs. endpoint, baseline vs. 5-week follow-up), and will be conducted on the whole sample, as well as stratified by group.

Aim 3b Hypotheses:

It is hypothesized that those in the ACT to RECOVER condition will demonstrate an increase in values-based living, quality of life, and MOUD compliance for Veterans in the ACT to RECOVER condition, as well as a decrease in behavioral avoidance, illicit opioid use, and substance-related problems.