Pilot Evaluation of "KeepWell" Using a Hybrid Effectiveness-Implementation Pragmatic Randomized Controlled Trial

Methods

Design We will evaluate the effectiveness and uptake of the KeepWell solution in a 6-month, pragmatic, hybrid effectiveness-implementation RCT for optimizing and sustaining the self-management of older adults with multimorbidity. We will use the type 2 design, which facilitates the simultaneously investigation of the effectiveness of an intervention while rigorously testing the implementation strategy. The implementation evaluation will be guided by the Reach, Effectiveness, Adoption, Implementation, Maintenance (RE-AIM) framework.

Theoretical basis of our work We used the Knowledge-to-Action (KTA) model to guide our methods. Our team has experience applying the KTA framework in creating technology-based interventions [refs]. We have also adapted an integrated KT (IKT) strategy whereby all our stakeholder team members who helped us to create KeepWell and prepare this protocol are going to continue to steer the execution of our study. This team helped to develop the objectives, project plan and timelines; discussed potential barriers and mitigating strategies, and monitored progress.

Population and recruitment Older adults will be identified with support from: (i) the University of Toronto Primary Care Research Network (UTOPIAN); (ii) the Retired Teachers of Ontario (RTO); and (iii) our clinician (geriatricians and family physicians) partnering sites: North York General Hospital, St. Michael's hospital of Unity Health, Sunnybrook healthcentre, and St. Peter's hospital. All these recruitment sources have a roster of older adults with multiple chronic conditions via their affiliated primary care and geriatric clinics. We have created a recruitment poster that will be administered at each site according to a procedure most feasible and practical for their practice flow and resources. For example, the poster may be administered directly to older adults by family physicians and geriatricians during a visit or through their clinic staff upon registering for a visit or posted on walls. In addition, the poster will be distributed via an email listserv from our patient partner organization, the RTO with a membership of more than 70,000 older adults. Study eligibility criteria are: (i) age 65 years; (ii) have one or more of the following chronic conditions: diabetes, heart failure, cardiovascular disease, dementia, chronic kidney disease, osteoporosis, osteoarthritis, rheumatoid arthritis, Chronic Obstructive Pulmonary Disease (COPD), depression, urinary incontinence, stroke; (iii) English speaking; (iv) access to a computer or tablet device; (v) have an email address; and (vi) able to consent. Consent-giving capacity will be assessed using the validated "MacCAT-CR" tool.

The recruitment poster includes a phone number and email address dedicated to this research project. Potential participants will self-refer after reviewing the poster, by contacting the research coordinator using one of two strategies: (1) if their mode of reply is by phone, the research coordinator will review the study information sheet and consent process document with potential participants and answer any questions. Study eligibility will also be assessed. Participants will have the option to consent over the phone or via an online survey (SurveyMonkey). Non-consenting respondents will be asked to provide a reason for their decision; (2) if the mode of reply is email, an automated, email response system will be initiated. The email will include the same study and consent process information as described for the phone option and a link to a survey (via SurveyMonkey) that will include questions to assess their eligibility and to obtain conent. Regardless of participants' mode of reply, they will be enrolled and assigned a unique identification number and randomization sequence as soon as the research coordinator receives confirmation of eligibility and consent. Once they do, participants will receive an email with a link to KeepWell and their unique login details (intervention condition) or to a link to an online survey to complete a risk assessment questionnaire (usual care control condition).

Randomization and blinding Consenting participants will be randomly allocated to the intervention (KeepWell) or control (usual care) using a random number generator. Consenting participants will be assigned a login and password by the research coordinator, which they can use to access the KepWell tool. Each individual login and password will be randomly allocated into intervention or control using a 1:1 ratio; participants will be the unit of randomization. The research coordinator will have the master list of logins and allocation. Allocation will thus be concealed because this list will not be shared with the research team. To protect against sources of bias, investigators, outcome assessors and data analysts will be blinded to the randomization sequence. Blinding of older adults will not be possible given that the intervention is a standalone, web-based tool and the control condition is usual care.

Intervention and Control KeepWell is a fully functional, standalone eHealth application aimed at supporting the self-management of older adults with multimorbidity. KeepWell can be used on any electronic device and has innovative features that most other chronic disease solutions don't have: (i) a multi-disease focus (it can generate lifestyle advice for any combination of the top 10 chronic conditions affecting older adults); (ii) an avatar health coach that walks users through a health prioritization and goal setting exercise; (iii) a health risk questionnaire (HRQ) covering three risk dimensions: health (chronic diseases), lifestyle (physical activity, diet, smoking, alcohol, caffeine, bladder health), and social and emotional well-being (social frailty, isolation, loneliness); (iv) an evidence-based Action plan with lifestyle advice customized to user's risks; and (v) other eHealth self-management tactics that have been shown to improve outcomes (interactive lifestyle tracker, journaling. A health coach avatar leads users through a health priority and goal setting exercise that allows them to create a customized action plan based on guideline recommendations for lifestyle changes. Users can then put their plan into action through a lifestyle tracking feature that allows them to track their progress in their identified lifestyle areas. KeepWell also has an extensive resource library, which has links to additional high-quality health and lifestyle information across topics of interest to older adults. Participants allocated to the control condition will receive care as usual but will be asked to complete the health risk questionnaire at baseline, 3- and 6-month follow-up via an online survey to collect outcomes data. The control group will receive full access to KeepWell at the conclusion of the study.

Outcomes Primary outcome: perceived self-efficacy for managing identified chronic diseases or risks measured at 6-months follow-up using a validated 6-item, self-efficacy scale, which is embedded within the health risk questionnaire of the KeepWell platform. We selected this outcome because increasing self-efficacy is a prerequisite for behaviour change which, through improved self-management may influence health and healthcare use. All participants will complete this outcome assessment via the health risk questionnaire of KeepWell at baseline, 3- and 6-month follow-up.

Secondary outcomes: (i) self-efficacy at 3-months follow-up; (ii) cost description analysis to assess the total cost of implementing the KeepWell overall and by stage (e.g., one-time costs vs. ongoing costs); (iii) implementation outcomes according to Proctor et al: reach (access), adoption (uptake, utilization); fidelity; maintenance; acceptability (satisfaction), appropriateness (relevance), feasibility; (iv) patient reported outcomes (self-reported disease management, perceived health and functional status, emotional well-being, QOL); (v) patient-reported experiences (self-reported patient activation, empowerment, patient-practitioner communication, shared decision-making). Outcomes will be measured using the health risk questionnaire embedded within KeepWell and an online survey at final follow-up.

Sample size calculation An increase in self efficacy of 40% is considered significant and has been achieved in other studies of self-management interventions including one of the highest quality studies of a chronic disease self-management program. In their Cochrane review of lay-led self-management interventions, Foster et al found 10 studies that looked at self efficacy and these interventions showed a small, statistically-significant improvement (standardized mean difference -0.30, 95% confidence interval (CI) -0.41 to -0.19). Using these estimates to calculate our sample size, targeting a power of 0.8 and assuming a dropout rate of 25%, we estimate that 220 older adults are needed per group for a total of 440 participants. In a previous interrupted time series study, an Osteoporosis self-management tool housed in a touch-screen laptop computer was implemented across 3 primary care practices for 12 months, with a total of 350 patients who completed the risk assessment questionnaire, so our proposed sample size of 440 patients is feasible, given our hospital and primary care partnerships.

Analysis We will perform descriptive statistics (binary outcomes) and means and standard deviations (continuous outcomes). For the primary outcome, a general linear model will be fit to investigate differences between groups in self-efficacy at 6 months. We will adjust for potential confounders (including baseline self-efficacy) and perform subgroup analyses (patient chronic conditions, risk factors, mode of KeepWell use [Tablet/Laptop], age group [65-74, 75-84, 85+], sex and gender). We will use the chi-squared and Fisher's exact test to compare binary outcomes and independent two sample t-tests for continuous outcomes. We will adjust for potential confounders in the final analysis (e.g., receipt of education about chronic diseases and risks or specialist referrals); and perform subgroup analyses (e.g., setting, gender, and age: 65-74 and 75+). We will perform visual investigation (through scatter plot displays) and analytical outcomes along with logistic regression analyses to determine whether unique patient characteristics predict thresholds of use for different KeepWell components. To examine the change in self-efficacy over time between groups (incorporating self-efficacy scores at baseline, 3- and 6- months), we will use a linear mixed-effects model. For the cost description analysis, we will estimate the cost to delivering KeepWell from the public healthcare payer perspective, including out-of-pocket costs for patients, technical costs for the clinic, efficiencies gained, estimated health care utilization increases / reductions; and exploration resource costs required (e.g., online data collection / analysis, web system testing / hosting). The KeepWell platform collects user data for these measures, so we will be able to track them longitudinally to observe how the solution impacts on outcomes. The findings from the economic analysis will represent the potential preliminary economic impact of KeepWell to the healthcare system. All outcomes will be measured at baseline, 3- and 6-months follow-up and assessed according to intention to treat. All statistical analyses will be carried out using the R statistical software.