Effectiveness of Interactive Voice Response for COVID-19 Vaccination Training in the Democratic Republic of the Congo

Background:

As COVID-19 continues to pose a significant threat worldwide, training health workers to administer the COVID-19 vaccine and encourage uptake of the vaccine is essential to limiting the number of deaths and ultimately ending the pandemic. Providing adequate training to health workers in low- and middle-income countries (LMICs) had already been logistically and economically challenging, but the onset of the pandemic has intensified the challenge of providing critical and rapidly evolving health information to health workers at a time when it is needed most, particularly in low bandwidth areas where access to accurate sources of information are limited.

A growing body of research suggests that effective and efficient training can be provided through mobile health (mHealth) solutions. The rapid expansion of mobile phone technology available throughout LMICs has enabled the scaling of health education delivery via mobile phone. However, while app-based learning has grown in popularity, drawbacks include limited access to smartphones and the data costs to access the internet, if available. Interactive voice response (IVR) systems pose an alternative solution by employing pre-recorded audio files in local languages, allowing users to receive information from mobile phones without the need for internet access.

Aims:

In this study, we investigate the effectiveness of an IVR training intervention for improving knowledge of COVID-19 vaccines and vaccine administration among health workers in the Democratic Republic of the Congo (DRC). We will also examine differences in effectiveness across subgroups including by geographic region, professional roles (e.g., nurses, physicians) and type of health worker facility.

Methods:

We are collaborating with Viamo, a global social enterprise that has established IVR systems in more than 23 countries, to conduct a randomized controlled trial to measure the effects of the IVR remote training on health workers' knowledge and beliefs about COVID-19 vaccines and preparedness to administer vaccines. In coordination with the DRC Ministry of Health, Viamo has identified roughly 9,000 facility-based health workers prioritized for participation. Participants will be randomized 1:1 into a treatment and a control group, stratified by geographic region and professional roles to ensure equal representation across regions and types of health workers.

The participants in the treatment group will receive the intervention at the start of the study while the participants in the control group will receive the intervention following a one-month delay, which will allow for the comparison of knowledge, attitudes and practices with a group that is controlled for bias introduced by time. The duration of the study for participants will be roughly one month, which is the time it takes to complete the 5 required IVR training modules and a baseline and endline survey.

Intervention:

Training content is being developed by Viamo in collaboration with the Stanford Center for Health Education and the DRC Ministry of Public Health. The intervention consists of 5 IVR-based modules conveying information about COVID-19 vaccines and vaccine administration best practices. Module topics include the following: 1) the importance of COVID-19 vaccines and recipient eligibility, 2) overview of and differences between approved COVID-19 vaccines , 3) surveillance of adverse vaccine events, 4) organizing a vaccination site, and safe storage and handling COVID-19 vaccines, and 5) micro planning and communication strategies for COVID-19 vaccination efforts. The training will be offered in five local languages: French, Lingala, Swahili, Kikongo, and Tshiluba.

Sample size calculation:

The DRC Ministry of Health identified roughly 9,000 health workers located in 13 focal provinces for prioritization of the training. With a total sample size of 9,000 (including both arms), we estimate a minimum detectable effect size (MDES) of 5.6 percentage points (at 80 percent power and a 5 percent significance level). Because of the voluntary nature of the IVR training participation, we anticipate a response rate of lower than 50 percent. At 80 percent power and 5 percent significance, a sample size of 2900 participants (over both arms) is needed to detect a 10 percent difference.

Data collection and training:

Data will be collected via the IVR platform. Treatment and control groups will complete a baseline and endline survey administered at the same time. The baseline survey will include a brief knowledge assessment, questions about participants' vaccination status, beliefs about COVID-19 vaccines, and preparedness to administer vaccines. The baseline also includes demographic and background questions about participants' prior work experience. The endline survey includes a more extensive knowledge assessment and questions about participants' vaccination status, beliefs about COVID-19 vaccines, and preparedness to administer vaccines.

All surveys will be completed directly by participants via IVR calls direct to participants' mobile phones. Surveys and training content will include audio prompts to overcome literacy barriers and will be offered in one of five local languages, selected by participants directly.

Participants will be trained on how to use the IVR technology on their mobile devices at the start of the study. Callers interact with pre-recorded audio files in their local languages and can choose what information they want to receive using their mobile phone keypads ("For more information on prevention, please press 1").

While the intervention group will be invited to begin the IVR training immediately after completing the baseline survey, the control arm will have a roughly one month delay before receiving the intervention. Besides the short delay, every aspect for treatment and control arms will remain the same.

Data Analysis:

The primary analysis will be based on intention-to-treat at the unit of the individual CHW. The analysis will include randomization strata fixed effects and control for baseline covariates to improve precision of estimates. Because we will have granular by-module completion data across participants, we will also conduct a treatment-on-the-treated instrumental variables analysis to examine the impact of the training on outcomes, among participants completing the training.