Impact Evaluation of Maternity Homes Access in Zambia

BACKGROUND: Although Zambia's maternal mortality rate has decreased from 591 maternal deaths per 100,000 live births since 2007 (2007 Zambia DHS), the maternal mortality rate in 2014 was unacceptably high at 398 maternal deaths per 100,000 live births. The WHO strongly recommends 'skilled care at every birth' as a strategy to reduce maternal deaths. However, it remains unanswered how to best deliver high-quality birth and postpartum care, particularly in rural and remote areas where distance and poor transportation services severely restrict access to care.

The Government of the Republic of Zambia (GRZ) is committed to improving maternal health and has initiated a large cooperative effort that aims to do so through the Ministry of Health (MOH), Ministry of Community Development, Mother and Child Health (MCDMCH), and several local and international organizations. Within this context, MHs-temporary lodgings located near health facilities with skilled care for pregnant mothers who are close to term-represent a potentially useful strategy to improve access to skilled obstetric care, particularly for those living farthest from the health facilities. MHs provide pregnant women with the option of planning ahead and traveling to health facilities well before labor begins, and have existed in various forms since the 1950s. However, there is insufficient evidence of the effectiveness of MHs in improving access to delivery services and improving maternal outcomes. While some evidence suggests MH utilization is correlated with improved maternal health outcomes (Gaym, Lori), a Cochrane review found no randomized, quasi-randomized or cluster-randomized trials assessing the effectiveness of MHs in low resource settings (van Lonkhuijzen L). Rigorous evidence on the impacts of MHs on maternal and perinatal health outcomes is needed.

The Maternity Homes Alliance (MHA) - a partnership between GRZ and implementing partners working in collaboration with academic institutions, and evaluation advisor, - seek to help women overcome the "distance problem" by implementing MHs, with the hypothesis that offering women access to quality MHs will bring women closer to facility-based delivery and postpartum care, ultimately improving health outcomes. If systematically implemented on a large scale with optimal maintenance, management and community acceptability and utilization, MHs have the potential to remove the distance barrier and increase access to safe delivery for women living farthest from the health facilities.

INTERVENTION: This project will implement high quality MHs in Southern, Eastern and Luapula Provinces in an effort to increase access to quality intrapartum care among the most vulnerable women. Following a formative evaluation that found that MHs could be an acceptable option to improve access to safe deliveries in rural Zambia, in-country partners, ZCAHRD and Africare, will implement a Core Maternity Home Intervention Model in 10 sites each, totaling 20 intervention sites.

OBJECTIVES: The main objective of the evaluation is to understand if MHs can effectively increase access to safe delivery, particularly for women living farthest from health facilities. Findings from this evaluation will be used to inform program decisions regarding implementation of the Maternity Home Model and, if applicable, to advocate for programmatic improvements as the Maternity Home Model is taken to scale beyond the districts proposed for this project.

Primary question:

1. Does the Minimum Core Maternity Home Model increase the number of facility deliveries among mothers living more than 10 km from the facilities compared to facilities with no improved maternity home services?

Secondary questions:

1. To what degree does the Minimum Core Maternity Home Model change maternal and neonatal health outcomes among those living more than 10 km from the facility compared to the standard of care?
2. How does the awareness and perceptions of MHs by communities located more than 10 km from the health facility change over the period of this study?
3. How does awareness and perceptions of HF associated safe delivery and HF delivery intention among pregnant women living in communities located more than 10 km from the health facility change over time?
4. What is the financial impact that use of the MH has on the families of women who utilize it compared to women who do not utilize it?
5. How does the perception of quality of care differ between intervention and comparison sites?

STUDY DESIGN: This is a quasi-experimental pre-post design wherein one implementing partner (BU/ZCAHRD) will use a cluster-randomized matched-pair design and one implementing partner (University of Michigan/Africare) will utilize a non-randomized matched-pair, comparison design. The framework of the study design is shown below:

Table 2: Quasi-experimental Study Design to Evaluate the Impact of MHs

BU/ZCAHRD UMich/Africare Overall R O1 X O2 NR O1 X O2 NR O1 X O2 R O1 _ O2 NR O1 _ O2 NR O1 _ O2 X = Minimum Core Maternity Home (see above) O = Observations at baseline (O1, in 2016) and endline (O2, in 2018) at intervention (X) and comparison (_) sites. R = cluster randomized; NR = not randomized.

Data will be pooled for analysis but sample sizes provide sufficient power to separately analyze the BU/ZCAHRD and UMich/Africare sites.

DATA SOURCES: We will collect data from two main sources:

1. Household Surveys
2. In-depth Interviews

Household Survey (O1, O2): A quantitative household survey will be conducted at baseline (2016) and endline (2018) among women who have delivered in the last 12 months) living more than 10 km from the intervention and comparison facilities. This approach will capture the experiences of those who utilized the facility in their catchment, other facilities, and those did who not access a facility for delivery, allowing us to more accurately estimate the impact of the MH intervention.

In-depth Interviews (IDI) (O1, O2): IDIs will be conducted among a sample of respondents identified for the household survey to gain a deeper understanding of community perceptions. Content will include perceptions of labor and delivery practices, barriers to accessing care, knowledge and awareness of MHs, perceptions of the quality of maternity homes, perceptions of respectful care at the facility, post-natal care, costs, and perceptions of MH ownership.

STUDY SITES: Selection and Randomization BU/ZCAHRD will select the 20 sites or 10 pairs matched on transfer time to CEmONC and clinic delivery volume. Pairs were randomized into the intervention group and control groups, yielding 10 intervention and 10 control sites.

UMich/Africare will purposively select 10 intervention sites and identify 10 comparison sites, matched on transfer time to CEmONC and clinic volume. This totals 20 intervention and 20 comparison sites across the Alliance. The unit of randomization (for BU/ZCAHRD) and matching (both partners) is the facility and its catchment area.

SAMPLING STRATEGY: We will employ multi-stage random sampling. First, we will randomly select a sample of 10 clusters (villages within the catchment area more than 10k from facility) from each study catchment area with probability proportionate to population size. Second, we will list all households within the selected villages, randomly order them, visit each in that order and screen for eligibility, continuing down the list until the nth eligible household in each village is identified. Third, we will randomly select, if applicable, one eligible woman from each household.

FOLLOW UP: At each observation point (O1 and O2) we will select a new cross-sectional sample using the same methodology. Therefore, households will not be followed up over time.

DATA QUALITY CONTROL STRATEGY: For quality control, first, enumerators will participate in a 5-day training. Second, enumerators will be overseen by experienced supervisors. These individuals will review surveys for quality nightly. Third, supervisors will randomly select a 5% subsample of households for each of the enumerators they oversee and revisit these households to re-conduct a subset of survey questions. Data collected by supervisors will be compared to data originally collected by enumerators to check reliability. Fourth, data will be uploaded and transferred nightly to the data analysis team in Boston (see Data Entry and Storage section below) and the analysis team will review in real time for consistency and quality.

DATA ENTRY AND STORAGE: Survey data will be captured on the tablets and saved to the tablets' internal memory. Each evening, a supervisor will review and encrypt surveys, then upload the surveys nightly to a secure server administered by SurveyCTO. The project team will download the encrypted data using the SurveyCTO Client software, and decrypt the data using a decryption key generated by the BU/ZCAHRD team themselves.

The project staff will oversee data entry, management, and storage for qualitative data. All qualitative data will be translated into English and transcribed within 1 month of being collected. Digital recorders and paper copies of qualitative notes will be kept in a locked cabinet until they are fully translated and transcribed, at which point audio files will be deleted and notes shredded. The electronic transcriptions will be kept in a password-protected file. The qualitative transcriptions will not contain identifying information, only a study id number. Only the study team will have access to identifiable quantitative and qualitative data.

SAMPLE SIZE: We will recruit 2,400 women into the survey, 1,200 from each study group per round for a study sample of 4,800 households plus 50 pilots at each round. After accounting for the clustered sampling design (ICC estimated at 0.04 based on previous work), and assuming an alpha of .05, this sample will provide us with 80% power to detect a minimum of 10 percentage point difference in the anticipated impact of the MH intervention on the primary outcome of facility delivery.

For the IDIs, we estimate a sample of 360 at baseline and endline, but will stop if we reach saturation or predictability before the full 360.

DATA ANALYSIS: All quantitative analyses will be conducted in SAS version 9.4 (SAS Institute, Cary, NC). Our quantitative analytic plan is threefold, yielding descriptive, bivariate and multivariate statistics. First, we will begin with a descriptive analysis of the sample for the household level characteristics and respondent level demographics.

Second, we will also look for differences in outcomes within the matched pairs. Categorical variables will be compared between intervention and control groups using the chi-squared test if cell sizes are sufficient or Fisher's exact test if cell sizes are small; continuous variables will be compared using t-tests if normally distributed or non-parametric Wilcoxon rank sum tests if distribution is non-normal.

Third, we will calculate crude and adjusted logistic or linear regression models depending on the outcome variable. We will calculate the relative risk or odds ratios comparing the intervention to control arms, adjusting for any imbalanced covariates.

All qualitative data will be analyzed in Nvivo 10 © (Doncaster, Australia). We will conduct a content analysis of the in-depth interview transcripts. Coding themes have been identified a priori according to the outline of the interview guide. Additional themes will be included if they emerge during analysis. We will triangulate findings with the quantitative data to identify consistencies, inconsistencies or additional themes to be explored.