Families First Program Evaluation in Indonesia

Aims and objectives The primary objective of this study is to compare, among caregivers of children aged 0-7 years in Cianjur district, Indonesia, the effectiveness of the Families First Program in reducing rates of violent discipline episodes in the home in comparison to rates observed under the standard community health and social service provision. The study will also identify unintended positive and negative effects of this program; determine facilitators and barriers to its implementation in this context; and assess facilitators, mentors, and families' satisfaction with the program.

Site eligibility Data will be collected across 20 villages or clusters that represent the general population in four sub-districts (Campaka, Ciranjang, Karang tengah, and Naringgul) in Cianjur district, West Java Province, Indonesia. Selected villages will consist of (a) rural and urban/peri-urban communities; (b) not been exposed to the Families First Program; (c) reachable within a 1-hour driving distance of the sub-district office and 1-8 hours driving distance of the district capital; (d) located at ample distance from other clusters; and (e) local political leaders provide approval for the intervention and the evaluation.

Randomization Stratified randomization will be used to select 20 clusters among the eligible 38 study clusters in rural and urban settings in the four sub-districts in a 1:1 ratio for the intervention and the control arms. Clusters will be randomized to avoid the resentment or contamination that might occur as a result of delivering the intervention to some families but not others in the community. Randomization will be done in a Random Allocation Ceremony led by the research team and attended by the chiefs from all participating villages as well as staff from the implementing agency. The ceremony will follow a step-by-step lottery and documentation procedure that includes the use of 20 sequentially numbered, opaque, sealed and stapled envelopes independently prepared by the lead investigator. Aluminium foil inside the envelope will render the envelope impermeable to intense light. The ceremony will be video-recorded. Eligible families will be identified before randomly assigning villages.

Measurement strategy The outcome measures for this study are given in the measurement section of the registration document. The original measurement strategy was to use full measures that have already been applied in this field. Piloting of the original plan indicated that this would not be feasible as there were too many items, many of which would not be appropriate or relevant to the context. Instead, the measurement strategy was to use outcome indicators: single items; indices by cumulating responses over several related items; or full measures/subscales with total scores validated using Rasch analysis.

Socio-demographic covariates Socio-demographic characteristics that will be documented include: a) Child: age, gender, orphanhood status, siblings, physical health, and school status; (b) Mother/caretaker: age, gender, marital status, education level, physical and mental health, ethnicity and cultural background, school/employment status; (c) Family: urban/rural location \& type of residence, household structure, family structure, economy, and functioning.

Qualitative and process evaluation assessments Linked qualitative methods will be used to develop and validate data collection measures (pilot) and to add depth and understanding of process and outcomes. During the randomized trial stages, they will explore caregivers' and implementers' experiences in the program and how service delivery, socio-economic, cultural, and other factors may impact the effectiveness and scalability of the intervention. Qualitative methods (in Bahasa and Sundanese) include semi-structured interviews with caregivers, implementers (Save the Children/non-governmental organization staff, mentors, and facilitators) and community leaders; focus group interviews with participating caregivers and implementers; observation of group sessions; record analysis; and visual methods.

Facilitator's tracking forms will be used to record attendance and level of engagement by caregivers, as well as the extent to which facilitators followed the program manual. Process evaluation outcomes will be assessed through checklists to track implementation fidelity, program exposure/attendance, and a brief program satisfaction questionnaire for caregivers in intervention sites, facilitators, and mentors post-intervention. Implementer records of program attendance and fidelity in the delivery of the intervention (as assessed by observation of sessions by mentors) will also be reviewed.

Statistical analysis and sample size calculation The study is powered to detect a change considered to be both significant clinically and plausible on the primary outcome. A sample size of N = 720 families will allow measuring a reduction of 15.2% in parent-reported use of violent discipline at home from the initial estimated prevalence of 30% (SKTA 2013) with an α level of 0.05 and 90% power, assuming 0.02 intra-class correlation and 80% participation rate. While each family will likely have more than one child, the outcome concordance across children is likely to be very high (>0.8) and so the sample size calculation considers the number of families, recognizing that the unit of analysis will be the child.

Types of analysis There will be only one analysis when all participants have been randomized. There will be administrative analyses done to ensure data accuracy on a continual basis. A baseline table reflecting the demographic profile of the households and caregivers by arm will be compiled.

The primary outcome is frequency of three types of punishment practices in the past month which will be aggregated into a cumulative rate of number of punishments over days. For comparative purposes, the primary outcome will be evaluated as any- vs. non-punishment. This rate will be ascertained for all participants in both trial arms three times. The main analysis will be logistic regression using generalized estimating equations (GEE) to nest families within villages to test the main hypothesis related to the superiority of the program in reducing the rate of physical and emotional punishment over the period of the study. Outcomes will be measured at baseline, end of intervention, and follow-up (i.e., 6 months after end of intervention). The primary comparison is at the end of the intervention; comparisons at follow-up are considered exploratory. The superiority of the program will be tested by including main effects for program and for time along with a program by time interaction. The linearity of the association with time will be considered to allow for the possibility of non-linear associations. Variables which reduce variance in the outcome will be considered for inclusion (sex and position (parent or child) of the reporting family member, number of children in the family, etc.). Although confounding is not expected to be an issue due to the randomization, villages will be compared on key factors and any variables showing large imbalances by randomization group will be added to the model. The investigators will adopt an intention-to-treat principle to analyze all outcomes and all persons will be analyzed in the groups to which they were randomized. A secondary analysis will estimate the impact of the program the other relevant outcomes. Program effects will also be examined using GEE to adjust for clustering of members within families and families within villages. Counts will be analyzed with poisson regression, binary outcomes with logistic regression and continuous scales with linear regression. The investigators will conduct a gender-based analysis (sex of the child) for exploratory purposes. The investigators will also explore whether families with more children or children of different age groups, or families with differently-abled children respond differently to the program. Another sub-group analyses will explore the impact of the intervention among people in single-headed households, first-time parents, and teenage parents. Analysis and reporting of the results will follow the CONSORT guidelines for reporting randomized controlled trials. Preliminary findings will be shared with the Advisory Committee and interpreted in the context of relevant policies.

Data management and monitoring Tablet-based questionnaires will be administered orally to caregivers in their homes. The Census and Survey Processing System (CSPro), a widely-used public domain software package that allows for collection of data and later uploading to the secure server, will be used for entering, editing, and tabulating data. An index child will be randomly selected at the beginning of the baseline interview.

The study will be monitored for quality, contextual appropriateness, and regulatory compliance. An independent advisory committee composed of academics and specialists with an expertise in child protection in Indonesia from government, non-governmental, and donor agencies will participate in the continuous monitoring of the study. Program implementers will monitor the intervention through facilitators and mentors' weekly meetings, and local visits by other program staff. Any adverse events observed by researchers or implementers will be assessed to decide whether additional investigation or a modification of the intervention may be indicated.