Using questionnaires that are designed to replicate the national 2014 Demographic Health Health Survey, CHPS+ will conduct baseline and endline demographic surveys for assessing impact. The baseline CHPS+ project survey sample is designed to obtain information to detect a 15% reduction in under-five mortality with 80% power at 5% level of significance in each of the two study regions. Overall, the interventions are taking place in eight districts (four in each region) while another eight districts (four in each region) are being used as comparison districts. Survey sampling is designed to draw a representative sample of the number of women of reproductive age (15–49 years) in three of the four treatment districts in each region and in four randomly selected comparison districts. Thus, a total of fourteen districts, seven per region, were included in the baseline survey. A two-stage stratified cluster sampling approach was used. In the first-stage, census enumeration area clusters were sampled, and in the second-stage, households will be sampled from the first-stage clusters. Because a minimum of 30 clusters is conventional for the first stage of cluster sampled household surveys [
41], a relatively conservative starting point of 40 clusters each for intervention and control districts in each region was selected.
Because the sample is designed to detect separate under-five mortality effects in each of two regions, sample size calculations were completed independently for each region. Final sample size and sampling parameters were based on region-specific estimates of census enumeration area intra-cluster correlation (ICC) and the number of children under five expected per woman. These parameters were calculated from publicly available data from the 2014 Ghana Demographic and Health Survey [
42]. In Volta Region, ICC was higher and the number of children expected per woman was lower than in Northern Region. Therefore, a larger number of clusters and a larger number of households per cluster will be sampled in Volta Region. Sampling requirements for each region were calculated using a software system for conducting power calculations in multi-level randomized experiments known as Optimal Design [
43].
Sample design
The sampling frame for the first stage was from the 2010 Population and Housing Census which provided a complete listing of Enumeration areas in the fourteen districts to be surveyed. The sampling frame contained information such as the location and estimated number of households. In each region, Enumeration areas were stratified by rural versus urban and by the estimated number of households in the cluster. Clusters were stratified into three groups (small, medium, or large yielding a total of 6 strata in each district. Enumeration areas were sampled from each stratum using probability proportional to population size (PPS). Weights will be applied at the cluster level to standardize probabilities of household selection based on the relative population size of clusters.
To obtain a sampling frame for the second-stage sampling, a household listing of all households in each cluster was compiled based on census information on household size and number women of reproductive age (15–49) in each household. Households were then stratified into three strata defined by the number of eligible women in each household.
To maximize sample efficiency for difference-in-difference estimation at the endline, baseline sample clusters will be reused without modification to permit longitudinal observation of clusters and the assessment of the average treatment effect arising from the timing of exposure of clusters to project interventions. The second-stage sampling procedure will be repeated, yielding a household sample for the longitudinal observation of children exposed versus unexposed to interventions over the duration of the project, with provision for statistical adjustment of baseline and endline differentials and changes in project endpoints that are unrelated to CHPS+ interventions.
Assessing impact
CHPS+ will have core, intermediate outcome, and process endpoints, each requiring systems of data capture, data management, and analysis to gauge project impact. These core endpoints and indicators will be consistent with the Ghana Ministry of Health core indicators of health improvement with instruments designed to maximize comparability with national Ghana Demographic and Health Survey instruments. This will involve indicators of under-five mortality, infant mortality, and neonatal mortality by gender of child; age specific and total fertility rates and proximate determinants that are relevant to policy; and indicators of parental health seeking behavior, such as skilled attendant delivery, care of sick children, exposure to community-based services, distance to health facility and utilization of facilities for essential care. Critical covariates essential to the understanding of equity and impact, such as educational attainment, household economic status, and distance to service point will also be assessed.
CHPS+ will utilize impact assessment strategies that have worked well for GEHIP. Routine compilation of time trends indicators will be supplemented with end-of-project
difference-in-differences calculations for each indicator. This econometric strategy has been applied elsewhere for the assessment of non-randomized plausibility trials [
44‐
46] and successfully applied to the evaluation of GEHIP [
47]. The procedures involve collecting data in baseline clusters, monitoring systems changes and the timing of these changes, and repeating the survey in baseline clusters with a separate endline stage two sample to gauge effects. Regression analysis is based on merged baseline and endline data for the estimation of parameters that control for baseline differences and contextual confounders, changes over time that are unrelated to interventions, and conditional effects of interventions that control for these confounding changes by estimating net intervention effects. A difference-in-difference estimate of program effects is given by the child survival effect of CHPS+ systems interventions (
S) where individual child i is scored 1 if the household is located in a cluster that is exposed to CHPS treatment and zero otherwise and z is scored 1 if the case
i is observed in the endline and zero if the case is observed in the baseline. If all covariates are equal to zero for child
i, (
i.e., X
1
=0,
X
2
=0,... X
k
=0, S=0, Z=0), the underlying mortality hazard is
h0(t) and the conditional multilevel proportional hazard model is:
$$ {\boldsymbol{h}}_{x\sim SZ}\left(\boldsymbol{t}\right)={\boldsymbol{h}}_{\mathbf{0}}\left(\boldsymbol{t}\right){\boldsymbol{e}}^{\left[{\beta}_0+\sum \limits_{k=1}^K{\beta}_k{X}_{ij k}+\gamma {S}_j+\delta {Z}_{ij}+\zeta {S}_j{Z}_{ij}+{\mu}_j+{\varepsilon}_{ij}\right]} $$
(1)
for all
t, where:
X̃
i
j
k
represents K characteristics of child i in cluster j;
S
j
indicates whether cluster j is in the treatment area (S = 1) or comparison area (S = 0);
Z
ij
indicates whether the person time of child i in cluster j occurs during the post-treatment period (Z = 1) or pre-treatment period (Z = 0);
S
j
Z
ij
is a cross-level interaction term of treatment and period for estimating the net effect of treatment in the difference-in-differences approach;
and
μ, ε represent residuals for cluster and individual levels, respectively.