Protocol for a randomized controlled trial evaluating the impact of the Nurse-Family Partnership’s home visiting program in South Carolina on maternal and child health outcomes

Millions of children in the USA live in households experiencing poverty [1], and nearly half of children born into poverty stay there for the remainder of their childhood [2]. Childhood poverty is linked to adverse children’s health and development outcomes, and early life experiences can shape children’s long-term outcomes well into adulthood [3‐5]. There is growing policy interest in addressing the challenges faced by low-income families during early childhood, with a recognition that effective policies and interventions will address maternal and child well-being in tandem [6]. Policy-makers increasingly seek to address the social determinants that may contribute to poor health outcomes for low-income mothers and young children, such as access to food and housing, environmental factors, and economic disadvantage [7, 8]. Moreover, there is an acknowledgement that traditional clinical settings may not be well-suited to meeting these challenges [9]. As a result, some policy-makers administering Medicaid are seeking to support programs that can reach beyond the capability of the traditional clinical approach to pregnancy, childbirth, and medical care [10‐17], to more comprehensively address the sources of disparities that are already entrenched by the time children enter kindergarten [18, 19].

One of the established models for reaching low-income families to address a range of challenges in early life is nurse home visiting during pregnancy and early childhood. From 1978 to 1995, three modest-scale randomized controlled trials were conducted in Elmira, New York; Memphis, Tennessee; and Denver, Colorado, to estimate the impact of the Nurse-Family Partnership (NFP) on the outcomes of low-income families over multiple decades [20‐22]. The evidence from these randomized controlled trials played a key role in the expansion of millions of dollars in philanthropic, local, state, and federal funding, including for home visiting services through the U.S. Department of Health and Human Services’ Maternal, Infant, and Early Childhood Home Visiting (MIECHV) program [23].

The extent to which evidence from these early trials of NFP applies within a modern context is an important policy question. Over the past decades, the health care and social safety net landscapes have changed substantially, as have the composition, health status, and experiences of low-income families. The scale of NFP has also increased substantially; earlier trials represent the impact of the program on a small scale within narrowly defined populations. Earlier trials were also conducted before current practices in pre-registration of outcomes for clinical trials and accounting for multiple inferences were developed. Recent evidence on the impact of similar nurse-led home visiting programs has been mixed [24, 25], and policy-makers want to understand whether scaling home visiting through public insurance can improve maternal and child well-being at the population level [26].

Our ongoing experiment in South Carolina represents a rare opportunity to understand the effects of NFP when scaled up to serve a broader population in today’s context. The South Carolina Department of Health and Human Services (SCDHHS) is offering NFP services to first-time, Medicaid-eligible mothers by leveraging Medicaid funding via a Medicaid 1915(b) Waiver and philanthropic funding. The financing operates through a Pay for Success (PFS) model, where program costs are initially covered via philanthropic and Medicaid funding, and later SCDHHS will make success payments if our randomized controlled trial provides impact estimates that exceed the impact thresholds that were defined in the PFS contract before the trial began [27].

Home visiting programs have been hypothesized to affect a diverse range of outcomes, including prenatal, maternal, and neonatal health, infant and child health and development, morbidity and mortality, mental health, substance abuse, family planning, nutrition, neglect and maltreatment, home environment and parenting skills, crime, educational attainment, public spending, family economic self-sufficiency, access to healthcare, community resource connections, and social competence [28]. This protocol paper focuses on defining health-related outcomes that may be observed during the period of pregnancy and through the child’s first 2 years of life, when the family would be eligible for nurse home visiting services through NFP. We will assess outcomes across a broader group of domains and longer time horizon in future work.

Low-income families face particular challenges and disadvantages during the critical period around pregnancy, birth, transition to parenthood, and early childhood. These include relatively worse access to and quality of health care [29, 30] and increased exposure to environmental, neighborhood, or housing-related risks [8, 31‐34]. The health risks associated with pregnancy, childbirth, and the postpartum period are significant, particularly for low-income households. Maternal mortality rates have doubled in the USA over the last two decades [35], and preterm birth rates have risen each year over the last 3 years [36]. Substantial racial disparities have been documented surrounding childbirth, including higher rates of maternal mortality and morbidity and a higher likelihood of babies born preterm and low birth weight for black women compared to non-Hispanic white women [37‐39]. Geographic disparities have also been documented; women living in rural areas are more likely to experience preterm birth, infant mortality, or maternal mortality than those living in urban areas [40, 41], and are more likely to face the closure of their nearest hospital [40]. Unintended and short-interval births are associated with adverse health and social consequences and are also substantially more likely for low-income families [42]. Pregnancy intervals of less than 6 months are associated with adverse newborn health outcomes including low birth weight, small for gestational age and infant mortality [43]. Both children whose birth occurred earlier than desired and their siblings receive substantially less maternal investment than those born at the desired time [44]. Substantial evidence suggests better access to contraception that allows for achieving desired timing of births can improve maternal educational achievement and economic outcomes [45] with corresponding reductions in child poverty [46]. Finally, the direct effects of the strain of experiencing poverty itself can limit adults’ attentional and cognitive resources [47‐50] available for parenting. Children from low-income families and communities with high rates of poverty are substantially more likely to experience adverse events including abuse or neglect [33, 51, 52], which is associated with long-term impacts on adult health [53, 54] and well-being [1, 55]. The leading cause of death of children under five is unintentional injury, which is more common in low-income neighborhoods [56]. Major unintentional child injuries are also more likely to occur in households of lower socioeconomic status [57, 58]. In this paper, we describe scientific outcomes which are designed to capture the potential impact of home visiting programs within the context of these challenges.

The study was approved by the Harvard T.H. Chan School of Public Health Institutional Review Board (IRB15-2939). Permissions were also obtained from cooperating institutions. The following IRBs have approved Harvard’s oversight of this research study: Massachusetts Institute of Technology, South Carolina Department of Health and Environmental Control (DHEC), IRB00000092 (which includes DHEC agencies covering the Upstate, Pee Dee, Lowcountry and Midlands regions), McLeod Health System, IRB00004313, Greenville Health System, and Spartanburg Regional Health System, IRB00001369.

NFP is a prenatal and infancy home visiting program for low-income, first-time mothers, and their families. Registered nurses enroll pregnant women who have completed less than 28 weeks’ gestation. NFP attempts to enroll women early in pregnancy so that home visits can take place throughout the pregnancy. This enables nurse home visitors to monitor pregnancy health at home, encourage utilization of high-quality health care during pregnancy, and help pregnant women make informed choices about their own health and the health of their baby. Nurses continue regular visits with the family until the child is 2 years old. Families may choose to discontinue participation in nurse home visiting services at any point. During the study period, mothers enrolled in the NFP program were eligible for up to 40 home visits divided into up to 15 visits during the prenatal period, up to 8 visits during the postpartum period (within the 60 days after delivery), and up to 17 visits during the child’s first 24 months of life that were covered by the Medicaid waiver (described below). When convenient for the mother, the nurses could conduct telehealth visits instead of doing visits in the home. Services are provided in English and Spanish and translation services are available for mothers speaking other languages.

With a limited Medicaid budget and the desire to improve early childhood outcomes statewide, SCDHHS sought to expand NFP to eligible Medicaid recipients by applying for a 1915(b) Waiver and establishing a “Pay for Success” (PFS) Contract. The Medicaid Waiver authorized South Carolina to expand its current postpartum home visitation services by scaling up NFP. Under the Waiver, an additional 3200 Medicaid beneficiaries and their children were made eligible to enroll in NFP services during the study enrollment period (between April 1, 2016, and March 31, 2020). As part of the Waiver, SCDHHS established a PFS contract with a consortium of local philanthropic agencies to secure the additional funding needed to scale up the program over the 5-year Waiver period. The philanthropic agencies provide the initial capital to scale up NFP (in combination with federal dollars from MIECHV and the Medicaid Waiver); based on point estimates of the program’s impact on four pre-determined outcomes, SCDHHS will make “success payments” back to the funders, who have agreed to reinvest the money to expand NFP in South Carolina in subsequent years. The four outcomes on which payments will be made are preterm birth, child injury, birth spacing, and the percent of served mothers who live in low-income zip codes. Additional file 2 describes the PFS outcomes and thresholds for payments. The signatories of the contract are SCDHHS, NFP, and the Children’s Trust of South Carolina (which administers funds from the philanthropic partners, MIECHV, and potential success payments). As independent evaluators, the research team was not a signatory of the PFS contract.

The first payments are scheduled to be made in April 2021 for all four study outcomes. Payment outcomes were chosen by the signatories to the contract to reflect state-level priorities and major drivers of Medicaid spending. This protocol paper describes health-related primary and secondary outcomes chosen by the research team which are informed by, but distinct from, the outcomes chosen by signatories to the PFS contract.

NFP has a well-established delivery system in South Carolina, and there are multiple pathways through which potential clients can be referred. NFP has operated in South Carolina since 2009 and is delivered by 10 different implementing agencies in 32 counties across the state. A map of NFP implementing agency locations in South Carolina is provided in Fig. 1. The catchment area covers both urban and rural South Carolina. Nine of the implementing agencies participated in this project, which represented a significant scale-up of NFP services throughout the state. Prior to the launch of the study, NFP served approximately 500–600 moms annually across the state. During the study enrollment, NFP scaled up to serve an average of 1200 women each year with a corresponding increase in staffing of nurse home visitors and supervisors.

Individuals are eligible for the study if they (1) are currently pregnant with a gestation period less than 28 weeks, (2) would be first-time mothers, (3) would be income-eligible for Medicaid during pregnancy, (4) are at least 15 years old, (5) live in a catchment area served by NFP nurses, (6) are not incarcerated or living in a lockdown facility, and (7) have enough language fluency that they would be able to benefit therapeutically from the program.

Potential study participants are identified through several referral channels. First, referral partners, such as local health care providers, schools, and Special Supplemental Nutrition Program for Women, Infants, and Children (WIC) agencies, directly refer potential clients to an implementing agency with the client’s permission. Second, referrals are made directly from the Medicaid eligibility database to NFP. Finally, during the scale-up of NFP services, NFP had in place an outreach team with outreach coordinators in four regional areas who worked to identify potential clients. In addition to these main channels, some clients are self-referred or referred by a family member or friend or through digital and print advertisements of the program throughout South Carolina.

Potential participants are assessed first for eligibility by NFP nurses and then provide written consent to participate in the study. NFP program staff are trained on how to implement the informed consent process by the study team. To minimize coercion and undue influence, program staff review the consent form with potential participants and allow them as much time as they need to read the consent form themselves and ask questions. The consent form (provided in Additional file 3) informs participants about randomization and participation in NFP and informs clients that researchers will track their data and their children’s data across a variety of administrative data records for up to 30 years. Electronic signatures are obtained from consenting study participants.

This study is a parallel group individually randomized controlled trial. Participant flow through study procedures is provided in Fig. 2. Study participants and program delivery staff are (perforce) not blinded to treatment-control status. After providing informed consent and completing a baseline survey, mothers are randomly assigned either to a treatment group that is offered access to NFP or to a control group that has access to the standard of care and other available community programs and services, but not NFP. Two thirds of the participants are randomly assigned to the treatment group and one third to the control group. This assignment ratio maximizes the use of existing resources while maintaining adequate statistical power.

Individuals assigned to the treatment group participate in typical NFP program activities. Individuals randomized into the control group receive the standard of care in South Carolina. They are not offered participation in NFP, but may receive all community and medical services to which they would otherwise be entitled, including up to two postpartum home visits routinely paid for by Medicaid in South Carolina. Both control and treatment group enrollees in the study are provided with a list of community resources available to low-income new mothers (sample list provided in Additional file 4).

Study enrollment, on-the-spot randomization, and the baseline survey are conducted by NFP staff using encrypted tablets and computer-assisted personal interview (CAPI) software. We use SurveyCTO, which provides full customization of the baseline survey described in the Baseline survey section, real-time randomization, built-in time-stamping, and audio-recording capabilities to monitor fielding quality and fidelity to protocols. The enrollment software automatically checks identifying information provided during the baseline survey to ensure that no individual can enroll in the study more than once, avoiding potential gaming of the random assignment mechanism.

NFP’s program strives to impact the lives of mothers and children in three central ways: (1) to improve pregnancy outcomes, (2) to improve child health and development, and (3) to improve economic self-sufficiency. NFP aims to achieve these outcomes through a strength-based approach—which is based on the idea that new mothers will be best able to make changes in their lives when building on their own knowledge and strengths [59]. NFP home visiting services are also delivered through a therapeutic, relationship-based model where nurse home visitors typically form long-term, trusting relationships with mothers [59]. Activities to achieve these outcomes center around five main domains: (a) maternal health, (b) the home environment, (c) maternal life course development or goals for the future, (d) mothers’ role in the child’s health and development, and (e) strategies for leveraging social support. Home visitors seek input from clients in choosing which areas to prioritize by routinely assessing mothers’ physical and mental health and social determinants of health (e.g., unsafe housing, food insecurity, or social isolation) to identify the mother’s primary concerns. Nurse home visitors utilize “facilitators”—a guided tool with pre-programmed content—to educate mothers and motivational interviewing techniques to empower mothers to advocate for themselves. Nurse home visitors also provide referrals and care coordination to needed health care providers and community resources. They may also perform monitoring services such as taking maternal weight or blood pressure. In many (but not all) implementing agencies, nurse home visitors have access to electronic medical records and may be able to access clinical information about patient’s health during pregnancy and postpartum.

NFP’s focus on maternal and newborn health is consistent with the focus of the federal MIECHV program (funded by the Health Resources and Services Administration) [60, 61] which lists maternal and newborn health as the first central objective of home visiting programs. Unlike many home visiting programs which target families after birth, NFP delivers home visiting services throughout pregnancy, maximizing the potential to influence maternal and newborn outcomes. Evidence on the impact of home visiting programs on maternal and newborn health is mixed. An early trial evaluating the impact of NFP in Elmira, NY, found that adolescent mothers randomly assigned to receive NFP services had babies with higher birth weights, and mothers in the treatment group who reported smoking during pregnancy experienced a reduced likelihood of preterm birth [62]. Another early evaluation of NFP in Memphis, TN, found that mothers assigned to receive NFP services were less likely to experience hypertensive disorders of pregnancy, but had no reduced likelihood of preterm birth or low birth weight [22]. More recently, a large-scale randomized trial conducted to evaluate the impact of the MIECHV program [63] and an evaluation of a home visiting program in the UK found no evidence of impacts on birth outcomes [24].

Second, home visiting programs, because they focus on improving the safety of the home environment and providing parents with a broad set of tools and resources to help them be more effective parents, have sought to reduce the risk of child injuries and indicators of child abuse and neglect. Home visiting programs have been cited as one of the few interventions where rigorous evidence has demonstrated the potential to improve child outcomes surrounding child maltreatment [55]. An early randomized trial evaluating the impact of the NFP program delivered in Memphis, TN, found 22% fewer health care encounters for child injuries and ingestions and a 78% reduction in days hospitalized for injuries or ingestions [22].

Finally, home visiting programs may alter maternal life course and economic opportunity through a variety of channels with the potential for these alterations to change the timing of subsequent births. Home visiting programs often serve a population of first-time low-income mothers who have greater likelihood of unplanned or mis-timed births and who may desire to avoid or delay subsequent pregnancies. Indeed, among the 5655 study participants, 82.7% reported on the baseline survey that their pregnancies were either unplanned or occurred earlier than the mother desired, and only 3.3% of mothers reported desiring less than 2 years of spacing between this birth and any subsequent birth. Home visiting programs may be effective in increasing birth spacing by coordinating continuity of care and ensuring that postpartum mothers have access to family planning services. Home visiting programs also focus on enabling mothers to alter their life course by taking advantage of educational and career opportunities, which may delay subsequent pregnancies. Evidence from previous evaluations of NFP have found substantial impacts on shaping the maternal life course by altering the patterns of subsequent births. In the early evaluation of NFP in Elmira, New York, mothers reported 19% fewer subsequent births by the time their first child reached the age of 15 [64]. In the trial of NFP conducted in Memphis, Tennessee, a subset of the sample consisting of adolescent mothers reported 16% fewer births and a significant increase in birth intervals between their first and second child at a 6-year follow-up [65].

We define three primary outcomes for this analysis, one for each of the three domains that home visiting programs, including NFP, present as central program objectives or areas where home visiting programs have demonstrated substantial health-related impacts. In addition to these three primary outcomes, we provide a list of pre-specified secondary outcomes in each of these domains in Table 1. We define each outcome with the mother as the unit of observation (instead of considering outcomes separately for multiple births). In future work, we plan to explore NFP’s impact on other domains and over longer time horizons.

First, we will assess the effect of NFP on the likelihood of having an adverse birth outcome. We define an adverse birth outcome as having a preterm birth (less than 37 weeks’ gestation) or a newborn being small for gestational age (less than 10th percentile of US births conditional on gestational age based on specifications provided by Talge et al. [66]), having low birth weight (less than 2500 g) or experiencing perinatal mortality (fetal death at or after 20 weeks of gestation or mortality in the first 7 days of life). Data for the adverse birth outcome will come from South Carolina birth certificates, fetal death, and mortality records. For mothers with multiple births, we define the outcome based on having any adverse birth outcome for any child. While we expect preterm birth and other adverse birth outcomes to occur more commonly among multiple births, we anticipate that rates of multiple births will be balanced across treatment and control arms. We will explore alternative specifications of this outcome that include only singleton births. While the composite includes outcomes with different severity, which may increase the difficulty of interpretation [67], it captures the efforts of NFP nurse home visitors to influence pregnancy outcomes through a variety of channels from direct provision of medical care to increased utilization of clinical services to generalized improvements in general well-being that may translate to reduced stress and anxiety during pregnancy. Our primary outcome capturing the objective to improve pregnancy outcomes focuses on the health of the infant, without considering the health of the mother. We designed this composite outcome in order to maintain a parsimonious list of primary outcomes and to ensure adequate statistical power. Secondary analyses (Table 1) will consider maternal outcomes that occur with lower frequency (including maternal morbidities, mortality, and maternal experiences of violence [68]). We have also planned detailed analyses that will allow for a better understanding of the changes in utilization that result from participation in NFP including antenatal care and mental health care utilization.

Second, we will assess the effect of NFP on the likelihood of experiencing injury, abuse, or neglect during early childhood. This will be defined by a composite outcome indicating a health care encounter or mortality associated with International Classification of Diseases (ICD) codes indicating either a major child injury or suspicion of abuse or neglect. We will identify children who have at least one health care encounter or experience mortality associated with an ICD code that indicates either major injury or suspicion of abuse and neglect. We will identify major injury through the observation of any medical claim or mortality case including an ICD code associated with injury excluding superficial injuries, injuries related to medical care, and injuries stemming from allergic reactions. ICD codes indicating suspected abuse and neglect are derived from Schnitzer et al. (2011) and Hooft et al. (2013) based on validated methods [69, 70]. Data on early childhood injury outcomes and suspected abuse and neglect will come from South Carolina all-payer hospital discharge records, Medicaid inpatient and outpatient claims, and mortality records. Previous work examining the impact of home visiting programs on abuse and neglect has considered the impact of the program on investigation for child abuse and neglect [71]. We focus on health care encounters and mortality in order to mitigate potential reporting bias introduced because NFP home visiting nurses are mandated reporters of child abuse and neglect. We will consider the impact of the program on investigations of abuse and neglect and confirmed cases of abuse and neglect using data provided by the Department of Social Services in exploratory analyses. In order to account for the possibility that control and treatment groups have different Medicaid eligibility and are therefore differentially likely to appear in Medicaid claims, we will also present a robustness check estimating the impact of NFP on major injuries and suspected abuse and neglect appearing solely in hospital discharge records. A list of ICD-10 codes that will be used to define major injuries are presented in Table 2. ICD-9 codes used to define suspected abuse and neglect are presented in Table 3. We use General Equivalence Mappings from the Centers for Medicare and Medicaid Services (CMS) to convert from ICD-9 to ICD-10 codes, as validated codes for suspected abuse and neglect are published using ICD-9 codes only [72]. For mothers with multiple births, we define the outcome based on having any code indicating major injury or suspicion of abuse and neglect for any child. We will also consider secondary outcomes which capture utilization of emergency care and adherence with preventative guidelines and well-child care (Table 1).

Third, we will assess the effect of NFP on birth spacing. We define the outcome based on whether a subsequent birth occurs less than 21 months after the birth of the child born from the pregnancy during which the mother was enrolled in the study. Studies examining various pregnancy interval lengths find stronger and more consistent effects on infant health for birth intervals of 6 and 12 months [43]. We selected a birth interval of 21 months, corresponding to a 12-month pregnancy interval, in order to capture the potential implications for infant health, ensure adequate statistical power for detecting effects, and reflect preferences of the vast majority of mothers enrolled in the study who report a desire to space births by at least two years (96.7% of currently enrolled sample). Data for this outcome will come from South Carolina birth certificate records. Secondary outcomes will consider utilization of family planning and contraceptive services. While South Carolina has not expanded Medicaid under the Affordable Care Act, South Carolina has a Medicaid Waiver to provide family planning services for individuals with the same income eligibility criteria as the state’s Medicaid eligibility threshold during pregnancy [73]. We provide a SPIRIT figure as Fig. 3 detailing activities surrounding enrollment, interventions, and assessments.

Our primary study outcomes are not defined identically to outcomes defined in the PFS contract. The outcomes in the PFS contract were defined by the signatories of the contract through the contracting process. We subsequently refined the definitions for the analyses in this protocol paper based on the literature on home visiting and an assessment of which outcomes our study would have sufficient statistical power to detect. Per the PFS contract schedule, the PFS outcomes will also be reported based on a partial sample, whereas the analyses described here will be reported on the full study sample. For these two reasons, we expect that the results of the analysis described in this protocol paper will likely differ from the point estimates we will provide to the PFS contract signatories.

Similar to previous NFP trials, we will examine differential impacts of the program among a subgroup of mothers whose circumstances at enrollment place them and their children at increased risk of adverse outcomes [22, 74, 75]. This primary subgroup includes mothers who have some indicator of poor mental health, are under 19 years old, or have not completed high school or received a General Education Development certificate by the time of study intake. We define the mental health indicator as either having received mental health treatment in the last year, or having elevated depressive symptoms as measured using the abbreviated Patient Health Questionnaire (PHQ-2), with scores of 3 or greater indicating high likelihood of a major depressive disorder [76]. Our focus on families experiencing greater perceived mental health challenges is consistent with previous subgroup analyses from the Memphis and Denver trials evaluating NFP that have focused on mothers with “low psychological resources” (including poor mental health outcomes). Our classification of mothers under 19 years old as high risk mirrors the criteria used to define the most vulnerable mothers in the trial of NFP conducted in Elmira [62]. We use the completion of high school as an indicator of potential economic mobility, whereas the Elmira trial measures low socioeconomic status using the Hollingshead Four Factor index [77].

We may also consider the program’s differential impact among other secondary subgroups of particular interest to policy-makers or stakeholders. For example, we may consider how program impacts on birth outcomes differ by maternal race, how outcomes on child injury and suspected abuse and neglect differ for a subgroup identified as experiencing substance abuse disorder, or how birth spacing differs for mothers who had expressed a desire to avoid closely spaced pregnancies. We will fully specify planned subgroups as we develop more detailed pre-analysis plans for specific academic manuscripts. When discussing heterogeneity in treatment impacts, we will clearly indicate whether subgroup analyses were planned and documented in a pre-analysis plan or are exploratory. Finally, for any subgroup analyses, we will correct for multiple hypothesis testing as discussed in “Statistical power” section.

Previous evaluations of NFP have been able to track families with in-person surveys over long periods of time with low to moderate attrition rates, with follow-up rates between 62 and 93% depending on the outcome of interest and the length of follow-up [64, 65]. Though our use of administrative data to measure study outcomes mitigates many challenges associated with long-term follow-up, our study faces other potential sources of attrition, in particular due to miscarriage or out-of-state migration. Specifically, we will not observe outcomes for mothers who experience a miscarriage early in pregnancy. Considering the distribution of gestational age at enrollment from our baseline survey and estimates of miscarriage risk by gestational week [78], we estimate that approximately 5% of mothers enrolled into the study will experience a miscarriage. In terms of out-of-state migration, South Carolina experiences a 3% out-migration over the course of 1 year (based on data from the census), and we anticipate that the rate of out-migration may be slightly lower among our sample of low-income first-time mothers. Evidence suggests that moving during the pregnancy and postpartum period often occurs over limited distances (< 10 km) [79]. We may also see attrition due to inability to match mothers to administrative records because of inadequate identifying information. We anticipate that we will be unable to match an estimated 3% of mothers enrolled in the sample for this reason. Finally, we anticipate that some mothers will withdraw their consent to participate in the study, though we anticipate this will account for < 1% of the enrolled sample.

For the purposes of statistical power calculations (“Statistical power” section), we calculate statistical power assuming a lower bound of 7.5% attrition and an upper bound of 15% attrition across both treatment and control groups through miscarriage, migration, imperfect identifiers for matching, or withdrawing consent. We anticipate that our use of administrative data will limit differential attrition across study arms. However, if attrition is higher than expected or differs substantially between treatment arms, we will conduct a bounding exercise to understand the robustness of our estimated treatment effects to possible attrition patterns [80].

We will estimate intent-to-treat (ITT) effects as the primary empirical specification. We will also report local average treatment effects (LATE). The ITT estimates capture differences in outcomes between those assigned to the treatment group and those assigned to the control group. The LATE estimates incorporate information on actual program participation, taking advantage of the randomization as an instrument for participation and allowing examination of average characteristics of those participating in the program [81]. We define program participation as receiving at least one visit from a nurse. Intervention group status is used as an instrumental variable for program participation. Consider an outcome, Y_i, such as an indicator for an adverse birth outcome. For subject i, the estimating equation is:

where “Enrolled in NFP” means having received at least one completed visit from NFP for service delivery. This model will be estimated using two-stage least squares (2SLS), where the first stage is:

where I(Treatment = 1)_i is an indicator variable equal to one if the subject was randomized to the intervention group and zero if the subject was randomized to the control group; X_i is a vector of covariates, specified in more detail below. These covariates should be uncorrelated with the treatment indicator because of the randomization. We include them in the model since they may increase the precision of the estimates. This linear model estimates the local average treatment effect (LATE) of NFP on intervention group members who actually participate in NFP relative to the services consumed by the control group. This estimated effect of NFP is of policy interest because it represents the impact of NFP on those clients who are likely to participate in NFP were the program to expand and offer additional program slots through a lottery. The source of non-compliance that it explicitly captures is that some mothers randomized into the intervention group may never receive NFP services (i.e., the “enrollment rate” is less than 1). According to the enrollment protocol, no mothers in the control group should be enrolled in NFP services. To the extent that some sample members in the control group receive services from similar home visiting programs that may also affect outcomes, this model estimates the effect of NFP relative to the mix of other home visiting programs that the control group receives, rather than relative to no home visiting service at all. We may also consider additional secondary specifications that may be of interest to policy-makers including an estimate of local average treatment effects where we consider two potentially relevant alternative definitions of “treatment:” (1) mothers who are still receiving home visits at the time of their anticipated delivery date as reported on the baseline survey and (2) mothers who receive home visits throughout the entire program period of pregnancy and the child’s first 2 years of life.

Existing literature and previous NFP trials have identified a number of characteristics that may be predictive of the three primary study outcomes. Such characteristics include maternal age, indicators of race and ethnicity, health behaviors (smoking and drinking), maternal socioeconomic status, relationship to father of the child, utilization of health services, and mental health. In our impact models, we will include variables from the baseline survey to measure these and other relevant sample characteristics (including gestational age at enrollment and indicators for implementing agency of NFP).

We calculate minimum detectable differences reported as the percentage point difference between treatment and control groups we will be able to detect, assuming a significance level (alpha) of 5% and a statistical power level of 80%. Based on current enrollment trends, we assume 96% of study participants randomized into the treatment group will receive NFP services. Furthermore, we assume that none of the control group receives NFP services. We will account for multiple hypothesis testing across three different outcomes and within one subgroup by calculating and reporting false discovery rate (FDR)-adjusted p values across all primary outcomes and subgroup analyses (6 hypothesis tests in total) using methods developed by Benjamini and Hochberg [82, 83]. We provide calculations based on a lower-bound and upper-bound estimate as discussed in “Subgroup analysis” section. We report both unadjusted and Bonferroni-adjusted minimum detectable differences for each outcome in Table 4. Because the FDR method improves power over the Bonferroni correction, Bonferroni-adjusted minimum detectable differences should be interpreted as upper bounds. Our estimates of control group means for our three primary outcomes are determined as follows. Adverse Birth Outcomes: We use vital records data from South Carolina to estimate that approximately 24.3% of study participants will have an adverse birth outcome in our control group. Birth Spacing: Assumptions about the control group mean for the birth spacing outcome come from recently published evidence on the incidence of shortly spaced births in South Carolina [84]. Early childhood injuries or suspected abuse and neglect: The assumptions about the control group mean for the early childhood injury outcome are estimated based on analysis from a sample of hospital discharge data from South Carolina. We anticipate that 18% of families in our sample will experience injury, abuse, or neglect.

Planned analyses will be pre-specified in a publicly archived analysis plan prior to un-blinding the data for analysis. With the exception of contractually required interim reporting on outcomes that are part of the PFS contract (scheduled for April 2021), data will be unblinded sequentially as outcomes can be observed for the full sample (Fig. 2). We may report on primary outcomes in separate manuscripts, for example reporting on birth outcomes prior to reporting on child outcomes which occur later in time. We will create and publish pre-analysis plans for separate planned manuscripts (including the analysis of pre-specified secondary outcomes and of additional domains and longer-term outcomes such as those in Table 1) before conducting analysis on outcomes using treatment assignment. This practice strengthens the integrity of the inferences drawn, facilitating multiple-inference adjustment and guarding against external pressures. We have registered the trial on the American Economics Association’s registry of social science RCTs and on Clinicaltrials.gov. Following the conclusion of the publication of primary study outcomes, we will make a de-identified public use dataset and replication code available to the maximum extent that is legally permissible under the terms of our data use agreements. As this study is not a traditional clinical trial, the study does not have a Data Monitoring Committee.

The study concluded enrollment earlier than expected on March 17, 2020, because of safety concerns related to the COVID pandemic. The originally planned study population was 6000 mothers total (4000 in the treatment group and 2000 in the control group) to be enrolled over 4 years (April 2016–2020), with NFP services delivered through the children’s second birthday (2016–2022). At the conclusion of enrollment, 5655 were enrolled in the study. This protocol was first drafted on January 30, 2020, and revised on October 21, 2020, in response to reviewer comments.