Community health worker intervention improves early childhood vaccination rates: results from a propensity-score matching evaluation

This study evaluated the effectiveness of a state-run CHW maternal and child health home visiting program in increasing early childhood vaccination uptake. Arizona’s Health Start Program (HSP) is embedded within local and tribal health departments and federally qualified health centers throughout the state to improve maternal and child health outcomes among historically disadvantaged groups, including Hispanic/Latinx and American Indian women, and women of low socioeconomic status. [18,19] Given the heterogeneity of the intervention population, we also examined the impact on several subgroups with either a history of disadvantage or low vaccine uptake.

Initially created in 1984, the HSP has been overseen by the Arizona Department of Health Services’ Bureau of Women’s and Children’s Health since 1992. The program was formalized through state statute in 1994 and has been funded by the Healthy Arizona Initiative lottery funds since 2004. HSP serves 14 communities across the state, “to educate, support and advocate for families at risk by promoting optimal use of community-based family health care services and education services through the use of CHWs who live in and reflect the ethnic, cultural and socioeconomic characteristics of the community they serve.” [5].

HSP is unique in that CHWs are the primary interventionists and home visitors. In 2016, 50 CHWs were employed at HSP, of which 82% identified as white, 62% Hispanic/Latinx, and 16% American Indian. Over half (58%) spoke Spanish and 16% spoke a Native language (Navajo, Hopi, or Apache). Majority (90%) had a high school degree and 10% had a college degree. The caseload of full-time CHWs may reach a total of 40 active clients, who are pregnant or up to two years postpartum. CHWs can meet with their clients up to four times per month through the duration of their enrollment in HSP (e.g. child’s second birthday). The CHWs facilitate prenatal care access and attendance, provide perinatal education, referral services, advocacy to access social services (i.e. food, housing, legal assistance), and emphasize timely childhood immunizations and developmental assessments.

To be eligible to participate in HSP, clients must meet the following criteria: (1) live in a targeted service area, (2) be pregnant or postpartum with a child under age two, and (3) have one or more social or medical risk factors. Social risk factors include marital and residential status (e.g. not married, living with parents or extended family), race/ethnicity (minority), low education level, low income, and insurance status (Medicaid or uninsured). Medical risk factors include presence of one or more chronic diseases, maternal BMI (under- or over-weight), and maternal age (≤ 18 or ≥ 35). [20] In 2016, approximately 51% of clients enrolled during pregnancy, and 49% enrolled postpartum.

Among HSP’s five goals, the program aims to “reduce the incidence of children affected by childhood diseases” and “increase the number of children receiving age-appropriate immunizations by two years of age”. [5] The CHWs are trained to motivate and support their clients through behavior-change activities that promote personal agency and self-efficacy. [19,20] CHWs also receive training from The Arizona Partnership for Immunizations every two years to improve immunization education practices, review state immunization requirements, and learn how to navigate the state immunization registry. CHWs thus support and encourage their HSP clients to complete age-appropriate immunizations for their children by age two. Specifically, during every well-child home visit, CHWs (1) educate families about Arizona school and care facility vaccine requirements, [21] (2) ask families about the child’s immunization status, (3) review and compare the child’s immunization cards to the CDC schedule and the child’s record in the Arizona State Immunization Information System (ASIIS) database (if available), and (4) collaborate with clients’ pediatricians by providing warm-handoffs when available, and assisting with immunization appointment scheduling when needed.

This retrospective study evaluates the effectiveness of the HSP intervention by comparing early childhood immunization outcomes among children whose mothers participated in the HSP between 2006 and 2016 to those of a statistically-matched cohort. A detailed description of the methodology can be found in Sabo et al. 2019 and 2021. Briefly, using Arizona birth certificate data, we employed propensity score matching to identify non-participants with the same observable demographic, geographic, and socioeconomic characteristics as the participants. The children of these matched mothers make up our synthetic control group(s). Vaccination information for these children was extracted from ASIIS. The differences in outcomes between the intervention and matched-control groups is often referred to as the average treatment-on-the-treated (ATT) effect, defined as the impact of the program among those who participated.

Our study population is comprised of the children of mothers who participated in HSP (intervention) and their matches (controls), as described above. For evaluation purposes, we focus on participants who enrolled in the program prior to the birth of the child, based on enrollment and birth dates. This criterion means that the intervention predates the administration of any immunization, thus the timing of events (enrollment, immunization) supports clear attribution of intervention-control group differences to the HSP intervention. This resulted in an intervention population of 7,218 HSP children, born during 2006–2016.

The population of candidate matches was the approximately 1,000,000 women with a recorded live birth in Arizona during the study period who never enrolled in HSP. Compared to all Arizona mothers who gave birth in this period, HSP mothers were more likely to identify as Hispanic/Latinx (59% vs. 42%) or American Indian (12% vs. 6%), have less than high school degree (33% vs. 22%), and be insured by Medicaid (83% vs. 54%). Given this composition, we also evaluate intervention impacts for the following subgroups: Hispanic/Latinx and American Indian children, children residing in rural border counties, children of teen mothers and those with less than a high school degree, and primipara mothers (Table 4). [18].

Our analysis focuses on the seven vaccination series intended for young children, for the following viruses: hepatitis B (HepB), diphtheria and tetanus toxoids and acellular or whole-cell pertussis (DTaP/DTP), Haemophilus influenzae type b (Hib), pneumococcal conjugate vaccine (PCV13), poliovirus, measles-mumps-rubella (MMR), and varicella. The CDC-recommended dose schedules for children by age two (plus catch-up time frames) are reproduced in Table 1. [3] We evaluate both completion and timeliness of completion for each series and for all seven series as a group.

Data for this project come from three administrative sources: HSP enrollment records, the Vital Records Birth Database (birth certificates), and ASIIS (immunization records). Using birth certificate data, the matching model included demographic indicators (mother’s age [< 20, 21–25, 26–30, ≥ 30], race/ethnicity, country of origin, marital and cohabiting status); health indicators (presence of pre-pregnancy diabetes and/or hypertension), primiparity (first birth); and socioeconomic indicators (mother’s education [less than high school, high school/GED, some college, 4 or more years of post-secondary education] and primary insurance payer [private/commercial, Medicaid, all others]). Other variables included the birth year, county of residence, and the median household income of mother’s zip code (using indicators for each decile across the distribution). Propensity scores (the likelihood of HSP enrollment) were estimated via logistic regression. This process identified 54,175 appropriate matches for the 7,218 HSP children; for all subsequent comparisons, control group statistics were weighted based on the number of matches and HSP participants within each propensity-score value (designed so to mimic a one-to-one match). Baseline equivalence, the statistical similarity of the intervention and matched-control groups, was verified through t-tests and standardized differences. [18] This matching process was repeated for several distinctive subgroups whose socioeconomic status (e.g. low education) or demographic characteristics (e.g. racial/ethnic minorities) are especially associated with high risk for adverse health outcomes. In each case the matching models are calibrated so that the comparison groups meet the criteria for baseline equivalence prescribed by the What Works Clearinghouse (WWC) Standards Manual (Version 4.1, 2020). [22] Results from these models are available upon request.

In Arizona, in order to access childcare facilities and schools, children aged 18 and younger must be vaccinated and immunizations administered to children must be reported and submitted to ASIIS. [21] The linking of records across all data sources (primarily using names and date of birth) was done through the cooperation of an Honest Data Broker and ASIIS administrators. A limited dataset including HSP enrollment (HSP Database), maternal and child demographics (birth certificates), and vaccination outcomes through 2021 (ASIIS) was then generated for analysis.

Vaccines were identified through ‘Codes for Vaccines Administered’ and ‘Current Procedural Terminology’ codes and descriptions. Age at vaccination was measured by comparing dates of administration and date of birth. Adherence is measured by comparing ages at administration against the CDC-recommended schedules. [3,23] Series are marked as completed if all required doses have been recorded within the appropriate time frames, including during the catch-up period (i.e. older than the recommended ages). Series are marked as completed on-schedule if the final dose was administered within the CDC-recommended age range, conditional on previous doses having happened at appropriate times. In order to standardize the catch-up reporting window across the different cohorts, we report these rates of completion by age five (approximately primary-school age).