Service-integration approaches for families with low income: a Families First Edmonton, community-based, randomized, controlled trial

Families were recruited by mail, referral, and community-engagement approaches from January 2006 to January 2008 in Edmonton, Alberta, Canada. Families were eligible if a) they included at least one resident child younger than 12 years of age and b) they accessed at least one of five government assistance programs: Income Support (welfare) [10], Alberta Child Health Benefits [11], City of Edmonton’s Leisure Access [12], Alberta Adult Health Benefit [13], and Capital Region Housing [14]. Families were allowed to self-identify, resulting in diverse structures and compositions including biological two-parent families, single-parent families, adoptive families, and grandparent-led families.

The study protocol approval came from a special interdisciplinary ethics committee at the University of Alberta and the Director of the Human Research Protections Office (file number Pro00000144). Interested families contacted the community-based research office to learn about the study and ask questions. Families in which English was not a first language were offered the option of an interpreter during all interactions (e.g., on the phone while booking appointments, during the consent process, and during data collection). At the first home visit, each family was given written information about the study, which the data collector reviewed verbally. Written informed consent was obtained prior to study entry and data collection. Families were given an honorarium at each data collection period ($25 in early data-collection periods and $30 at the final data-collection period).

One adult respondent per family (“primary parent”) was selected based on familiarity with the children. To measure child outcomes, one “focus child” was randomly selected among the children within the household at screening. Limiting the measurement to one child per family was necessary to limit the measurement burden on families and reduce costs to the project. Families were ineligible if a) they could not commit to a 5-year data collection period, b) they were unwilling to allow the researchers access to the focus child, and c) a relevant interpreter was not available for families that did not speak English well. After families were recruited, they were free to withdraw from the study at any time and were no longer eligible to participate if they moved outside of Edmonton.

The investigators, in collaboration with community partners, developed three parameters for the service delivery models: 1) the service-integration foci of family recreation and family healthy lifestyle; 2) the practice principles of family-centeredness, cultural sensitivity, capacity building, and reflection; and 3) the hours of direct service. These parameters formed the foundation of the two service-delivery models tested: Family Healthy Lifestyle and Family Recreation. Full details of the interventions and their development are described in Table 1 using the Template for Intervention Description and Replication (TIDieR) checklist [15]. Qualitative analyses of the development and implementation of these interventions have been published [16‐18].

Participant families were randomized to four groups: Self-Directed (no intervention), Family Healthy Lifestyle, Family Recreation, and Comprehensive (Family Healthy Lifestyle plus Family Recreation programs). Families were randomized after baseline-data collection using a 1:1:1:1 allocation ratio. Computer-generated lists of permuted blocks of eight and 12 were created by a statistician. In order to limit the number of stratification variables [19], we selected the two variables directly involved in the selection of the families. Randomization lists were stratified by type of financial support (employment income versus income support or other government assistance programs) and age of the randomly selected focus child (0 to 3.9, 4 to 6.9, 7 to 9.9, or 10 to 12.9 years). Intervention assignment was concealed in sequentially numbered, opaque, sealed envelopes. Data collectors, who went to families’ homes, were blind to intervention assignment. Research assistants, who performed and explained the random intervention assignment and scheduled interviews, intervention providers, and families were not blind to intervention assignments. Families who were assigned to the intervention groups received up to 24 months of service-integration intervention. All families were followed by researchers for 3 years (reduced from 5 years due to funding), with a minimum of eight face-to-face interviews for data collection: two at baseline (separated by a month), and two each at year 1, year 2, and year 3.

The primary outcome was the number of total family linkages to health and social services as defined by an author-created tool called the “Family Services Inventory” (FSI). The FSI adopts a societal perspective and measures the public and private expenses for families. The FSI was developed to maximize precision and minimize participant burden and recall bias. An FSI toolkit comprised in-service training materials for data collectors, a user manual, a codebook, and a calendar that served as a memory aid to reduce participant recall bias. During annual interviews, data about family linkage use was collected. At the first visit of each data collection point (baseline, years 1, 2, and 3), data collectors explained how to use the FSI tool, and the family recorded their health and social service linkages for a month. At the second visit, approximately 1 month later, service linkages that occurred in the previous 28 days were collected. Each family service experience was considered an encounter. Repeated encounters within a single service were considered a single linkage (e.g., two family doctor visits within a 28-day period at the same clinic would count as two encounters but only one linkage).

For the purposes of the analysis reported here, the service linkages are subdivided into the following categories: basic needs, family challenges (e.g., addiction supports, parole, and mediation), child development, health care, childcare, and other services (i.e., nature of the service was unclear). These categories were developed through consultation with experienced social workers in Edmonton to capture the broad spectrum of services commonly accessed by vulnerable families. Each service is listed in only one category as shown in the Additional file 1. These categories were treated as secondary outcomes. Other outcomes were measured but are not reported in this paper. A full list of outcomes is provided in our published protocol [8].

Data on the following family characteristics were collected and described: the age and gender of the primary parent, age and gender of the focus child, family structure (number of adults and number of children), citizenship history (e.g., born in Canada (Aboriginal and not Aboriginal), immigrant, or refugee), geographic residence in Edmonton (northcentral, northwest, west, southwest, and southeast), the number of residence relocations, educational attainment (university, college, secondary school, and less than secondary school), current employment, and after-tax income. Other family characteristics were derived: child-to-adult ratio and household depth of poverty (DOP). The families’ household DOP was calculated as their annual household income after tax (from a maximum of two adults) divided by Statistics Canada’s 2005 low-income cut-offs [20], which are based on family and community sizes. The characteristics of the randomized families are presented in Table 2.

Analyses were performed using Stata/MP 11 (www.​stata.​com). To compare the intervention assigned to the intervention received, we measured the frequency of the intervention contact, length of intervention contact (in hours), and intensity of intervention contact (total hours divided by months of service divided by number served). In-person visits, phone conversations, messages left on voicemail, and emails were all considered forms of intervention contacts.

A number of sensitivity analyses were performed. First, the data were analyzed according to the intervention the participating families received (“per protocol”) rather than the intervention they were assigned (“intention-to-treat”). In addition, to account for families missing before the year 1 visit, we regressed missing outcome status against family characteristics using a generalized linear mixed regression: a binomial distribution with a logistic link. Family characteristics significantly associated with missing outcome were then adjusted for in the primary outcome. To account for missing outcome data, we imputed outcome data using the last-value-carried-forward method. Visit year was modelled as a linear variable. To determine whether the association between outcome and intervention was modified by visit year, we used an interaction term for visit year and intervention group. Lastly, the number of encounters (per month) were analyzed as the dependent variable rather than the number of linkages. Childcare encounters were not included in this sensitivity analysis because their frequent use precluded the value of this measure.

Projecting a moderate (f = 0.25) and even a small (f = 0.10) effect size (mean divided by standard deviation), given an alpha of 0.05 and a 25 % attrition rate, the trial proposed an initial sample size of 300 families per group to detect any significant difference between the four groups (power was 0.99 and 0.72, respectively). The trial randomized approximately 290 families to each group. The overall alpha value was not controlled for multiple comparisons. No interim analyses were planned.

Of the 1749 families screened for eligibility, 1282 families were enrolled, and 1168 were randomized: 293 to Comprehensive, 293 to Family Healthy Lifestyle, 291 to Family Recreation, and 291 to Self-Directed (Control). The remaining 114 participants were unavailable for randomization (Fig. 1). Table 2 provides descriptions of the family characteristics by assigned intervention groups. More than 85 % of the primary parents were women; the median age was 35 years. Their median health-related quality of life score as measured by EQ-VAS was 75 % [21]. The EQ-VAS is a visual analog scale used by individuals to self-report their current global health status; e.g., 100 % would indicate perfect health. Single parents headed almost 60 % of the families. Approximately 60 % of the primary parents were born in Canada. Of those, 25 % were Aboriginal. Of those not born in Canada, 15 % were refugees, and 57 % immigrated to Canada in the last 5 years. A third received income support. Almost half were currently employed. The median after-tax income was $22,000; the median household depth of poverty was 75 %.

Of those families randomized, 50 % participated in all the data-collection points (see Fig. 1). With regression analyses, we found that the following family characteristics (from Table 2) were associated with more missed data collection visits: younger primary parents, male primary parents, greater numbers of children, Northcentral residence, having relocated in the last year, lower educational attainment, and a less healthy primary parent (as measured by the EQ-VAS).

The interventions as delivered were different from the interventions as planned. The full details of intervention fidelity are described in Table 1 using the TIDieR checklist. In summary, the dose in each intervention was diluted and the Comprehensive intervention no longer equated to the additive components of the Family Healthy Lifestyle and Family Recreation interventions (Table 3). As such, the data are analyzed as a four-group randomized trial rather than a factorial randomized trial.

The mean number of linkages during the 28-day baseline period were 0.05 for basic needs, 0.01 for family challenges, 0.02 for child development, 0.79 for health care, 0.19 for childcare, and 0.06 for other services.

Table 4 (and Fig. 2) show the rate ratios of linkages to health and social services for the Comprehensive, Family Healthy Lifestyle, and Family Recreation groups versus the Self-Directed group. All three interventions were not significantly different in terms of the total number of linkages from the Self-Directed group (Comprehensive 1.15 (95 % CI 0.98–1.35), Family Healthy Lifestyle 1.17 (0.99–1.38), and Family Recreation 1.12 (0.95–1.32) rate ratios), although all three showed a trend toward favoring an intervention.

For the specific types of services, the rate ratios were not significantly different between interventions groups and the Self-Directed group except for two. Significantly more healthcare service linkages occurred in the Comprehensive group compared to the Self-Directed group (1.27 (1.06–1.51)), and significantly more linkages occurred with child-development services in the Family Healthy Lifestyle group compared to the Self-Directed group (3.27 (1.59–6.74)).

When participant families were analyzed according to the intervention they received rather than the intervention they were assigned, the rate ratio of total service linkages did not change (Additional file 2: Table S1). The results were similar when we imputed missing outcome data with previous visit outcome data and when we adjusted the results for the covariates (younger and male primary parents, greater number of children, Northcentral Edmonton residence, relocations in the last year, less education, and lower health score for the primary parent according to the EQ-VAS) associated with the missing outcome data.

When the number of encounters rather than the number of linkages were modelled, the rate ratio was attenuated for the Comprehensive group compared to the Self-Directed group (Additional file 2: Table S1). The results for Family Healthy Lifestyle were slightly attenuated, and the results for Family Recreation remained similar.

In Additional file 2: Figure S1, the top panel shows the observed mean rates for each randomized group over the time points (baseline, and years 1, 2, and 3). In general, all groups decreased their rate of service linkage between baseline and year 1 (P < 0.001), and then, the rates (including the rate from the Self-Directed group) were attenuated between years 1 and 2 and between years 2 and 3. Since the baseline rates for Family Healthy Lifestyle and Self-Directed groups are greater than those for the Family Recreation and Comprehensive groups, we show the mean rates over time adjusted for the baseline rate (Additional file 2: Figure S1).

When the year of the visit was modelled as a linear variable rather than two indicator variables, the results did not change (Additional file 2: Table S1). In order to assess the impact of the year of the visit, following the initiation of the intervention (which occurred over a median of 22 months), we tested the interaction between the year of the visit and the interventions. None of the interactions were significant (data not shown).

Along with a threefold increase in linkage to child-development services, the strengths of our study include the diversity of our families across ethnicity, citizenship, family structure, education, employment and income, and a measured quality of life for low-income parents that can be compared to that of any number of populations. An EQ-VAS score of 75 shows that low-income parents on average display a highly compromised health state [32].

Our study also had a number of limitations. First, as previously mentioned, neither the full dose nor the ratio of doses between intervention groups was delivered as planned. Thus, the size of effect was expected to be lower and since the groups could no longer be collapsed in order to carry out the planned 2 x 2 factorial trial, there was less statistical power. The origins of this dose dilution are found in three areas related to the decision to deliver the service-integration approaches through a collaboration of existing agencies. One, social service agencies are used to being evaluated on the numbers of families served, not on the quantity or quality of their practice nor on the outcomes from that practice. Two, and related to the first, practice is often principle-based, as these service-integration vehicles were as opposed to the describable best behavioral practices upon which the practitioner, supervisor, and family can reflect. Three, professionals (nurses and likely social workers) are more likely than paraprofessionals to persist in delivering behavioral practices with low-income families [33, 34]. Constraints on public funding have led to the use of paraprofessional staff in the delivery of community-based service. These workers are poorly paid and thus turnover in employment is high, with expensive retraining required for new staff. They also need more skilled supervision because they are less persistent than professional workers.

A second limitation is that the interventionists were matched to the three intervention groups rather than to the two service-integration vehicles, each with a separate supervisor. The rationale was that the former approach allows the workers to focus on building relationships with their families so that more accurate assessments could be made and more teachable moments optimized. Such an approach should have given the Comprehensive group an advantage because they could build relationships using the relatively nonthreatening Family Recreation vehicle and go on to transfer and build on behaviors learned to the Family Healthy Lifestyle vehicle. Alternatively, the use of two supervisors, one for each vehicle (Family Recreation or Family Healthy Lifestyle), would have necessitated the use of two family workers for the Comprehensive group. This is not the practice of “real world” service agencies. The issue of workload and case management are practical considerations in dividing supervision. Therefore, because of the importance placed by the delivery agencies on developing enduring interpersonal relationships with clients, we may have confounded the effects of the interventions with the effects of the three intervention groups.

Third, as shown in Additional file 2: Fig. S1, our study may have exhibited a form of observer bias, a so-called Hawthorne effect [35]. The number of baseline links to services was high and then dropped for all four groups, only to climb back by the third year of measurement. The act of being observed (our data collectors while visiting and collecting data) might have temporarily decreased links to services in the first year only to return to the baseline rates over subsequent years. However, given that all four groups, including the Self-Directed group, exhibited the same downward-then-upward trend, observer bias did not likely impact the comparative effects between groups. Alternatively, in order for the families to balance their schedules, the data collection periods (the 28-day period prior to their visits) may not have overlapped sufficiently with the time allocated for intervention when the families may have accessed more services.

Fourth, since the families were not blinded, contamination bias may have diluted the effects of the interventions. Families may have transferred some of their learning to each other.

Fifth, the Family Services Inventory tool, given the short 1-month window of data captures and that the tool itself has not yet been validated, may not have adequately captured the linkages to service, thus diluting the effects of the interventions. For example, transportation supports are often received in conjunction with other services and would have been categorized with the primary service rather than as a basic need transportation service. In addition, since the data captures were retrospective, the results are subject to recall bias.

Similar to WTBB, the last limitation of our study was that 50 % of the enrolled families were lost to service and/or follow-up. Both high- and lower-functioning low-income families tend to avoid participation in social interventions such as service integration [36]. As well, low-income families tend to move frequently and are busy keeping their lives organized. The lives of lower-functioning low-income families can be complicated by issues such as mental health challenges, substance-dependency issues, and intergenerational family functioning problems. In addition, both higher- and lower-functioning low-income families are the most likely to be dropped from service by agencies or “have their files closed.” Although files were not to be closed by Families Matter in this study, losing track of “easy” families or “difficult” cases was observed to be a passive way to manage caseloads.

This study was a community-based trial on the effectiveness of principle-based practices for linkage to services by low-income families. The poor intervention fidelity in this study emphasized the existing challenges in modern community-based service delivery. Research is required to determine the processes the service-delivery sector needs to have in place in order to support full delivery of service practices. Consideration should be given to developing and documenting processes that provide behavioral description of practice; develop behavioral and attitudinal qualities of supervisors of paraprofessionals; improve retention of family participants; and provide early and ongoing audits of progress in the delivery of service.

Policymakers and funders have a couple of challenges. First, if funds are to be expended on service-integration approaches, then, given the intervention fidelity found in this study, policymakers need to insist, and therefore fund a) better-described practice, b) the auditing of that practice, c) retention of family participants, and d) the examination of family service use and outcomes. Second, if child-development services are widely required and are difficult for low-income families to access, then current policy needs to be examined. The following two questions need evaluation: 1) Are scarce and stigma-laden services best accessed through the provision of service integration? and 2) Is a system of accessible universal child-development programming more useful to the long-term health and economic productivity of the population?