Supporting Vulnerable People During Challenging Transitions: A Systematic Review of Critical Time Intervention

We followed the systematic protocol recommended by Preferred Reporting Items for Systematic Review and Meta-Analysis for protocols (PRISMA-P) (Shamseer et al., 2015). We performed a standardized search of the literature using the following databases for peer-reviewed articles dating from 1990 to August 2020: CINAHL, EMBASE, PubMed, and Cochrane. The following search terms were used to identify studies testing the effectiveness of CTI: critical time intervention, critical time, time limited intervention, transitional support, transitional intervention, and transitional assistance. To identify articles not included in our original search, we reviewed reference lists of studies that met inclusion criteria, reviewed existing relevant systematic reviews and general literature reviews (de Vet et al., 2013; Herman, 2014; Herman & Mandiberg, 2010; Herman et al., 2007; Hopkin et al., 2018; Vigod et al., 2013), searched Google and Google Scholar by using different combinations of the terms, and then retrieved unpublished data and gray literature reports from existing networks.

A screening tool was specified in advance. Reports were considered eligible for inclusion if they examined CTI or a modified version of CTI using experimental or quasi-experimental designs. Given the broad application of CTI, we did not specify inclusion criteria with respect to populations, transition types, or outcomes. We included reports regardless of their country of origin that were written in or translated into English. We excluded studies without a comparison group, review articles, editorials, commentaries, theoretical articles, and case reports. In addition, we also excluded studies that described CTI as a secondary or partial focus in multi-component interventions.

After we conducted the search, we exported all identified references into EndNote X7. We removed duplicates both automatically using EndNote’s duplicates removal function and manually. Two raters independently screened titles and abstracts of the retrieved publications, excluding reports that did not match our inclusion criteria, and then independently evaluated the full-text publications to confirm eligibility based on our inclusion criteria. Two raters extracted data for all included studies using a standardized Excel form, and a third rater checked each extraction for accuracy. Information was extracted on study design, sample size, follow up, intervention duration, comparison group type, sample population, transitional setting, fidelity rating, and reported outcomes. We found a high level of heterogeneity among studies on these study characteristics, which led us to conduct a narrative systematic review. We categorized and evaluated all outcome measures that were included in randomized controlled trials and quasi-experimental studies.

To assess risk of bias within each study, we used the Risk of Bias tool (RoB) in the Cochrane Handbook for Systematic Reviews of Interventions (Higgins & Green, 2011). Before beginning the risk of bias assessment, we consulted the Cochrane Handbook to clarify bias categories and criteria for judgements (Higgins & Green, 2011). A spreadsheet was created that contained a separate, identical table for each study, comprised of a set of categories for each element of potential bias: Selection Bias (Random Sequence Generation), Selection Bias (Allocation Concealment), Performance Bias (Blinding—participants & personnel), Detection Bias (Blinding—outcome assessment), Attrition Bias (Incomplete Outcome Data), Reporting Bias (Selective Reporting). We did not assess for Other Bias (Other Sources of Bias), though we acknowledge the possibility of other sources of bias for some studies. Based on the Cochrane Handbook, descriptions of each bias category and rating anchors were provided to each rater. Notes, including direct quotes from studies and rater comments and justifications, were documented along with each rater’s independent assessment.

To establish consistency and minimize the impact of rater subjectivity, we used an iterative process involving independent reviews by multiple raters. Three raters conducted the risk of bias review (JIM, MN, DB). During the first round of reviews, each rater reviewed the same three studies independently then went through each rating of each study together. If ratings did not align, raters gave justifications and again reexamined the study together until consensus was reached. The second round of reviews included another three studies with the same process. The third review consisted of two raters (JIM, MN) assessing the remainder of the studies independently. After completing the independent ratings, all three raters met to discuss discrepancies in ratings of each study until consensus was achieved for all studies. If raters identified a need for more information, study authors were contacted. If a study was rated high or unclear on at least two bias domains, they were given an overall rating of high risk of bias.

The results of the systematic search and selection process are summarized in Fig. 1. The search identified 5242 citations from database searching and nine citations were identified through other sources. Of these, 890 citations were duplicates. A total of 4361 abstracts were screened and reviewed, and 4275 ineligible studies were excluded. The full text of the remaining 86 articles was assessed further for inclusion. A total of 68 full-text articles were excluded. We excluded articles that were non-English; classified as conference abstracts, commentary, reviews, protocols, fidelity pilots or qualitative studies; or described transition interventions where CTI was not a central component of the intervention. A total of 18 articles were included in the review, representing data from 13 original studies. One of the included studies was from the grey literature, which includes unpublished reports, working papers, government documents, and evaluations.

Table 1 describes the characteristics of the included studies. Of the 13 original studies, 10 were randomized controlled trials and three were quasi-experimental studies. The majority of studies were conducted in the United States (n = 8), one in Brazil, two in the Netherlands, one in the United Kingdom, and one in Canada. Sample sizes ranged from 71 to 484 participants and follow-up periods ranged from 3 to 24 months. Ten studies reported attrition at follow-up, ranging from 2.7 to 44%. Nine studies discussed fidelity monitoring, of which 4 studies reported an overall fidelity score of 3 out of 5 or better (indicating at least fair model fidelity). Two studies used a modified fidelity scale that included 12 items representing compliance fidelity, or the degree to which the key elements of CTI were implemented (8 items) and the extent to which the intervention is well-documented (4 items) (de Vet et al., 2017; Lako et al., 2018).

Included studies represented a range of populations and transition types, including individuals with serious mental illness and chronic homelessness, individuals with co-occurring mental health and substance use disorders, veterans, individuals who were being released from prison or jail, homeless families, homeless women, and women with history of intimate partner violence. CTI was implemented for a variety of transitions, including from homeless shelters, transitional living communities, inpatient psychiatric hospitals, emergency room, and prisons or jails. Most studies also involved a shortened intervention period from the original 9 months to 3 to 6 months in duration. Outcomes also varied widely between studies, with housing and service engagement as most prominent. Other outcomes examined were hospitalization or emergency services, mental health, substance use, family and social support, and quality of life. Most often, different instruments and operationalizations of outcomes were used across the studies (see Online Appendix A). It is important to note that the designation of outcomes as primary or secondary varied and depended on the study goals and population or transition setting. As such, we report all outcomes by domain without such designation in the results.

Details of the risk of bias assessment are provided in Table 2. All 10 experimental studies were rated as exhibiting low risk of bias and all three quasi-experimental studies received ratings of high risk of bias. For Selection Bias domains (Random Sequence Generation and Allocation Concealment), even if procedures were not described in great detail, if groups were found to be comparable at baseline, a low bias rating was given. Of note, Silva et al. (2017) had a relatively small sample size, which is a limitation, and though the study appeared to follow appropriate, unbiased procedures, it ended up with unbalanced groups with respect to gender. Nevertheless, Silva et al. (2017) was still rated as low risk. The publications with a high-risk rating were primarily limited methodologically due to absence of randomization and high risk of selection bias. Performance Blinding, wherein both participants and personnel were blind to knowledge of which intervention the participant received, was impossible in nearly all studies due to the nature of the intervention. According to Cochrane guidelines, if raters acknowledge no blinding or incomplete blinding, but do not believe that such would have affected the outcome, then a low rating is appropriate as long as the judgment is justified (Higgins & Green, 2011). The same rating logic applies to Detection Bias when outcome assessors have knowledge of the participants’ intervention status; if the raters believe that lack of blinding to the intervention that the participant received by outcome assessors has not influenced the outcome measurement, then a low rating is appropriate as long as the judgment is justified (Higgins & Green, 2011). Study attrition levels were relatively low with only three studies reporting more than 25% attrition at follow up. Crampton et al. (2020) had a shortage of specific details regarding missing data, which accounts for our only unclear rating.

Table 3 presents the overall impact in each outcome domain for each study, indicating when two or more measures of an outcome were reported. We summarize each outcome domain below, highlighting results from experimental and quasi-experimental studies.

This is the first systematic review to our knowledge summarizing the efficacy of CTI across multiple domains. This review extends the earlier narrative reviews of CTI (Herman et al., 2007; Herman & Mandiberg, 2010; Herman, 2014), which reported on fewer randomized controlled trials with different target populations. We found 18 reports based on 13 studies, most involving randomized controlled trials. Like the original CTI trials (Herman et al., 2011; Susser et al., 1997), the majority of studies recruited participants with current or a history of homelessness. Despite the original 9-month intervention timeframe, most studies used abbreviated timeframe ranging from 3 to 6 months. Justifications for an abbreviated model range from maximizing program capacity (Kasprow & Rosenheck, 2007; Nossel et al., 2016) to focusing specifically on continuity of care outcomes (Dixon et al., 2009; Nossel et al., 2016; Shaffer et al., 2015; Shaw et al., 2017; Stergiopoulos et al., 2017). Other adaptations offered CTI alongside with ancillary services and supports, including trauma-informed care (Crampton et al., 2020); the Strengths Model (Lako et al., 2018); and family-based care (Samuels et al., 2015). Nossel et al. (2016) employed peer specialists as CTI workers as a way to increase the acceptability of the intervention to study participants. Eleven studies described the extent to which the interventions were delivered in accordance with model criteria, although only four studies reported an overall fidelity score.

Three key findings emerged from our review. First, there is consistent evidence that CTI has a positive impact on housing and service engagement use outcomes in the United States. These outcome domains were among the most commonly studied. These results align with CTI’s focus of linking individuals to services and supports in the community to address areas of need critical for successful transition. These findings highlight the practical and adaptable nature of CTI in which the model can be implemented successfully across varied settings and populations (Herman, 2014).

Notably, the studies conducted outside of the U.S. revealed inconsistent homelessness and service engagement findings. This inconsistency may be due to cross-country differences or implementation challenges. For example, de Vet et al. (2017) suggest that the lack of impact on housing outcomes they observed in the Netherlands may be because recurrent homelessness was rare in both study conditions. The authors also believed the impact of CTI on housing outcomes might have been stronger had the intervention been delivered with greater model fidelity. Stergiopoulos et al. (2017) suggest that the complex needs of the frequent emergency room users with mental illness or addiction they studied in Canada may not have been adequately addressed by CTI. The authors also noted that the delivery of the intervention may have had a greater impact had it more closely followed the model’s core components and tasks.

Less consistent results were found with respect to other domains (i.e., hospitalization or emergency services, mental health, substance use, family and social support, and quality of life). The inconsistent findings may be a function of the priority areas defined by individuals in collaboration with their CTI case managers. If the intervention work is centered on the priorities of obtaining and retaining housing and service engagement, especially during the early phases of the intervention, then other issues like family and social support and quality of life may receive less attention. Perhaps if studied interventions had focused on engaging more informal resources, they would have had more positive gains in other domains. Furthermore, it may take longer to see the intervention’s impact on these ancillary domains given the chronic nature of homelessness and mental illness prevalent in many sample populations. Finally, these domains are likely more resistant to change using a time-limited care coordination approach, especially in studies using the abbreviated model of CTI (i.e., 3–6 months) applied in six of the studies. Nevertheless, studies with long-term follow-up assessments are needed to assess whether changes in CTI are sustained over time.

While the optimal duration of CTI is unclear, a second key finding of this review is the successful use of the 9-month model with a variety of populations and settings. Results on the implementation of CTI using shorter durations are less clear, suggesting the need for studies to compare the impacts of CTI of varying duration and intensity. Additionally, we know little about for whom the intervention works and under what conditions. For example, research is needed that moves beyond examining the average treatment effects by identifying the intervention effects across subgroups with respect to demographic (i.e., age, gender, race/ethnicity) and clinical (i.e., symptom severity) characteristics, and for whom might varying durations and intensity work best.

Third, like many other empirically supported models, we found limited evidence that addresses how CTI achieves its positive impacts. The specific mechanisms of how CTI leads to reductions in homelessness and other outcomes remain unclear. Indeed, only one of the randomized trials (Tomita et al., 2014) examined potential mediating paths through which CTI may operate. More research is clearly needed to map CTI’s program components onto specific mediators and investigate whether and how these mediators affect specific outcomes.

Findings from this review should be interpreted with caution given the high degree of variability in the samples and methodological design and rigor of the studies reviewed. Furthermore, making comparisons between studies is limited due to a lack of standardization, inconsistent fidelity assessment and reporting, varying lengths of intervention, diversity in samples and outcomes, inconsistency of measures, and variability in methodological design and rigor. Five of the 10 randomized controlled trials were conducted outside the US, including Brazil, Canada, Netherlands, and United Kingdom. While several of these studies are small, making it difficult to detect effects, it is important to note the substantial differences between countries with respect to the strength of the safety net and organization of health care systems that make it challenging to compare results across these studies. The studies also relied heavily on participant self-report data, which may lead to inaccuracies in the reporting of treatment effects. Although most of the outcome measures in the studies reviewed have strong psychometric properties, future studies should consider including real-time measures of behaviors and experiences (i.e., ecological momentary assessments) or more objective measures (e.g., electronic health records) when possible.

We note several limitations to our review. First, we did not examine the costs of CTI, although one of the included studies found CTI to be cost effective (Jones et al., 2003; described in Online Appendix A). Second, the included studies were implemented with a variety of populations and settings using varying durations, making it difficult to compare across studies. Third, the number of outcomes examined, as well as the lack of consistency in most, precluded a formal meta-analysis. As the body of CTI evidence grows, we hope to conduct such an analysis on selected outcomes. Another limitation is that the bias assessment was conducted for each study overall and did not separately assess the risk of bias for each outcome of each study. Therefore, while a study's overall risk of bias might be rated as low, the risk of bias for different outcomes within each study is not reflected in the overall ratings. The Cochrane assessment scheme excludes key implementation processes (e.g., program specification, training and supervision, fidelity, intervention dose/attrition) that also may impact the quality of studies.

CTI showed a fairly consistent positive impact on homelessness and increased service engagement across diverse populations, suggesting the versatility of CTI for different populations and transitions. To properly inform policymakers and practitioners, future studies should be carefully designed with an aim for greater consistency in outcomes evaluated and enhanced standardization of measurements. Additionally, several ancillary, longer-term outcomes of CTI, including mental health, substance use, family and social support, and quality of life, require greater study. Future studies are needed to examine structural adaptations to the model, such as the intervention duration and delivery approaches (e.g., the use of peer specialists). Future adaptations of CTI should be guided by implementation science to ensure contextual factors are considered when tailoring the model to a specific population or transition setting. Additionally, we need greater attention to how these models will be sustained, especially in fragmented systems of care like that of the U.S.

Another important implication of our review is the need to predict which model components facilitate favorable outcomes. Formal testing of mediating and process variables need to be incorporated into future research. More research is also needed to understand subgroup effects and the specific adaptations needed to better serve those for whom the current model is less effective. The articles we reviewed contained limited information on important implementation processes, including training, supervision, and fidelity assessment processes. More consistent use of the CTI fidelity scale in future research is critical for exploration of key components of the model. As CTI continues to be broadly disseminated into routine practice settings, it would also be fruitful to study the cost-effectiveness of intervention duration and delivery approaches, as well as the challenges and strategies in its uptake.