Review Article
Agency for Healthcare Research and Quality Evidence-based Practice Center methods for systematically reviewing complex multicomponent health care interventions

https://doi.org/10.1016/j.jclinepi.2014.06.010Get rights and content

Abstract

Objectives

The purpose of this Agency for Healthcare Research and Quality Evidence-based Practice Center methods white paper was to outline approaches to conducting systematic reviews of complex multicomponent health care interventions.

Study Design and Setting

We performed a literature scan and conducted semistructured interviews with international experts who conduct research or systematic reviews of complex multicomponent interventions (CMCIs) or organizational leaders who implement CMCIs in health care.

Results

Challenges identified include lack of consistent terminology for such interventions (eg, complex, multicomponent, multidimensional, multifactorial); a wide range of approaches used to frame the review, from grouping interventions by common features to using more theoretical approaches; decisions regarding whether and how to quantitatively analyze the interventions, from holistic to individual component analytic approaches; and incomplete and inconsistent reporting of elements critical to understanding the success and impact of multicomponent interventions, such as methods used for implementation the context in which interventions are implemented.

Conclusion

We provide a framework for the spectrum of conceptual and analytic approaches to synthesizing studies of multicomponent interventions and an initial list of critical reporting elements for such studies. This information is intended to help systematic reviewers understand the options and tradeoffs available for such reviews.

Introduction

What is new?

  • Information addressing the challenges of systematic reviews of complex multicomponent interventions (CMCIs) is presented in one document.

  • This information is a synthesis of the current literature and the practices of leading international experts.

  • We provide a framework to consider when systematically reviewing CMCIs and analyzing the results.

  • We provide an initial list of elements that should be reported in research reports and systematic reviews of complex multicomponent health care interventions.

The Agency for Healthcare Research and Quality (AHRQ)'s Effective Health Care program receives requests to synthesize evidence regarding the effectiveness and harms of complex multicomponent health care interventions, such as quality improvement, infection control, and health care informatics interventions. Systematic reviews of complex multicomponent interventions (CMCIs) provide clinicians, policymakers, and others with information about the benefits and harms of interventions for decision making about implementation; however, because of their complexity, conducting reviews on multicomponent interventions is challenging.

A number of features distinguish CMCIs from other interventions such as devices or pharmaceuticals. First, CMCIs are commonly implemented at the level of an inpatient unit, outpatient clinic, hospital, or health system rather than individual patients. Because setting characteristics may interact with the intervention, it is critical to understand and enumerate those aspects that may modify the intervention's effects and synthesize the available interventions in context of those factors. Second, it may be uncommon for studies to examine identical combinations of components. This is particularly true as CMCIs evolve over time, and researchers add or remove components based on ongoing experience. Third, the “complexity” of CMCIs implies that the interventions cannot be reduced to the sum of their individual components but rather should be analyzed as “systems.” This challenges reviewers to delineate essential from nonessential components. Ultimately, the generalizability and usability of findings from syntheses of CMCI studies may be limited unless the reviewer can address these challenges [1].

Our purpose was to outline approaches to the challenges of conducting systematic reviews of CMCIs as a step toward the development of guidance.

Section snippets

Methods

Information was gathered through two complementary activities: a literature scan and key informant (KI) interviews of researchers, systematic reviewers, and health systems leaders involved in the development or use of CMCI reviews. Detailed methods and results are in the full report [2].

Results

Challenges identified by both literature reviews and interviews include terminology used to describe CMCI studies, framing the reviews, searching for literature, study designs, analytic considerations, and reporting elements.

Discussion

Ultimately, the goal of systematic reviews is to synthesize information for (1) clinicians and patients as they decide which interventions would be most effective for their situation, (2) hospital leaders to inform which interventions are most likely to achieve better results in their system, and (3) policymakers to make sound decisions to improve societal health and the health care system. Achieving these goals, when synthesizing evidence for CMCIs, is daunting. As recently demonstrated, the

Conclusion

Reviews of CMCIs are valuable, despite the field's evolving nature and unresolved challenges. Reviews can summarize an evidence base, which is valuable to researchers, implementers, and policymakers. However, reviewers are constrained by limited reporting in the evidence base. The list of reporting elements generated by KIs in this article provides a starting point. Next steps include the development of agreed on terminology for CMCIs and comparing and exploring new framing and analytic

Acknowledgments

The authors extend deepest appreciation for the contributions of the key informants who agreed to be interviewed for the project and to Stephanie Chang and Yen-Pin Chiang for their contribution to the project.

References (35)

  • J. Guise et al.

    Methods for conducting systematic reviews of complex multi-component health care interventions

    (2013)
  • Centre for Reviews and Dissemination

    Systematic reviews: CRD's guidance for undertaking reviews in health care

    (2009)
  • P. Craig et al.

    Developing and evaluating complex interventions: the new Medical Research Council guidance

    BMJ

    (2008)
  • T. Greenhalgh et al.

    Effectiveness and efficiency of search methods in systematic reviews of complex evidence: audit of primary sources

    BMJ

    (2005)
  • S. Shepperd et al.

    Can we systematically review studies that evaluate complex interventions?

    PLoS Med

    (2009)
  • N. Jackson et al.

    Criteria for the systematic review of health promotion and public health interventions

    Health Promot Int

    (2005)
  • Medical Research Council

    A framework for development and evaluation of RCTs for complex interventions to improve health

    (2000)
  • Cited by (0)

    Funding: This project was funded under contract numbers HHSA290201200010i, HHSA290201200012i, HHSA290201200011i, HHSA290201200015i, HHSA290201200008i, and HHSA290201200004C from the Agency for Healthcare Research and Quality and U.S. Department of Health and Human Services.

    Disclaimer: The authors of this document are responsible for its content. Statements in the report should not be construed as endorsement by the Agency for Healthcare Research and Quality or the U.S. Department of Health and Human Services.

    View full text