We adopted the Joanna Briggs Institute (JBI) methodology for umbrella reviews [
18], and we used the Preferred Reporting Items for Systematic Reviews and Meta-Analyses-Protocol (PRISMA-P) guidelines to structure this protocol (Additional file
1) [
19]. The study protocol was registered with the International Prospective Register of Systematic Reviews (PROSPERO, CRD42020183774).
Eligibility criteria
We will address all types of evidence matching the “PICO” criteria (Participants, Intervention, Comparator, Outcome) and “PICo” (Population, phenomena of Interest and Context) to capture the evidence from quantitative and qualitative reviews.
Participants or population
All reviews that include primary studies focusing on individuals (e.g., patients or caregivers, health care providers), or health systems, services, or organizations that have been exposed to the scaling of a health or social care innovation.
There will be no restrictions based on socio-demographic factors (e.g., age, ethnicity, socio-economic status) or general health conditions (e.g., comorbidities).
Interventions/phenomena of interest
No restrictions. We will consider all types of scaling, vertical and horizontal. This umbrella review will not only explore the scaling of health and social care innovations but also any aspect of or any topic relating to such scaling, including (but not limited to) concepts, models, analytical models, tools, cost or impact assessments, and user engagement.
Comparator
No restrictions.
Outcomes
We will consider all outcomes reported in the included reviews (i.e., no restrictions), including health outcomes. We will seek outcomes such as (but not limited to) patients/caregivers and/or health care providers and/or policy makers’ perceptions and experiences of barriers, facilitators, acceptability and feasibility of scaling innovations, impact (e.g., adaptability, efficacy, effectiveness), coverage (e.g., proportion of the target population that is reached by the scaling, adoption, fidelity, penetration, or maintenance of the innovation), health outcomes (e.g., impact on morbidity, mortality), patient reported outcomes (e.g., quality of life, satisfaction), and health care resources (e.g., cost-effectiveness of the scaled innovation, cost of staff resources).
Types of study
We will consider all types of review (quantitative, qualitative, and mixed-methods) that address, synthesize, and summarize evidence in the field of scaling. We define a review as a knowledge synthesis of evidence that includes a clear research question, describes the methods used (which are reproducible) to identify and select the primary research studies, and synthesizes data from its included studies [
20,
21]. Reviews can include studies with any research design. We will include all types of review that have been rigorously conducted according to their chosen methodological approach. We will exclude reviews that do not describe their search strategy and inclusion criteria explicitly at the stage of full-text screening. We will also exclude primary research studies, conference abstracts, comments, opinions, letters, and editorials.
Context/setting
We will include reviews regarding any type of health and social care setting (e.g., home care, community, hospital, primary care, specialized care) in any geographical setting (e.g., rural or urban regions, low-, middle- or high-income countries).
We define “health and social care services” as follows:
Health care consists of services provided in institutional or community settings, any form of access to a health-related service (such as dental, podiatry, or optical services), and access to health care practitioners (such as nurses, physiotherapists, or general practitioners) [
22].
Social care consists of interventions that support frail or vulnerable individuals by meeting needs or enabling them to meet needs that arise as a result of physical, mental, or emotional impairment [
23].
Search strategy
Our information specialist (NR) will develop an Ovid-MEDLINE and Web of Science strategy with input from the project team and a second information specialist. An iterative revision process will be conducted by the members of the research team. Research keywords will include the following: “scaling,” “reviews,” and “health and social care”. We will use the search strategy for “scaling up” previously developed by members of our team. The Canadian Agency for Drugs and Technologies in Health (CADTH) search filter for Systematic Reviews/Meta-Analysis/Health Technology Assessment was adapted for this project [
24]. A second information specialist will review the search strategy using the Peer Review of Electronic Search Strategies (PRESS) tool [
25]. Comments will be integrated in a final version of the search strategy. The final version will be approved by the team members. Once approved, this search strategy will be translated into the other databases mentioned below. The exact search terms will be recorded as the search strategy is refined. This protocol only includes the search strategy conducted in one database (Additional file
2). A further systematic literature search will be performed to identify published studies in the following electronic bibliographic databases: MEDLINE (Ovid), Embase, PsychINFO (Ovid), CINAHL (EBSCO), Web of Science, The Cochrane Library, Sociological Abstract (Proquest), Academic Search Premier (EBSCO), and Proquest Dissertations & Theses Global. No language restriction will be applied. We will search from inception onwards.
Additionally, to identify the grey literature, we will search the websites of relevant organizations such as the World Health Organization (WHO), Global Reporting Initiative, the UK’s National Institute for Care and Excellence (NICE), Australia NSW Government, the Institut National d’Excellence en Santé et Services Sociaux (INESSS) in Quebec, Canada, the Canadian Foundation for Healthcare Improvement (CFHI), Canada’s IDRC, the CADTH’s Grey Matters checklist, and clinical trial registries. We will contact experts in the field by email for additional data. In addition to searching databases, the reference list of each included review will be reviewed.
Selection process
The search results will be imported and stored in an EndNote X9 library for reference management and duplicate removal [
26,
27]. The resulting records will be exported to the Internet-based system Covidence for the selection process [
28]. The team will develop a selection grid. It will be adjusted, if necessary, before the screening of titles and abstracts of all articles. All selection criteria will be discussed by reviewers to ensure common understanding. A pilot screening (titles and abstracts) of 2.5% random sample reviews will be completed. Discrepancies will be resolved by consensus or by a third reviewer if necessary.
First, two independent reviewers will screen titles and abstracts of identified reviews against the eligibility criteria. Articles with abstracts that do not appear to meet the criteria for exclusion, or are ambiguous, or have missing abstracts, will be retained and reviewed in full. Second, full-text examination of the remaining reviews will be assessed for eligibility. The reviewers’ full understanding of the selection criteria will be validated again before beginning this stage. In case of an “unclear” response regarding the eligibility of studies, authors will resolve disagreements through discussion and, if necessary, consult a third senior author. Any reasons for exclusion will be recorded in Excel.
Articles that are not available electronically will be ordered via interlibrary loan. We will contact the corresponding author if an email is available.
Once the reviews have been selected for inclusion, two reviewers will independently conduct the data extraction. We will develop a data extraction form based on the JBI form for review of systematic reviews. A standardized, pre-piloted form will be used to extract data from the included reviews for assessment of study quality and evidence synthesis. A pilot data extraction will be conducted on 5% of selected reviews until conclusive results are agreed on between reviewers. Any disagreement between the reviewers will be resolved through discussion; if consensus is not reached, they will consult a third senior reviewer. We will not extract data from the primary studies included in the reviews. We will summarize the review findings but we will not re-synthesize the results of primary studies. Extracted information will include:
-
Review characteristics: author, year of publication, country, type of review; aim of the study; language; PICO or PICo; number and characteristics of participants (e.g. sexas a biological variable, age); setting and context; number and type of databases sourced and searched; date range of database searching; citation index; number and study design of primary studies included in each review; instrument used to appraise the primary studies and the rating of their quality; method of synthesis/analysis used to synthesize the evidence; synthesis/summary of results; heterogeneity if applicable; mention by the authors of research needs or gaps in the review.
-
Focus of the reviews: infrastructure (e.g., policies, guidelines, human and material resources); concepts and models (e.g., definition, conceptual model, framework); measurement (e.g., tools, scalability, cost); analytical methods (e.g., mathematical approach); descriptive (e.g., perceptions, barriers, facilitators, acceptability); patient engagement (e.g., authorship, study’s reporting on patient advocates) and scaled interventions (e.g., strategies, training, process, coverage, feasibility, effectiveness).
Assessing quality of reviews
Two independent reviewers will use AMSTAR2 [
29] to assess the methodological quality of included reviews and ROBIS [
30] for the risk of bias. Based on the nature of the included reviews, we will explore the relevance of using the GRADE assessment system [
31,
32]. Disagreements between the reviewers over the quality or the risk of bias in particular studies will be resolved by discussion, with involvement of a third author where necessary. AMSTAR2 is a validated, widely used 16-item instrument for assessing reviews of reviews. ROBIS contains 24 questions divided into three phases. Phase 1 assesses relevance (optional) and verifies if the research question matches the umbrella review’s PICO; phase 2 identifies concerns with the review process; and phase 3 judges risk of bias in the review. Phase 2 and 3 questions are answered with the following options: yes, probably yes, probably no, no, no information. The concerns regarding phase 2 domains and phase 3 are classified as high, low, or unclear.
Data synthesis
We will use the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) flow diagram to describe the process of study selection [
33]. We will use tables to describe characteristics of included studies (e.g., first author’s name, year of publication, study aim) and analysis details (e.g., appraisal instrument used, methods of analysis, results summary, authors comments, heterogeneity of the results), and we will synthesize the data narratively using categories of scaling evidence that have emerged from the included reviews (e.g., descriptive reviews of scaling, reviews of barriers and facilitators, reviews of determinants, reviews of concepts, reviews of frameworks, reviews of economic evaluation, reviews of scaling interventions, reviews of scaling measures, reviews of effects of scaling interventions, reviews of scalability).
A meta-analysis will not be performed, given that pooling the results of qualitative and quantitative reviews can introduce significant overlap and bias. We will clearly indicate the overlap between primary studies within the reviews in the tables, and we will develop a citation matrix. Because our purpose is to identify evidence about scaling in health and social care in the literature, we will present the whole body of knowledge and include the results of all the reviews, regardless of the overlap across primary studies [
34].