The use of the PARIHS framework in implementation research and practice—a citation analysis of the literature
verfasst von:
Anna Bergström, Anna Ehrenberg, Ann Catrine Eldh, Ian D. Graham, Kazuko Gustafsson, Gillian Harvey, Sarah Hunter, Alison Kitson, Jo Rycroft-Malone, Lars Wallin
The Promoting Action on Research Implementation in Health Services (PARIHS) framework was developed two decades ago and conceptualizes successful implementation (SI) as a function (f) of the evidence (E) nature and type, context (C) quality, and the facilitation (F), [SI = f (E,C,F)]. Despite a growing number of citations of theoretical frameworks including PARIHS, details of how theoretical frameworks are used remains largely unknown. This review aimed to enhance the understanding of the breadth and depth of the use of the PARIHS framework.
Methods
This citation analysis commenced from four core articles representing the key stages of the framework’s development. The citation search was performed in Web of Science and Scopus. After exclusion, we undertook an initial assessment aimed to identify articles using PARIHS and not only referencing any of the core articles. To assess this, all articles were read in full. Further data extraction included capturing information about where (country/countries and setting/s) PARIHS had been used, as well as categorizing how the framework was applied. Also, strengths and weaknesses, as well as efforts to validate the framework, were explored in detail.
Results
The citation search yielded 1613 articles. After applying exclusion criteria, 1475 articles were read in full, and the initial assessment yielded a total of 367 articles reported to have used the PARIHS framework. These articles were included for data extraction. The framework had been used in a variety of settings and in both high-, middle-, and low-income countries. With regard to types of use, 32% used PARIHS in planning and delivering an intervention, 50% in data analysis, 55% in the evaluation of study findings, and/or 37% in any other way. Further analysis showed that its actual application was frequently partial and generally not well elaborated.
Conclusions
In line with previous citation analysis of the use of theoretical frameworks in implementation science, we also found a rather superficial description of the use of PARIHS. Thus, we propose the development and adoption of reporting guidelines on how framework(s) are used in implementation studies, with the expectation that this will enhance the maturity of implementation science.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Contributions to the literature
• Describes how a well-established theoretical framework—PARIHS—has been operationalized in the scientific literature and provides examples of its use in implementation studies.
• The findings underline that descriptions of the use of the framework generally were not that transparent and often partial.
Anzeige
• Findings also point at difficulties in using the framework, such as lack of guidance on key steps to overcome barriers and support implementation
• Identifies the need of common guidelines on how theories, models, and frameworks should be reported in research articles.
Introduction
There has been an increased use of theoretical frameworks in the field of implementation science in the last decade, with most developed in the last two decades [1, 2]. Tabak et al. identified 61 theoretical models used in dissemination and implementation science [3]. However, while theoretical frameworks are increasingly being cited, more research is needed to understand how they are chosen and applied, and how their use relates to improved implementation outcomes [1, 4]. Variously described in the form of theories, frameworks, or models, all strive to provide conceptual clarity on different aspects of implementation practice and research. For consistency, we will refer to these as theoretical frameworks, or simply “frameworks.”
The Promoting Action on Research Implementation in Health Services (PARIHS) framework is a multi-dimensional framework which was developed to explicitly challenge the pipeline conceptualization of implementation [5]. The PARIHS framework is a commonly used conceptual framework [1, 4] that posits successful implementation (SI) as a function (f) of the nature and type of evidence (E) (including research, clinical experience, patient experience, and local information), the qualities of the context (C) of implementation (including culture, leadership, and evaluation), and the way the implementation process is facilitated (F) (internal and/or external person acting as a facilitator to enable the process of implementation); SI = f(E,C,F). The framework was informed by Rogers’ Diffusion of Innovations [6] and various organizational theories and theories from social science [7] and generated inductively by working with clinical staff to help them understand the practical nature of getting evidence into practice. The PARIHS framework was initially published in 1998 [5] and updated based on a conceptual analysis in 2002 [8] and further primary research [9]. A further refinement was undertaken in 2015 [10], resulting in the integrated or i-PARIHS. Articles using the revised version are not included in the citation analysis reported here. The PARIHS framework has been described as a determinant framework in that it specifies determinants that act as barriers and enablers influencing implementation outcomes [2]. Skolarus et al. [1] identified Kitson et al. [5] as one of the two primary originating sources of influence in their citation analysis of dissemination and implementation frameworks.
Anzeige
Despite the growing number of citations of theoretical frameworks in scientific articles, the detail of how frameworks are used remains largely unknown. Systematic reviews of the application of two other commonly used frameworks [1], the Knowledge to Action framework [11] and the Consolidated Framework for Implementation Research [12], both reported that use of these frameworks, beyond simply citation, was uncommon. While PARIHS has been widely cited, it has also been scrutinized; in 2010, Helfrich et al. published a qualitative critical synthesis of studies that had used the PARIHS framework [13], finding six core concept articles and 18 empirical articles. One of the reported findings was that PARIHS was generally used as an organizing framework for analysis. At the time, no studies used PARIHS prospectively to design implementation strategies [13]. A systematic review applying citation analysis to map the use of PARIHS (similar to those undertaken for the Knowledge to Action framework (KTA) [11] and the Consolidated Framework for Implementation Research (CFIR) [12]) has not yet been performed.
Systematic reviews can contribute to the development of existing theoretical frameworks by critically reviewing what authors state as their weaknesses and strengths; they can also direct future and current users of frameworks to examples of using the frameworks in different ways. To contribute to this development from the perspective of the PARIHS framework, we undertook a citation analysis of the published peer-reviewed literature that focused on the reported use of PARIHS (and its main elements), in what contexts the framework has been applied, and what scholars who have used the PARIHS framework (and its main elements) report as its strengths, limitations, and validity.
Methods
The method used for this study is citation analysis, i.e., the examination of the frequency and patterns of citations in scientific articles, in this case articles citing the core PARIHS framework publications. A team of researchers with engagement in the development and/or use of the PARIHS framework was constituted. Initially, the group decided on the core publications for the citation analysis. Four articles were selected as they represented the key stages of the framework’s development, namely the original paper that described PARIHS, plus three subsequent papers that informed and outlined revisions to the framework:
1.
Kitson A, Harvey G, McCormack B. Enabling the implementation of evidence-based practice: a conceptual framework. Qual Health Care. 1998;7(3):149-58.
2.
Rycroft-Malone J, Kitson A, Harvey G, McCormack B, Seers K, Titchen A, et al. Ingredients for change: revisiting a conceptual framework. BMJ Quality Saf. 2002;11(2):174-80.
3.
Rycroft-Malone J, Harvey G, Seers K, Kitson A, McCormack B, Titchen A. An exploration of the factors that influence the implementation of evidence into practice. J Clin Nurs. 2004;13(8):913-24.
4.
Kitson AL, Rycroft-Malone J, Harvey G, McCormack B, Seers K, Titchen A. Evaluating the successful implementation of evidence into practice using the PARiHS framework: theoretical and practical challenges. Implement Sci. 2008;3:1.
Citation search
Citation searches were performed by an information specialist (KG) to retrieve published articles citing any of the four core articles. The searches were performed in two citation databases: Web of Science and Scopus. The first searches were performed between 31 March 2016 and 1 April 2016. Later, 6 September 2019, additional searches were performed in respective databases. These searches were limited to citations that were published 1 April 2016–31 August 2019 to update the result from the first searches. All citations that were published September 1998 (i.e., when Kitson et al 1998 was published)–31 August 2019 (i.e., prior to the search date) in respective databases were collected in EndNote Library. Endnote was used for checking duplicates and retrieving full texts. To manage the scope of the citation analysis, we opted to only include articles in English published in peer-reviewed scientific journals. The searches in Web of Science were, because of the subscription, limited to Web of Science Core Collection without Book Citation Index.
Data extraction
The Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) flow diagram [14] for the data extraction is provided in Fig. 1. Initially, an assessment to identify the articles that used the PARIHS framework in any other way than merely referencing one or more of the core articles was performed (Additional file 1). For this initial assessment, all articles were read in full. After identifying articles where the PARIHS framework was used, data extraction was undertaken using a tailor-made data capture form (Additional file 1). The data capture form was developed and piloted in iterative cycles by the research team. Apart from capturing information about where (country/countries and setting/s) and with whom (professional groups and roles) PARIHS had been applied, the form included questions on whether PARIHS was used in one or more of the following ways:
1)
In planning and delivering an intervention,
2)
In data analysis,
3)
In the evaluation of study findings, and/or
4)
In any other way.
Fig. 1
Adapted PRISMA flow diagram
×
Each of these questions was followed by an open-ended item for extracting information on how this was reported [15]. To enhance reliability and data richness, each reviewer copy-pasted sections of the article corresponding to the open-ended reply into the data extraction form when appropriate and indicated page, column, and row. Two additional items captured whether the PARIHS framework had been tested or validated, as well as any reported strengths and weaknesses of the framework. Thus, we report on what the authors of the included articles claim to have done, rather than a judgment as to how and to what extent they actually used the PARIHS framework.
For data extraction and validation, the research team was divided into four pairs, ensuring that each article was assessed separately by at least two research team members. The pairs received batches of 20 articles at a time. Variations in the assessments were discussed until consensus was reached within the pair(s). Further, queries detected within the pairs were raised and discussed with the whole research team, until consensus was achieved. Regular whole-team online meetings were held to consolidate findings between every new batch of articles and throughout the development and analysis process. In total, the group had > 20 online meetings and four face-to-face meetings from the initial establishment of the group in January 2015.
Data analysis
Categorical data were analyzed using descriptive statistics, whereas the open-ended items were analyzed qualitatively [16], including the collated extractions of data to illustrate each of the four types of use (i.e., how the PARIHS framework was depicted in terms of (1) planning and delivering an intervention, (2) analysis, (3) evaluation of study findings, and/or (4) in any other way).
Anzeige
Applying a content analysis approach [17], members of the research team worked separately with the texts extracted from the reviewed articles. The extracts for each open-ended item were read and reread, to get a sense of the whole. Next, variations were identified and formed as categories. Findings for each question were summarized in short textual descriptions, which were shared with the whole team. In a face-to-face meeting, the data relating to each question were critically discussed and comparisons were made between the findings for each question, to identify overlaps and relationships about how PARIHS has been used.
Results
After duplicate control, 1613 references remained. These were sorted by language and type of publication. In this phase, 131 references categorized as books, book chapters, conference proceedings, and publications written in non-English language were excluded. Also, three of the four core articles (i.e., the three citing Kitson et al. [5] which was the starting point for development of the PARIHS framework and therefore did not appear in the citation search) were excluded from the database [8, 9, 18], as were four articles expanding and refining PARIHS [19‐22]. Accordingly, 1475 articles remained, and after the assessment excluding those merely citing PARIHS, a further 1108 articles were excluded, leaving 367 articles that cited one or more of the core articles, and made explicit use of the PARIHS framework (see Fig. 1 and Table 1).
Table 1
List of articles for data extraction for citation analysis of the use of the PARIHS framework. The table is sorted on type of article, type of use of the PARIHS framework, author and year of publication
A complex culturally targeted intervention to reduce Hispanic disparities in living kidney donor transplantation: An effectiveness-implementation hybrid study protocol
Implementing a knowledge application program for anxiety and depression in community-based primary mental health care: A multiple case study research protocol
Implementation of a knowledge mobilization model to prevent peripheral venous catheter-related adverse events: PREBACP study-a multicenter cluster-randomized trial protocol
Prioritizing Responses Of Nurses To deteriorating patient Observations (PRONTO) protocol: Testing the effectiveness of a facilitation intervention in a pragmatic, cluster-randomized trial with an embedded process evaluation and cost analysis
Monitoring and managing metabolic effects of antipsychotics: A cluster randomized trial of an intervention combining evidence-based quality improvement and external facilitation
Accessibility and implementation in UK services of an effective depression relapse prevention program - mindfulness-based cognitive therapy (MBCT): ASPIRE study protocol
Implementing health research through academic and clinical partnerships: A realistic evaluation of the Collaborations for Leadership in Applied Health Research and Care (CLAHRC)
Protocol: Adaptive Implementation of Effective Programs Trial (ADEPT): Cluster randomized SMART trial comparing a standard versus enhanced implementation strategy to improve outcomes of a mood disorders program
Comparison of high and low intensity contact between secondary and primary care to detect people at ultra-high risk for psychosis: Study protocol for a theory-based, cluster randomized controlled trial
Implementing the I-DECIDED clinical decision-making tool for peripheral intravenous catheter assessment and safe removal: protocol for an interrupted time-series study
Evidence-based intervention to reduce avoidable hospital admissions in care home residents (the Better Health in Residents in Care Homes (BHiRCH) study): Protocol for a pilot cluster randomized trial
Study protocol for the translating research in elder care (TREC): Building context - An organizational monitoring program in long-term care project (project one)
The prevention and reduction of weight loss in an acute tertiary care setting: Protocol for a pragmatic stepped wedge randomized cluster trial (the PRoWL project)
Study protocol: Addressing evidence and context to facilitate transfer and uptake of consultation recording use in oncology: A knowledge translation implementation study
Improving quality of care through routine, successful implementation of evidence-based practice at the bedside: An organizational case study protocol using the Pettigrew and Whipp model of strategic change
Exploring the interpersonal-, organization-, and system-level factors that influence the implementation and use of an innovation-synoptic reporting-in cancer care
Sustaining Transfers through Affordable Research Translation (START): Study protocol to assess knowledge translation interventions in continuing care settings
Process evaluation of a knowledge translation intervention using facilitation of local stakeholder groups to improve neonatal survival in the Quang Ninh province, Vietnam
Facilitating successful implementation of a person-centred intervention to support family carers within palliative care: a qualitative study of the Carer Support Needs Assessment Tool (CSNAT) intervention
The Influence of Context and Practitioner Attitudes on Implementation of Person-Centered Assessment and Support for Family Carers Within Palliative Care
Why is it so hard to implement change? A qualitative examination of barriers and facilitators to distribution of naloxone for overdose prevention in a safety net environment
Designing and implementing two facilitation interventions within the "Facilitating Implementation of Research Evidence (FIRE)' study: a qualitative analysis from an external facilitators' perspective
The inter-play between facilitation and context in the promoting action on research implementation in health services framework: A qualitative exploratory implementation study embedded in a cluster randomized controlled trial to reduce restraint in nursing homes
A realist process evaluation within the Facilitating Implementation of Research Evidence (FIRE) cluster randomised controlled international trial: An exemplar
Optimizing the mobility of residents with dementia: A pilot study promoting healthcare aide uptake of a simple mobility innovation in diverse nursing home settings
Cluster randomized adaptive implementation trial comparing a standard versus enhanced implementation intervention to improve uptake of an effective re-engagement program for patients with serious mental illness
Action-oriented study circles facilitate efforts in nursing homes to go from feeding to serving: Conceptual perspectives on knowledge translation and workplace learning
The role of organizational context and individual nurse characteristics in explaining variation in use of information technologies in evidence based practice
A model (CMBP) for collaboration between university college and nursing practice to promote research utilization in students' clinical placements: A pilot study
“If really we are committed things can change, starting from us”: Healthcare providers’ perceptions of postpartum care and its potential for improvement in low-income suburbs in Dar es Salaam, Tanzania
Facilitating Implementation of Research Evidence (FIRE): An international cluster randomized controlled trial to evaluate two models of facilitation informed by the Promoting Action on Research Implementation in Health Services (PARIHS) framework
Pilot study for evidence-based nursing management: Improving the levels of job satisfaction, organizational commitment, and intent to leave among nurses in Turkey
A collaborative project to improve identification and management of patients with chronic kidney disease in a primary care setting in Greater Manchester
Implementing point of care "e-referrals" in 137 clinics to increase access to a quit smoking internet system: The Quit-Primo and National Dental PBRN HI-QUIT Studies
Change in Patient Outcomes After Augmenting a Low-level Implementation Strategy in Community Practices That Are Slow to Adopt a Collaborative Chronic Care Model A Cluster Randomized Implementation Trial
Developing leadership in managers to facilitate the implementation of national guideline recommendations: A process evaluation of feasibility and usefulness
Evaluating implementation of methicillin-resistant Staphylococcus aureus (MRSA) prevention guidelines in spinal cord injury centers using the PARIHS framework: A mixed methods study
Knowledge translation in Uganda: A qualitative study of Ugandan midwives' and managers' perceived relevance of the sub-elements of the context cornerstone in the PARIHS framework
Organizational readiness to change assessment (ORCA): Development of an instrument based on the promoting action on research in health services (PARIHS) framework
Effects of computer reminders on complications of peripheral venous catheters and nurses' adherence to a guideline in paediatric care-a cluster randomised study
Importance of clinical educators to research use and suggestions for better efficiency and effectiveness: results of a cross-sectional survey of care aides in Canadian long-term care facilities
Barriers and Enablers to Implementation of a New Zealand-Wide Guideline for Assessment and Management of Cardiovascular Risk in Primary Health Care: A Template Analysis
Examination of the utility of the Promoting Action on Research Implementation in Health Services framework for implementation of evidence based practice in residential aged care settings
A qualitative study to identify barriers and facilitators to implementation of pilot interventions in the Veterans Health Administration (VHA) Northwest network
Role of "external facilitation" in implementation of research findings: A qualitative evaluation of facilitation experiences in the Veterans Health Administration
Associations between perceptions of evidence and adoption of H1N1 influenza infection prevention strategies among healthcare workers providing care to persons with spinal cord injury
The relationship between baseline Organizational Readiness to Change Assessment subscale scores and implementation of hepatitis prevention services in substance use disorders treatment clinics: A case study
A formative evaluation of organizational readiness to implement nurse-initiated HIV rapid testing in two veterans health administration substance use disorder clinics
Promoting evidence-based urinary incontinence management in acute nursing and rehabilitation care—A process evaluation of an implementation intervention in the orthopaedic context
Factor Structure, Reliability and Measurement Invariance of the Alberta Context Tool and the Conceptual Research Utilization Scale, for German Residential Long Term Care
A qualitative study of a primary-care based intervention to improve the management of patients with heart failure: The dynamic relationship between facilitation and context
Empowering nurses with evidence-based practice environments: Surveying magnet®, pathway to excellence®, and non-magnet facilities in one healthcare system
Applying the RE-AIM framework to evaluate two implementation strategies used to introduce a tool for lifestyle intervention in Swedish primary health care
Promoting continence in nursing homes in four European countries: The use of PACES as a mechanism for improving the uptake of evidence-based recommendations
Comprehensive assessment of chronic pain management in primary care: A first phase of a quality improvement initiative at a multisite community health center
A qualitative study on clinicians' perceptions about the implementation of a population risk stratification tool in primary care practice of the Basque Health Service
Health system context and implementation of evidence-based practices-development and validation of the Context Assessment for Community Health (COACH) tool for low- and middle-income settings
South Africa, Vietnam, Uganda, Nicaragua, Bangladesh
Engaging and developing front-line clinical nurses to drive care excellence: Evaluating the Chief Nurse Excellence in Care Junior Fellowship initiative
Outcomes of a clinical nurse specialist-initiated wound care education program: Using the promoting action on research implementation in health services framework
Training Substance Use Disorder Counselors in Cognitive Behavioral Therapy for Depression: Development and Initial Exploration of an Online Training Program
Implementation and evaluation of the VA DPP clinical demonstration: protocol for a multi-site non-randomized hybrid effectiveness-implementation type III trial
A checklist for identifying determinants of practice: A systematic review and synthesis of frameworks and taxonomies of factors that prevent or enable improvements in healthcare professional practice
Systematic Realist Review of Key Factors Affecting the Successful Implementation and Sustainability of the Liverpool Care Pathway for the Dying Patient
Context matters in implementation science: A scoping review of determinant frameworks that describe contextual determinants for implementation outcomes
Exploring How Knowledge Translation Can Improve Sustainability of Community-based Health Initiatives for People with Intellectual/Developmental Disabilities
The NIHR collaboration for leadership in applied health research and care (CLAHRC) for Greater Manchester: Combining empirical, theoretical and experiential evidence to design and evaluate a large-scale implementation strategy
Promoting physical therapists' of research evidence to inform clinical practice: Part 1 - Theoretical foundation, evidence, and description of the PEAK program
Solving the wicked problem of hospital malnutrition
N/A
Not reported
✓
Opinion/ theoretical paper
aProtocols planned to use PARIHS in the analysis
Of these 367 articles, 235 cited Kitson et al. [5], 208 cited Kitson et al. [18], 136 cited Rycroft-Malone et al. [8], and 92 cited Rycroft-Malone et al. [9]. In total, the 367 articles consisted of 35 protocols [25, 28, 29, 31‐38, 40, 42, 44‐50, 52‐54, 56, 57]. A further 255 articles reported empirical studies:
▪ 91 where PARIHS guided the development of the intervention [58‐82, 84‐143, 145, 146, 386‐388],
In addition, the database included 28 empirical review studies [3, 13, 313, 314, 316‐338, 389] and 49 opinion/theoretical articles [2, 144, 339‐385]. In terms of professional focus, about 65% of the included articles involved nursing.
In the following sections, references have been added to the categorical items in the data extraction while we have opted only to provide examples of references to the findings from the qualitative exploration of how the PARIHS framework was operationalized in detail.
The types of articles published using the PARIHS framework changed over time, with an increase in the number of empirical studies from 2004 onwards, as illustrated in Fig. 2. As the search for articles for this review only included the first eight months of 2019, the graph is limited to full years (i.e., 1998 through 2018).
Fig. 2
Types of articles published before December 31 using the PARIHS framework
×
Use of PARIHS
Figure 3 depicts how PARIHS was used by type of article. Although authors frequently claimed that PARIHS was used in one or more ways, details as to how the framework was used were often lacking.
Fig. 3
Use of the PARIHS framework by type of article
×
The application of PARIHS to plan and deliver an intervention
In total, 117 (32%) articles claimed to use the PARIHS framework to plan and deliver an intervention [23‐46, 58‐82, 84‐143, 145, 146, 339, 340, 386‐388]. Predominantly, these were empirical studies (n = 91) [58‐82, 84‐143, 145, 146, 386‐388] but also two opinion/theoretical articles [339, 340] and 24 protocols [23‐46]. Of the 117 articles, about half stated that the framework was used for theoretically informing, framing, or guiding an intervention (e.g., [82, 103, 105, 134, 386]). However, in these studies, PARIHS was referred to only in a general sense, in that the core elements of the framework were said to have informed the planning of the study. There was a lack of detail provided about what elements of the framework were used and how they were operationalized to plan and deliver an intervention. In the other half of the 117 articles, it was described more specifically that one or more elements of the framework had been used. Most commonly the facilitation element (e.g., [58, 80, 92, 98, 110]) was referred to as guiding an implementation strategy. The articles that provided explicit descriptions of interventions using facilitation employed strategies such as education, reminders, audit-and-feedback, action learning, and evidence-based quality improvement, and roles including internal and external facilitators and improvement teams to enable the uptake of evidence (e.g., [23, 79, 125, 142]). Some articles drew on the PARIHS framework more specifically, to understand the role of organizational context in implementation (e.g., [34, 63, 145, 340]).
The application of PARIHS in data analysis
There were 184 (50%) articles where the PARIHS framework was reported to be used in the analysis [2, 13, 23‐35, 47‐55, 58‐82, 84‐94, 149‐242, 313, 314, 316‐322, 339, 341‐358, 386, 389]. Most of these involved empirical studies (n = 131) [58‐82, 84‐94, 149‐242, 386] where PARIHS often was described as guiding or framing the data collection, e.g., developing an interview guide, and/or analysis, but with no further details. In articles that provided more detailed information, PARIHS was used to guide or frame qualitative analyses in about 50 studies (e.g., [67, 94, 173, 178, 207]). Of these, around 20 used a deductive approach in that they used the elements and sub-elements to structure the analytic process (e.g., [150, 170, 188, 215, 242]). About 35 studies applied PARIHS for quantitative analysis, (e.g., [69, 168, 174, 190, 211]). In half of these, the Alberta Context Tool (e.g., [155, 165, 180, 195, 229]) and the Organizational Readiness to Change Assessment Tool (e.g., [74, 159, 219, 240]) were used; both these tools being derived from PARIHS. Empirical studies using the PARIHS framework in the analysis encompassed primarily all three main elements of PARIHS (e.g., [166, 181, 193, 221]) and the context domain (e.g., [78, 152, 153, 207]), and in lesser extent the evidence (e.g., [185, 208, 214]) and the facilitation domain (e.g., [58, 67, 79, 182]).
Eleven review studies [13, 313, 314, 316‐322, 389] used the framework for the analysis; findings were mapped to PARIHS elements in a few studies [316, 317, 389]; one described that their data had been “analysed through the lens of PARIHS” (p1) [389]. A couple of the review studies had PARIHS as the object for analysis, comparing it with other frameworks [318, 322]. This approach was also common in the 20 opinion/theoretical articles [2, 339, 341‐348, 350‐358], where the PARIHS framework itself was the focus of the analysis (e.g., [341, 349, 354]). In these articles, the analysis was performed in different ways, primarily through mapping and comparing PARIHS to other frameworks or models or even policies, but also for general discussions on implementation and evidence-based practice. Among the 185 articles that reported using the PARIHS framework in the analysis, there were also 22 protocols where authors reported that the intention was to use the framework in the analysis [23‐35, 47‐55].
The application of PARIHS in the evaluation of study findings
A total of 203 (55%) included articles provided information on how the PARIHS framework was used in the evaluation of study findings, in terms of contributing to the discussion and interpretation of results [13, 52, 58‐82, 84‐89, 95‐121, 149‐214, 243‐259, 261‐284, 313, 314, 316‐320, 323‐331, 339, 341‐350, 359‐365, 386, 389]. The majority (n = 167) of these were empirical studies [52, 58‐82, 84‐89, 95‐121, 149‐214, 243‐259, 261‐284, 386].
We found two main approaches to how the PARIHS framework was used in the evaluation of study findings. First, PARIHS was used to organize the discussion of the findings (e.g., [73, 87, 109, 166, 214]), where the framework and/or its elements were used to provide a structure for reporting or generally discussing the findings, or both, for example in stating that the key elements of PARIHS were reflected in the study findings. Second, the framework was used to consider the implications of the study’s findings (e.g., [81, 84, 170, 207, 361]), where the framework or its elements (varying between one (e.g., [75, 195, 211]), two (e.g., [71, 86, 105]), and all the three main elements (e.g., [80, 261, 269])) enabled authors to elaborate on findings, or reflect on the implications of their study to evaluate the PARIHS framework itself. Specifically, we found some empirical articles reported evaluating the PARIHS element “context” by means of context tools (e.g., [155]). In addition, an evaluation of the study findings using the framework was identified in 18 opinion/theoretical articles [339, 341‐350, 359‐365] and 18 empirical review studies [13, 313, 314, 316‐320, 323‐331, 389]. Among the opinion/theoretical articles, there were papers evaluating other theoretical constructions in relation to the PARIHS framework (e.g., [364]).
The application of PARIHS in any other way
A total of 136 (37%) reported using PARIHS in some other way than directly informing the planning and delivery of an intervention or analyzing and evaluating findings [3, 23‐25, 36, 37, 47‐50, 56‐62, 90, 95‐97, 122‐127, 144, 149‐170, 215‐224, 243‐256, 285‐313, 323, 324, 332‐338, 351, 359‐361, 366‐385]. A majority of these articles (n = 89) were empirical studies [58‐62, 90, 95‐97, 122‐127, 149‐170, 215‐224, 243‐256, 285‐312], and about half of these described the use of PARIHS as an overall guide to frame the study (e.g., [58, 60, 168, 222, 285, 303]). A similar finding was apparent in the 11 protocols [23‐25, 36, 37, 47‐50, 56, 57]; about half of these also referred to the use of PARIHS to guide and frame the study design (e.g., [47, 48, 50, 57]).
An alternative use of PARIHS in empirical studies involved focusing on one of the three PARIHS elements (n = 17) and investigating them in greater depth, most notably context (n = 10) (e.g., [155, 232]) and facilitation (n = 7) (e.g., [307, 312]). A total of 25 opinion/theoretical articles [144, 351, 359‐361, 366‐385] reported using the PARIHS framework in some other way, including a discussion about PARIHS as part of presenting a general overview of theories and frameworks to inform implementation (e.g., [369, 376, 378, 384]), using PARIHS to augment, develop, or evaluate other implementation models and frameworks (e.g., [318, 359, 367, 374, 382]), and informing education and learning and teaching initiatives [144, 372]. Empirical review articles (n = 11) included reviews of implementation frameworks [3, 313, 323, 324, 332‐338], including PARIHS, a review of the facilitation dimension of PARIHS and a discussion of the potential to combine implementation and improvement methodologies.
Testing and providing views on the validity of the framework
▪ Identifying gaps in the framework (e.g., [64, 170, 326])
Over the review study period (1998 to 2019), among empirical studies, there was a shift from primarily studying the context element of the framework to more articles evaluating the whole framework. This was also evident in the pattern found in the protocols, which mostly focused on testing facilitation (e.g., [58, 182, 206]). Opinion/theoretical articles tended to critique the whole framework (e.g., [319, 326, 342, 349, 369]). Of the 15 empirical reviews, the majority focused on the whole framework (e.g., [13, 322, 333]), then on context (e.g., [316, 318, 335]) and then on facilitation (e.g., [323]). Of note is the lack of attention in the literature to the element of “evidence” in the PARIHS framework (examples of articles paying attention to evidence include [208, 255]).
The articles varied in detail, depth, and quality in terms of descriptions of how they went about testing the validity of the PARIHS framework. Approaches ranged from general observations of whether the research teams/users found the elements and sub-elements easy to use (e.g., [62, 188, 203]), to studies that used elements of context described in the PARIHS framework to validate new context measures across settings and groups (e.g., [150, 155, 207]). As one example, the Alberta Context Tool started from the PARIHS conceptualization of context to include dimensions of culture, leadership, and evaluation.
Regarding the strength and limitations of the PARIHS framework, about one third of the included articles reported on its strengths and about 10% commented on perceived limitations. The identified strengths included:
▪ Holistic implementation framework (e.g., [141, 164, 209, 258]) that is perceived as intuitive and accessible.
▪ Both practical and theoretical and therefore feasible to use by both clinicians and researchers; also seen as intuitive to use and accessible (e.g., [117, 209, 255]).
▪ Can be used as a tool: diagnostic/process/evaluative tool; predictive/explanatory tool or as a way to explain the interplay of factors (e.g., [93, 205, 255, 285, 379]).
▪ Can accommodate a range of other theoretical perspectives (including approaches such as social network theory, participatory action research, coaching, change management and other knowledge translation frameworks) (e.g., [93, 105, 245, 246]).
▪ Can be used successfully in a range of different contexts (low- and middle-income countries) [150] and services and for various groups of patients (disability, aged care) (e.g., [80, 113, 248, 312]).
Limitations of the PARIHS framework included:
▪ Poor operationalization of key terms leading to difficulties in understanding and an overlap of elements and sub-elements (e.g., [165, 285, 376]).
▪ Lack of practical guidance on steps to operationalize the framework (e.g., [209, 254]) with a subsequent lack of tools.
▪ Lack of information on the individual and their characteristics (e.g., [209, 361]) and their lack of understanding of evidence (e.g., [204, 390]).
▪ Too structured and does not acknowledge the multidimensionality and uncertainty of implementation (e.g., [143, 214]).
▪ Lack of acknowledgement of wider contextual issues such as the impact of professional, socio-political, and policy issues on implementation (e.g., [115, 143, 285, 354]).
▪ Not providing support in how to overcome barriers to successful implementation (e.g., [88]).
Discussion
In a recent survey among implementation scientists, the PARIHS framework was found to be one of the sixth most commonly used theoretical frameworks [4]. Yet, in our review, about 23% (n = 367) of the identified 1614 articles citing any of the four selected core PARIHS articles used the framework in any substantial way. Similarly, a review of the CFIR found that 26/429 (6%) of articles citing the framework were judged to use the framework in a meaningful way (i.e., used the CFIR to guide data collection, measurement, coding, analysis, and/or reporting) [12]. A citation analysis of the KTA framework found that about 14% (146/1057) of screened abstracts described using the KTA to varying degrees, although only 10 articles were judged to have applied the framework in a fully integrated way to inform the design, delivery, and evaluation of implementation activities [11].
PARIHS has been used in a diverse range of settings but, similarly to other commonly used implementation frameworks, most often superficially or partially. The whole framework has seldom been used holistically to guide all aspects of implementation studies. Implementation science scholars have repeatedly argued that the underuse, superficial use, and misuse of implementation frameworks might reduce the potential scientific advancements in the field, but also the capacity for changing healthcare practice and outcomes [4]. The rationale for not using the whole PARIHS framework could be many, including the justified reason of only being interested in a particular element. As such, partial use cannot always be considered as inappropriate. Simultaneously, many researchers entering the field might be overwhelmed with the many frameworks available and the lack of guidance about how to select and operationalize them and using their elements [2, 4, 391]. The current citation analysis can thus help remedy a gap in the literature by revealing how the PARIHS framework has been used to date, in full or partially, and thus provides input to users of its potential use.
The use of theoretical frameworks in implementation science serves the purpose of guiding researchers’ and practitioners’ implementation plans and informing their approaches to implementation and evaluation. This includes decisions about what data to gather to describe and explain implementation, their hypotheses about action steps needed, how to account for the critical role of context, and providing a foundation for analysis and discussion [7]. The advancement of theoretically informed implementation science will, however, depend on much improved descriptions as to why and how a certain framework was used, and an enhanced and better-informed critical reflection of the functionality of that framework. This review shows that the PARIHS framework has rarely been used as a whole; rather, certain elements tend to be applied, often retrospectively as indicated in Fig. 3 underlining the use of PARIHS in the evaluation of study findings, which resonates with the findings of reviews about the use of the KTA [2] and CFIR [11] frameworks. This could be as a result of a lack of theoretical coherence of some frameworks making them difficult to apply holistically, and/or a function of a general challenge that researchers face in operationalizing theory. However, this could also be a result of publishing constraints. While the PARIHS framework may have guided implementation or been implicitly used in the study design, it was rarely the focus of the publications. Further, the aims and scopes of scientific health care journals have historically prioritized clinical outcomes over implementation outcomes where one could expect a more detailed description of the use of theoretical frameworks. This may have resulted in authors not fully reporting their use of, e.g., the PARIHS framework.
The number of empirical studies using the PARIHS framework has steadily increased over the review period. There is also evidence to show that more research teams have contributed to critiquing the framework in terms of reporting on its strengths and limitations and its validity. The pattern of investigation is moving from studies on context, to more systematic explorations of facilitation, thus contributing to a more detailed understanding of the elements and sub-elements of the framework. The lack of focus on “evidence” identified in this review highlights the need for researchers and clinicians to focus on the multi-dimensionality of what is being implemented. Common patterns emerging in this review support the changes made to the most recent refinement of the PARIHS framework [359].
Consistent with other reviews of the use of theoretical frameworks in implementation science, we found that PARIHS was often not used as intended. Further, it was not always clear why the particular framework was chosen. Frequently, authors merely cite a framework without providing any further information about how the framework was used. The lack of clear guidance on how to operationalize frameworks might be one of the underlying reasons for this. Lastly, to enable a critical review of frameworks and further build collective understanding of implementation, we urge authors to be more explicit about how theory informs studies. Development and adoption of reporting guidelines on how framework(s) are used in implementation studies might assist in sharpening the link between the used framework(s) and the individual study, but could potentially also enhance the opportunities for advancing the scientific understanding of implementation.
Limitations
To increase study reliability during the review process, more than one person identified, assessed, and interpreted the data. We had regular meetings to discuss potential difficulties in assessing included articles, and subsequently, all decisions were resolved by consensus to enhance rigor. We used a rigorous search strategy, which was undertaken by an information specialist. The standardization of our processes across the team was also enhanced by the creation of an online data extraction form via Google. However, as the form was not linked to other software (e.g., Endnote), this added time-consuming processes.
As we did not include articles that were not written in English, we may have limited the insights about the application of the PARIHS framework, particularly with relevance to different country contexts. Additionally, we did not search the grey literature for practical reasons concerning the size of the literature, which may also have provided some additional insights not reflected in this publication. We also limited our search to two databases, which may mean we missed some relevant articles. However, we are confident that we found the majority of relevant published evidence to address the review questions because of a rigorous approach to retrieval. Thus, we think the findings of our citation analysis on the use of PARIHS are generalizable for studies in English published in peer-reviewed journals.
Conclusions
The importance of theoretically underpinned implementation science has been consistently highlighted. Theory is important for maximizing the chances of study transferability, providing an explanation of implementation processes, developing and tailoring implementation interventions, evaluating implementation, and explaining outcomes. This review of the use of the PARIHS framework, one of the most cited implementation frameworks, shows that its actual use and application has been frequently partial and generally not well described. Our ability to advance the science of implementation and ultimately affect outcomes will, in part, be dependent on better use of theory. Therefore, it is incumbent on theory developers to generate accessible and applicable theories, frameworks, and models, and for theory users to operationalize these in a considered and transparent way. We propose that the development and adoption of reporting guidelines on how framework(s) are used in implementation studies might enhance the maturity of implementation science.
The authors wish to acknowledge Sayna Bahraini, Heledd Owen Griffiths, and Veronica Costea for partaking in the initial assessment of retrieved articles.
Ethics approval and consent to participate
Not applicable.
Consent for publication
Not applicable.
Competing interests
We acknowledge that GH, AK, and JRM were all involved in the development of the PARIHS framework. Further, IDG, JRM, and LW are all members of the BMC Implementation Science Editorial Board.
Open AccessThis article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
