Organizational participatory research: a systematic mixed studies review exposing its extra benefits and the key factors associated with them

Participatory research is an umbrella term for a host of collaborative approaches to research where academics conduct research with non-academic partners [1] and has been defined as a “systematic inquiry, with the collaboration of those affected by the issue being studied, for purposes of education and taking action or effecting social change” [2]. This research approach produces theoretical or evaluative knowledge, similar to conventional quantitative and qualitative methodologies, and also blends research with action, thereby producing knowledge that can inform healthcare practices, services, and organizations [3]. The origins of this family of research approaches have been referred to the Northern and Southern traditions [4]. The Northern tradition is rooted in the work of Kurt Lewin on action research in the 1940s. In his seminal paper, Lewin called for collaborations between academic researchers and practitioners to effect sustainable change, writing that action research “needs the best of what the best among us can give, and the help of everybody” [5]. Action research engages academic researchers and non-academic partners in a democratic and iterative research cycle of planning, action, and fact finding about the effect of the action [5, 6]. A central tenet of action research is that working collaboratively will lead to changes in the academics and non-academics alike; thus, the approach integrates research, action, and education [5, 6]. In the 1980s, Chris Argyris and Donald A. Schön built on this tradition with their work on organizational learning and action science [7] and reflective practice, or “kind of action research that enhances common sense, a form of inquiry that builds on and feeds back to modify what we already know-in-practice.” [8] The Southern tradition, for its part, stems from the work of Paolo Freire, in Brazil [9]. This tradition emphasizes empowerment and emancipation of marginalized or oppressed populations. Much of community-based participatory research follows from this tradition [1, 10].

The research literature demonstrates the dynamic evolution of participatory research with numerous constructive debates, methodological papers, reviews, and books regarding similarities and nuances between action research, community-based participatory research, and several related approaches such as collaborative action research, cooperative action research, participatory action research, transformative research, emancipatory action research, and so forth [11‐13]. For instance, participatory research does not necessarily follow the planning, action, and fact finding cycle outlined by Lewin [5]. Also, what exactly does it mean to ‘engage’ or ‘collaborate’ with non-academic partners in communities vs. in organizations? Does this collaboration occur during the action phase (e.g., design or implementation of an intervention) or during the research phase (e.g., definition of the research question or data collection and analysis)? In their large systematic review of what the authors refer to as action research, Waterman et al. [3] documented non-academic partners’ participation in four stages of the research (information gathering, planning, implementation, and evaluation) according to six categories of participation (co-option, compliance, consultation, cooperation, co-learning, and collective action). Notably, they documented that these stakeholders’ category of participation often varies throughout the phases of a given study, a finding which corroborates previous work [14]. Moreover, the many labels for the various approaches of the family of collaborative research are not always used to mean the same thing [3, 11, 15]. In line with other typologies of stakeholder participation in research [14, 16, 17] (Table 1), and a previous review by co-authors on community-based participatory research [18], we submit that participatory research refers to non-academic partners participating with university researchers in at least three key research decisions: (a) identifying the research question(s); (b) setting the methodology, collecting and/or analyzing the data, or interpreting the findings/results; and (c) implementing or disseminating the research findings. All types of qualitative, quantitative, and mixed methods may be used with this research approach [19].

In health, organizational participatory research (OPR) is carried out with organizations, where an organization is a “context of action in which relationships of cooperation, exchange, and conflict between actors with divergent interests are being established and managed” [20] with rules and hierarchies, and which fluctuates in response to changes in the environment. A health organization is any organization offering health-related services such as hospital wards, primary care clinics, or long-term care facilities. In health organizations, OPR can be used to develop research capacity and reflective practice (e.g., practitioners not only collect facts regarding their practice, but reflect on their practice to uncover and understand tacit knowledge), and it produces organizational learning (i.e., practitioners not only reflect on and amend organizational activities, but also the governing strategies behind them) [7]. It is used to implement organizational changes and practice improvement by, for example, addressing challenges or solving clinical and non-clinical problems, or to develop and implement innovations or interventions [3, 21‐23]. Additionally, OPR empowers health professionals and can contribute to their professional development (e.g., increased skills and knowledge in areas such as management, clinical practice, education, and research), and results in improvements in healthcare (e.g., improved patients knowledge about medication, improved infection control) and patient satisfaction [3, 15, 24, 25]. Indeed, this research approach is recommended because it “appears to have the potential to assist practitioners, managers and policy makers in their efforts to provide high-quality healthcare” [3].

To date, four literature reviews have focused on OPR in health, including studies with varying degrees of organization participation throughout the research (from cooperation to full engagement in the research and/or ensuing actions) [3, 15, 24, 25]. These reviews are each narrow in focus examining only nursing [24], adult intensive care unit settings [15], UK settings [3], or only practice-based nursing [25]. They conclude that OPR empowers health professionals and contributes to their professional development. However, the association of OPR outcomes with processes of participation remains unclear because these reviews included OPR studies without distinguishing between (or among) types of participation. Furthermore, the challenges associated with this research approach such as increased workload of organization members, time constraints and conflicting commitments, and disruption, such as resistance to change or interpersonal tensions [3, 25], further underscore the need to understand if the benefits of this approach outweigh the costs. Participatory research approaches are also a more time-consuming research approach, compared to non-participatory research [26], and may require extra effort to work through challenges, among them practitioners’ research values and skills, and communication difficulties [25]. Thus, understanding outcomes in terms of the added value of OPR, if any, is important.

Systematic mixed studies reviews are an emerging form of literature review that use mainly textual data to combine qualitative and quantitative evidence extracted from qualitative, quantitative and mixed methods studies [27, 28], and can provide a highly practical understanding of complex health interventions and programs [23, 29‐31]. In this review, we synthesized qualitative and quantitative evidence to test the hypothesis that the likelihood a co-construction type OPR study yields extra benefits is greater than for studies using consultation type OPR, and that five other variables are influencing factors (Table 2). Our synthesis design combined quantitative content analysis method and a multivariate logistic regression.

To help ensure our work is relevant to health organizations, we used a participatory approach, partnering with managers from a variety of health organizations (Table 3) to define our research objectives, refine our data collection and analysis, and interpret and disseminate findings. Below we present our methods in line with the PRISMA statement [32]. As per PROSPERO inclusion criteria, this review is not registered.

The search of the peer-reviewed and gray literature led to the retrieval of 8873 unique records. Based on our aforementioned eligibility criteria, we identified 992 potentially relevant articles, 140 of which we selected for further appraisal. At this stage, we excluded an additional 33 studies due to a lack of clear description of the link between participatory processes and outcomes. Owing to the fact that several studies are described across multiple publications, our final sample of 107 studies consists of 177 publications. The flow diagram is presented in Fig. 1. The raw kappa values for the extra benefits, participation, and research initiator variables were 0.506, 0.590, and 0.534, respectively. These coefficients all indicate moderate agreement [42].

Most (n = 83; 77.6%) of the included studies represent co-construction type participation in research decisions. Nurses were the organization members most commonly involved as research partners (n = 77 studies; 30.7%). Other partners included physicians (n = 31; 12.4%), support staff (n = 13; 5.2%), and managers (n = 35; 13.9%). Many studies (n = 22; 20.6%) included more than one type of organizational member. The means and standard deviations for number of organization groups in the studies with no extra benefit or one or more extra benefits were 2.4(1.8) and 2.2(1.6) respectively (p = 0.66). Patients and their family or caregivers were research partners in only 15 (13.6%) and eight (7.3%) studies, respectively (Additional file 3: Table S1). We were able to determine the project duration for 89 studies. Among these, equivalent proportions of co-construction and consultation studies lasted 1 year or longer (65.3%; 47/72 and 64.7%; 11/17 respectively). No significant association was found between project duration and presence of extra benefits (Table 8).

Regarding the settings, nearly two thirds of studies took place in hospitals or hospital wards (n = 66; 61.7%). Other settings include nursing homes, primary care clinics, specialized treatment facilities, pharmacies, and community health centers. For the most part, this type of research has been conducted in the UK (n = 38; 35.5%), Australia (n = 22; 20.6%), and the USA (n = 14; 13.1%). The majority of the included studies used only qualitative methods (n = 89; 83.2). Quantitative and mixed methods were used in four (3.7%) and 14 (13.1%) studies, respectively. Additional file 3: Table S1 presents the full description of all studies in the review sample.

Two thirds (n = 70; 65.4%) of included studies reported at least one extra benefit of OPR. The only variable in the model that was significantly associated with extra benefits was the study initiation variable (Table 8). Contrary to our hypothesis, the logistic regression revealed no difference between the type of organization members’ participation (co-construction vs. consultation) and the likelihood of exhibiting an extra benefit. However, analyses revealed that the likelihood of a co-construction or a consultation study exhibiting at least one extra benefit is quadrupled when the impetus for the study comes from the organization, as opposed to the university researcher, or the organization and the university researcher together (OR = 4.11, CI = 1.12–14.01; Table 9).

The sub-analysis defined five broad types of extra benefits. First, organization members exhibited leadership development or improvement (specifically, autonomy and confidence) exemplified in such things as empowerment/emancipation; becoming aware of skills, knowledge, and/or power; feeling ownership; advocacy; and confidence in role, skills, and/or power. Second, general workforce development was observed in terms of (a) reflective practice (e.g., improved understanding of the rationale behind certain tasks; more creative or critical evaluation of practice; increased awareness of patient needs), (b) development and/or use of new skills or tools (e.g., professional, problem solving, and/or research skills), (c) personal development (e.g., enroll in graduate school, job promotion), (d) positive changes in relations with service users (e.g., more patient centered care, learning from service users); and (e) new understandings about workplace, conditions that shape practice, and contributions of colleagues. Third, group benefits were observed in the form of improved collaborations, relations and communication among organization members, as well as staff learning from one another. Fourth, broad systemic developments or changes were observed. For example, the intended practice changes often extended beyond the target setting (e.g., transferred from the initial hospital ward to the whole hospital) or beyond the intended time frame (e.g., researchers and organization members continue to work together on subsequent OPR projects). Finally, in some studies, extra benefits were noted for the university and/or service user partners. Table 10 presents the distribution of extra benefits observed, together with specific examples from included studies. Figure 2 illustrates the distribution according to mode of participation. It is noteworthy that the majority of studies with two to four types of extra benefits are co-construction (32/36; 88.9%), whereas the majority of studies with one type of extra benefit represent consultation type participation (23/34; 67.6%). Owing to small sample sizes, no further analysis was possible.

Our results indicate that when the impetus of the OPR (consultation or co-construction) comes from the organization, the likelihood of the study resulting in at least one extra benefit is quadrupled. Although the confidence interval was fairly large, possibly due to the moderate agreement between coders, this may suggest that, where extra benefits are concerned, it is not the degree of organization members’ participation in research decisions (consultation or co-construction) that makes the difference, but rather their participation up-front. It may therefore be important for academic researchers to be open to modifying their OPR objectives to align with those of their organization partners. Moreover, it may be important for researchers to establish relationships with organizations pertinent to their research program (that is, organizations with a common interest) such that organizations’ questions may emerge from an existing relationship. Indeed, in their study of partnered-research grant applicants submitted to The Canadian Institutes of Health Research (CIHR) Sibbald, Tetroe [43] found that 35% (n = 17) of respondents indicated their partnerships grew from a common interest, and over 75% had established relationships.

Beckett et al. [44] found that clinicians contemplating participation in clinical research studies follow decision making processes similar to those outlined by Prochaska and DiClemente [45] for people contemplating changes in health behaviors: pre-awareness, awareness, information-gathering, first protocol (“action” stage in [45]), and maintenance. Seen through this lens, organization members who express research needs may be at the action stage. Should they, then, complete an OPR study and benefit from it, they may progress to the “maintenance” stage, exhibited by, for example, extra benefits such as extending the reach of their study into other sectors of their organization or conducting further research (or broad systemic change as illustrated in Table 10).

With their 2001 systematic review of action research in the UK, Waterman and her colleagues concluded: “the level of participation of those being investigated (the co-researchers) can vary. The minimum level of participation needed to guarantee success is not yet known.” To build on this, we tested the hypothesis that extra benefits (a potential indicator of success) are positively and significantly associated with co-construction type participation compared to consultation, but found no association. However, it is interesting to note that the confidence interval is approaching significance even with moderate agreement between coders; perhaps with additional data, the results would be different. Additional data is contingent upon the descriptions of processes authors provide in publications. As with previous reviews [3, 24], we found publications lacking in this regard. We encourage OPR authors to clearly describe the mode and timing of non-academic partners’ participation such that future reviews of OPR may re-examine the relationship between co-construction type participation and extra benefits. Moreover, given the importance of the OPR being initiated by the organization, examining whether initiation differs between partnerships where organizations participate in research-related decisions via consultation or co-construction.

Previous reviews on the topic included studies with a variety of collaborative approaches in specific contexts, including between 21 and 62 publications [3, 15, 24, 25]. While our review provides an update, it is also more comprehensive including 107 studies from any geographic location and health setting. Our review is also more precise regarding participation; some studies included in extant reviews did not meet our participation inclusion criterion. Our focus on, and distinguish between, two precise modes of participation in health research is unique and helps to define the field. The two forms of participation we explored in this study differ with respect to research governance (with or without co-governance), but both require that organization members are involved throughout the research process. It may be this continued involvement of non-academic partners that is important, rather than the decision-making process per se. It is also possible that the consultation mode of participation is sufficient to achieve extra benefits. This may be of importance to managers who make resource allocation decisions.

Defining and describing the incidence and type of extra benefits of OPR is a significant contribution of our work. Previous reviews regarding collaborative work involving academic researchers and health professionals have documented outcomes similar to those we report herein, such as practitioners’ increased confidence [3], knowledge, awareness, empowerment [3, 15], and skills [3, 46], effective communication and collaboration among staff [15]; and continued effects at the same, or another, location [3]. Yet, these reviews do not clarify whether these outcomes were intended or not. Moreover, they do not document an association between the collaborative research process and the outcomes. We have documented five types of extra benefits (unintended), that included studies' authors link with OPR processes.

Our results underscore that an OPR approach contributes to increased capacity, a commonly cited effect of participatory research [1, 18] We have identified and documented specific outcomes according to a capacity building inspired framework [38]and illustrate the type of capacity that can be built. Most studies included in this review report at least one extra benefit of the process at the individual, group, or organization level. Moreover, our framework of extra benefits, inspired by the New South Wales capacity building framework [38], is not unlike the benefits for service users that Spector [46] identified in her review of 13 participatory research projects with health and social service professionals. She notes that through collaborative work with university partners, professionals “gain research knowledge and skills, professional development, social support, and professional relationships that can help build capacity in their agencies through training, program evaluation, and securing funding” [46].

As with previous reviews [3, 24, 25], our work indicates that published OPR in health continues to engage non-academic partners according to a co-construction mode of participation more often than a consultation mode of participation. Also common to previous reviews is the finding that most research using this approach is carried out with nurses in hospital settings in the UK [3, 15, 24, 25]. Yet, our review suggests the feasibility of undertaking partnered research in other health organizations such as pharmacies, long-term care centers, and primary care clinics, and with a variety of health professionals and other organization members. Thus, to improve the use of research findings across all health organizations and among a host of health care practitioners and other organization members, promotion of, and advocacy for, OPR may be needed to increase its uptake. Finally, while one review found that multidisciplinary groups helped improve the translation of knowledge to practice [15], we found no association between the number of types of professionals and staff and extra benefits.

According to this review and others [3, 15, 24, 25], OPR rarely involves patients and/or their family or caregivers. This is striking given that in the UK, INVOLVE was established in 1996 to support active public involvement in public health and social care research (http://​www.​invo.​org.​uk/​) and that, more recently, the Patient-Centered Outcomes Research Institute (http://​www.​pcori.​org/​) and the Strategy for Patient-Oriented Research (http://​www.​cihr-irsc.​gc.​ca/​e/​41204.​html) were implemented in the USA and in Canada, respectively. Research exploring the benefits and drawbacks, and the facilitators and barriers to partnering with patients and the public in OPR could provide a valuable contribution to the OPR literature and provide guidance to enhance future OPR.

One of the strengths of this work is our synthesis design which illustrates a novel way to complete a complex synthesis of a large number of quantitative, qualitative, and mixed methods studies. Unlike meta-regression that uses extracted factual data from primary studies, three of our variables were developed using quantitative content analysis. Although the centennial quantitative content analysis method faces validity and reliability issues (no independent objective measurement), it can overcome a limitation of meta-regression, namely missing data [47]. Combining quantitative content analysis and multivariate regression modeling is a novel aspect of our review and particularly relevant given the large number of primary studies included and the limited number of variables is examined.

Our results are based on what authors reported in their research publications, and our data of interest were not the research findings, but rather authors descriptions and reflections on the OPR processes they used and OPR outcomes observed. There are, thus, some limitations inherent in our study design. First, our study design assumes no extra benefit was experienced if none was reported; however, this is not necessarily true given extra benefits, as we have defined them, were not the focus of the included studies. Moreover, given the cyclical and iterative nature of OPR, intermediate, or more proximal, extra benefits may occur (e.g., improved staff relations) that may contribute to achieving the study objective. It can be difficult for stakeholders to separate interdependent benefits from each and to attribute the various changes and learnings that occur, or actions that are taken during the research to the OPR processes themselves. We sought to overcome the limitations of what is reported in studies by using a rigorous quantitative content analysis approach to define extra benefits and identify them in the included studies. Furthermore, while it is possible that some eligible studies were not picked up with our search strategy, or lost during the selection processes, the large sample size lends strength to our results.

Second, although the dichotomy of participation we used does not fully capture the nature of stakeholders’ participation throughout an OPR, it did allow us to compare two distinct levels of organization member participation. Such a comparison is otherwise impossible given it is not feasible to, experimentally, compare OPR with non-participatory forms of organizational research. Additionally, a more nuanced analysis of participation was not feasible given that authors do not often describe the fluctuating level of participation of various partners throughout the research process [3, 24]. Again, we used content analysis, excluding studies that did not clearly describe organization partners’ participation, to attempt to overcome this reporting limitation and generate a reliable participation variable. Despite the moderate inter-rater agreement for the participation, extra benefits and study initiation variables, the fact that the likelihood a study yields at least extra benefits is quadrupled when the OPR is initiated by the organization indicates this relationship warrants further exploration. We suggest future OPR publications clarify non-academic partners’ participation in the research decisions, that is, for each main research phase, describe whether organization members work actively and co-govern research with academic partners, or are consulted by academics (providing research input without research co-governance). We also suggest that publications clearly distinguish between decision-making in research-related processes vs. intervention-related processes. Clear descriptions of OPR processes and outcomes are required to improve knowledge regarding the added value of organization-university research partnerships.

While less resource-intensive means may be used to respond to organizational issues, this review focusses on the potential extra benefits of the OPR process, providing some insight into whether the investments in time and energy are worth it. We would like to acknowledge that merely quantifying extra benefits does not do justice to the richness of OPR. Indeed, stakeholders may judge the relevance of undertaking OPR by the quality and meaning of the benefits experienced. In the second phase of this work, we will use qualitative synthesis to generate a deeper understanding of the processes of OPR and the outcomes to which they contribute.

With this unique review, we have provided a framework for reporting and assessing OPR processes and outcomes, and have suggested potential key factors associated with extra benefits of OPR that can be tested in future research. We hope this contribution may influence organizations and academics alike to embark on satisfying OPR partnerships and to report their work in such a way that analyses of OPR processes and outcomes may be completed in the future to enhance our understanding of this research approach.