2.1 Study Design: The Realist Review Method
We synthesized knowledge on patient, family, and caregiver engagement in healthcare decision making using a realist synthesis approach. As there is currently no consensus on the best approach to conduct a knowledge synthesis, Kastner et al. [
5] proposed a scoping review to identify the most appropriate knowledge synthesis method. Of the 25 approaches listed, the realist synthesis approach developed by Pawson et al. [
6] and Greenhalgh et al. [
7] is most appropriate for this study. Realist syntheses address limitations of more traditional approaches to systematic reviews and meta-analyses. Such traditional approaches address effectiveness, often narrowly defined, but do not consider why, for whom, and in what circumstances an intervention or policy works [
8]. This method provided rich information and explanation that can guide real-world decision making.
The basic phases of a realist review are similar to those followed in a conventional Cochrane review but involve more sub-steps and may be overlapping and iterative rather than sequential [
6]. Realist synthesis involves identifying a theoretically based framework (“initial rough theories” [
9]), which is then populated with evidence that is used to enrich and refine the theory. The refined theory then becomes the basis for practice and policy recommendations. The search methodology was informed by the framework for realist syntheses put forward by Wong et al. [
10].
The processes of scope clarification, stakeholder involvement, systematic search and review, and development/dissemination of recommendations are consistent with accepted practice for creation of best practice guidelines [
11]. The synthesis encompassed peer-reviewed and grey literature; conceptual/theoretical as well as empirical work; research conducted using qualitative, quantitative, and mixed methods; and expert opinion, including the opinions of seniors, older patients, and their social support networks. The realist synthesis comprised the five phases described below and illustrated in Table
1. A more detailed description of the methods can be found in the protocol paper by Stolee et al. [
12].
Table 1
Phases of the realist review
1. Clarify scope | Refine question and purpose; search for major theories and frameworks Interview key informants to help identify theories and frameworks Consult with patients to discuss project and review theories and frameworks |
2. Search for evidence | Develop search strategy with library scientist |
3. Appraise primary studies and extract data | Develop data abstraction table |
4. Synthesize evidence and draw conclusions | Review articles, searching for context, mechanisms, outcomes, and patterns related to meaningful engagement Compare information against “initial rough theory” (candidate framework) |
5. Disseminate, implement, evaluate | Develop program theory; workshop participants review |
2.1.1 Phase One: Clarifying Scope
We first conducted in-depth discussions with stakeholders to refine the review question and purpose, and to find and articulate relevant theories. We initially focused on Canadian information at provincial and national levels, but identified frameworks used internationally through hand searching papers from countries with similar healthcare systems.
We conducted two key informant interviews with research leaders recognized as experts in patient, family, and community engagement in Canada. The 30-min interviews were audio-recorded and later transcribed. The interviews helped to identify frameworks commonly used in research and practice. Data were coded using the line-by-line coding technique by Lofland et al. [
13]. A group discussion was held with older adults, patients, and their caregivers (
n = 8) from the Seniors Helping as Research Partners (SHARP) network: a network created by the Geriatric Health Systems (GHS) research group (University of Waterloo) [
14] that engages older adults and caregivers in meaningful partnerships through discussions about healthcare issues and research. We also conducted a full-day workshop with participants (
n = 17) from Patients Canada who discussed the meaning of patient engagement and reviewed the frameworks we had identified.
The consultation components of this review involving patients and caregivers received ethics clearance from the University of Waterloo Office of Research Ethics (ORE# 19094).
2.1.2 Phase Two: Search for Evidence
We conducted an extensive purposive search of peer-reviewed literature.
Inclusion and Exclusion Criteria Papers were included if they reported description, assessment, or evaluation of strategies for engagement of adults (patients or citizens aged ≥18 years), families, or caregivers. All papers that discussed outcomes that could distinguish an actual experience or process of engagement were included. Papers containing strategies relevant to older adults (aged ≥65 years) were highlighted in the abstraction. Papers focused strictly on engagement of children (aged <18 years) were excluded.
Search Methodology A systematic search of the following licensed databases was conducted: MEDLINE, Embase, CINAHL, Sociological Abstracts, Scopus, and the Cochrane Database of Systematic Reviews. The search included the following key concepts: healthcare; decision making; healthcare decision making; patient-centered care; public; engagement; public engagement. The date limits of the literature search ranged from the earliest coverage of individual databases to the date of the final search, January 2014. The review included both English and French language content. The search results were exported to RefWorks, a reference management system, and duplicate results were deleted. The search strategy was modified to conduct a grey literature search using Google. Following advice from the librarian, the reviewers went through the Google results and retrieved any relevant documents. The grey literature search also included searching relevant government and organizational websites (e.g., Patient Voice Network, UK Department of Health).
The reviewers applied the inclusion and exclusion criteria. To assess inter-reviewer agreement on peer-reviewed article retention, two reviewers independently reviewed a sample of articles, and results were compared using a kappa statistic until a score of “good” was achieved using Altman’s [
15] criteria (>0.60).
2.1.3 Phase Three: Appraise Primary Studies and Extract Data
In realist syntheses, data abstraction is an ongoing, iterative process that is dependent on information gathered. Data abstraction, including an assessment of relevance and rigour was conducted following guidelines outlined by Pawson et al. [
6] and by Wong and colleagues [
9,
10]. The data abstraction table was developed through consultation with stakeholders and frequent research team meetings. Relevance was assessed by asking questions similar to those suggested by Kastner et al. [
16], such as, “does the research address the theory under investigation? In what context does the engagement occur? Does the engagement involve older patients, family, or caregivers?”. Rigour is used in a realist synthesis to apply judgement to the articles being reviewed to assess their quality—does the research support the conclusions drawn from it? [
6].
2.1.4 Phase Four: Synthesize Evidence and Draw Conclusion
As each article was reviewed and re-read, the reviewer created and iteratively revised codes to capture themes or concepts related to both the initial rough theories and the engagement experiences and processes that emerged from the data. The investigators (JE, HM, JA, KH) independently conducted article abstraction and line-by-line coding using an approach that allowed for themes and patterns to emerge from the data [
13]. Data were coded until saturation was reached. The analysis process was guided by a process similar to that used by Wong et al. [
17]. Nvivo 10 was used to code themes that emerged through the reading of the data. We then examined these themes for information related to the context, potential mechanisms, and outcomes (CMO). The CMO structure aims to explain, within a particular context, what underlying process (mechanism) occurs to achieve a specific outcome. The researchers also looked for patterns that aim to explain how to achieve meaningful engagement. The research team met frequently to discuss the emerging CMO patterns by applying realist logic to the analysis. For a realist review, Pawson et al. [
18] suggest that “the reviewer should aim not for encyclopaedic coverage of all possibly relevant literature but for a concept borrowed from qualitative research, that of theoretical saturation … stop searching at the point when no new information is added”. The emerging CMO structures were compared against the candidate framework, which confirmed and refined components of the framework in an effort to explain how meaningful engagement can be achieved. The findings either supported or refined the original theories/framework. In line with a realist review, this was an ongoing and iterative process, using information from all phases of the process.
2.1.5 Phase Five: Disseminate, Implement, and Evaluate
During the final phase of the study, we conducted a half-day workshop with participants (n = 11) from Patients Canada to discuss the findings from the realist review. Notes from the discussions were recorded by three researchers. Information was amalgamated to finalize the principles for engagement of older adults and their caregivers in healthcare decision making.