Strategies for incorporating patient-reported outcomes in the care of people with chronic kidney disease (PRO kidney): a protocol for a realist synthesis

Based on findings from preliminary database searches and readings, the focus of our realist synthesis was refined. As the scope of the review was being defined, our international methodology expert (JG) facilitated a number of brief methodology workshops for the research team prior to the realist synthesis being conducted.

Engaging stakeholders is an integral part of the realist synthesis process to refine the scope of the review [27]. Thus, invitations for people living with kidney disease to participate in the Patient Advisory Committee were electronically circulated through the Kidney Foundation of Canada (KFOC), Northern Alberta and Territories Branch membership, as well as through local KFOC public presentations. Over a period of multiple months, 13 kidney patients (dialysis and transplant) and spouses volunteered. The primary investigator (KSM), a co-investigator (ST), and research assistants also met with the Patient Advisory Committee (a total of 15 people, including a patient Co-Chair) in May 2017 for a full-day training on practical strategies of how to engage in our patient-oriented research, sponsored by Strategies for Patient-Oriented Research [28]. The Patient Advisory Committee met again in August 2017 to give an update on the review process, to seek input on study objectives, and to discuss future development of knowledge translation materials that they would find useful.

Searching in realist synthesis comprises two main stages: (1) searches to identify existing implementation theories and frameworks and (2) searches for evidence to test theories relevant to the review questions [24]. From data from the first search, we will identify an initial program theory (or theories) that explains how an intervention is intended to work. The initial program theory will encompass context-mechanism-outcome hypotheses to provide a theoretical explanation about how PROs are used at individual and aggregate levels across healthcare practices.

To locate these theories, we (a) systematically searched the literature to identify existing theories and (b) consulted our methodological experts and research team practitioners. Research team members identified three exemplar papers a priori [29‐31]. A research librarian was consulted to design search strategies. A preliminary search strategy was developed for Ovid MEDLINE [32]. It was further expanded to focus on published literature, indexed in the following bibliographic databases: Ovid MEDLINE, Ovid Embase, EBSCOhost CINAHL, Web of Science, and Scopus. The search was conducted on January 20, 2017. The search contained two main concepts: patient reported outcome measures and theory. Appropriate subject headings and keywords were used in the search and were modified for each specific database, in order to retrieve literature about each of these concept areas [32]. The search was not limited by language, type of literature (primary research, theoretical, or review) or date to ensure breadth of scope. In total 13,412 articles were retrieved. Of these, 7117 were duplicates (1516 were automatically de-duplicated in Ovid and 5601 were de-duplicated using endnote). An endnote library containing 6295 records was provided to team for screening. Titles were screened and 1210 abstracts proceeded for screening by the team (with 10% double-screened for relevance). Inclusion and exclusion criteria for objective 1 are outlined in Table 3. Rater agreement between reviewers was 91.7% (Table 4). Thirty-four full-text articles were included for extraction for the initial program theory (Fig. 1).

An initial program theory will be built to map out the main components of using PROs, anticipated outcomes, elements that contribute to these outcomes, and sequencing of these elements. The initial program theory (or theories) will be presented to various stakeholders (e.g., interdisciplinary nephrology practitioners). We will ask them to identify and discuss which context-mechanism-outcome hypotheses are most important. After this meeting, the initial program theory will be refined prior to synthesis of the renal literature [22].

The following four stages address objective 2: To develop a kidney-specific program theory about use of PROs in nephrology that may enhance person-centered care by testing and refining the theory through a realist synthesis of the empirical (renal) literature.

The purpose of the second search is to identify relevant bodies of literature as “data” from which to test and refine our initial program theory for a kidney context. The search strategy will be guided by the initial program theory/theories, and it will examine the relevant bodies of literature pertaining to theoretical use of PROs, as well as observed quantitative and qualitative research on the use of PROs in clinical nephrology care at individual and aggregate levels.

In collaboration with the library scientist on the research team, we have developed an initial database search strategy for Ovid MEDLINE [32]. The search strategy was reviewed by PRESS (Peer Review of Electronic Search Strategies). Searches in other databases (Ovid Embase, EBSCOhost CINAHL, Web of Science, and Scopus) will be based upon this search strategy using appropriate subject headings and operators for each database. The search will not be limited by language, type of literature (primary research, theoretical, or review), or date to ensure breadth of scope.

We will also search the gray literature, and consider white papers, editorials, reports, and guidelines describing the use of PROs in clinical nephrology practice. This will be done by searching in the bibliographic databases that index gray literature, using web-based search engines such as Google and DuckDuckGo, clinical practice guideline indices, and looking at the websites of organizations involved in kidney healthcare such as the Kidney Foundation of Canada and the National Kidney Foundation. The practitioners on the research team will help refine search terms to ensure that they are sufficient to capture the broad manner in which PROs are utilized in different practice environments and healthcare systems.

As the search results are screened, additional key articles will be identified. Through this iterative process, we will use forward citation chaining (articles that cite a given article) and backward citation chaining (searching the reference list of a given article) [33] using Google Scholar, and in-depth exploration of the relevant studies’ relationship to other research articles. Searching for additional data to enhance development of the program theory is a critical step in realist synthesis. We anticipate that additional searches will be undertaken to test and develop specific sections of the program theory. Thus, the research team will refine the screening processes based on future searches.

We have developed the inclusion and exclusion criteria (for the second objective) for abstract screening (see Table 5). In review of this literature, we will ask the question, “Does this provide any evidence, discussion, or conceptual/theoretical perspectives that will enable us to test and refine our program theory for PRO use in clinical nephrology care to enhance person-centered care?”

The selection of the articles will be based on inclusion/exclusion criteria, relevance to theory building and/or testing, and rigor [25]. After all abstracts are screened, the research coordinator and research assistants will screen full-texts. To attend to rigor, 10% double-screening and discussion within the research team will be undertaken to discuss any discrepancies. Study quality will be judged according to quality standards appropriate for the type of research and on relevance (whether the article contributed to theory building). The quality will be assessed using McGill Mixed Methods Appraisal Tool (MMAT), a critical appraisal tool developed for appraising the methodological quality of studies included in systematic reviews [34]. Unlike many appraisal tools, the MMAT has documented evidence of reliability [34]. Studies will not be excluded based on MMAT scores because relevance will be based on contribution to context-mechanism-outcome configurations and provision of relevant evidence to help test the theory/theories.

Appraisal continues as evidence is extracted for its relevance to theory testing and the rigor with which it has been produced [35]. In many instances, it is only a subset of findings from each study that relate specifically to the theory being tested that are included in the synthesis. Therefore, quality appraisal relates specifically to the validity of the causal claims made in this subset of findings, rather than the study as a whole. Trust in these causal claims is also enhanced by the accumulation of evidence from a number of different studies which provide further lateral support for the theory being tested, discussed in more detail below.

Given that any section of a document may be pertinent to the theory enhancement, detailed data management processes are essential. NVIVO, a qualitative software system may be used to create a filing system and coding database. The research coordinator and research assistants will complete data extraction. To ensure consistency, a random selection of articles will be independently coded by the principal applicant (KSM), research team members (ST, RF), the research coordinator and assistants. KSM will join as a reviewer to resolve differences. The coding will be deductive (codes created in advance of data extraction and analysis based on the initial program theory) and inductive (codes created to capture data reported in the included studies). Both qualitative and quantitative data are compiled. In addition, the inferences and conclusions drawn by the authors of the studies are extracted as data within realist synthesis, as they often permit the identification of sub-theories which can then be further tested with empirical evidence. Different fragments of evidence are sought and utilized from each study. Data will also be coded to identify if it pertains to context, mechanism, or outcome in order to explain and understand how, and in what circumstances, PROs enhance person-centered nephrology care at individual and aggregate levels.

Synthesis of the extracted evidence will focus on testing and refinement of the initial program theory to create a kidney-specific theory about use of PROs at individual and aggregate levels in nephrology care. Data will be synthesized across the literature into context-mechanism-outcome configurations seeking to explain how PROs are used to enhance person-centered nephrology care at individual and aggregate levels. For example, for the purposes of understanding how context influences outcomes, we will compare use of PROs in nephrology practice where it was “successful” against those which were not, or where PROs were completed or implemented electronically or through an electronic medical record versus paper. Guided by the literature, the research team will then identify which mechanisms are “key” in this process.

As extracted evidence is synthesized, the initial program theory/theories will be refined to reflect evidence from the renal literature. During this process, we will ask the following questions: “What does this evidence suggest about this aspect of our theory? Does it support it? Does it disprove it? Does it suggest an amendment to it?” [22]. The initial program theory/theories containing context-mechanism-outcome configurations will then be amended in response to these questions. New context-mechanism-outcome configurations may be created, some initially hypothesized configurations may be removed, and others may be combined, separated, or revised.

These refinements include both summary and analysis, but do not yet offer synthesis in its fullest sense. Moving up a level of abstraction to “make sense” of the patterns of findings involves using a priori middle-range theories or formal theories from the field in which the analysis may be situated [22]. Examples may include theories of change, learning theories, systems theories, or complexity theories. With the addition of formal theories, syntheses may bridge the evidence provided in the renal literature with theoretical “sense-making” of the patterns.

Although the processes of searching, screening, selecting, extracting, and synthesizing have been described in a linear fashion, the review process is iterative. During the analytic steps, we will move iteratively between analysis of examples in the data, revision of the program theory, and refined searching of the data to test specific sections of the theory. To attend to rigor, refinement of the theory will include consultation with research team nephrology practitioners and the Patient Advisory Committee. Testing and refinement of the theory is the purpose of the synthesis itself, rather than “validation” per se. The overarching purpose is to build an explanation of how context shapes the mechanisms through which the intervention works. The emerging explanation will be substantiated in evidence, and stakeholder involvement throughout the synthesis will support the usefulness of the emerging explanation. However, it would be beyond the scope of the synthesis to directly test its usefulness. The next and last stage addresses objective 3: To develop KT products for kidney practitioners and knowledge users that will facilitate the optimal utilization of PROs in nephrology care.

Our integrated KT products will (1) provide evidence-based strategies for PRO use in clinical nephrology practice that enhances person-centered care and (2) provide guidance to policy makers on effective, strategic processes for PRO use in kidney settings. The development of KT products will be created with input from the Patient Advisory Committee and CIHI’s Patient-Reported Outcome Measures Renal Care Working Group.

The KT products from this project will be circulated and disseminated widely from the team to knowledge user groups such as CIHI’s Patient-Reported Outcome Measures Renal Care Working Group; Canadian Association of Nephrology Nurses and Technologists; Canadian Association of Nephrology Social Workers; Canadian Society of Nephrology, as well as through publications and conference presentations. Our intention in the translation of our findings into knowledge products is that the results may be both meaningful and useful for practitioners and policy makers integrating PROs in renal settings.

Throughout the course of the project, our team will work towards creation of the following products:

These products will be disseminated widely through our knowledge users in their own areas of influence, as well through publications and conference presentations.