Background
The transition from cancer treatment to follow-up care is often challenging for the nearly 15 million cancer survivors in the United States [
1]. To facilitate survivors’ transitions, the Institute of Medicine (IOM) recommends that cancer care providers develop and deliver to survivors and their primary care providers survivorship care plans (SCPs). SCPs are written documents that are often developed in cancer programs and, ideally, include plans for follow-up care, such as surveillance and preventive services, and supporting information such as survivor’s diagnosis, stage, and cancer treatments received. SCPs are intended to facilitate communication and coordination of care among survivors, cancer care providers, and primary care providers [
2]. SCP use is increasingly advised and required in guidelines issued by cancer care quality improvement organizations (e.g., Commission on Cancer) [
2]. Despite increasingly pervasive guidelines for SCP use, a recent survey indicated that only 20% of United States oncologists reported always/almost always providing SCPs [
3]. Further, many providers develop SCPs without delivering them to survivors or their primary care providers [
4].
Limited SCPs use in practice may relate, in part, to poor quality of guidelines for SCP use. Guidelines are tools that are intended to promote the use of recommended practices. High-quality guidelines reflect the perspectives of relevant stakeholder groups; provide clear guidance for implementation; are based on empirical evidence, explicit in their methods of development, critically reviewed by experts, and free from conflicts of interest; and are specific and unambiguous [
5]. Evidence suggests that providers’ intentions to implement guidelines are stronger when guidelines are clear and unambiguous [
6,
7]. Clear, unambiguous guidelines for the use of SCPs might define for whom and by whom SCPs should be developed, when and where SCPs should be developed and delivered, to whom SCPs should be delivered, and what survivors and primary care providers should do with SCPs once they receive them. Evidence of limited and inconsistent SCP use may suggest that cancer care providers lack clear guidance regarding these questions [
3,
8]. Optimal responsibility and timing for developing and delivering SCPs are unclear [
3,
9]; the utility of electronic SCPs has been debated [
3,
9-
11]; and questions remain regarding where SCPs are optimally delivered (e.g., survivorship clinic, final treatment visit) [
12,
13].
The purpose of this study was to assess the quality of guidelines for SCP use. Results may offer perspective on why SCP use has been limited to date. If the quality of guidelines for the use of SCPs is low, then developing clearer, less ambiguous guidelines may represent a first step toward promoting SCP use. If the quality of guidelines for the use of SCPs is high, then efforts may need to focus on multifaceted interventions to promote the implementation of guidelines for SCP use in practice.
Methods
Literature search
For the purposes of this study, we used the IOM’s definition of guidelines: “statements that include recommendations intended to optimize patient care” ([
14], p. 25). For inclusion in this study, we required guidelines to include recommendations regarding the development, delivery, and/or use of SCPs during follow-up care.
A broad range of literature was gathered to identify guidelines for the use of SCPs. The following electronic databases were searched for references to guidelines related to SCPs published through April 15, 2014: MEDLINE/PubMed (1946–2014), EMBASE (Excerpta Medica Database) (1947–2014), and CINAHL (Cumulative Index to Nursing and Allied Health Literature) (1981–2014). In addition to the databases for indexed scientific publications, we searched grey literature sources, including websites of professional organizations and guidelines groups (see Additional file
1). The search used a broad strategy that combined terms for ‘survivorship care plans’ and ‘guidelines’.
Our initial search yielded 175 unduplicated records. An additional 39 references were identified through searches of guidelines groups’ and professional organizations’ websites and publications. Additional file
1 identifies these websites and publications and depicts our process of excluding records that did not contain recommendations regarding the use of SCPs. Eighty-six duplicates were removed, yielding 128 records. From this, we eliminated 111 records that represented guidelines for the use of clinical procedures, non-English publications, childhood cancers, adult survivors of pediatric cancers, models, programs, tools, editorials, dissertations, and templates not accompanied by explicit SCP recommendations. This yielded 17 unique records.
We sent the titles of the 17 records to six experts in the field of cancer survivorship for review. These experts, along with SB and DM, also experts in survivorship, were asked to review the list to ensure accuracy and comprehensiveness. Experts suggested that the authors investigate six additional resources that they believed might reveal additional guidelines; three of these met the inclusion criteria applied in previous rounds of the search, resulting in 20 records that were included in our final full-text review.
SB and SE independently conducted full-text review of the 20 records, using two criteria for inclusion: Records were required to constitute a guideline per the IOM’s definition [
14] and to include recommendations regarding the use of SCPs. Disagreements were resolved through consensus and review by DM. Four records were excluded during this process, yielding 16 guidelines to be evaluated.
Data abstraction
Guidelines evaluated in the study included recommendations regarding topics other than SCP use. Since the purpose of this study was to evaluate the quality of guidelines for SCP use, we did not abstract data regarding recommendations that did not relate to SCP use. To extract data from guidelines related to SCP use, SB developed a data extraction form (see Additional file
2) based on domains specified in the AGREE II instrument (
Appraisal of
Guidelines for
Research and
Evaluation, 2nd edition;
www.agreetrust.org; domains described in detail in the analysis section), a tool developed by an international group of scientists to advance the quality of clinical practice guidelines [
15]. The form was reviewed and edited by all authors.
SB and LD began by collaboratively extracting data from one guideline. Then, we independently extracted data from a second guideline and met to resolve discrepancies. SB and LD then independently extracted data from the remaining 14 guidelines. SE synthesized SB and LD’s extracted data into a single form.
Analysis
We assessed the overall quality of the 16 included guidelines using the AGREE II instrument. The validity and reliability for assessment of practice guidelines using the AGREE II have been established [
15,
16]. The instrument includes 23 items that address six quality domains: (1) scope and purpose, (2) stakeholder involvement, (3) rigor of development, (4) clarity of presentation, (5) applicability, and (6) editorial independence. Two additional assessment items (Overall Guideline Assessment) pertain to an overall judgment of the guideline. Each item is rated on a seven-point Likert scale, with 1 assigned for items with no clear discussion and 7 for exceptional quality of reporting. DC and LD read the entire AGREE II user’s manual and then independently rated all included guidelines.
Not all of the AGREE II items were applicable to guidelines for SCP use. Items 11 (“The health benefits, side effects, and risks have been considered in formulating the recommendations.”) and 16 (“The different options for management of the condition or health issue are clearly presented.”) are limited in their applicability to SCPs; evidence of benefits, risks, and alternatives is limited. As such, these items were excluded from analysis. DC, LD, and SB worked together to revise item 12 (“There is an explicit link between the recommendations and the supporting evidence.”); in light of limited empirical evidence regarding SCPs, DC and LD interpreted item 12 as “Evidence is described, and recommendations follow from it”.
SB conducted the final scoring, according to the instrument protocol, by adding together DC and LD’s respective ratings for items in each domain and standardizing the total score out of 100%.
The AGREE II manual does not provide guidance regarding how to interpret scores. To promote consistency with extant studies that have used the AGREE II instrument [
17,
18], we adopted their method: Guidelines receiving a standardized score of 50% or greater on all domains were
strongly recommended, guidelines receiving an overall assessment of 50% or greater were
recommended. and guidelines that neither received a standardized score of 50% or greater on all domains nor received an overall assessment of 50% or greater were
not recommended.
Discussion
Overall, AGREE II domain scores suggest that the quality of guidelines for SCP use is low. This finding is consistent with that of other studies that have shown poor quality of guidelines in cancer care [
34]. Guidelines were generally definitive in their recommendations to use SCPs, but these recommendations were often not explicitly linked to evidence. Further, guidelines for SCP use offered little clarity regarding why, when, where, and how SCPs should be used; who should use them; and for whom they should be used. Only half of the guidelines explained the purpose of SCPs [
20,
24-
30]. Recommendations regarding when, where, and by whom SCPs should be used varied across guidelines. Most guidelines recommended that SCPs be developed and delivered upon survivors’ transition to follow-up care, but they differed in their definitions of the transition period; only four guidelines indicated who should develop and deliver SCPs [
2,
25,
31,
32]; and just six specifically identified the cancer survivors to whom their guideline applied [
19,
20,
22-
24,
32].
The lack of clarity was underscored by the challenges of abstracting guideline data. Two investigators abstracted data, and a third reconciled abstractions. In multiple instances, the independent abstractions led to two reasonable but distinct interpretations of the guidelines. That two informed individuals could develop such distinct interpretations of one guideline suggests that busy clinicians may have difficulty culling relevant information that is consistent with their peers’ practices. This lack of guideline clarity may partly explain limited and inconsistent SCP use in practice [
4,
35].
As we suggested might be the case in the introduction, the poor quality of guidelines for SCP use may contribute to limited and inconsistent SCP use in practice [
3,
8]. Clear and unambiguous guidelines for SCP use may promote the effectiveness with which SCPs are implemented [
6,
7]; in effect, high-quality guidelines are an implementation strategy [
36]. In particular, guidelines that use behaviorally specific terms may be the most effective way of increasing implementation [
6]. Guidelines for the use of SCPs may be improved with precise and consistent definitions of which templates are best to use; to whom the guideline applies; to whom the guideline does not apply; and when, where, and by whom SCPs may be most effectively developed and delivered. Clear specifications may facilitate evaluation of adherence to guidelines. Future research should assess the relationship between guideline quality and the effectiveness of SCP implementation.
Many of the guidelines included in this study scored highly in some, but not all, domains. Efforts to promote SCP use may benefit from combining elements from multiple guidelines into a single clear, unambiguous resource. SB, JW, LD, and colleagues are currently synthesizing information from the guidelines that scored highest in each AGREE II domain in this study to create a “meta-guideline” for SCP use that practitioners may reference to facilitate their decision regarding adopting SCPs and, should they adopt SCPs, to facilitate their implementation. By leveraging the work of existing guidelines, this effort represents an efficient approach to facilitating practitioners’ decisions around SCP adoption and implementation.
Study limitations should be considered when interpreting results. The search was conducted by a librarian with expertise in systematic reviews; however, the search may have excluded some guidelines. For example, we excluded guidelines that were not published in English. And although the AGREE II instrument is a useful tool for evaluating guideline quality, it has limitations. In particular, it does not offer guidance for interpreting scores. Our criteria for strongly recommending, recommending, or not recommending guidelines were intended to promote consistency with extant studies using the AGREE II instrument [
17,
18] and to clearly and unambiguously identify potentially useful guidelines for the use of SCPs. In most cases, this resulted in overall assessment scores that were commensurate with scores received on individual domains. For this reason, the field of guideline assessment may benefit from future AGREE II studies using a similar method. Further, AGREE II only assesses the quality of guidelines’ structure and content; it does not assess the quality of guidelines’ recommendations. A case in point, despite the variation in guidelines’ structural and content quality that we found in this study, we found almost no variation in guidelines’ recommendations in favor of SCP use. As such, although our assessment of the guidelines indicates room for improvement to facilitate implementation of recommendations, neither our study nor the AGREE II instrument offers insight into whether the included guidelines’ recommendations are correct.
Another challenge of AGREE II relates to applying criteria across several guidelines. Scores are intended to be based on a guideline’s quality irrespective of other guidelines’ quality. However, as scoring proceeds, reviewers’ evaluations of guidelines becomes biased by familiarity with previously scored guidelines. The quality of future AGREE II evaluations may benefit from randomizing the order in which each reviewer evaluates guidelines to average out the effects of increased familiarity.
Acknowledgements
Dr. Birken’s effort was funded by grant number 5 R25 CA57726 from the National Cancer Institute. Ms. Check’s effort was funded by grant number R25CA116339 from the National Cancer Institute. The authors thank Kirsten Nyrop and Amanda Nelson for their suggestions; Patricia Ganz, Eva Grunfeld, Michael Jefford, Mary McCabe, Daniel McKellar, and Julia Rowland for their expertise regarding survivorship care plan use guidelines; and Mary McCabe, Daniel McKellar, and Brian Mittman for their feedback on the manuscript.
Competing interests
The authors declare that they have no competing interests.
Authors’ contributions
All authors made significant contributions to the manuscript. JW and SB collected the data. SB, SE, LD, DC, and DM analyzed the data. SB, JW, SE, LD, DC, AG, and DM drafted and critically revised the manuscript for important intellectual content. All authors have read and gave final approval of the version of the manuscript submitted for publication.