Trends in the development process of clinical practice guidelines: a questionnaire survey for the guideline development groups in Japan

Clinical practice guidelines (CPGs) are statements that include recommendations based on a systematic review of evidence and an assessment of the benefits and harms of alternative care options in order to assist the decision making of practitioners and patients [1, 2]. CPGs are representative methods for promoting the standardization of healthcare and improving its quality. In October 2021, more than 29,000 articles indexed as “practice guideline” for publication type were listed in PubMed. In Japan, the Ministry of Health, Labour and Welfare has encouraged academic societies to develop CPG for major diseases using public research funds since 2000. Currently, academic societies and research groups are involved in developing and managing CPGs, and approximately 60 CPGs, including newly developed and revised CPGs, are being published every year.

Numerous development manuals and more than 40 appraisal tools have been published to ensure the quality of CPGs [3, 4]. The general steps involved in the development of CPG are as follows: i) identifying and refining the CPG subject area, ii) running development groups, iii) identifying and assessing the research evidence, iv) translating the evidence into CPGs, and v) reviewing and updating the CPGs [5]. Because the evidence and resources that can be used differ among the CPG developers, the actual CPG development processes may vary substantially [2]. Therefore, information regarding the methods used and the difficulties encountered in the actual development processes could help in improving the environment (available methodological guidance, tools, support systems, etc.) of CPG development process. Additionally, given the changing environments of CPG development, the trends of the CPG development process can provide valuable information. The descriptions of the methods and processes of CPG development are usually included in the CPGs, and there are some case reports [6, 7] and cross-sectional surveys [8‐10] on the actual CPG development processes. These reports show only a snapshot of the development process and not the trends of the methods used by CPG developers. Some previous studies have reported the trends of the CPG development through conducting systematic reviews [11‐13]. However, it is not possible to obtain information on the detail of ingenuity or difficulties in the CPG development process from the description in published CPGs only.

Based on the questionnaire surveys conducted on the development groups of the CPGs published by 2006 [14‐16], Hasegawa revealed the problems in the CPG development process in Japan. These problems were associated with the involvement of experts (e.g., epidemiologists, librarians, or health economists) and patients, efficient evidence collection and appraisal in systematic review, and paucity of evidence on Japanese patients [16]. The US Institute of Medicine pointed out the importance of expert and patient involvement in the “Guideline Development Group Composition” for developing trustworthy CPGs [2]. Participation of patients as stakeholders in the CPG development process is expected to enhance the validity and usefulness of published CPGs [17]. Because conducting systematic reviews can be a time-consuming and cost- and resource-intensive task, the efficacy of systematic reviews becomes especially problematic when the CPG developers make recommendations expeditiously [18]. CPGs are developed to support patients and practitioners in each country or region based on the evidence gathered from around the world, and the evidence used in the CPGs may often not include patients of interest of the CPG developers. These problems were revealed on examination of the CPG development processes in Japan, but these could be common problems for CPG developers worldwide. Hasegawa et al. conducted a questionnaire survey on development groups of the CPG published by 2011, addressing the CPG development process. Their questionnaire and data can be used in this study for the trend analysis of the methods of CPG development and the difficulties encountered by the CPG development groups [19].

This survey involving CPG development groups in Japan revealed that among the problems noted in CPG development in a previous study [16], there has been significant improvement in patient involvement and coping with the paucity of Japanese evidence, but not in expert involvement, efficiency of evidence collection and appraisal.

Patient involvement has increased across many fields in Japan, such as clinical practice, health policy making, and clinical research [26‐28]. With respect to CPG, the Japanese CPG development handbook published in 2007 [25] served as the basis for CPG development in Japan for a considerable time [7]. It suggested that patient involvement was “desirable” in the CPG development process, but the manual, which was the updated version of that handbook in 2017, emphasized the importance of patient involvement by calling it “essential” [29]. The efforts in the dissemination of the importance of patient involvement might have enhance the validity and usefulness of the CPGs through improving patient involvement in the CPG development process.

Basing on the Grading of Recommendations Assessment, Development and Evaluation (GRADE) approach [30], the manual disseminated the assessment methods of “indirectness,” referring to the difference between the population, interventions, comparisons, and outcomes intended by the CPG developers and those in the obtained evidence [29]. Even if there was insufficient evidence of Japanese patients, the CPG developers could evaluate the body of evidence using the criteria for the certainty of evidence, including indirectness, and formulate recommendations based on such evidence. The progress in the CPG development methodology might have contributed to coping with no Japanese evidence in the CPG development process.

The CPG development methodologies, such as the GRADE approach that was developed by some of the original founders of the evidence-based medicine (EBM) movement, have improved over the last decade or more. Introducing the GRADE approach, the Japanese CPG development manual has emphasized on the comprehensiveness, transparency, and unbiasedness rather than the efficiency of systematic reviews in the CPG development process [29]. However, these methodologies may be highly technical and beyond many organized CPG efforts, including some who have formally endorsed the use of these methodologies [31]. No improvement was observed in the efficiency of evidence collection and appraisal in the results. Recently, rapid reviews, which accelerated the process of a traditional systematic review through streamlining or omitting various methods, have been conducted to produce evidence for stakeholders in a resource-efficient manner [32]. Cochrane, a global leader in the production of high-quality systematic reviews and methodological guidance, provided methodological recommendations for conducting rapid reviews [33]. Additionally, some tools for efficient systematic review have been developed and available [34, 35]. Although many challenges to the conduct of rapid reviews were addressed [36], the authors of CPG development manuals should consider introducing the methodology of rapid review and these tools to CPG developers in addition to rigorous methodologies.

With regard to expert involvement, some organizations offer support to the CPG development groups. In Japan, The MINDS Guideline Library assigns experts of CPG development to these groups [37], and several organizations, including the Japan Medical Library Association, offer support in searching for evidence in the CPG development process [38, 39]. Our results suggest that these forms of support might not fully meet the needs of the CPG development groups. Collaborating with McMaster University, Guidelines International Network (GIN), which is a global network of guideline producing organizations and guideline participants, has initiated a comprehensive, evidence-based, and up-to-date training program for CPG development group members [40]. In this program, GIN prepares the course for expert methodologist. It could increase the importance of and promote the standardization of experts. The organizations for support in the CPG development should introduce the CPG development groups about the importance of expert participation and establish an expert referral system that meets the CPG developers’ needs.

In addition to the problems noted in the previous study [16], “composing development group” has become a difficult process in CPG development. This survey revealed an increasing trend in the total number of CPG development group members and patients involved. Regarding the number of CPG development group members, Murphy et al. suggested that although having more group members increased the reliability of the group judgment, it caused coordination problems within the group [41]. As for patient involvement, Blackwood et al. recently revealed the paucity of knowledge on how to identify, incorporate and report patient preferences in CPGs through an international cross-sectional survey of CPG development organizations [42]. The lack of relevant knowledge, inability to separate personal experiences from systematic methods and analytical rules, and misunderstandings about EBM in patients makes it difficult to find an appropriate person who can consider the evidence objectively and make recommendations free of preconceived views or self-interests [2]. The increasing number of members and the progress of patient involvement in the CPG development groups might have increased the difficulty of group composition for CPG developers in the coordination and assignment of patient representatives. Piggott et al. suggested the contribution and participation of CPG development group members has become more demanding, although more guidance for CPG development is available. Therefore, they developed a guide for CPG development group members containing 33 items for consideration before, during and in follow-up to CPG group meetings and a description of each participant role [43]. The researchers on CPGs could help CPG development groups through clarifying the appropriate group composition based on the actual CPG development processes. Additionally, the support organization for the CPG development and dissemination should introduce the findings on the effective group composition including the above tool and description to CPG development groups.

This is the first study to reveal the long-term trends in CPG development activities through a cross-sectional questionnaire survey. However, it has several limitations. The response rate was not high, and there was a time lag between the CPG development and the current survey, as well as recall, selection, and unresponsive bias. The results might be biased towards the responses of the CPG development groups which have been involved in the CPG development and dissemination actively. Therefore, the results should be not interpreted as the trends for the general CPG development organizations.

Although there has been improvement in the patient involvement and coping with the paucity of Japanese evidence in the CPG development process in Japan, expert involvement and efficiency of evidence collection and appraisal have not. Our results revealed the need for methodological improvement in the efficient collection and appraisal of evidence and in the supporting system assigning experts to the CPG development groups.