Support of personalized medicine through risk-stratified treatment recommendations - an environmental scan of clinical practice guidelines

We started by forming a framework for developing risk-stratified treatment recommendations. Figure 1 outlines the major steps for developing risk-stratified treatment recommendations, each of which requires high quality evidence from observational studies (development and validation of risk assessment tools), randomized trials (evidence about treatment effects) and studies to elicit patient preferences (using various study designs, for example, discrete choice experiments). It is well known for all guidelines that evidence about treatment effects on benefit and harm outcomes must be available. In addition, a risk assessment tool should be available that allows the assigning of patients to different risk categories. A method is required to estimate how treatment evidence applies to patients at different risks and how the benefits compare to the harms in patients at different risks. As a result of such a benefit-harm assessment, treatment thresholds can be defined for patients with different risk profiles that maximize the chance for benefits while minimizing harms. In addition, patient preferences for outcomes would ideally be explicitly considered for the development of risk-stratified treatment recommendations or their application in practice.

We performed an environmental scan of CPGs, which included a limited literature search (described below) but not a comprehensive, systematic review of all CPGs. We focused on CPGs for major chronic diseases and from the United States (US), Canada, or the United Kingdom (UK) National Institute for Health and Clinical Excellence (NICE). The completed PRISMA checklist is available as Additional file 1.

We searched the US National Guideline Clearinghouse (NGC) database on February 5 2011 for CPGs with treatment recommendations for five major chronic diseases. The top five chronic diseases in the US are heart disease, cancer, stroke, COPD and diabetes, accounting for more than two-thirds of all deaths [14]. In the NGC database, guidelines were categorized by disease topics that were linked to a specific term derived from the US National Library of Medicine's Medical Subject Headings classification.

For heart disease and stroke, we performed our search within the Cardiovascular Diseases section of the database (n = 442) and considered CPGs specific for primary prevention of heart disease and stroke, that is, the prevention of an event in persons free of established cardiovascular diseases. For cancer, we chose to examine three types of cancer with the highest mortality rates in the US (lung cancer, prostate cancer and breast cancer) [15]. We searched for CPGs within the Lung Neoplasms (n = 53), Prostatic Neoplasms (n = 26) and Breast Neoplasms (n = 52) sections, respectively. For COPD, we considered CPGs specific for COPD within the Respiratory Tract Diseases section (n = 102). For diabetes mellitus, we considered CPGs for type II diabetes within the Diabetes Mellitus, Type 2 section (n = 44).

We included CPGs that recommended using risk assessment tools to inform treatment decisions. Risk assessment tools are tools to calculate the probability of developing an event or a disease based on a prediction model (binary outcome), or tools that make projections about the course of disease measured by patient-reported or other continuous outcomes (for example, decline of functional status over time). We excluded CPGs if they were not from the US, Canada or NICE (UK); focused on childhood diseases; made recommendations on screening, genetic counseling or diagnostic work-up alone; or did not use any risk assessment tools to inform risk-stratified treatment decisions. This latter excluded category involved guidelines that recommended treatments according to diagnostic criteria, as for example based on pathological staging, rather than according to prognostic information (for example, the risk stratification scheme proposed by D'Amico et al. in prostate cancer guidelines [16]).

We developed a standardized form to extract data from the included CPGs and the background documents detailing the methods used in developing CPGs when available. We extracted general items such as guideline title, bibliographic source, date released and guideline developer. We then extracted information related to five key components for developing risk-stratified treatment recommendations (Figure 1). We extracted the following information on risk assessment tools: the name of the prediction model, the outcome and the timeframe (for example, 10 years) used in the model, and whether validation of the model (for example, assessment of discrimination and/or calibration) was discussed in the CPGs. We extracted information on the type of evidence used to determine the effects of treatments on benefit and harm outcomes (observational studies, single or several randomized controlled trials (RCTs), or meta-analyses). We recorded the methods to link risk prediction and evidence on treatment effects (for example, applying relative risk reductions to different absolute risks calculated from the risk assessment tool). We recorded the way the treatment benefits and harms were assessed and how treatment thresholds (based on risk assessment tools) were determined. We also extracted information on assumptions made for linking risk prediction and treatment evidence (for example, assumption of constant relative risk reductions across the risk spectrum) and on assumptions made for the assessment of benefits and harms (for example, assumption that benefit and harm outcomes can be put on a single scale and the overall net benefit expressed as a single number indicating benefit or harm). Finally, we noted whether patient preferences (for example, relative importance of different benefit and harm outcomes) were considered for developing risk-stratified treatment recommendations. Because some CPGs were very brief, without detailing the development process but referring to other documents, we considered those documents for data extraction to avoid underestimating the rigor of the development process of a CPG. Two reviewers (TY and DV) independently extracted all relevant information from each CPG and the discrepancies were resolved by discussion or third-party (MP) arbitration. We constructed a table for comparison of recommendations from each of the included CPGs.

A large proportion of CPGs (16 out of 20, 80%) were on type II diabetes or on primary prevention of heart disease and stroke [10, 18‐32], and the remaining four CPGs were on breast cancer management [33‐36]. All 16 CPGs on diabetes or cardiovascular disease recommended one or several risk assessment tools to assess 10-year cardiovascular disease risk. The Framingham Risk Score was explicitly suggested in 12 CPGs [10, 19‐29]; the UK Prospective Diabetes Study Risk Engine was suggested in four CPGs [21, 24, 30, 31] for patients with diabetes, and the Prospective Cardiovascular Münster and the Systematic Coronary Risk Evaluation risk tools were used in one CPG [29]. Of the 16 CPGs on diabetes or cardiovascular disease, two (13%) did not clearly specify the prediction model used for calculating 10-year cardiovascular disease risk [18, 32]. Among the four CPGs on breast cancer [33‐36], two recommended using the National Cancer Institute breast cancer risk assessment tool to calculate 5-year risk of invasive breast cancer [33, 35]; one recommended using the Nottingham Prognostic Index to calculate 10-year survival [36]; one mentioned different risk assessment tools but did not clearly define risk categories [34]. Information on the validation of the risk models was reported in seven (35%) of the 20 included CPGs (Table 2) [10, 19, 20, 22, 29, 30, 35].

Of the 16 CPGs for type II diabetes and primary prevention of heart disease and stroke, nine (56%) suggested specific target lipid levels for each risk category when making recommendations about lifestyle management or pharmacotherapy (for example, aspirin, statins and antihypertensive drugs) [10, 19‐21, 26‐29, 31]. The four CPGs on breast cancer [33‐36] provided recommendations on surgery or pharmacotherapy (for example, tamoxifen, raloxifene and aromatase inhibitors) according to risk levels.

Sixteen (80%) of the 20 CPGs reported using evidence on treatment benefits from RCTs or meta-analyses or other guidelines [10, 18‐25, 27, 29‐32, 35, 36]. One CPG (5%) did not report quantitative information on treatment benefits [28] and three (15%) did not clearly specify the type of studies considered [26, 33, 34]. Treatment harms were only reported in 13 of the 20 CPGs (65%) [10, 18, 22, 23, 25‐27, 29, 30, 32, 33, 35, 36]. The source of evidence on harms was specified in nine out of these 13 CPGs (69%) and included observational studies, RCTs and meta-analyses [10, 18, 32, 23, 25, 27, 30, 35, 36]. Heterogeneity of treatment effects were assessed in eight (40%) of the included 20 CPGs (Table 2) [10, 18, 22, 23, 25, 26, 32, 35].

In reviewing how CPGs made the link between risk prediction and treatment effects, we found fewer than half of the CPGs (eight out of 20, 40%) explicitly or implicitly stated that they applied evidence of relative risk reductions from RCTs and/or meta-analyses to different absolute risks [10, 18, 23‐27, 32]. For instance, the U.S. Preventive Services Task Force (USPSTF) guideline [25] applied a 32% risk reduction of myocardial infarction (in men) and a 17% risk reduction of strokes (in women) with regular aspirin use to absolute outcome risks and assumed that the effects were constant across risk levels and age categories. One (5%) of the 20 included CPGs [33], instead of applying treatment evidence to all risk levels, used the evidence from RCTs with the same risk profile (high breast cancer risk) population for which the recommendation was made. Eleven (55%) of the included CPGs did not report the way in which they linked risk prediction to treatment effects (Table 2) [19‐22, 28‐31, 34‐36].

Only a small proportion (two out of 20, 10% [25, 35]) of CPGs explicitly stated that they planned to perform benefit and harm assessment as the basis for making risk-stratified treatment recommendations. To define treatment thresholds, only the USPSTF guideline quantitatively weighed treatment benefits and harms by putting the expected benefit and harm outcomes on the same scale (events per 1,000 persons treated over 10 years). The USPSTF guideline recommended using aspirin when the treatment benefits (number of myocardial infarctions or strokes prevented per 1,000 persons treated over 10 years) outweigh the treatment harms (number of gastrointestinal bleedings or hemorrhagic strokes per 1,000 persons treated over 10 years). For example, the expected number of myocardial infarctions prevented by aspirin was estimated to be 16 per 1,000 men of age 60 to 69 years if men had a 10-year risk for myocardial infarction of 5%, while the expected number of excess gastrointestinal bleedings was 24 and hemorrhagic strokes was one. Because the number of excess events exceeded the number of prevented myocardial infarctions, the USPSTF recommended against the use of aspirin in men at 5% risk for myocardial infarction and an age of 60 to 69 years. Based on observational studies, the USPSTF assumed different risks for gastrointestinal bleeding with aspirin according to age. Finally, the USPSTF presented their benefit-harm assessment and the resulting treatment thresholds as a matrix table with categories for age and risk for myocardial infarction defining each cell.

Three (15%) of the 20 CPGs qualitatively weighed the treatment benefits and harms [23, 29, 32]. Nine (45%) of the 20 CPGs made the recommendation on thresholds based on expert consensus or referred to other guidelines [18, 19, 21, 22, 26‐28, 33, 33]. Seven (35%) of the 20 CPGs did not report how they determined the treatment thresholds when making recommendations [10, 20, 24, 30, 31, 34, 36]. With regard to involving patient preferences when developing treatment recommendations, only three (15%) of the 20 CPGs explicitly reported that they considered patient preferences in the process (Table 2) [25, 30, 36]. For example, the USPSTF focused on major benefit (myocardial infarction) and harm events (gastrointestinal bleeding and hemorrhagic stroke) and assumed equal preferences (that is, importance) for those outcomes.