Validation of a new predictive risk model: measuring the impact of the major modifiable risks of death for patients and populations

The risk model computes an individual’s total risk of mortality over the next 10 years based on exposure to 12 major risk factors (Table 1) for adults aged 30 years or older. These risk factors were included based on reviews of the scientific literature and represent a parsimonious set of the most substantial, modifiable risk factors that contribute to the probability of dying [2]. All risk factors selected had to be (a) actionable: subject to modification by clinical or behavioral interventions, (b) substantial: contribute at least 0.20 years to mortality risk, and (c) evidence-based: supported by recent meta-analyses [9]. We envisioned that the full survey instrument (provided in Additional file 1) could be completed in multiple settings, ranging from clinical visits to online surveys.

While the need for accurate and actionable information on modifiable health risks is well recognized, the risk models currently available have important limitations, largely because they either focus on a specific, important cause of death [24] or because they are based on proprietary algorithms that may not be based on the most up-to-date evidence and have not been validated in the general US population. Individuals, clinicians, and others therefore currently lack a broadly available, evidence-based tool that would support more accurate identification of risks, assessment and comparison of the potential impact for individuals of different risk-modification strategies, or identification of high-risk subgroups of the population. The risk model described and validated here attempts to overcome these limitations.

We developed a new risk model based on 12 major behavioral and biometric risks to health that predict an individual’s probability of dying over the next 10 years compared to a population with an optimum distribution of risk factors. The model is based on current scientific evidence on risks to health and contemporaneous, high-quality data on the distribution of risk exposures and mortality rates in the US population. The model has excellent discrimination in a sample of the US population [23]. The excellent and widely used Framingham Index [24‐26] typically has areas under the curve of between 0.75 and 0.80 for predicting cardiovascular endpoints only [27]. This newly developed model has areas under the curve of greater than 0.80 for both men and women for the more difficult task of predicting mortality from any cause and is constructed using the best available evidence on the major modifiable causes of mortality.

The risk model offers promise as a tool to support individual decision-making. A recent systematic review demonstrates the benefits of providing cardiovascular risk information to individuals for discussion with their families and physicians [28]. The results suggest that if individuals understand the magnitude of their risk, they are more likely to adopt or maintain healthy behaviors. The review also found that information on overall risk is likely to have a greater impact when it is paired directly with education or counseling. As illustrated in Additional file 3, this risk model can facilitate counseling and decision-making by providing a systematic way for patients and clinicians to compare how different behavioral or clinical interventions would likely influence risk. For example, the importance of smoking cessation for most smokers becomes obvious. The risk model can show patients how they might avoid medication use by increasing physical activity or reducing weight, or conversely, the lost benefits of failing to take prescribed medications. The risk model also takes into account the effect of behavioral modification on other biological risk factors included in the risk score, e.g., the effect of reductions in body weight on blood pressure. Presenting accurate, holistic, and balanced information about the risks that patients face, in conjunction with counseling about the importance of different options for change, could help align decision-making with patients’ preferences – an important national aim [28]. Implementing the model in a way that provides an attractive and accessible tool for the general population, e.g., through Internet or phone-based applications, would also provide a way for individuals to self-assess their risk and encourage contacts with healthcare providers to decrease risk.

Broad adoption of the risk model could offer other important benefits. First, clinical practices, health systems, and workplace health programs that obtain completed surveys from their populations can accurately stratify people according to their level of risk and develop epidemiologically informed programs to reduce risks. In the highest-risk subpopulation, individualized multiple-risk-factor interventions, such as case management and health coaching, could be a wise investment [29]. Second, the measure could contribute to clinical and public health research by providing a validated composite endpoint for clinical trials of multiple-risk-factor intervention programs. Third, the measure offers a potential improvement over current quality indicators that focus largely on intermediate outcomes (e.g., levels of blood sugar, blood pressure, and cholesterol) that have little intuitive meaning to patients and encourage well-recognized hazards among providers. For example, if the proportion of diabetics with well-controlled HbA1c is used as a quality measure, some may be encouraged to label those with pre-diabetes as diabetic (inflating the denominator), and treatment efforts may be focused on those with mild disease, in whom achieving target levels is easier but less important. Performance measured on the basis of improvement in predicted risk could give greater credit to meaningful progress for those at greatest risk, even if specified targets were not achieved. In addition, the risk measure would in all likelihood be more parsimonious (i.e., one broad measure of risk status versus many narrow measures) [30]. Finally, if broadly adopted within specific geographic regions and mapped in ways that preserve confidentiality, the measure could provide a basis for collaboration among providers and community stakeholders on initiatives to improve population health.

At the same time, the risk model has important limitations. We made judgments about which risks to include. For example, we did not include depression, believing that screening and intervention (especially for those at risk of suicide) has a different and more pressing time horizon, and we judged trans fats to be akin to an environmental risk that is difficult for an individual to control or modify. The risk model is based on the average American, and, although differential risk exposure explains a large fraction of geographic, racial, and socioeconomic factors, there are likely to be residual differences in the underlying risk of death between these groups. Although the proportional hazards of risk exposure are largely generalizable across different populations, [31] several risk factors are based on self-report, and these responses may not be accurate or comparable across populations. While all included risks have strong evidence as predictors of mortality, the strength of evidence for risk factor modification on reduction in mortality differs across risks, with a greater body of evidence available for risk factors such as high blood pressure. Related to this, some of the evidence for these risk factors is based on observational studies, which are prone to potential confounding. Although we split the development and validation datasets, our use of the same broad data source (NHANES) for the analysis may have caused us to overestimate the predictive validity. The risk score appears to be mainly driven by cardio-metabolic factors and has not yet been validated either for specific clinical populations (e.g., cancer, arthritis, dementia) or for different racial, ethnic, or socioeconomic groups. The risk score also does not include a prediction of morbidity, which is an important consideration particularly among older-aged individuals, nor does it presently include a quantification of the potential adverse effects, for example of medication, to reduce exposure to risk factors. It will be important for patients and clinicians to understand these limitations, both about the relative magnitude of specific risks and the potential benefits of risk factor modification and to conduct further validation studies and to update the model as new evidence accumulates. Finally, while the focus of the use of the risk score presented in this paper is on individual-level modification, this should also be balanced against population-wide approaches for reducing risk exposure.

Some of these limitations can be addressed or mitigated by implementing the risk model in diverse practices and populations and by linking respondents’ original and subsequent risk factor scores to data on survival. This would facilitate continued improvement and validation of the model. This will be increasingly possible as electronic health record-enabled environments are adopted for large patient populations [32].

The need to balance downstream treatment of diseases with upstream prevention is well recognized [6, 17, 29]. Legislative and other upstream actions are now a cornerstone of risk reduction and in some countries they play a key role in reducing non-communicable diseases. Patients, clinicians, and employers are faced with a wide array of commercial health risk appraisal tools that aim to catalyze prevention and health promotion. Incorporating a standardized, validated, freely available, transparent, and continuously refined method to measure, summarize, and track the most important modifiable risks of death within these tools would offer benefits to patients, clinicians, and policymakers.