Coronary heart disease policy models: a systematic review

Improving population health through effective interventions remains a fundamental challenge for policy makers. Decision-makers at the population and individual levels each need to choose the 'best intervention' for a specific health problem. However, limitations on resources, time and information can make this decision process difficult. This is particularly true for cardiovascular disease, its diversity of manifestations and wealth of effective interventions are potentially complex and confusing. Assessing the potential population benefit of a health intervention requires consideration of many elements including disease prevalence and population characteristics, effectiveness and cost [1]. Clinical trials will never provide all the answers, particularly since their study groups are restricted with inclusion and exclusion criteria; therefore generalisation is always an issue [2]. Weinstein has usefully defined a model as, "a logical mathematical framework that permits the integration of facts and values to produce outcomes of interest to clinicians and decision makers" or, alternatively as: "an analytical methodology that accounts for events over time and across populations based on data drawn from primary or secondary sources"[3].

Models have been increasingly used in policy making and resource allocation, because they permit policy makers to simulate the effects of different scenarios within a population[4] and hence examine future policy options.

An appropriate model for coronary heart disease (CHD) could thus potentially allow users to simultaneously consider all the key issues when evaluating diverse options for intervention.

A remarkably wide variety of CHD policy models exist. Some simply consider risk factors alone[5], while others include selected cardiovascular treatments[1, 6, 7] secondary prevention such as cholesterol lowering treatment[8] or estimates of general practice workload [9]. However, model construction and development is complex and difficult. Thus, few CHD models have attempted a comprehensive consideration of all standard treatments and all major risk factors[10]. Furthermore, the variable quality and utility of different models may not always be appreciated.

In this paper we have systematically reviewed and evaluated the strengths and limitations of existing CHD policy models. Critiquing and providing a comprehensive overview of all CHD models is a challenging task. Although it is useful to have a broad summary of the similarities and differences of models, it is virtually impossible to capture the complexity and subtlety of a range of models in a single paper. However, this paper represents our best efforts, and will hopefully serve as an overview and introduction to this complex field. It will also provide sources and routes to further information on specific models.

For this systematic review, we defined a CHD policy model as any mathematical tool that may help to explain or predict the outcome of CHD interventions: specific treatments or cardiovascular risk factor changes or the implementation of a new strategy at a population level.

From 4,531 initial papers, a total of 75 articles describing 42 different CHD policy models were finally included (Figure 1). Each was originally used to address one or more specific health policy questions and all were based on large populations. Due to space restrictions, we present here details of the six principal CHD policy models, which generated more than one publication that selected for this review (Table 1). Critical appraisals of all 75 papers and all 42 models (including 6 models presented here) are provided in Appendix 3 [see Additional file 1].

This is the first comprehensive systematic review of CHD policy models. Previous reviews were restricted to a particular type or application [44‐46]. The increasingly wide use of CHD modelling has thus far resulted in few attempts to evaluate model quality. We therefore aimed to systematically assess the quality of the modelling methodology rather than simply comment on the reported results.

A wide variety of CHD policy models have been developed with over 70 publications now available. CHD models have become more complex and comprehensive as a result of improving computer technology and wider usage[10]. Although, on the surface many of the models may appear similar, one must remember that they were often originally designed to address very different policy questions.

In general, the quality of the models has also improved over time so that more recent papers tend to explicitly report assumptions, limitations and sensitivity analyses.

Quality criteria assessing for publications are well described, especially for randomised controlled studies[47]. However, there are no widely accepted quality criteria for modelling papers in general nor specifically for CHD policy models. We therefore first had to develop an evaluation framework using logical quality criteria including sensitivity analyses, validity, and comprehensive reporting of assumptions and limitations (Appendix 3) [see Additional file 1]. These criteria explicitly reflect the main quality components suggested in the recent ISPOR Guideline [3].

Models can allow a large amount of evidence to be considered simultaneously, by combining and integrating into a coherent whole different types of data from controlled trials, routine surveillance and expert consensus[10]. Models have been extensively used in policy making and resource allocation, since they permit policy makers to examine future options, or to simulate the effects of different scenarios within a population [4]. However, improved technology potentially increasingly enables practitioners and policy-makers to use these models without necessarily understanding the inherent assumptions or data limitations [10].

Models require considerable data input. The data sources therefore need to be appropriate and credible. However, the availability of comprehensive high quality data remains a problem. The data are therefore usually obtained from a variety of sources including clinical trials, meta-analyses, surveys, clinical databases and registries, medical records, audits, routine statistics, official tariff lists for health care resource use and Delphi panels (to elicit expert opinion)[48]. Every CHD modelling paper should therefore explicitly report and critically discuss data quality, methodological limitations and the assumptions used to address these deficiencies. However, few of the papers reviewed here did so.

Uncertainties about data are a perennial problem in modelling studies. Sensitivity analyses are therefore essential to quantify the degree of uncertainty. In general, CHD models have only recently started to report sensitivity analyses [14, 40, 49]. The most common approach is where one or more parameters of an evaluation are varied across a plausible range[50]; 95% confidence intervals can easily be included. One-way sensitivity analysis(where only one parameter is changed at a time while the others retain their base-case specifications) is obviously less rigorous than multiway-sensitivity analysis(where more than one parameter is changed at the same time). However, multiway sensitivity analyses remain very uncommon [6, 10, 37]. Lately, the use of probabilistic sensitivity analysis has been suggested alongside with traditional sensitivity analyses. Probabilistic sensitivity analyses offer the opportunity to make statistical statements about the impact of parameter uncertainty in models. Although, this approach has been criticized for its arbitrariness in choosing the statistical distribution it may be used more often in the future [51]. Probabilistic sensitivity analysis was not used in any of the reviewed models in this paper.

In conclusion, CHD models offer a potentially valuable tool for policy development. However, existing models vary widely in their depth, breadth and quality. Few models have been calibrated, replicated or validated against minimum quality criteria. Before being accepted as a policy aid, any model should explicitly include a statement of its aims, assumptions, outputs, strengths and limitations.