1 Introduction
Simulation modelling is a useful tool in health economic evaluation, especially for interventions targeted at treating chronic diseases such as diabetes mellitus. Clinical trials of new therapies and behavioural interventions in type 2 diabetes often involve estimation on changes in intermediate-risk factors such as glycosylated haemoglobin (HbA
1c) [
1,
2]. For these interventions to be evaluated using common health economic outcomes such as quality-adjusted life-years (QALYs), the potential benefits of observed improvements in metabolic control need to be extrapolated over longer time periods to capture impacts on the rates of complications of diabetes as well as mortality.
There is a long history in the use of simulation models to evaluate interventions for people with type 2 diabetes. The first simulation model for type 2 diabetes was built in 1997 by Eastman et al. [
3,
4], which was based on clinical data from both type 2 and type 1 diabetes. Many models have been developed since then [
5], and while many are based on the United Kingdom Prospective Diabetes Study (UKPDS) Outcomes Model [
6], there are differences in terms of model structure, particularly when they incorporate additional epidemiological and trial evidence to capture additional complications (e.g. impact of hypoglycaemia). A common feature of all these models is that they extrapolate changes in intermediate outcomes such as HbA
1c to metrics such as QALYs or life expectancy (LE), which are most commonly used to capture outcomes in economic evaluations.
While most diabetes simulation models have been developed independently, the field has benefited from the diabetes simulation modelling conference ‘Mount Hood Diabetes Challenge’, which has been regularly held since 2000 to compare and contrast the outputs of models in a set series of simulations [
7]. The last Mount Hood Diabetes Challenge meeting was held in 2014 and developers of 11 type 2 diabetes models participated, highlighting the development of simulation modelling in type 2 diabetes in recent years [
8]. Diabetes simulation models are now widely used in cost-effectiveness studies and play an important role in defining clinical guidelines and the evaluation of new drugs [
9].
Former reviews in type 2 diabetes cost-effectiveness studies have focused on summarising and describing the characteristics of different models [
5,
10]. Currently, there are no studies that use quantitative techniques to evaluate the relationship between treatment-effect assumptions (e.g. a reduction in the level of HbA
1c) and simulated outcomes (e.g. a gain in QALYs or LE). In this article, we systematically review cost-effectiveness studies of glycaemic control interventions that use type 2 diabetes simulation models to estimate long-term outcomes of QALYs or LE. Using data extracted from those studies, we use regression analysis to estimate the relationship between initial changes in HbA
1c following the commencement of an intervention and model outputs. The objective of this research is to explore whether there is a consistent relationship between a widely reported intermediate outcome that is often used as an input in simulation models and model outputs across type 2 diabetes cost-effectiveness studies, and across different models.
4 Discussion
This study used regression analysis to estimate the association between changes in risk factors (e.g. HbA1c), which are common inputs for simulation models, and the estimated outcomes. The analysis is based on data from 76 studies obtained through a systematic review of published cost-effectiveness studies of blood glucose-lowering interventions for people with type 2 diabetes. Based on multiple linear regression that adjusted for a variety of metabolic risk factors, it found that the marginal effect of a 1% HbA1c decrease could result in life-time increases in QALYs and LE of 0.371 and 0.642, respectively. There was no evidence of heterogeneity between models. Studies reporting small differences in HbA1c tend to report larger gains for QALYs than LE, which implies that when the treatment effect on HbA1c is limited, the increase in QALYs mainly comes from utility gain, rather than longer life-years.
In this study, we mainly focus on the treatment effect of changes in HbA
1c, as it is used in almost all analyses of blood glucose control interventions and is an input for all diabetes simulation models. Our review result suggests that recent economic evaluations of blood glucose-lowering interventions now use a wider variety of risk factors. For example, hypoglycaemic events, which have been incorporated into several models (CORE, Cardiff and ECHO) as an outcome of interest, were not captured in economic evaluations prior to 2006, but influence the outcomes of over 60% of the model simulations of blood glucose lowering in published studies during 2013–15 (Figure S1 in the Supplemental Material). We included these treatment effects into our multivariable regressions and found a significant relationship between them and model outputs as well. Furthermore, the relationship between the ratio of increase in QALYs and LE and a difference in HbA
1c could be explained by the treatment effect on BMI and hypoglycaemia events. In addition to the impact on complications and death, the decrease in BMI and avoidance in adverse events themselves could also increase people’s quality of life [
34] and appear to play a significant role in the outcomes of some recent economic evaluations of blood glucose control therapies [
35,
36].
There are several potential practical applications of the relationship between a change in initial HbA1c and model outcomes found in this study. First, it can be used as a diagnostic tool or benchmark for decision makers, enabling them to identify analyses that deviate from the general trend and investigate whether there are other factors that may have led to the discrepancy and whether they are reasonable. Second, with limited information and resources to run a diabetes simulation model, the regression estimated in this study can be used to give a rough prediction of the long-term effectiveness that could be expected from an intervention in its early stages. In addition, beyond the specific results, this study provides a potential methodology for a meta-analytic approach to combining the results of cost-effectiveness studies based on simulated outcomes in the future.
This study has also highlighted inconsistencies in the reporting of assumptions regarding the treatment effect and in results of model simulations. To make cost-effectiveness simulation results transparent, the effect of treatment on all major risk factors should be reported over time. Currently, there is no standard way of reporting assumptions regarding the duration of an intervention effect on risk factors (e.g. some studies report an annual decay [
37], while others a change at some future time [
26]). The lack of consistent reporting has made it hard for us to incorporate this information into our regressions at this stage, thus we have been limited to using the initial change in HbA
1c and other risk factors in our regression models. Three studies were excluded because the initial change in risk factors was not clearly reported. However, the main issue with the reporting of outcomes was that 40% of studies included in this analysis did not report their undiscounted results. Studies from different countries usually use different discount rates for their base case results, which makes the comparison between studies difficult to conduct. To address this issue, we calculated an average ratio between discounted and undiscounted ∆QALYs and used this to infer undiscounted outcomes when these were not reported. Basic cohort characteristics (age, duration of diabetes, proportion of male, ethnicity and other baseline risk factors) and model assumptions (time horizon) were also sometimes not reported. In this analysis, six studies were excluded from the multivariable regression because of a lack of information on age, diabetes duration or post-treatment HbA
1c level in the control group [
18,
38‐
42]. There is a clear need for general reporting standards of diabetes cost-effectiveness studies to be developed to promote transparency and facilitate future model comparisons. A starting point for this is the use of the Consolidated Health Economic Evaluation Reporting Standards [
43].
This study is subject to a number of limitations. First, the difference in HbA1c collected in this study is the initial difference between two simulated cohorts, which is often the only measure of glycaemia that many simulation modelling studies included. While we note some simulation models make additional assumptions about the relative trajectory of HbA1c, most economic evaluations of diabetes therapies do not report these in a uniform manner and with insufficient details to be incorporated into the current analysis. While there is a strong association between initial HbA1c and long-term outcomes, the consistency of regression relationship depends on independence of the initial HbA1c and the error term. It would be useful to re-examine this assumption in future work, particularly as the transparency of reporting of simulation models improves over time.
Second, a lack of statistical power (124 pair of comparators) meant that we were unable to include many variables in the multivariable regression. Although other factors such as sex percentage, ethnicity and baseline values for other risk factors were also collected they were not included. Further, the limited number of studies using simulation models other than CORE meant we were unable to investigate the consistency between models separately. We found no evidence of heterogeneity between models by comparing studies that used the CORE model and studies that used any of the other type 2 diabetes simulation models. Again, there is scope for future work to include more control variables and further explore the consistency between models when more studies are available.
Finally, we have not taken account of reported uncertainty surrounding model estimates because of the limitations in the reporting of these measures in published studies. A future analysis could focus on comparisons of uncertainty by different simulation models.