PRIMEtime CE: a multistate life table model for estimating the cost-effectiveness of interventions affecting diet and physical activity

There are well established methods for modelling the cost-effectiveness of healthcare interventions. For example, the International Society for Pharmacoeconomics and Outcomes Research-Society for Medical Decision Making (ISPOR-SMDM) guidelines for modelling research have published multiple best practice guidelines [12]. Furthermore, The National Institute for Health and Care Excellence (NICE) has guidance on methods for health economic modelling [13]. The ISPOR-SMDM guidelines and NICE guidance aim to standardise methods for health technology assessments (HTAs) so that the results from different studies can be directly compared with each other against a decision framework, such as the NICE cost effectiveness threshold of £20,000 to £30,000 [13]. However, economic evaluations of public health interventions have their own specific challenges compared with HTAs such as quantifying long term outcomes, wider societal consequences and the impact on inequalities, and the effects of multicomponent interventions [14‐24].

In 2005, NICE started producing guidance on public health interventions, including economic modelling [13, 25, 26]. The NICE reference case now includes advice on how to address some of the challenges of public health economic evaluations: for example, the time horizon should be long enough to incorporate all important costs and effects; the perspective on costs may be public sector, societal, or any other as appropriate; and non-health benefits to local government and other settings may also be included [13]. However, the guidance is not prescriptive, meaning that different public health economic modellers often use different model structures, time horizons, health and economic perspectives, epidemiological data, and outcome measures [24]. Even within academic fields with a well-established history of health economic modelling, such as diabetes, chronic obstructive pulmonary disease (COPD), and cancer, different model structures and assumptions can produce very different outcomes despite modelling the same intervention [27‐29]. Therefore, often it is not possible to compare results directly when prioritising different public health policies. As such, standardised processes are required for assessing and modelling the cost, health impact, and cost-effectiveness of public health interventions.

In this paper we describe the PRIMEtime Cost Effectiveness (PRIMEtime CE) model. PRIMEtime CE addresses some of the challenges outlined above by being able to directly compare the cost-effectiveness of public health policies aimed at improving population diet or levels of physical activity.

The PRIMEtime model is a multistate life table model which quantifies the effect of changing 14 dietary risk factors (the 10 risk factors included in PRIME plus processed meat, red meat, free sugars, and fibre from cereals) on morbidity and mortality from ischaemic heart disease (IHD), type two diabetes, stroke, seven cancer subtypes (breast, lung, colorectal, stomach, liver, kidney, and pancreas), and liver cirrhosis in the UK population [31]. The model combines the dietary risk factors included in the PRIME model [30] with multistate life table methods developed by Cobiac and colleagues in Australia and New Zealand [32‐34]. It simulates a closed adult population cohort (aged 15 and above) by single year of age and by sex over the lifetime of the cohort or until individuals reach 100 years of age. It uses UK specific data by age and sex (where available) on baseline disease incidence, prevalence, case-fatality rates (the annual mortality rate among prevalent cases), and disease trends.

As with PRIME, the relationships between diet and disease are parameterised using meta-analyses of randomised controlled trials or prospective observational studies and are modelled either as direct effects or via one of three intermediate risk factors: blood pressure, BMI, and total cholesterol. PRIMEtime is built in Microsoft Excel and uses Ersatz and EpiGearXL add-ins from EpiGear International to run Monte Carlo probabilistic sensitivity analyses and quantify the uncertainty in model input data [35‐37]. The 2.5th and 97.5th percentiles of multiple model runs are used to estimate 95% uncertainty intervals (UIs, usually 2000 runs by which point uncertainty ranges have converged on a stable value). Tables 1, 2, 3, 4, 5 and 6 show the sources of input data and uncertainty distributions included in PRIMEtime and PRIMEtime CE.

PRIMEtime CE is developed from PRIMEtime by adding healthcare and social care costs, estimates of morbidity based on age, sex, and disease state, and physical activity as a behavioural risk factor. The relationship between free sugars and total cholesterol is currently not included in PRIMEtime CE, although this could be added.

The conceptual modelling framework for public health economic models published by Squires’ et al. was used to guide the development of a conceptual model [81]. The initial conceptual model was shared with multiple stakeholders who provided feedback on the proposed model outcomes, the relationships described, and its face validity. Stakeholders included national governmental organizations, local governmental organizations, charitable organizations, health professional and academic organizations, and patients and public (see Table 7). The final conceptual model is shown in Fig. 1.

Population physical activity levels by age and sex are taken from the Active People Survey, 2010–2011 (APS) [68]. The APS collects information on sport participation rates from a representative English population sample and includes the degree of participation in a variety of activities. Physical activity data from APS were used to calculate the total number of metabolic equivalent of task (MET) minutes per week for each respondent. A MET is defined as consuming 3.5 ml of oxygen per kg of body weight per minute, the resting metabolic rate. Levels of physical exertion can be measured in relation to this resting metabolic rate, for example running at seven miles per hour is equivalent to 11 METs [82]. The APS includes data for 166,275 adults aged 16 years and over after outliers reporting over 200 MET hours/week were removed.

The relationships between physical activity and IHD, stroke, type two diabetes, breast cancer, and colorectal cancer are included in PRIMEtime CE. The beta-coefficient describing the dose-response relationship between physical activity and disease was derived using two recent meta-analyses of observational studies, both conducted by the same research group using the same methodology [69, 70]. The parameters are unadjusted for obesity as this is assumed to act on the causal pathway (the effect of physical activity on disease is modelled directly rather than via BMI).

Each age and sex group in the population is divided into four categories: sedentary (zero minutes of moderate physical activity per week), under active (less than 60 min of moderate physical activity per week but not sedentary), active (60–150 min per week), and recommended (meeting the UK Chief Medical Officers’ recommendations of 150 min of moderate physical activity per week or more) [83]. Examples of moderate physical activity include brisk walking and cycling. The mean and standard error MET hours/week is calculated from APS for each age and sex group in each physical activity category. These can then be applied to results from the meta-analyses to estimate the relative risk of disease compared to being sedentary. These relative risks are used in PRIMEtime CE to predict the change in disease burden following an intervention.

Furthermore, in PRIMEtime CE, decreasing the prevalence of type two diabetes reduces the risk of the modelled cohort developing IHD and stroke. Using the relationships describing the effect of physical activity on IHD, stroke, and diabetes derived using Wahid et al. would over-estimate the effect of physical activity on IHD and stroke as falls in diabetes would result in additional CVD reductions beyond those estimated in the meta-analyses. Therefore, the relative risk of physical activity on IHD and stroke from Wahid et al. was adjusted downwards using results from Cobiac and Scarborough such that the overall effect of physical activity on CVD is the sum of the direct effect plus that mediated via diabetes [84].

Uncertainty in the size of the relationship between physical activity and disease (the beta-coefficient) was estimated from 95% confidence intervals reported in the meta-analyses. Both this uncertainty and uncertainty in the mean physical activity level for each age and sex group based on the standard error reported in APS are included in PRIMEtime CE.

A limitation of this method is that Wahid et al. include domains of physical activity such as occupational activity and household activity that are not captured in the APS. Therefore, the disease relative risks used by PRIMEtime CE are based on higher levels of physical activity than those reported by the APS, where only leisure time physical activity is recorded. As a result, the overall health benefit modelled by PRIMEtime CE may be an over-estimate because the effect of physical activity on health is non-linear with greater health benefits accruing from gains at lower baseline physical activity levels.

Annual disease specific NHS England costs per prevalent case that are comparable between different diseases are derived from 2013/14 programme budgeting data, [77] detailed methods and model costs can be found in Briggs et al [85]. In summary, a macro top-down approach was taken where total NHS England expenditure in 2013/14 is disaggregated to the level of ICD-10 code for diseases included in PRIMEtime CE. Total disease specific costs were divided by the estimated 2014 prevalence of each individual disease in PRIMEtime CE to give the annual cost per prevalent case, averaged across the entire population irrespective of time since diagnosis. The remaining NHS England budget not accounted for by diseases included in PRIMEtime CE was used to calculate the age and sex specific annual healthcare spend on diseases unrelated to those explicitly modelled by PRIMEtime CE that accrues as people age. A summary of included model costs is shown in Table 8.

Social care costs are estimated using the wider societal costs tool developed by the Department of Health [78]. The tool estimates the age and sex specific effect on production (paid and unpaid) and consumption following a change in quality of life (quantified using utility values measured by the EQ-5D-3L questionnaire) and ICD-10 code. Consumption includes social care (described as formal care by the tool), informal care (care provided by family and friends), private paid (goods and services purchased for consumption), private unpaid (benefit from goods and services not paid for, such as domestic work), and government (services provided by the government not included in other categories). Social care costs are estimated as a function of age and quality of life, in PRIMEtime CE they are included from the age of 75 years (as per the Department of Health tool) and assume an average monthly cost of £4826 per person (the 2013/14 monthly local authority residential care cost [87]) [79]. This reflects the total societal costs of adult social care rather than the direct costs to local authorities because many people are required to fund a proportion of their care from personal savings.[88]

In PRIMEtime CE, both the change in social care costs arising as a result of changes to modelled diseases and from unrelated diseases due to increasing longevity are included. Age and sex specific utility values for quality of life at baseline and for each disease are calculated using the methods described under ‘Estimating utilities’ below. Productivity gains and wider societal costs can be readily included as sensitivity analyses. A key assumption underlying the Department of Health wider societal costs tool is that quality of life is the key driver of costs, irrespective of diagnosis (stroke and dementia are exceptions to this). In reality this may underestimate the effect of certain diagnoses on an individual’s ability to care for themselves or to be productive.

Morbidity is estimated using Sullivan et al.’s catalogue of EQ-5D utility values [80, 89]. The EQ-5D results were collected by Sullivan et al. from annual US medical expenditure surveys between 2000 and 2003, and included 79,522 unique responses with linked clinical data for the preceding year covering 135 ICD-9 codes; an equivalent representative UK source of EQ-5D results does not exist. The EQ-5D survey results were then valued using time trade off methods by a representative UK population sample (as recommended by NICE) [90].

Matching the ICD-9 disease codes published by Sullivan et al. to ICD-10 codes used by PRIMEtime CE was done with the website www.​icd10data.​com [91]. The calculation of baseline utility values by age and sex, and disease specific utility decrements followed guidance published by Sullivan et al. in their UK and US papers [80, 92]. To use these utility values to estimate the change in quality adjusted life years (QALYs) in PRIMEtime CE following an intervention, the age and sex specific utility values change with age based on number of years that the model has been running for, and on changes to the proportion of the population in different disease states. If thought important, as with other input parameters to PRIMEtime CE, it is possible to change the utility decrements used within the model either for a primary analysis or for sensitivity analyses. Baseline age and sex specific utility values and the disease specific utility decrements used in PRIMEtime CE are presented in Tables 9 and 10.

The additional change in utility decrement based on an individual’s number of chronic conditions (as estimated by Sullivan et al.) was not included in PRIMEtime CE because of the inability to accurately estimate the baseline prevalence of co-morbidity and how it changes following an intervention. To test the impact of this limitation on results, the model was run firstly using a theoretical scenario without any decrement arising from co-morbidity, and secondly, assuming that everybody had the maximum possible additional decrement. Results were not significantly different from one another.

Sullivan et al. EQ-5D scores are sampled from a non-English patient population survey and there is likely to be underrepresentation of those at very early and late stages of disease as these population groups may be too unwell or unwilling to participate. In order to identify whether utility values published in the Sullivan et al. catalogue were similar to those identified in other populations using different methods, we compared Sullivan et al. with EQ-5D derived breast cancer utility values that were systematically identified from the literature. We used a pre-defined protocol following NICE guidance (see Additional file 1 for protocol) [90].

The systematic review identified 196 studies for full text review from which 23 studies were included for data-extraction. Four studies received the joint highest study quality and applicability score (see Additional file 1 for scoring method) and the extracted utility values from these four studies are shown in the Additional file 1: Table S1. The utility values from each of these four studies overlap with the mean utility value among breast cancer patients reported by Sullivan et al [80].

Interventions affecting any of the risk factors included in PRIMEtime CE can be modelled based on the intervention in question. Examples of this and a discussion of the model’s validation can be found in Briggs et al [93].

Following stakeholder feedback, the primary outcomes of PRIMEtime CE are cost-effectiveness and return on investment (for interventions that are cost-saving) from an English health and social care perspective over a 10-year time horizon. Cost-effectiveness is calculated as: (C_b – C_a) / (E_b – E_a); where C_b is the sum of intervention costs and expenditure on health and social care in the 10 years following the intervention; C_a is the 10 year costs of health and social care in the scenario where there is no intervention; E_b is the total number of QALYs experienced by the modelled population in the 10 years following the intervention; and E_a is the same but where no intervention is modelled. In the case where (C_b – C_a) is negative and therefore the intervention is cost saving compared with no intervention, return on investment is the money saved for every £1 spent: (C_b – C_a) / C_i; where C_i is the cost of the intervention. Included are all costs of the intervention, whether they are incurred by government organisations such as the NHS and local authorities, or by industry where appropriate, as these costs may be relevant to decision makers. Costs and health outcomes are discounted at 1.5% as recommended by NICE for interventions likely to have long-term health benefits [13] (see Table 11). However, there is flexibility within the PRIMEtime CE model to manipulate all of these parameters to either change the model’s primary outcomes, or as sensitivity analyses (see Table 12 for a list of potential PRIMEtime CE sensitivity analyses).

Sensitivity analyses can be used with PRIMEtime CE to explore the effects on cost-effectiveness of changing various assumptions including time horizon, discount rate, and removing industry related costs (Table 9).

PRIMEtime CE addresses some of the challenges of public health economic modelling, namely quantifying long term health outcomes, wider societal consequences, and the effects of multicomponent interventions [15, 17, 23].

Multistate life table models are well suited to modelling long terms health impacts and PRIMEtime CE can model outcomes over a population’s lifetime. Furthermore, both health and social care costs are included, thereby incorporating some wider societal consequences of the interventions modelled. There is a broader debate in the literature about how best to value outcomes from public health interventions where they may have societal and environmental costs and benefits beyond health that are not quantified using cost-utility analyses [14‐18, 20, 81, 94, 95]. However, using cost utility analyses and reporting the cost per QALY is a useful method for health policy makers in England because it is consistent with the NICE approach to assessing medical interventions and - as things stand - is therefore relatively easy to interpret and compare with other health economic assessments [13].

A limitation of multistate life table models (and cohort models more generally) is that they are less flexible than some other modelling approaches (for example, using microsimulation) at modelling heterogeneous populations and quantifying the impact of an intervention on inequalities. In order to obtain results by population subgroup using PRIMEtime CE, the user would need to conduct multiple runs, each with a different population subgroup and using subgroup specific model input parameters.

Additionally, PRIMEtime CE does not fully address the challenges of modelling multicomponent interventions, for example, an intervention including both social marketing and legislative changes. Some more complex model structures, such as system dynamics models and discrete event simulation, might be better suited to this as they can simulate interactions between population subgroups, or between the population and the environment [24]. However, multistate life tables do allow for additional disease outcomes to be included without having to construct a new model meaning that they can be readily adapted if multicomponent interventions include diseases that are not already simulated. And PRIMEtime CE allows for the effect of an intervention on multiple different dietary and physical activity risk factors to be modelled simultaneously.

It is intended that PRIMEtime CE will be used to produce comparable cost-effectiveness estimates of different public health policies, including ranking interventions, to help decision makers prioritise resources. Similar rankings have been compiled in the UK and elsewhere; for example, the NICE physical activity return on investment tool, [96, 97] the ACE Prevention programme of research in Australia, [98] and the BODE³ Programme in New Zealand [99]. The NICE physical activity return on investment tool compares how different physical activity interventions would affect health and costs for a population of interest. The tool is based on a Markov model that estimates how changes to levels of physical activity affect the prevalence of CHD, stroke, and type two diabetes. The tool has a user-friendly interface whereby various model inputs can be manipulated and the outputs analysed. Unlike PRIMEtime CE, however, the disease costs and utilities used by the model are from a variety of different sources and population groups.

The research programmes in Australia and New Zealand both used systematic and comparable approaches to estimating the cost-effectiveness of different public health interventions, with the explicit aim of informing policy. As with PRIMEtime CE, these were developed using country specific routine data where possible, with standardised approaches to estimating utilities and costs. The BODE³ programme of research derived age, sex, and disease specific health costs from Health Tracker - national data that individually links costs with health events, [100] and New Zealand specific disability weights are taken from the Global Burden of Disease study [101]. The ACE programme used the DISMOD tool and data from the Netherlands to estimate disability weights based on Australian burden of disease data (see Begg et al. for details [102]), and costs were taken from a national dataset with disease specific healthcare cost estimates [103]. The use of country specific data sources mean that although results from the ACE and BODE³ programmes of research may not be directly applicable to England, both can rank interventions.

PRIMEtime CE has several strengths. Bias is minimised by adopting a consistent and systematic approach to identifying model input parameters, in particular disease costs and utilities; model uncertainty is quantified; there are options for multiple sensitivity analyses; and changes to costs and quality of life from diseases unrelated to those modelled can be estimated.

There are some important methodological limitations of both the data sources used by PRIMEtime CE, and the model’s structure. Firstly, multistate life table models assume diseases are independent of one another. This means that the proportion of the cohort existing in one disease state does not affect the probability of the cohort developing any other disease states - the model cannot distinguish between whether the population with breast cancer is the same as that with heart disease. Multistate life table models may therefore over- or under-estimate utilities and costs due to differences occurring between two diseases existing in two separate individuals or being co-morbid within the same individual. To help counter this limitation, PRIMEtime CE has been developed such that there is a dynamic relationship between type two diabetes and cardiovascular disease (an increase in diabetes prevalence increases the risk of the modelled cohort developing IHD and stroke, and the relative risk of IHD and stroke incidence due to changes in BMI or physical activity is adjusted to prevent double counting due to a concomitant rise in diabetes). However, this adjustment does not occur for other diseases which, depending on the intervention simulated, may over- or under-estimate the resulting cost per QALY. Secondly, no interactions between individuals within the population or between the population and its environment are simulated. Including these interactions requires data on their effects and would generate more uncertainty in model results, but conversely it may also mean the model better represents reality and is more accurate. Thirdly, PRIMEtime CE only models the effect of changing a risk factor on disease incidence and not case fatality rates. If simulated interventions lead to reductions in case fatality rates, the model may under-estimate health and cost outcomes arising from increased longevity (both for modelled and unrelated diseases).

Further limitations are introduced through using NHS England programme budgeting data to estimate disease costs where miscoding of NHS England expenditure may mean that some annual disease costs are overestimated, and others underestimated. It is not possible to quantify any misallocation of costs, however the effect on outcomes will be limited by the fact that the majority - 78% - of the NHS England budget is accounted for through disease specific programme budget categories. Disease costs are also only estimated per prevalent case whereas in reality costs vary by time since diagnosis and proximity to end of life. This is a limitation of using a top-down method for estimating costs that may over- or under-estimate total costs, particularly where there are changes to case fatality rates. Further details on the limitations of the costing methods, as well as the PRIMEtime model and social care costing can be found elsewhere [31, 79, 85].

Finally, there are limitations of the utility decrements estimated by Sullivan et al [80]. In particular, EQ-5D data are sampled from a 2000–2003 US population rather than a more recent UK population; and individuals at very early or late stages of their disease are likely to be under-represented due to being too unwell or unwilling to participate meaning disease specific utility decrements might under-estimate the true average reduction in quality of life. The systematic review of breast cancer utilities provides some reassurance that the values reported by Sullivan et al. are reasonable estimates. Furthermore, baseline age and sex utility values are derived based on all diseases in the population rather than only unrelated diseases (those not explicitly modelled by PRIMEtime CE). Therefore, utility decrements that accrue due to increased longevity following an intervention may over-estimate the true utility decrement. However, the effect of this on model results is limited due to discounting and the relatively smaller proportion of the modelled cohort living to old age.

Future intended developments of PRIMEtime CE are to include further testing of the model’s validity [93] and to expand PRIMEtime CE’s scope. This may include consideration of how the model can be used to quantify the effects of policies on inequalities; the addition of further behavioural risk factors such as alcohol and tobacco, and the use of drugs in primary prevention; how the model can be made more user-friendly by the development of non-technical interface; and whether results can be quantified at the local authority level.