Decision modelling of non-pharmacological interventions for individuals with dementia: a systematic review of methodologies

A literature search was conducted on the 7th July 2017 on PubMed, Scopus, Science Direct, Cochrane, NHS EED, Embase, EconLit and Psychinfo databases. Detailed search strategy for each database is presented in Additional file 1. The search was limited to studies published in English. The search did not include studies published prior to 2000, for a number of reasons: there were few NPIs for dementia developed before the year 2000 [10], studies before 2000 were covered by broader reviews and do not reveal many decision models on NPIs [17‐19], and modelling techniques used prior to 2000 are likely to have been significantly improved and perfected [22].

Study eligibility was established on initial screening of title and abstract. Studies passing initial screening were subject to full text review against inclusion and exclusion criteria, outlined in Table 1. Studies were included if they modelled the onset and/or progression of dementia, and evaluated the cost-effectiveness of a NPI aimed at people at risk of or with dementia. All study designs were considered for review. Studies were excluded if they evaluated interventions aimed only at caregivers of patients with dementia, or if they only included economic evaluation alongside an RCT without additional modelling.

The nature of the intervention should, at least in part, dictate the structure of the model, and, therefore the necessary inputs such as transition probabilities, utilities, costs etc. [22]. Therefore, the analysis of studies selected for this review is focused on three categories of NPIs: primary, secondary and tertiary prevention, as defined previously. Primary prevention includes interventions aiming to preventative or delay onset of dementia, secondary prevention interventions include screening and early identification initiatives, whilst interventions in the tertiary prevention category focus on aiding the symptoms and/or slowing the progression of the disease after a diagnosis has been made. Key features of the included studies were summarised, with specific focus on model structure, outcome measures and characterisation of disease onset and progression. These were selected to reflect the most significant aspects of modelling dementia-related interventions [18, 23], with particular relevance to evaluation of NPIs.

Methodological approaches were described with a view to identify common challenges and possible improvements for future model-based economic evaluations of NPIs for dementia. This was done by examining the studies against two widely-used best-practice guidelines for decision modelling in economic evaluation [24, 25]. Data used in reviewed models were assessed for the relevance in terms of date, setting, sample size and method of collection. In cases where studies utilised secondary/published data, we reviewed the data at the cited source.

Structure of the model should be transparent and justified given the aim, scope and perspective of the model [25]. It should also be possible to reproduce the model from the technical information provided to the reader [24].

Of the ten reviewed studies, five [26‐29] used Markov modelling techniques to simulate dementia or AD progression. The others included a Monte-Carlo model [30], three decision trees [27, 31, 32] and a regression-based simulation model [33]. There was no evident pattern of model preference in primary, secondary or tertiary prevention studies. In all reviewed studies, little or no rationale was given for the choice of model, and, while model choices did appear appropriate for purpose, it is advisable to include a justification or rationale for the choice of methods [24, 25].

The state-transition Markov models utilised in the five studies were reasonably simple in structure, consisting either of three to four health states (‘non-demented/non-AD’, ‘Mild cognitive impairment’, ‘demented/AD’ and ‘dead’) or five to six states, where ‘demented/AD’ was further broken down into various degrees of severity. While maintaining a simple approach to modelling does require fewer data inputs and, therefore, also exposes the findings to fewer biases, it may also limit the accuracy of the model predictions. It is possible that the choice of health states included is most likely influenced by data availability. Four interventions utilised 12-month cycles [26, 27, 29, 34]; McMahon and colleagues modelled disease progression in 6-week cycles [28]. Cycle length has an impact on model predictions: shorter cycles may improve accuracy, in particular, with regards to survival time, but do require more detailed data.

Two decision tree-based models evaluated screening methods for dementia [31, 35] and one evaluated a preventative treatment [36]. The model by Silverman and colleagues [31] is not transparent, as only a schematic flow of decisions during a screening process is presented, not the actual model structure. Similarly, Dixon et al. only provide a textual description of the model [35]. It is questionable whether the two models could be reconstructed on the basis of the information provided. Tsiachristas and Smith [32] present a stochastic decision model for evaluating the cost-effectiveness of preventative treatment with B-vitamin. The model is not described in great detail, nor represented schematically, although parameters used provide some guidance as to how the model is structured [37].

A parametric Monte Carlo model was built to evaluate the cost-effectiveness of early diagnosis and intervention in AD [30]. The model is described in some detail, and with the aid of specified parameters, it is feasible to presume the model to be reproducible.

McDonnell et al. [33] assessed the cost-effectiveness of a potential treatment compared to standard care by simulating two cohorts of patients with AD using two regression-based simulation models. The models simulated the changes in cognitive decline as measured by Mini-Mental State Examination (MMSE), care setting and mortality of people with AD and compared the costs and effects of the strategies. The models were based on primary data from an epidemiological trial [38] and reproducing the model predictions without access to this data may prove difficult.

As both costs and health outcomes are dependent on disease onset and severity, characterisation of disease progression is, arguably, the most critical model input for producing accurate estimates. In many types of decision models, disease progression is directed by progression probabilities, i.e. the probability of shifting from one health state to another. It is advised that transition probabilities should be derived from “the most representative data sources for the decision problem” [24].

Four studies based their disease onset and progression parameters on a single study. McDonnell and colleagues [33] used primary data from a large epidemiological trial conducted in the 1990’s to inform disease onset and progression parameters [38]. These were reported by age, gender, residential status and MMSE score. Tsiachristas and Smith [32] based their disease onset and progression parameters on a national report on AD [43]. Zhang et al. [26] synthesised progression probabilities from a single source [44], although it is not transparent how progression probabilities were arrived at. The model by Weimer et al. [30] used disease progression data from a single study of 134 patients on cholinesterase inhibitors [45].

Five studies synthesised this information from two or more studies. McMahon et al. [28] and Martikainen et al. [29] utilised data from US-based studies [40, 46]. Martikainen and colleagues combined data from both papers, but do not provide more detail on how this was done. Another study [34] also utilised data from Neumann et al. [46] and another US-based study [47]. Similarly, Saito et al. also used data on progression of AD from Neumann et al. [46], combining it with another study, reporting on onset of mild cognitive impairment from pre-dementia [48]. Silverman et al. [31] also synthesised onset probabilities from two studies and provided little detail on how this was done.

Finally, Dixon et al. did not specify how disease onset and progression was characterised in their model [35].

Survival, or mortality, from dementia was integrated into the reviewed papers in a range of ways. McMahon and colleagues [28] calculated the probability of death from each stage of AD from a Neumann et al. [40], and the probability of death for the non-AD group from US life tables; these were not stratified by age. Weimer and Sager [30] applied a hazard ratio (2.1) to a non-AD population mortality rate to simulate higher rates of mortality. The ratio was obtained from a cardiovascular health study of the US population. Zhang and colleagues [26] populated the model with age-specific mortality for both AD and non-AD groups, obtaining the data from Statistics Sweden; the study assumed the same mortality rate for both groups.

Three papers used the same data sources for mortality/survival as for other disease progression probabilities. Saito et al. obtained mortality data from the same source as the other transition probabilities, by combining two previously published studies [46, 48]. Insufficient details were provided on how these probabilities were combined. Another model [29] also used the same sources for probabilities of death as for other transition probabilities [40, 46]. These were stratified by severity of AD (mild, moderate and severe), but not by age and gender. Mirsaeedi-Farahani et al. [34] also used the same two studies for mortality as for other disease progression probabilities [46, 47]; these were stratified by severity, but not age or gender.

One study [32] used life expectancy data to estimate mortality in their model; UK life tables were used to calculate life expectancy of non-AD group, and AD mortality was based on life expectancy obtained from another UK study [49]. McDonnell et al. calculated probabilities of death based on the results of a study reported in the paper. The probability of death is reported by age, gender, residential status and MMSE score [33].

Finally, two studies did not account for mortality in their models. The study by Silverman et al. [31] did not appear to extend far enough to calculate the impact of screening on mortality, while the other [35] did not include mortality in their parameters, stating that it was ‘beyond the scope of the model’, although a lifetime time horizon was utilised.

Decision modelling guidelines recommend that costs included in the model should be consistent with the selected perspective, reflective of all outcomes included in the model and should also be incorporated from data sources in a consistent manner [24, 25]. Overall, costs were consistent with the studies’ perspectives. In terms of consistency of data use, two studies estimated costs from primary data, such as administrative registries and databases [29, 33], three utilised secondary costs [27, 32, 34], and a further four studies used a combination of secondary and primary data for costs, mostly complementing any data missing from primary sources with published information [26, 28, 30, 35]. Silverman and colleagues utilised Medicaid reimbursement rates to estimate their costs [31]. Studies utilising secondary sources included generalised average per patient costs for dementia, of which two used average total annual costs [32, 34] and one included costs per stage of dementia [27]. Studies utilising any primary cost data used unit costs for individual aspects of diagnosis and treatment.

Dementia manifests itself in a wide range of ways, including cognitive decline, behavioural and psychological symptoms, as well as deterioration in ability to perform everyday tasks. NPIs for dementia have broad effects, which include improving behavioural and cognitive facets of dementia, as well as improving the overall wellbeing of a patient (and, potentially, their caregiver).

However, this breadth of both impacts of dementia, and interventions to address the disease, was not reflected in the characterisation of disease progression or the breadth of main outcome measures used to measure intervention effectiveness in the reviewed studies. The majority of the interventions utilised measures of cognition, such as MMSE, to model progression of dementia. The use of such measures to model disease progression in NPIs may be problematic, as such interventions often not only focus on slowing progression of the disease, but also on independence of patients and quality of life (both patients’ and caregivers’) such as those assessed in Martikainen [29].

Single-faceted measures of dementia, such as cognition, are also unlikely to fully capture the breadth of the burden imposed by dementia, nor the equally broad impacts of NPIs. This is likely to affect cost and outcome estimates in decision models. An alternative approach could be to characterise disease progression using an alternative concept, such as dependence, or a multi-domain model of disease progression [50, 51]. Broadly, these concepts represent progression of dementia as a function of cognitive, behavioural and activities of daily living aspects of dementia. These allow to classify the burden of disease, and associated costs and outcomes more accurately. It has also been proposed that QALYs and costs associated with each stage of disease can be mapped on to the measure and used in decision modelling [50].

However, the application of these concepts would require additional work and further development of instruments, as neither of the concepts are yet fully developed and ready for application [50‐52]. The.

MMSE is a commonly used measure for cognitive ability, it is easy and relatively cheap to administer and can be categorised to define stages of cognitive deterioration. However, MMSE and other measures of cognitive decline are surrogate end-points, and their use in cost-effectiveness analysis has been questioned [53], as it is difficult to establish a cost per incremental change on such a scale. Demographic characteristics of a patient can also be a determinant of cognitive capacity, and this is often not accounted for. Furthermore, measures of cognitive decline do not reflect patients’ or caregivers’ preferences.

A number of reviewed studies used QALYs as an outcome measure. HRQoL measures may capture the effect of an intervention on both morbidity and mortality, and provide a common denominator for comparison of economic evaluations across diseases; however, they also are subject to a number of limitations. For the purposes of brevity, this discussion will focus solely on issues relating to the subject of dementia.

Increase in severity of dementia corresponds with decreased cognitive function, memory loss, and, eventually, physical decline. These changes have a significant effect on the quality of life of a person with the disease, and should be accounted for in an economic evaluation. Two of the four reviewed studies that used QALYs as an outcome measure did not differentiate between stages or severity of dementia, simply providing utility values for either having or not having dementia. This is likely to significantly impact on the accuracy of the findings.

Furthermore, some studies combined utilities from a range of different sources, without providing a justification for why, or an explanation of how. Another important aspect in measuring HRQoL in people with dementia is the loss of cognitive function, which impairs judgement and ability to complete these relatively complex questionnaires. In order to avoid this, proxy-completion by caregivers is often used. However, there is evidence of significant discrepancy between self- and proxy-completed HRQoL measures for people with dementia [54, 55]. It is important to recognise these potential biases in outcome measures for future studies.

Adapting a model initially designed for evaluation of a pharmacological intervention could be the reason behind the choice of an outcome measure [28, 29]. Such models utilise clinical end-points which are used to measure the effectiveness of a pharmaceutical, which may not be sufficiently broad for NPIs, as previously discussed. Models developed for pharmacological interventions also focus on the period during which medication is taken, often resulting in short time horizons. When applied to a NPI for dementia, this is likely to omit the longer lasting effects of the intervention, and associated costs. In this review, purposefully-designed models tended to have longer time horizons, and also measure outcomes with QALYs, rather than cognitive-based measures.

Decision modelling in NPI for dementia should be conducted using models developed for the purpose. Purpose-developed model are likely to better reflect the features of NPIs, such as broad impacts, more appropriate time-horizons to capture all relevant costs and outcome, and more appropriate characterisation of disease progression.

A major concern is the availability and accuracy of parameters for disease onset and progression, as well as mortality data. The majority of data on disease onset, progression and mortality used in the included studies were obtained from literature, and, in many cases, from multiple foreign (often US-based) sources.

Synthesising progression probabilities from a range of published sources may be problematic for a number of reasons. First, mortality varies dramatically across the world due to a range of socioeconomic, health care, educational and other factors. Secondly, some of the data were collected in the 1990’s, and both mortality patterns and treatments available have changed significantly since then. This generalisability issue may result in inaccurate predictions of disease-related mortality. However, it is fair to note that this arises due to the lack of appropriate data on disease progression, rather than the quality of research presented. Finally, the methodological differences in calculation of probabilities for disease onset, progression and mortality, and assumptions made during the calculations, may mean that combining data from different studies may reduce the reliability of these inputs.

Data generalisability issues are also highly relevant for mortality data. There is strong evidence to indicate that age, gender, and severity of dementia affect mortality rates [39, 56]. Only one reviewed study parametrised mortality by age, gender, and severity; others focused only on severity. Two studies omitted survival altogether, with one declaring a lifetime horizon for their study. Future decision models of interventions in dementia could consider including mortality and stratifying it by age, gender and severity of dementia for more accurate predictions.

This indicates a need for more detailed and localised data on disease onset, progression and mortality, and stratifying such data by demographic factors such as age and gender. Alternative ways of characterising disease progression, such as dependence and health state utilities, should also be given consideration in future research. Overall, most studies would benefit from employing a more robust approach to economic evaluation in this field, by following a checklist or guidelines for economic evaluation studies such as those outlined by Philips et al. [25] and Caro et al. [24].

The findings of this review may be limited by the restriction of the review to English-language studies only and only assessing the methodology, and not the findings, of the included studies.

The studies included in this review were relatively heterogenous, representing a range of NPIs as well as modelling methods. However, as we are focusing on methodology, not results, of studies, this should not be a critical factor. We reviewed ten models on NPIs, of which only three evaluated tertiary prevention interventions. This is in stark contrast to the large number of NPIs targeting cognitive, behavioural and functional aspects of dementia [13, 15]. It is difficult to isolate one reason for this disparity; although the issues of data availability could be contributing factors.

This study followed robust methodology outlined by the Cochrane Handbook for Systematic Reviews of Interventions [20]. To the best of our knowledge, this is the first study to review modelling in NPIs for dementia. It is a growing field and the number of new interventions requiring economic evaluation is growing rapidly. Our study provides an insight into methods and data requirements for decision modelling in this area.