Background
Methods
Literature search
Study selection
-
The paper had a reference in the title and/or abstract to the methods of economic evaluation of public health interventions.
-
The paper identified key challenges of conducting economic evaluations of public health interventions.
-
The paper made recommendations for conducting economic evaluations of public health interventions.
-
The paper was from a national source (e.g. Government or Advisory Group policy documents and reports) or published in a peer-review journal.
-
They were not specifically related to the economic evaluation of public health interventions.
-
They did not provide guidance on the economic evaluation of public health interventions.
-
They were published in a language other than English.
Data collection process
Assessment of theoretical underpinning
Results
Guidance relevant to the economic evaluation of public health interventions
UK guidance
Source of published guidance | Key points | Theoretical paradigm underpinning guidance |
---|---|---|
HM Treasury Green Book (2003) [22] | Care should be taken to avoid spuriously accurate figures; costs and benefits should be based on market prices where possible; where not possible techniques such as ‘willingness to accept’ (for costs) and ‘willingness to pay’ (for benefits) should be used; wider social and environmental costs should be considered; use previous studies to capture value of benefits (‘benefit transfer method’). | The approach used here is a welfarist based approach whereby outcomes relate to a notion of utility. |
Joseph Rowntree Foundation (Byford et al. 2003) [23] | This is a practical guide to conducting economic evaluation in the social welfare field, but many of the principles can be applied to public health interventions. There is guidance on how to measure costs and benefits along with a standard assessment of the different types of cost-effectiveness techniques (e.g., cost-effectiveness, cost-utility, cost-benefit etc.). | A welfarist approach is the theoretical underpinning here. There is no discourse as to the measuring of wider costs and benefits. |
Public Health Research Consortium (Drummond et al., 2006) [7] | Acknowledges additional challenges of applying traditional techniques of economic evaluation to a public health context. Undertook a methodological review of main challenges and proposed some solutions (underpins Weatherly et al. 2009) i.e. breadth of costs and benefits; need to consider effect on inequalities in health; attribution of costs and outcomes to stakeholders; measuring and valuing outcomes across different sectors. | An extra-welfarist approach is taken here where outcomes are not confined to a welfarist utility framework, but rather a specific importance is put upon measuring wider costs and benefits which are wider than QALYs. |
MRC Guidance on developing and evaluating complex interventions (2008) [32] | Defines complex interventions as “interventions with several interacting components”. Many public health interventions can be described as complex in this sense. Provides a frame work ranging from feasibility, evaluating (including assessing cost effectiveness), implementation and development. Economic evaluation is not dealt with specifically in this guidance which is focused on effectiveness. | There is no underlying theory here in terms of economics as the guidance is mainly concerned with effectiveness rather than cost effectiveness, as found in the evidence based medical movement. |
NICE Public Health Guidance (2009) [25] | Economic evaluation only relevant if there is expected net benefit, taking into account number of individuals affected; uncertainty in the cost-effectiveness literature and the likelihood that economic analysis will clarify matters. Economic evaluation is not needed if: it is not possible to estimate costs and benefits; if it is obvious the resources required are modest in comparison to health gains and if robust economic evaluation is already available. Assumes that objective of public health interventions is to maximise QALY. Cost per QALY calculations recommended using a £20,000-30,000 threshold. Also cost-consequence or cost-benefit analyses recommended. | This guidance is based on a conceptual framework for public health which comprises four vectors–population, environment, society and organisations all of which affect human behaviour. An extra-welfarist approach is taken whereby QALYs are viewed from an extra-welfarist perspective. |
Cabinet Office Social Return on Investment Guide (2010) [11] | There are two types of SROIs: evaluative and forecast; assigning a value to things that do not have a market price: contingent valuation–willingness to pay or accept compensation, revealed preference–hedonic pricing and average household spending–from sources such as the Family Spending Survey; SROI ratio is calculated as the total present value divided by the value of inputs–this gives a ratio of £x of value per £1 of investment; sensitivity analysis needs to be carried out–by how much would the parameters need to be changed to get an SROI of £1:£1. | SROI analysis has a microeconomic theoretical underpinning whereby evaluation of specific interventions is done on a one-by-one basis and no attempt is made to aggregate the results into a macro framework. |
NICE guidance for assessing cost effectiveness and ROI (2011) [26] | A cost impact project set up by NICE in 2010. The aim is to meet public sector demands to demonstrate potential returns on investment of public health interventions. It produces tools for financial planning to support local implementation of NICE’s recommendations. This report assesses the literature in terms of cost-effectiveness analyses and their adherence to NICE guidelines. They provide a number of recommendations to changes to NICE’s methods and processes. | Wider costs and benefits are considered here rather than concentrating on QALYs alone. Insofar as this is true it could be said that an extra-welfarist theoretical approach has been taken. |
International guidance
Source of published guidance | Key points | Theoretical paradigm underpinning guidance |
---|---|---|
World Bank (1993) [27] | The World Bank produced guidance focused on using techniques of economic evaluation to select a cost-effective package of healthcare interventions in developing countries. | The authors here write from the perspective of extra-welfarism, as they promote the use of cost-effectiveness analysis in their study. |
World Health Organisation (2003) [28] | The World Health Organisation advocate use of a generalised cost-effectiveness analysis approach, where the comparator is ‘doing nothing’, as opposed to the more conventional ‘usual practice’. | The authors here write from the perspective of extra-welfarism as they promote the use of cost-effectiveness analysis in their study although they propose an alternative approach in terms of a generalised cost-effectiveness analysis. |
Honeycutt et al. (2006) [29] | A practical ‘step-by-step’ guide to conducting cost-effectiveness analyses of public health interventions. These steps include defining the study question, identifying the study perspective, determining the time frame and analytic time period and selecting the type of economic study to conduct. | The authors focus on cost-effectiveness analysis and so it can be said that the theoretical underpinning here is extra-welfarism. |
Mont and Loeb (2008) [30] Beyond DALYs: developing indicators to assess the impact of public health interventions on the lives of people with disabilities. | Aims to address a gap in the DALY approach by offering two public health relevant outcome measures 1) Activity limitation score 2) participation limitation score, based on data from Zambia. | The theoretical underpinning here is extra-welfarist since the report is concerned with improving health outcomes by using DALYs rather than mortality on its own. |
OECD (2008) [31] The prevention of lifestyle related chronic diseases: an economic framework. | Focuses on the premise that prevention of chronic disease may increase social welfare and enhance health equity. This approach links chronic disease to the performance of markets and rationality failures, preventing individuals achieving the best possible outcomes. Focused on influencing choices available to individuals and seeking preventative mechanisms where market failures exist. | This approach is based on a behavioural economics theoretical framework focusing on rationality and market failure. |
Observations of key commentators relevant to the economic evaluation of public health interventions
Source of published guidance | Key points | Theoretical paradigm underpinning guidance |
---|---|---|
Kelly et al. (2005) [3] | Limited evidence of what interventions will work to reduce inequalities in health; evaluating single initiatives may fail to capture effects that rely on multiple interventions; behaviour needs to be changed in order to secure uptake of the intervention; difficult to isolate cause and effect due to multi-faceted nature of public health interventions; biological variation in clinical trials are much narrower than social variation in which public health interventions take place; public health interventions can change during their implementation; at what point is an intervention judged to have succeeded? | As they advocate the use of cost-benefit analysis it could be said that the authors come from a theoretical base of welfarism. |
Weatherly et al. (2009) [4] | Attribution of effects: likely to be fewer controlled trials of public health programmes–other approaches will be necessary; measuring and valuing outcomes: other outcomes, apart from QALYs, may be relevant e.g. external outcomes not confined to the health sector alone; identifying inter-sectoral costs and consequences: costs and benefits may fall on many parts of the public sector; incorporating equity considerations: in many cases the main objective of the intervention is to reduce health inequalities. | An extra-welfarist theoretical paradigm is used here in terms of concentrating on measuring benefits not adequately captured by QALYs. |
Payne et al. (2012) [5] | Valuation and Evaluation Research Theme (VERT) demonstrates that complex public health interventions have broader objectives than just health gain; maximising health gain is not a sufficient objective to achieve once cost and benefits outside the healthcare sector are recognised; public health guidance is more pragmatic than for clinical interventions and includes using alternative outcome measures (e.g. life years gained, cases averted) as well as QALYs; the authors suggest a move away from defining health benefits in terms of utility or QALY maximisation to consider non-health benefits and a measure of capability (or empowerment). | A strong theoretical basis to this guidance is the capability approach. This is coupled with an extra-welfarist perspective where wider costs and benefits are considered. |
Marsh et al. (2012) [34] | Reviews methods that could be employed to capture wider range of benefits generated by public health interventions; economists need to embrace a wider set of modelling techniques to capture the effects of public health interventions the selection of which should be facilitated by the production of better data on behavioural outcomes; more valuation paradigms should be explored such as the capabilities and subjective well-being approach. | The authors take an extra-welfarist perspective whereby wider costs and benefits are considered along with a wider range of modelling techniques. Behavioural economics is also a key theoretical underpinning where modelling of behavioural outcomes is considered. |
Discussion
Health economics coming full circle
The QALY/SROI dilemma
Should we expect public health interventions to be cost saving?
Design conduct and reporting issues of economic evaluations of public health interventions
1 | What is the appropriate theoretical framework for analysis e.g., welfarist, extra-welfarist, capability theory? |
2 | What is the setting of the public health intervention under evaluation? (e.g., environmental change; infectious disease control; screening; supporting behaviour change; supporting government legislation or policy) |
3 | Is this best described as a primary, secondary or tertiary prevention intervention i.e. upstream or downstream? |
4 | What is the main agency (government; health service; local government; voluntary sector) responsible for implementation and who are the key stakeholders? |
5 | If this is an intervention aimed at behaviour change, what are the key levers of change (legislation; price; changing social norms; choice architecture and nudging)? |
6 | What is the appropriate time horizon of analysis and what is the most appropriate discount rate for costs and outcomes? |
7 | If the public health intervention aims to “shift the curve”, are we most interested in the centre or tails of the distribution? |
8 | How is this public health intervention likely to impact on inequalities in health? |
9 | Will subgroup analysis help identify the range of cost effectiveness estimates across different settings, delivery methods and population groups? |
10 | What are the main outcome measures of interest e.g. QALYs/DALYs or a large range of outcome measures relating to health and wider social outcomes? |
11 | How important is it to value costs, benefits and returns in monetary terms? Is it reasonable to expect the intervention to be cost saving in the short, medium or long term? |
12 | How relevant will it be to compare an ICER with the NICE threshold of £20,000-30,000 or an international equivalent? |