What can be concluded from the Oxcheck and British family heart studies: commentary on cost effectiveness analyses

Introduction

The reports of the main findings of the Oxcheck study1 2 and the British family heart study3 show that it is possible to achieve some change in coronary risk factors in primary care based, nurse led personal health checks. The reaction to both sets of results was generally negative,3 4 suggesting that the improvements were too small to justify the intervention. This reaction ignores the fact that an intervention that is relatively cheap and achieves a little can be more cost effective than one that achieves a lot at a high cost. Before empirical data from the Oxcheck and British family heart studies became available, a recent study attempted to model the cost effectiveness of screening for cardiovascular risk factors.5 The empirically estimated costs and cost effectiveness of the two interventions have now been reported.6 7

Our aim here is to provide a commentary on these two papers: firstly, to compare directly the relative cost effectiveness of the two interventions; secondly, to make some cost effectiveness comparisons with other interventions; and, thirdly, to consider some of the important practical and methodological problems encountered in carrying out these kinds of analyses.

Comparison of methods

The Oxcheck and British family heart studies were separate but concurrent attempts to explore the usefulness of health checks in primary care to reduce heart disease risk. Both studies were population based and nurse led, and both screened several risk factors including blood pressure, cholesterol concentration, smoking habit, weight, and alcohol consumption. The philosophies of the studies were, however, quite different, with a much more intensive follow up in the British family heart study. In the Oxcheck intervention, follow up was negotiated between the nurse and participant on an individual basis governed by a protocol for each risk factor, whereas participants in the British family heart study intervention were invited for follow up according to a strict protocol for both high overall risk and separately for individual high risk factors. Both studies provided intervention for one year only; some of the subjects in the Oxcheck trial were given additional annual screening, but these screens did not influence the observed clinical effect.2 Another difference was that the Oxcheck intervention targeted middle aged individuals but the British family heart study, while targeting middle aged men, invited whole families to attend.

The two independent study teams came together to coordinate the economic evaluations so as to avoid artefactual differences resulting from arbitrary variations of methodological detail or cost information. The costing methods of the two studies were very different, as they were constrained by the data collection strategies developed earlier in the trials' histories. The British family heart study used a bottom up approach, ascribing a cost to the observed trial resources deemed to be necessary in a service setting, whereas the Oxcheck trial adopted a top down approach, necessitating the removal of research costs from the actual trial expenditure. Together the two teams determined which of the resources used were service costs and which were research costs. Where service resource use was determined to be the same for both interventions, such as for the cost per nurse of overheads, an identical cost was used.

In assessing the cost effectiveness of health checks in reducing the burden of coronary heart disease it is important also to consider the costs of differences in treatment consequential on the findings of the health check. This may represent an increase (if, for example, the health check leads to the prescription of lipid lowering drugs) or a decrease if some costs of acute episodes of illness are avoided. In principle the only way to evaluate these costs accurately is to measure prospectively the use of health services over a long period for the subjects in each arm of the trial. The Oxcheck study collected data on general practitioner consultations attended by a sample of subjects and also recorded the prescribing of antihypertensive and antihypercholesterolaemic drugs. The British family heart study recorded the names and numbers of all drugs prescribed as well as the frequency of a variety of health service visits at five of the centres. The estimated mean costs of this broader resource use had large confidence intervals which included zero. Neither study followed up the pattern of such resource use after the interventions had finished. In comparing the two interventions we must therefore ignore the broader and longer term, assuming that the similarity of the studies may imply a similarity in their effects on other sectors of the health service.

In addition to coordinating the costing methodology, the study teams agreed to use a single indicator of effectiveness. Given the relatively short follow up in the studies, it was agreed that this would be the reduction in population coronary risk as derived from the measured changes in Dundee risk score8 as originally reported for the British family heart study in the clinical results3 and calculated for the cost effectiveness paper by the Oxcheck study team.6 Percentage changes in the Dundee risk score are not easily interpreted in themselves but can be used to feed into calculations of population coronary risk to show the likely effects of the interventions on extending life.

Comparison of results

In terms of the clinical effects, a comparison of the British family heart study at one year with those for Oxcheck at three years shows that although the British family heart study had a greater effect on blood pressure, the Oxcheck study was more effective in terms of reductions in both smoking prevalence and cholesterol concentration (table 1). The effectiveness of the Oxcheck study was particularly evident for the female participants. It was estimated that about half of the reduction in blood pressure observed in the British family heart study was due to accommodation bias and that all of the observed reduction in smoking prevalence could be explained by non-returner bias. The Oxcheck study results were also subject to potential non-returner bias. The estimates of coronary risk reduction presented in table 1 have been adjusted to account for these biases and have been converted from an individual risk to a population risk.

The British family heart study achieved a 13% reduction in coronary risk for men and a 10% reduction for women, whereas the Oxcheck study achieved a 7% reduction for men and a 13% reduction for women. There was little difference in their overall effectiveness: the Oxcheck study had a 13% reduction in risk compared with 12% for the British family heart study. Although the Oxcheck intervention seems to be slightly more effective overall, in terms of relative change in risk, if effect is defined as change in life expectancy then it is likely to be slightly less effective than the British family heart study intervention as the majority of the change was for women who are at a lower absolute level of risk.9

The Oxcheck intervention was more effective for women than the British family heart study, which was more effective for men. The Oxcheck study concentrated equally on men and women; in the British family heart study the primary target population was men, although a family centred approach was adopted. A health promotion programme where people are strictly required to attend follow ups regularly and where the whole family is involved may be a more effective way of treating men, whereas negotiating follow up may be more effective for women.

The less intensive Oxcheck programme cost substantially less than the British family heart study: £29.27 per participant (male or female) compared with £66.01 for men and £57.82 for women. For the Oxcheck intervention the cost of a 1% reduction in coronary risk was £4.18 per man and £1.72 per woman, compared with £5.08 per man and £5.78 per woman for the British family heart study. Hence the ratio of the programme cost to the percentage reduction in risk was better for Oxcheck than the British family heart study both for men and for women.

Difficulties in carrying out cost effectiveness studies in these contexts

The two studies have provided important lessons for the economic evaluation of clinical programmes. It was found that even when trials kept detailed financial records, a number of assumptions were required to translate these into routine service costs, firstly, because of the difficulty of separating service costs from research costs, and, secondly, because financial records often do not include important economic costs such as overheads.

The British family heart study found that the broader costs to the health service, as opposed to the narrower programme costs, may be substantial and are therefore important when it comes to estimating cost effectiveness. Such costs are highly variable, more variable than narrowly defined clinical variables, and therefore much larger sample sizes will be required if they are to be measured with any amount of precision in the future. The broader clinical effects, like the broader costs, may have a considerable influence on the cost effectiveness of the programmes. The change in coronary mortality brought about by the interventions will be augmented by changes in other types of mortality and by changes in morbidity, both of which may need to be measured if the effectiveness and cost effectiveness of health promotion programmes are to be assessed methodically.

Health check interventions are likely to have effects in the longer term as well as broader short term effects. The duration of the observed reduction in coronary risk is unknown, although the Oxcheck study suggests that it can last at least three years. The relation between risk factors and the number of events, in this study defined by the Dundee risk equation, is still not certain. Future coronary events will inevitably impose further costs on the health service as well as contributing to mortality and morbidity. Fully understanding the cost effectiveness of health promotion programmes requires that we measure reliably broader and longer term costs and benefits. This poses a challenge for future clinical and economic evaluations.

We thank our clinical colleagues from these studies for their advice, support, and comments but emphasise that the views expressed in this paper are those of its authors alone.