Testing the Validity of Cost-Effectiveness Models

A growing body of recent work has identified several problems with economic evaluations undertaken alongside controlled trials that can have potentially serious impacts on the ability of decision makers to draw valid conclusions. At the same time, the use of cost-effectiveness models has been drawn into question, due to the alleged arbitrary nature of their construction. This has led researchers to try and identify ways of improving the quality of cost-effectiveness models through identifying ‘best practice’, producing guidelines for peer review and identifying tests of validity.

This paper investigates the issue of testing the validity of cost-effectiveness models or, perhaps more appropriately, whether it is possible to objectively measure the quality of a cost-effectiveness model. A review of the literature shows that there is much confusion over the different aspects of modelling that should be assessed in respect to model quality, and how this should be done.

We develop a framework for assessing model quality in terms of: (i) the structure of the model; (ii) the inputs of the model; (iii) the results of the model; and (iv) the value of the model to the decision maker. Quality assessment is investigated within this framework, and it is argued that it is doubtful that a set of objective tests of validity will ever be produced, or indeed that such an approach would be desirable. The lack of any clearly definable and objective tests of validity means that the other parts of the evaluation process need to be given greater emphasis. Quality assurance forms a small part of a broader process and is best implemented in the form of good practice guidelines. A set of key guidelines are presented.