Do economic evaluation studies inform effective healthcare resource allocation in Iran? A critical review of the literature

A total of 258 articles were identified by the literature search, of which 150 articles were excluded at the initial stage because they violated basic inclusion criteria (Figure 1). The full-texts of 108 articles were then reviewed and a further 78 articles were dropped because they failed one or more inclusion criteria. The remaining 30 articles [30‐59] met the inclusion criteria of the study. Details of the reviewed studies are presented in Additional file 2: Table S1 and Additional file 3: Table S2.

High quality economic evaluation data on interventions targeting high burden of diseases are important to aid informed decision-making and resource allocation. This study enumerates the availability of economic evaluation studies and critically assesses the quality of those data in the Iranian context. The findings showed that the number of economic evaluation studies in Iran is limited. This is comparable with settings such as Bangladesh [60], Nigeria [26], Saudi Arabia [61], Zimbabwe [62], and lagging behind countries such as Thailand [29], South Korea [24], India [63] and South Africa [64]. The number of available studies remains very low compared with high-income countries where economic evaluation is a relatively well-established and formal part of the policy making process [21, 65]. The findings from this review also showed that many published Iranian economic evaluation studies did not meet current international standards and were of sub-optimal quality.

The review showed that CEA was the most frequently published economic evaluation in Iran. The relative ease of obtaining effectiveness data in the form of intermediate outcomes or natural units, and the application of arguably more straightforward computational methods in these types of evaluations [60] are two features that make CEA a practical method of analysis in many settings. This approach may be particularly attractive, when collecting comprehensive outcome measures such as DALYs and QALYs are time intensive and expensive to collect. CEA is useful in situations where outcome measures of interventions under investigation are similar. However, since the outcome measure may differ for different interventions, CEA can seldom be used to make comparisons across a broad set of interventions [1]. Taking into account this limitation, CUA or CBA arguably provide better tools for policy makers allocating resources across different health care programmes or even across different sectors [1].

Considering resource constraints, it would be expected that economic evaluations focus on interventions for diseases that have a significant impact on population health. However, the findings of the current review showed that the majority of economic evaluation studies concentrate on a small number of disease categories which do not include some of Iran’s major health problems. For example, injuries and mental disorders, two disease categories which contribute more than 40% to burden of disease in the country [6] and are likely to increase in the future [6], are strongly underrepresented in the Iranian economic evaluation literature. In addition, the findings show that researchers have paid a great deal of attention to curative interventions, while relatively little attention has been given to preventive interventions and care delivery strategies. These findings are consistent with previous studies [2, 28, 29]. A mismatch of economic evaluations with disease burden does not necessarily indicate inefficient resource allocation. It is possible that studies do not focus on high burden illnesses because there are few effective interventions for some health conditions. It is also possible that there exist a number of well-established, cost-effective interventions for some disease categories and as such new work in the area is not required. Difficulties in obtaining data on the effectiveness of interventions in some areas, such as mental disorders, can be another barrier to the conduct of evaluations for high burden illnesses. The interest of researchers and funding agencies could also play a role as the majority of research in Iran is funded by universities or other governmental research centres. Multiple criteria including equity and social justice are frequently used to determining health priorities and the burden of disease is only one of those criteria [2, 28, 66]. Nonetheless, we believe that comparing the priorities reflected in the evidence regarding the burden of disease, with those reflected in the economic evaluation literature in Iran can provide a useful starting point for discussing future priorities for economic evaluation research.

The review indicated that the Iranian economic evaluation studies had low adherence to good practice criteria for the reporting of economic evaluations and suffer from several methodological flaws. The perspective of an economic evaluation study is an important issue as it affects the measurement of both costs and outcomes of interventions. Yet few of the reviewed studies (only 23%) specified their perspective. This suggests that many authors are unaware of the importance of the perspective adopted and its effect on the costs and outcomes. Among the studies which stated their perspective, many failed to include all appropriate costs associated with their chosen perspective. For example, two studies [30, 58] stated their perspective as societal but they didn’t measure indirect costs associated with their interventions and those studies that had taken a healthcare system or provider as their perspective, failed to include capital or equipment costs and some recurrent costs (e.g. overhead costs). In the context of Iran, obtaining data to estimate capital and overhead costs, particularly in hospital settings where most of the reviewed studies are based, might be difficult. Moreover, some part of these costs is usually incorporated in the inpatients and outpatients’ costs or hospital bills, which many of the reviewed studies included in their analysis. Using hospital charges/bills (and also health care tariffs) in economic evaluation have been criticised since they may not reflect the actual costs [1, 67]. In the context of Iran in particular, hospital charges do not reflect true hospital costs due to government subsidies for hospital services and medicines [68]. Moreover, the effects of adverse events associated with interventions on use of resources and outcomes were also rarely [37, 53] included in the reviewed studies. Lack of transparency in reporting intervention costs, was another important shortcoming of the reviewed studies. In some studies [31, 40, 43, 44, 55] it becomes impossible to ascertain what authors had actually done. Many studies failed to describe clearly how they measured and valued costs and did not provide details on type and quantities of resources and their price, which limits the possibility of replicating the evaluation in other settings.

Moreover, few studies calculated and reported ICER. Instead of an ICER, some studies reported the average cost-effectiveness ratio (i.e. total cost divided by total effect for the interventions being compared), which can flaw the conclusion of the evaluation and limit direct comparison between interventions. This is because an average ratio implies the comparison of each alternative with a hypothetical intervention with no costs and no effects [69].

A major weakness of the economic evaluation studies in the Iranian setting was the limited use of sensitivity analysis to explore the effect of uncertainty on findings. Only 27% of studies performed some sort of sensitivity analysis. Sensitivity analysis helps to assess reliability of the findings for the context of the study and can also facilitate consideration of the generalisability of findings to other settings [70]. Moreover, the review also showed that very few Iranian economic evaluation studies discounted costs and/or outcomes when the study period was more than one year. None of the studies provide justification for the discount rate used, even if that rate was zero, and none performed a sensitivity analysis on the rate used.

Low adherence to good practice for the conduct of economic evaluations and the other methodological shortcomings discussed above are not unique to the Iranian setting. Systematic reviews of economic evaluation studies in some other settings (both developed and developing countries) have reported similar shortcomings [21, 23, 24, 26, 29, 60‐65]. For this reason, many countries have developed formal and informal guidelines to standardize and improve the quality of economic evaluation in health care [71].

The findings of this review indicate that the literature on economic evaluation in Iran is still at an early stage and these evaluations suffer from significant methodological flaws. Furthermore, the review reveals that economic evaluation studies have not focused on Iran’s most significant health problems i.e. those contributing most to the country’s burden of disease. Although interest in using inputs from economic evaluation and HTA studies has increased in Iran, evidence still points to scarce demand for and utilisation of these inputs by policy makers. This itself might undermine incentives to improve the quality of economic evaluation studies.

The findings from this study suggest that Iranian evaluations might benefit from the establishment of clear national guidelines on the conduct of economic evaluations. In addition, capacity building of local scientists in the conduct of economic evaluations may be a priority area in the future. This would require additional investment in the teaching of economic evaluation in Iranian universities and the training of health professionals and policy makers in the use of economic data. Moreover, priority areas for future economic evaluation should be established by collaboration between researchers across disciplines, and in communication with policy makers – taking explicit account of the national burden of disease.