Key actors’ perspectives on cost-effectiveness analysis in Uganda: a cross-sectional survey

In the past 20 years, cost-effectiveness analysis (CEA) has become increasingly popular as a means of assessing the value of health care treatments. CEA endeavors to guide decision making such as to compare the health benefits of competing treatments within the context of cost. While these methods have been refined and gained traction in developed nations, there has been little use of CEA in developing nations [1]. Countries with scarce health care resources would logically stand to gain most from these methods to optimize allocation of those scarce resources. In Sub-Saharan Africa (SSA), adverse factors ranging from civil wars and extreme poverty to high rates of infectious disease strain health care budgets and delivery systems, making optimal allocation of health care especially important. Taken in the context of an expected doubling of the population in the next 25 years [2], it becomes even more urgent that CEA be more widely conducted and used to inform policy and practice. Accordingly, the World Health Organization (WHO) recommends incorporation of CEA in policy formulation and practice [3]. Despite all this, CEA publications in developing countries are few [1], and some have employed inconsistent and often flawed methodology [4].

Additional potential barriers to the conduct and impact of CEA in developing nations include lack of trained local personnel, lack of local data, and poor infrastructure [5]. Other hypothesized barriers include a shortage of CEA experts and lack of familiarity with CEA among policy makers and opinion leaders.

Despite the presence of barriers, evidence from developing countries continues to show that CEA is perceived as important for decision making. A study done among Ugandan health workers and policy makers showed that about 90% of all respondents recognized both disease severity and cost effectiveness as very important in health care decision making [6]. However, we have not come across studies which have attempted to document key actors’ perspectives in regard to the barriers which attenuate the influence of CEA in policy formulation and practice in Sub-Saharan Africa.

In a commentary on the role of pharmacoeconomics in developing countries that was recently published in PharmacoEconomics, Babar and Scahill [7] argue that “there is a need for a model that could aid in determining the perceived need and benefits of using pharmacoeconomics in formulary development in a given developing country”. By providing primary research on the exposure, interest and barriers to CEA in the context of a developing Sub-Saharan African nation, our study attempts to make a contribution to this effect.

This study aims to identify barriers to the conduct of CEA and its impact on policy and practice in a Sub-Saharan African country, Uganda. It also assesses the interest in and exposure to CEA methods, as well as the frequency of use of CEA in actual health care policy and practice.

We carried out a cross sectional survey of the different health care delivery actors in Uganda. Self-administered questionnaires were used. In order to increase the response rate, we used both paper-based and online questionnaires. After two reminders (chosen arbitrarily) were given following no response on the paper version, we would send the online version. This approach was not effective and was applied to only six participants. We eventually opted to invite all those who had not completed the paper version to complete an online questionnaire. The online questionnaires were designed and administered using REDCap Survey Software (version 1.3.10; Vanderbilt University, Tennessee, USA).

Questionnaires were sent out to health care workers; technical officers at the Ministry of Health (these are directly involved in policy formulation), post-graduate students training in health- related fields, and professors in the medical schools.

Health care workers and students were drawn from the two main teaching hospitals namely: Mulago teaching hospital for Makerere University and Mbarara University teaching hospital for Mbarara University of Science and Technology. The two hospitals were selected because they are the two main teaching hospitals in Uganda; it is the students trained in these hospitals that end up being the main decision makers in health care.

The study participants were selected as follows: For the Ministry of Health Officials, we contacted a senior officer and after we described the study to him, he advised us to send questionnaires to specific individuals from 3 departments that are involved in economic evaluations, namely Planning  − 5 respondents; Clinical Services-5 respondents; and Community Health- 5 respondents. For the students in the two medical schools, we obtained a list of names of postgraduate students from the program administrators and randomly selected those to whom the questionnaires were sent. We obtained a similar list for health care workers at the level of medical officer and above and randomly selected respondents. We also sent questionnaires to directors or deputy directors of Non-governmental organizations involved in health care delivery. All participants were fluent in English, so it was not necessary to provide translations.

Data was collected between July and October 2010. Bi-weekly e-mail reminders were sent to participants.

The study received IRB approval from the Case Western Reserve University Institutional Review Board (IRB) and the Makerere University Faculty of Medicine Scientific and Ethics committee, with final approval from the Ugandan National Council of Science and Technology. This study was conducted under standard ethical practices.

A total of 134 questionnaires were sent out. Of these 75 (57.2%) were paper-based and 65 (46.4%) were online. Six of the 65 online ones were from those who received the paper-based version and never responded. The remaining 59 were all sent directly online. Of the 75 paper- based ones, 69 (92.3%) were returned, but only 26 (40%) of the 65 online participants responded. Results are derived from a total of 95 (68%) returned questionnaires.

Majority of the respondents were employees of tertiary/referral hospitals and universities. These accounted for 34% and 28% respectively. The mean age of respondents was 36 years and males represented 61% of all respondents (Table 1).

Seventy eight percent (74/95) of the respondents had no exposure; the remaining 22% (21/95) only had some exposure to cost effectiveness analysis. Nobody had a lot of exposure. Among those with a bachelor of medicine degree, only 19% (11/59) had some level of exposure to cost effectiveness analysis compared to 81% (48/59) who had no exposure to CEA. None (0/8) of those with master of medicine degree (MMED) had exposure to CEA and only 25% (3/12) of those with MS or MPH had some exposure to CEA (Table 2).

Looking at exposure to CEA by work place also showed interesting findings. Among those working for the ministry of health, 12/15 (80%) had no CEA exposure. For tertiary/referral hospitals workers and university faculty, only 11/32 (34%) and 6/27 (22%) respectively, had some exposure to CEA. None (0/20) of those working for a non-governmental organization had any exposure to CEA.

Barriers to CEA identified by more than 50% of the participants were: lack of local experts in the field of CEA; lack of or limited local data; limited CEA training in schools; equity or ethical issues; lack of training grants incorporating CEA and lack of IT infrastructure. This meant the actual lack of or much limited supply of hardware such as computers and lack of or limited licenses to software packages such as TreeAge, which are used in economic evaluations such as CEA.

There was no statistical difference as to whether these were more of barriers in relation to policy or clinical practice. Participants also overwhelmingly agreed that improving these barriers will go a long way in increasing the use of CEA in policy and clinical practice (Table 3).

Majority of the respondents overwhelmingly agreed that CEA should play an important role in policy and clinical practice. Again there was no statistical difference as to whether perception of the importance of CEA was in relation to policy or clinical practice (Table 4).

Among those who had no exposure to CEA, 43% (32/74) had a lot of interest in conducting CEA compared to 57% (42/74) who did not have a lot of interest in conducting CEA. Among those who had some exposure to CEA, 30% (6/20) had a lot of interest in conducting CEA compared to 70% (14/20) who did not have a lot of interest in conducting CEA.

The picture was slightly different when we looked at CEA exposure versus interest in learning to conduct CEA. Although not statistically significant, respondents who had no exposure to CEA compared to those who had some exposure were slightly less likely to be interested in learning to conduct CEA, 58% (43/74) vs. 65% (13/20), p = 0.08.

In general, most of the respondents frequency of using CEA in making clinical or policy decisions ranged between sometimes to rarely or never (Table 5).

In the resource-constrained countries of Sub-Saharan Africa (SSA), where policy and decision makers are frequently faced with tough decisions regarding resource allocation, cost effectiveness analysis (CEA) can be a useful tool for optimal allocation of scarce resources. This study shows that the health care actors in Uganda perceive cost effectiveness analysis as an important part of the decision making process.

This study identified the lack of exposure to CEA and the lack of trained individuals to conduct CEA as the two main barriers to CEA playing a substantial role in health care decision making in Uganda. Just like in Thailand and other developing countries [8], the majority of the health policy decision makers in Uganda are medically trained. Systemic introduction of CEA methods into medical training is necessary, and also requires the creation of training grants for CEA.

Lessons learned from the approaches that developed nations have used to increase the knowledge and skills to carry out CEA should be applied to developing countries. Noteworthy is the fact that the approaches adopted should be modified in ways which make them appropriate in the local context. This will enhance the impact of CEA in the decision making process. A review by Alan Stewart, Jordana K. Schmier, & Bryan R. Luce, 1999 [9] has documented some of these strategies.

Jauregui et al., 2011, describes the Pan-American Health Organization (PAHO) ProVac Initiative’s approach. This initiative was in response to the request by country managers of vaccine programs for additional technical support in the use of cost-effectiveness analysis and other forms of economic evaluations of competing interventions. Its implementation inter-alia resulted into new models for cost-effectiveness analysis; innovative data collection tools; CEA trainings at regional levels; technical support to individual nations that ask for it and development of web-based based technical support systems [10]. These lessons can be applied to SSA.

In addition, the WHO-CHOICE(CHOosing Interventions that are Cost-Effective) project provides key information needed to standardize CEA methods, and also takes a sectoral perspective where the costs and effectiveness of a wide range of interventions are compared in order to identify those that maximize health from a given set of resources. The WHO-CHOICE project also assembles regional databases on the costs, and impact on population health. It also provides cost-effectiveness of key health interventions, as well as a contextualization tool, enabling adaption of regional results to the country level [3].

In this study, less than one in four had any exposure to CEA, including the subset of Ministry of Health employees. Furthermore, more than 80% identified lack of local experts as a barrier to the use of CEA. Similar findings have been observed among policy makers in Thailand, where only 29% were able to define the concept of economic evaluations [8]. Following these findings, interventions such as capacity building using various approaches like short-term trainings, workshops, and conferences about CEA methodologies and approaches have been widely implemented in Thailand [5]. With necessary modifications to make them locally appropriate, these should be adopted in Uganda and other developing or poorest countries.

This study identified other barriers similar to those identified by a range of policymakers in a Thailand study [5]. Among these is the lack of infrastructure and physical tools like computers and associated software.

In addition, there is limited local data and policy makers are routinely using clinical data from other countries.

In Thailand, the Health Intervention and Technology Assessment Program has adopted certain principles, such as technical excellence and policy relevance to enhance the utilization of its research findings by policy makers and other stakeholders [5].

There is a need for establishing cost effectiveness analysis institutes in SSA to guide research involving CEA and also carry out capacity building activities. While some developing nations like Thailand have been able to do this on their own, for the extremely impoverished countries of Sub-Saharan Africa, it may be necessary to establish a regional body. This may start by establishing a think tank comprised of key actors in health care decision making and experts in local institutions such as the economic policy research center in Uganda, the Kenya Medical Research Institute, and the Botswana institute of social policy to guide capacity building for CEA in SSA. This approach should be part of an overall strategy and can help define a better, sustainable process for increasing awareness and capacity to conduct the analysis.

Furthermore, fairness and ethical concerns were also cited as significant barriers to CEA. Societal values such as equity may sometimes conflict with the conclusions of CEA evaluations. For example, providing universal access to second-line antiretroviral therapy (ART) for treating HIV/AIDS patients may not prove to be as cost-effective, but may be preferable to society [11].

On the other hand, the fact stands that affordability of an otherwise cost effective intervention can be a barrier to wide spread use of knowledge gained from CEA. The additional cost of the program (affordability) when fully implemented may be prohibitive for a given country [12],[13] despite a favorable ICER per outcome for one person.

A high proportion of those who had no CEA exposure had a lot of interest in learning to conduct it, yet only 35% of those who had some exposure had a lot of interest in learning to conduct CEA. This may be because people who have had some exposure to CEA may better realize how complex CEA can be, and may prefer not to make the investment of time to learn to do it themselves. They still found it important to do CEA. Indeed, developing capacity to conduct CEAs will likely require the teaching of these methods in quantitative health-related disciplines such as epidemiology and biostatistics and might be augmented by the Ministry of Health requiring staff be knowledgeable and skilled in CEA.

Also noteworthy was the finding that none of the twenty individuals employed by NGOs reported any exposure to CEA. This may be because many NGOs have specific well-funded missions, and are not conducting CEA and formulating policy.

It is worth noting that although CEA may be essential for guiding resource allocation, its use might be limited in countries like Uganda, where resource allocation for health care is governed by a number of stakeholders including government and development partners such as Global fund, that may be interested in funding specific programs, e.g., Malaria, HIV/AIDS, medicines and diagnostics etc. [14].

In such settings, the lack of a single unitary agency e.g., the ministry of health that governs resource allocation limits the use of CEA tools to inform resource allocation at a sector level. This creates inefficiencies. However, within a sector program e.g., HIV/AIDS Anti-retro-viral drugs, there might be circumstances where CEA tools might be applicable.

The biggest limitation of this study is the small sample size, with limited power to detect significant differences between subgroups. However, the sample size was adequate to support the exploratory aims of our study which will guide future studies and interventions. Having mentioned that however, participants in this study belonged to categories that were identified as the main health care actors in Uganda [6] and elsewhere [15], therefore the information they provide is assumed to be both relevant and reliable. As with any self-administered questionnaire, we also cannot rule out the possibility of poor question interpretation [16]. Nevertheless, systematic analysis of our questionnaire results did not identify any specific questions that appeared to be interpreted illogically.

This study demonstrates that despite substantial interest, significant barriers have to be dealt with if cost effectiveness analysis is to have a significant impact on policy and practice in Sub-Saharan Africa. More locally generated data is needed, as is improved infrastructure, and increased local expertise. The study also highlights the unfortunate reality that despite acknowledgement of the importance of CEA and interest in learning to conduct it, very few health care actors in Uganda have sufficient exposure to CEA for these methods to impact policy. If cost effectiveness analysis is to take root in Sub-Saharan African countries like Uganda, there must be concerted effort to increase exposure to CEA at all levels of training.

Since political pressure also plays a role in decision making [8], there is a need for effective dialogue with all the involved stakeholders, particularly political representatives so that they understand the appropriate role for CEA in policy making.

JSM participated in the study development, developed data collection tools, analyzed the data, and was involved in manuscript development. MES participated in the study development, supervised data analysis, and contributed to manuscript writing. AM participated in study development and participated in manuscript writing. AK Participated in the study development supervised data collection in Uganda and contributed to manuscript writing. All authors read and approved the final manuscript.