Stakeholders’ perception on including broader economic impact of vaccines in economic evaluations in low and middle income countries: a mixed methods study

This study offers unique insight into the reasoning and justification employed by decision makers, funders and experts in the field of vaccine introduction.

According to respondents, the Ministries of Health, Finance and NITAGs play a crucial role in the current decision making process around new vaccine introductions. Although the numbers of NITAGs globally have been growing, the level of expertise in health economics has been limited and lacking in most LMICs [31]. Hence, explicit guidance on the inclusion of BEIVs by international organizations such as WHO would be most timely and well received.

Our study results indicate that evidence on burden of disease, safety and cost-effectiveness is regarded as most important for vaccine introduction decisions. This is in contrast to findings from another qualitative study which reported that only burden of disease evidence, and on occasion affordability and safety, was essential for immunization program decision making in seven LMICs [32]. More importantly, our study affirms that BEIVs belonging to ecological effects and macro-economic impact domains are considered to be equally as important as the more traditional outcome measures. Indirect effects and care-related productivity gains were considered less important to include in economic evaluation. Quite similarly, Burchett et al. [32] also reported that serotype replacement effects and, impact on non-health outcomes were rarely mentioned whereas equity considerations were only acknowledged in South Africa. Although productivity gains have often been cited in other studies [14,15,18,22,33], one explanation why this domain ranks lower on some studies [17] could be its relatively high importance to economists rather than clinicians and public sector officials who may be less familiar with its causal relationship. Cultural and contextual differences could be another possible explanation [4]. Indeed we observe that respondents dealing in an LIC context and representatives of government bodies tend to give higher importance to BEIVs than other stakeholders.

Considerations not covered by the survey were also brought up by interviewees. For example, the potential negative impact of wastage during immunization programs or the idea of using vaccines as a platform for providing other health interventions. The concept of combining vaccination with other interventions is not new [23]. However it remains difficult to translate the potential benefits of this synergistic effect into economic terms. Furthermore, interviewees highlighted the importance of using a broader perspective particularly when deriving the value of newer and costlier vaccines. This can be explained by the substantially higher list prices for new developed vaccines [34]. Several interviewees also reported the existence of different audiences. This finding underlines the necessity for flexible guidelines and the importance of combining different evaluation techniques to accommodate different needs and evaluation perspectives.

There are also several limitations to our study worth highlighting. Firstly, as most of the NUVI participants are actively involved in vaccine advocacy at a global as well as country level, the results may reflect biased viewpoints in favor of promoting the wider benefits of vaccines. Since this study was exploratory in nature, it was important to select a group that was actively engaged in decision making. It may be useful to conduct the survey within a more neutral group in the future. Secondly, the study suffered from a low response rate. Only 26 out of 140 respondents participated, which could be explained by the estimated 30 minute completion time. As a result none of the subgroup analyses performed were statistically significant. Future versions should attempt to be shorter. Offering rewards to attract participants could also be considered. Furthermore, the data collection could be organized in another setting. For example, the survey could be performed in workshop setting with direct feedback to the audience. Thirdly, the placement of the different types of impacts in the framework can be discussed. This is especially the case with school absenteeism which can also be placed under productivity-related gains and governmental savings which can be interpreted as a macro-economic impact. A separate analysis was performed to give insight in the outcomes for this alternative framework. It was found it would not change the results except for the macro-economic impact, which would be valued as even more important. Fourthly, using a Likert-scale has several methodological disadvantages [35]. These can be overcome by using a discrete choice experiment which would allow for a more explicit analysis of trade-offs between the various BEIVs and might offer greater discriminatory power than Likert-scales [36]. Finally, the qualitative part was supposed to add a layer on top of the survey findings, by providing illustrative anecdotal experiences and thoughts on feasibility of including BEIV in economic evaluation studies. However, it was difficult to conduct a sound comparison of the qualitative and quantitative part of the research, because not all interviewees filled out the survey. This may have resulted in recall bias as participants that contributed to both were already familiar with the different types of BEIVs outlined in the survey. To get more insight in this bias we checked for overrepresentation of both components in the results. No indications of this were to be found.