Pediatric Vaccines and Cost-Effectiveness Thresholds: How Much is Too Much to Pay for Prevention?

This article is published with digital features, including a summary slide, to facilitate understanding of the article. To view digital features for this article go to https://​doi.​org/​10.​6084/​m9.​figshare.​13135730.

Over the last few decades, economic evaluations have come to play an increasingly important—and often prominent—role in decision making regarding the adoption and use of new medical technologies. In countries such as England, Australia, and Canada, estimates of cost-effectiveness are a key input into decisions regarding the allocation of public sector healthcare resources [1‐3]. In the US, however, notwithstanding the fact that health economists, decision scientists, and health services researchers have long played an important role in the development, refinement, and practical application of methods of economic evaluation in medicine, explicit use of cost-effectiveness criteria in healthcare decision making remains the exception rather than the rule. Indeed, the Center for Medicare and Medicaid Services (CMS), which is the primary insurer for approximately 50 million Medicare beneficiaries in the US, is explicitly barred from considering questions of cost-effectiveness in its coverage decisions; CMS is supposed to make such decisions based simply on whether treatments are “reasonable and necessary,” without consideration of whether they provide value for money [4].

The Advisory Committee on Immunization Practices (ACIP) of the US Centers for Disease Control and Prevention (CDC) represents a notable exception to the generally limited use of economic evaluations and cost-effectiveness criteria in healthcare decision making in the US. The Committee, which consists of a group of medical and public health experts, advises the CDC about how vaccines should be used to control preventable diseases in the US [5]. The CDC, in turn, sets the childhood and adult immunization schedule based on the Committee’s recommendations. Vaccines that are part of the childhood schedule are covered under the Vaccines for Children (VFC) program, which provides vaccines free of charge to children who otherwise might not be immunized because of economic circumstances as well as applicable health plans under the Affordable Care Act [6]. Although the recommendations of the ACIP do not carry legal mandates, they are used by both the federal government and private insurers to determine vaccine coverage and reimbursement [7].

While the ACIP typically considers various factors in its deliberations concerning the use of new vaccines as well as expanded indications for existing ones, such as the incidence and severity of the disease in question, and the safety, efficacy, and effectiveness of the vaccine under consideration, cost-effectiveness—i.e., if a vaccination program is a reasonable and efficient allocation of resources—is one of the criteria that the Committee explicitly considers [8]. Over the past decade, the ACIP has cited findings of favorable cost-effectiveness in its support of the use of a number of vaccines, including the human papillomavirus vaccine (HPV) in 12-year-old girls ($3000–$45,000 per quality-adjusted life-year [QALY] gained) [9]; the 13-valent pneumococcal conjugate vaccine (PCV-13) in infants and children aged < 6 years (cost-saving versus the 7-valent vaccine [PCV-7] [10] and $20,200 per QALY gained as a single supplemental dose in selected patients) [10]; influenza vaccine among persons aged 50–64 years and ≥ 65 years ($28,000 and $980 per QALY gained, respectively) [11]; and the recombinant zoster vaccine (RZV) among immunocompetent adults aged ≥ 50 years ($31,000 per QALY gained) [12]. However, it is also true that the ACIP has recommended use of vaccines in the absence of a favorable cost-effectiveness ratio, because either a ratio had not been reported or the estimated ratio was found to be unfavorable.

One of the more recent economic evaluations considered by the ACIP concerned the use of a quadrivalent vaccine against invasive meningococcal disease (IMD), an acute infection of the bloodstream and the membranes surrounding the central nervous system. The disease is caused by the bacterium Neisseria meningitidis (or meningococcus), which is a leading cause of bacterial meningitis and sepsis in the US today [13‐15]. The rate of fatality in IMD is approximately 10%, even with appropriate treatment, and approximately 30% of survivors experience permanent sequelae, including neurologic damage, hearing loss, renal failure, paralysis, and amputation [16]. While N. meningitidis remains susceptible to many commonly used antibiotics, treatment is often initiated too late to significantly alter the course of the disease and to prevent complications and death; prevention is therefore a much better clinical strategy than treatment [13, 14].

Almost all cases of IMD in the US are caused by strains from one of four N. meningitidis serogroups (B, C, Y, and W) or non-groupable strains, although the breakdown of cases by serogroup varies by age [17]. In 2005, the first quadrivalent vaccine that was effective against serogroups A, C, W, and Y (MenACWY) was licensed for use among persons aged 11–55 years. In that same year, in response to rising disease incidence (especially among persons aged ≥ 11 years) and disease severity and based—at least in part—on the results of cost-effectiveness analyses, the ACIP recommended routine vaccination with one dose of MenACWY for persons aged 11–12 years, persons entering high school if not previously vaccinated with MenACWY, and other persons at increased risk of meningococcal disease, including college freshmen living in dormitories [17‐19].

In recommending the use of MenACWY in adolescents, the ACIP cited a cost-effectiveness ratio of $212,000 per QALY gained [17]. While there is no explicit cost-effectiveness threshold that the ACIP uses to define how much is too much to pay for prevention of communicable diseases, the estimated ratio for MenACWY in adolescents is substantially higher than that of other vaccines also recommended by the ACIP in recent years. The estimated cost-effectiveness ratio for MenACWY also substantially exceeds: (1) the threshold of $50,000 per QALY gained that frequently has been used in the US to identify interventions that provide “value for money,” notwithstanding the fact that this figure is > 30 years old [20, 21]; (2) the threshold of $100,000 per QALY gained, which has been suggested as a more appropriate figure to use in the US today [22]; and (3) the World Health Organization’s (WHO) suggested rule of thumb of three times the per-capita GDP [20], which would yield a current threshold for the US of almost $160,000 per QALY gained. Notably, in June 2015, the ACIP provided guidance that adolescents and young adults aged 16–23 years may be immunized with MenB vaccine to provide short-term protection against most strains of serogroup B meningococcal disease and reported estimates of cost per QALY gained ranging from $3.7 to $9.4 million [23]. While the ACIP did not render a routine recommendation for use of MenB vaccine in this population, the reasons cited included low disease prevalence and lack of important requisite data; an unfavorable cost-effectiveness profile was not mentioned.

This question about the right value to use as a cost-effectiveness threshold is especially interesting in the case of pediatric vaccines. If a threshold did not exist above which ACIP would deem a particular vaccine too expensive to use in relation to the clinical benefits it would confer, then economic evaluations of vaccines would be a largely academic exercise. The fact that these evaluations are undertaken and accorded weight in the ACIP’s deliberations suggests—in principle, at least—that such a critical value indeed must exist. A central question, therefore, particularly from a societal perspective, is how much is too much to pay for a pediatric vaccine? This issue is of more than just theoretical interest, since the answers impact how public health policy is determined, which, in turn, impacts the lives and well-being of entire populations as well as the budgets of payers. This article is based on previously conducted studies and does not contain any studies with human participants or animals performed by any of the authors.

In numerous respects, the pace of advancement in medical science has been dizzying in recent years and promises only to increase with expanding knowledge about the biologic basis of disease and our growing ability to target specific mechanisms. Our desire to reap the benefits that modern medicine increasingly can confer undoubtedly will increase apace. As it does, it will inevitably continue to bump up against the limits of affordability and cause a central question to be posed again and again: How much of our economic resources—either as individual consumers or as a nation as a whole—do we want to devote to medical care? It is almost inevitable that some medical interventions will be seen as providing good “value for money,” and others will be seen as being priced too high relative to the clinical benefits they confer. A fundamental problem of resource allocation in medicine is to figure out how to distinguish between the two.

Cost-effectiveness analysis is an exceedingly attractive tool for dealing with such problems. In a cost-effectiveness analysis, clinical outcomes and economic costs are compared under two alternative scenarios: one that assumes that an intervention of interest would be used in a particular fashion and another that assumes that it would not be used in the contemplated manner. An incremental cost-effectiveness ratio (ICER) is then obtained by dividing the estimated difference in costs between the two scenarios by the corresponding expected difference in health outcomes. The ICER represents the estimated “bang for the buck”—or, more accurately, the “bucks for the bang”—to move from the less effective treatment to the one that is more effective. All else equal, lower ICERs are preferable to higher ones, as they indicate that a given amount of health can be “purchased” at a lower price with one intervention versus the other. Healthcare decision makers thus can increase aggregate health benefits for any given level of expenditure by allocating resources to therapies with lower ICERs versus those with higher ones [24]. This criterion is sometimes referred to as economic efficiency.

Techniques of cost-effectiveness analysis have been applied to a wide range of diseases and interventions—from asthma inhalers [25] to drugs to treat cancer [26]. Many areas of medicine have their own preferred measures of clinical benefit, making it challenging to compare economic value across a diverse range of diseases and therapies. In some settings, it may be useful to know how much it costs to “produce” an additional symptom-free day in patients with asthma, while in others it may be useful to know the cost of delaying disease progression by 1 month in patients with advanced cancer. Determining which of these two interventions provides better value for money, however, requires a yardstick capable of translating qualitatively different health benefits to an identical measurement scale. While some may cringe at the thought of having to decide between a new drug for patients with asthma versus another one for patients with advanced cancer, if we cannot agree that such decisions inevitably must be made, then we are effectively conceding that no amount of money spent on healthcare could ever be too much, no matter the sacrifices that this might entail in other aspects of our lives. When framed in this fashion, we suspect that most people would concede that such comparisons are necessary.

One of the key milestones in the development and refinement of methods of cost-effectiveness analysis was the concept of the quality-adjusted life-year, or “QALY.” Over the past 2 decades, QALYs have emerged as the standard metric for measuring health outcomes in economic evaluations [27]. The QALY is a composite measure that takes into account both the length of a person’s life and its quality. QALYs often are calculated by segmenting the time that a person is alive into intervals that he or she spends in different health states. Each such health state is assigned a preference weight (or “utility value”) that reflects its desirability in relation to a state of “perfect health” (which is assigned a value of 1) and death (which is assigned a value of 0). QALYs are tallied by multiplying the time that patients spend in each health state by its corresponding utility value [28] and then summing these products. In this manner, QALYs facilitate comparisons of interventions across a wide range of conditions [29], effectively making the incomparable comparable.

While QALYs represented an important step in the development of methods of economic evaluation in medicine, in that they recognize that individuals and society value both length and quality of life, one must also consider how well the so-called QALY model truly reflects preferences from the individual and/or societal perspectives. While it may not be immediately apparent, if asked about how they think limited healthcare resources should be allocated, many people would express or reveal a preference for additional considerations or attributes. To the extent that these additional factors affect our choices in the real world, the QALY model may provide an incomplete picture of how we value different interventions.

For example, one important attribute of some decisions that is ignored by the QALY model is risk. While attitudes toward risk are complicated, context-dependent, and subject to cognitive bias [30], it is well established that people often are willing to pay a great deal of money to avoid it. Examples abound of people willingly spending large amounts of money to avoid financial risk, whether in the form of insurance premiums or acceptance of a reduced rate of return to ensure the safety of their investments. With a bit of introspection, this behavior can easily be confirmed (Example #1).

Given that catastrophic outcomes are an unavoidable part of many healthcare decisions, one might surmise that risks and attitudes toward risk would be an important factor in any decision framework. However, in the standard QALY model and therefore in the standard cost-effectiveness analysis framework, risk is treated as an irrelevant consideration. The use of cost-effectiveness analysis based on the QALY model explicitly assumes that people are risk-neutral with respect to outcomes [31]. In other words, methods of cost-effectiveness analysis assume that for any given amount of QALYs, people are indifferent about risk.

However, there are a plethora of examples (one of which is summarized in Example #2) that demonstrate the importance of ensuring that widely shared attitudes toward risk are incorporated into decision making paradigms. While the core assumption of risk neutrality has long been identified as a proper target for criticism of standard methods of cost-effectiveness analysis [31], there has been relatively little discussion about how attitudes toward risk should be incorporated into healthcare decision making. Nonetheless, some key points may be identified. Cost-effectiveness analyses of decisions that involve risk—especially where some outcomes may be disastrous—cannot simply focus on the expected overall number of QALYs produced by different treatment strategies without giving consideration to the range of possible outcomes and their likelihoods. If risk aversion is widespread, then decision makers should give greater priority to strategies that involve the avoidance of catastrophic outcomes, such as life-threatening diseases that strike in childhood [32].

In addition to ignoring the element of risk, traditional methods of cost-effectiveness analysis also disregard how health benefits are distributed among different members of society and focus simply on the expected total number of QALYs gained across all persons who would be impacted in a relevant accounting or social unit [33]. No consideration is given to how those QALYs are distributed. In the parlance of health economics, “a QALY is a QALY;” to whom it belongs is irrelevant (Fig. 1).

But is it indeed the case that we do not care how health benefits are distributed? Consider, for a moment, two persons with terminal cancer—one, a 12-year-old boy; the other, a 75-year-old man. Suppose that, with optimal palliative and supportive care, each could experience one additional QALY than would otherwise be the case. If care could only be provided to one of these two patients, and the decision were yours, would you be able pick a patient, and if so, which one would you pick? Most people, we believe, would express a clear preference for treating the young boy. Yet again, such behavior would be inconsistent with the conceptual foundation and framework of cost-effectiveness analysis. If a QALY is a QALY, then it should not matter to us who gets them.

Why do we seem to care how QALYs are distributed? One possible explanation can be found in the principles of fairness and equity. In particular, it seems especially unjust that some of us get to live to a ripe old age, while others are robbed of the chance for a long life for inexplicable reasons. Most of us find these kinds of inequalities upsetting, and by treating the older man instead of the boy, we would be widening the gap in QALYs between them. Treating the boy instead of the older man, on the other hand, feels right to us because we would be closing the gap, if only slightly. Evidence for this attitude can be found in the everyday language of death. People seem to view death very differently depending on when in life it occurs. When a child dies, it is very commonly referred to as a tragedy, yet when a person dies in old age, we often readily comment that he or she had a “full life” (usually, irrespective of how they actually spent it and the enjoyment they derived from it). While such events are still unhappy, we acknowledge that death is inevitable, and there is solace to be found when it occurs at an advanced age and in a comfortable setting. These are not thoughts that typically come to mind when a child dies.

The so-called “fair-innings argument,” coined by the British economist Alan Williams, builds on this attitude by making the claim that we are all “entitled” to a certain normal span of life and state of well-being [34] and that an important goal of healthcare decisions should be to reduce inequalities in the distribution of lifetime health outcomes. To the extent that outcomes-based equity considerations are important to us, such as the “fair-innings argument,” decision makers would be wrong in focusing simply on the total number of QALYs produced by a particular intervention without regard to how the benefits are distributed. The distribution of health benefits associated with interventions must be considered, and there should be recognition that, in some instances and in particular circumstances, we would willingly trade economic efficiency for equity (i.e., we would accept a lower total amount of QALYs in return for a distribution of benefits that we thought was more equitable). The implication is that in some situations and circumstances, policy makers might willingly accept treatments that offer fewer QALYs (i.e., have higher cost-effectiveness ratios) if the distribution of benefits is deemed more equitable [34]. In the present context, the implication is that we might want to accord greater value, all else equal, to interventions that can prevent diseases that rob children of “innings” that we think should be their due.

We note that the objective of this review article is not to address limitations inherent in QALYs and their measurement—including ethical, methodologic, and contextual concerns (e.g., variability/validity of instruments, numeracy challenges), especially in pediatric populations—since doing so would take us beyond our intended scope [35]. However, it is important to note that to the extent some of these limitations (e.g., imperfect information on future interventions and the timing of these interventions that may impact health) may modify perspectives on risk taking and thus decision making at a given point in time, the QALY concept may be further challenged. We also note our review article does not address the other component of cost-effectiveness ratios and its associated limitations, namely costs.

Ultimately, the application of cost-effectiveness thresholds comes down to how much we are willing to pay to achieve outcomes that we desire and/or reduce the likelihood of experiencing outcomes that we would rather avoid. Conventional cost-effectiveness thresholds, expressed in terms of a cost per QALY gained, accord value only to the length and quality of life, irrespective of risk or how benefits are distributed. As we have discussed in the preceding section, attitudes toward risk and how outcomes are distributed are often important considerations in decision making about healthcare. To the extent that these concerns and issues are real elements of social preference, we should observe that people are willing to pay a premium to avoid catastrophic health outcomes, especially in children. An obvious question is whether people in fact behave in this manner?

Many surveys have examined how the value we place on an intervention may depend on factors beyond its expected total health benefits. Some studies have examined these choices as a function of risk preference at an individual level and asked respondents whether they would accept a greater risk of early death to improve their life expectancy. Hellinger [36], for example, surveyed 50 employees of the US Department of Health and Human Services about whether they would prefer a life expectancy of 30 years or an 80% chance of a life expectancy of 40 years and a 20% chance of immediate death. If people were not risk-averse, they should prefer the latter choice, as it involves an expected gain of 2 years of life in comparison with the risk-free option (i.e., [80% × 40 years + 20% × 0 years] > 100% × 30 years). Hellinger reported, however, that respondents were almost unanimous (96%) in their preference for the choice that did not involve a risk of instant death. In a similar study, Eraker and Sox [37] surveyed 523 persons about whether they preferred treatments with certain or uncertain effects with the same expected number of life-years. They examined cases where the expected gain ranged from 1 to 3 life-years, and in each case, they found that a statistically significant proportion of respondents preferred the “sure thing.” If risk aversion is apparent even when questions are hypothetical and stakes are as small as the loss of a single year of life, it is not hard to imagine that it could play a much greater role when the choices are real, and the stakes are much higher.

Other surveys have adopted a more societal perspective and asked respondents to choose between healthcare programs targeted at different age groups. Johannesson and Johansson, for example, surveyed a random sample of 1000 persons in Sweden, aged 15 years or older, about whether they would prefer a hypothetical program that saves 100 lives among 50 year olds or one that saves 50 lives among 30 year olds [38]. The number of persons saved in the 50-year-old group was varied to determine the point at which respondents thought the two programs were of equal value. They then used general-population mortality rates and utility values to estimate the implied QALYs gained by each program. In doing so, they found that preferences expressed by survey respondents implied that a QALY gained for a person aged 30 years was “worth” 3.9 times that of a QALY gained for someone aged 50 years. They then repeated the survey using an older comparison group and reported that a QALY gained for a person aged 30 years was accorded a value 16 times greater than that of QALY gained for someone 70 years of age. Such extreme results are obviously difficult to reconcile with a model that assumes that a QALY is a QALY, regardless of whose it is, and one wonders whether the result might have been even more extreme if children rather than 30 year olds had been considered. A study by Mooney, Jan, and Wiseman asked respondents whether they would prefer a program that offered 1000 units of “health gain” to people aged 20–30 years or a similar program that offered the same benefit to persons aged 60–70 years [39]. While a plurality of respondents (45.9%) answered that these programs were equally valuable, far more respondents preferred the program for younger persons (39.9%) than the program for older persons (11.3%). While this result does not support widespread preference for the young, it does suggest that such preferences exist in the aggregate.

In addition to surveys that ask purely hypothetical questions, evidence of social preferences also may be found in real-life decisions that are made by patients, parents, and regulators on a daily basis. If a preference for risk aversion and equitable distribution of health outcomes is common, then we should expect to find examples of decision makers willingly spending large amounts of money to avoid even small risks of catastrophic outcomes or to achieve even modest health gains in children. A potent example of this may be found in the private banking of umbilical cord blood, which has shown promise in treating various diseases, including metabolic disorders, leukemia, and cerebral palsy [40]. The potential therapeutic value of cord blood has led to the creation of for-profit banks that offer to store cord blood collected at birth in case it is needed later in life. The likelihood that banked cord blood will actually be used, however, remains extremely low—reportedly, as low as 1 in every 250,000 “deposits” [41].

A cost-effectiveness analysis found that, even with extremely generous assumptions, the cost-effectiveness of banking cord blood was well in excess of $1,000,000 per life year gained [40]. Yet its high price has not stopped parents from banking the cord blood of their children; currently, an estimated 1.7 million cord blood samples are being held in private storage banks [42]. The fact that so many parents believe that the banking of umbilical cord blood offers reasonable value for money suggests that there is considerable demand for healthcare interventions that hold out the promise of reducing even small risks of catastrophic outcomes in children, even if their cost-effectiveness ratios vastly exceed any current benchmark of how value should be assessed in healthcare. Other examples in which similar decisions have been made include child safety regulations (Example #3) and homeland security (Example #4).

A large portion of healthcare spending goes toward extending and improving life among older adults during their final years [46, 47]. Vaccines are our first line of defense against a number of devastating diseases, but represent a relatively small portion of preventive healthcare expenditure [48]. Vaccine-preventable diseases in children can result in unnecessary morbidity and mortality, begging the question whether we really believe that the value of preventing these diseases should be measured by the same cost-per-QALY “yardstick” as adding another year of life to someone who already has many under his or her belt. Do we really believe that “a QALY is a QALY,” and are we indeed indifferent about risk? If not, then the yardstick that is commonly used in cost-effectiveness analysis may not be measuring what we sometimes think it is, and reliance upon it may yield outcomes that are inconsistent with our preferences.

With cost-effectiveness an explicit factor in ACIP’s Evidence to Recommendation framework [8], successful vaccine development requires not only evidence of safety and efficacy but also the foresight to predict trends in disease incidence many years into the future. In the case of IMD, the annual incidence rate has declined precipitously over the last 2 decades (from about 0.5 to 1.1 cases per 100,000 persons prior to 2000 to 0.1 cases per 100,000 in 2018) [49], and many have struggled with how to appropriately estimate the value of preventing a catastrophic disease that is rare but variable in its occurrence, subject to fluctuations over time and spontaneous outbreaks [50]. Unpredictable trends in disease incidence can yield important changes in the cost-effectiveness profile of a vaccine candidate from the time antigens are identified to when it is later appraised by regulatory agencies and reimbursement officials. Given the challenges of vaccine development and the consequences of incorrect decision making, it would be of paramount importance to ensure that cost-effectiveness thresholds that are applied to vaccines accurately reflect societal preferences.

We therefore believe that caution should be exercised when making use of conventional cost-effectiveness thresholds in deliberations concerning the use of vaccines against rare and unpredictable diseases. While the cost-per-QALY framework is unquestionably a useful one in framing problems of value for money, there are many potential sources of value that this metric does not consider. We note that while our review focused on the US and ACIP, the issues and challenges discussed in this review article apply to other countries around the world, notwithstanding differences in approaches to healthcare decision-making. Accordingly, we encourage similar reviews focusing on these alternative (i.e., ex-US) settings.

As we have shown, societal preferences for risk avoidance and more equitable distributions of health outcomes may affect how healthcare decisions are made in practice. While available evidence is more suggestive than definitive, we believe that it is sufficient to warrant a serious re-examination of how cost-effectiveness thresholds, either implicit or explicit, should be applied when decisions involve the prevention of devastating diseases in children and adolescents, such as IMD. Do we really know how much is too much to pay in these instances? At this point in time, unfortunately, the answer is probably “no.”