We had hypothesized that the SLV-I program would become more efficient in Year 2 of implementation, but found that the costs and cost-effectiveness of SLV-I were comparable in Year 2, a routine seasonal influenza vaccination year, compared to Year 1, the year of pandemic H1N1. Project coordination costs (Component B) were higher in Year 2 than 1, more than offsetting improved efficiencies in other costs. A detailed comparison between the two years offers useful policy implications for SLV-I and can help set future goals to improve the overall cost-effectiveness of SLV-I during a more typical season. Additionally, analyses of both Years 1 and 2 showed that SLV-I could be cost-saving to society, compared to “no vaccination,” if savings from the increase in disease prevention under the SLV program (Component E) were included.
Comparison of year 2 trial with other studies
Our Year 2 intervention had a moderate impact on influenza vaccination uptake with an improvement of 12 percentage points (pp). This impact was higher than that found in other trials, such as those using text message reminders (3.7 pp) [
43], mail reminders (6.5 pp) [
44], and provider prompts (4.0 pp but statistically insignificant) [
45].
The overall vaccination coverage rates found in our trial was in the range of rates noted in other studies. For example, SLV-I trials at the state level in Hawaii and the three-county level in Minnesota resulted in very high vaccination rates of 46 % [
25] and 41 % [
46], respectively; however these were not clinical trials, no control schools existed, and the papers did not report what percent of vaccinations were delivered in school versus in physician offices. In an SLV-I trial at nineteen elementary schools in California, coverage varied by school, with 26.9 %-46.6 % of children in each school receiving at least one dose of influenza vaccine with a large impact due to SLV-I; however in this setting control schools had virtually zero vaccination rates [
47].
Unlike other SLV-I trials, our SLV-I trial billed insurers (or parents if insurance coverage was unknown) for vaccines and vaccine administration. We contracted with a for-profit vendor that delivered the in-school vaccinations, which were purchased through routine channels or obtained through VFC program. Consequently, the cost estimates from our study are greater than those in other SLV-I trials that did not include vaccine cost and\or the cost to bill insurance or Medicaid for vaccines or vaccine administration. The cost estimates in other studies in this section were all adjusted to 2010 US $ with medical care CPI [
31]. For instance, our estimate of $59.88 per vaccinated-child was much lower than that by Kansagra et al. ($80.92) [
48], but considerably higher than that by Schmier and colleagues ($6.98) [
40], Hull and associates ($10.11) [
46], Effler et al. ($15.66) [
25], and Kemp and colleagues ($24.69) [
49]. Among these five studies, only Kansagra et al. and Effler et al. reported the detailed cost items within the administration cost. Concerning the labor cost estimates, our estimate ($39.13 per vaccinated-child) was slightly larger than $33.17 estimated by Kansagra et al. [
48], and much larger than $12.19 estimated by Effler et al. [
25]. The latter lower estimate may have been partly due to economies of size of their large state-wide program, vaccinating 63,153 children after targeting all children aged 5–13 in Hawaii [
25]. Additionally, the studies by Effler et al. [
25]
or Hull et al. [
46] did not seek third party reimbursement, which was included as part of the vendor’s administrative cost (for the billing process) in our study. Parents were not billed for any fees in the study by Kemp et al. [
49], although our cost estimates includes the vendor’s billing process costs for parents.
Extensive project staff time was needed to manage parents’ consent forms–all done on paper -- which included details of about patient insurance. More efficient consent systems could reduce future SLV-I program costs [
50,
51].
Material costs incurred by schools and the project coordinators were $4.69 per vaccinated-child, which was similar to $5.72 (adjusted to 2010 US $ with medical care CPI) reported by Effler et al. [
25]. However, our estimate of the vendor’s material cost, $13.43 per vaccinated-child, was much higher than that of $1.64 by Effler et al. [
25]. This difference can be partly explained by our study’s broader cost definition including items such as the refrigerator for vaccines and non-medical supplies.
Other studies also have concluded that SLV-I may be cost saving to society, when considering broader indirect costs [
40,
52]. Using secondary datasets only, White and associates estimated that group-based influenza vaccination was cost-saving, i.e., saving $6.40 and $55.82 per vaccination, as compared to individual vaccination at a medical practice and no-vaccination, respectively [
52]. Schmier et al. analyzed their primary data to conclude that SLV-I is cost-saving to society, saving $170.31 on average among all households in intervention schools [
40].
In summary, our study, based upon a real-world demonstration and including billing of third party payers, had higher program costs than most prior SLV-I studies, resulting in lower cost-effectiveness. Our findings regarding indirect parent or societal costs were in line with those of other studies.
Potential limitations
There was uncertainty in cost estimates in Year 1 that may affect the comparison with Year 2. As discussed in the paper describing Year 1 [
28], it is difficult to accurately allocate the fixed costs between first clinics and second clinics during Year 1 due to the limited available data.
Another limitation is the potentially limited generalizability of our estimates, which may have been affected by multiple factors. The effectiveness of SLV-I is sensitive to the proportion of local children vaccinated by medical practices prior to the school vaccine clinics. For instance, in an area where medical practices vaccinate a high proportion of children, a SLV-I program may have a smaller impact on vaccination coverage. Hence, our SLV-I effectiveness estimates are likely to be most applicable to other areas where the vaccination rates achieved by medical practices are similar to those in our study site, i.e., less than one-third of children were vaccinated pre-intervention. Second, since influenza vaccination rates may be influenced by a host of seasonal influenza factors (e.g., disease severity [
53], vaccine availability [
54], media coverage [
55]) SLV-I vaccination coverage in school and, consequently, effectiveness estimates could differ from year to year.
Different methods were used to ascertain the vaccinated status between intervention schools (based on the vendor’s records and New York State Immunization Information System (NYSIIS)) and control schools (NYSIIS records only). This difference could affect the effectiveness measure within a year, but would not affect the comparison between Years 1 and 2.
Finally, we derived indirect costs from the literature, not from our trial. Since we utilized a national-level median hourly wage among working adults for estimating the indirect cost component D, averted parents’ costs, these estimates are expected to be reasonably generalizable. Since the magnitude of the other indirect component (E, costs saving from disease prevention) might be sensitive to the estimation methods and the seasons analyzed, component E-related results were not presented in Tables. Our analysis excluded some relevant, but unmeasured, indirect costs such as costs due to disruptions of the school day by SLV-I, and cost savings from decreased absenteeism.
Policy implications
Our findings from a real-world demonstration project indicated that while SLV-I is effective in improving influenza vaccination rates in school-aged children, project coordination costs (Component B) remained high during a second project year. High project coordination costs (Component B), driven by a substantial amount of effort needed to obtain informed consent and to manage implementation of the project, more than offset lower school and vendor costs (Components A and C). Project coordination costs (Component B) would need to be reduced through strategies such as an efficient parent consent and communication system. Overall, the per-vaccinated child cost estimates of our SLV-I were higher than those in medical practices and also higher than typical reimbursement rates. While our current cost estimates favor SLV-I over medical practices when we account for averted parental costs to visit medical practices (Component D), such cost-savings to parents may not be considered by health systems responsible for SLV-I. Thus, while costs are not the only consideration in setting up and sustaining SLV-I [
56,
57], the program costs for SLV-I should be lower than practice-located costs, or at least lower than or equal to reimbursement rates for SLV-I to be sustained.
Finally, achieving higher in-school vaccination coverage would improve cost-effectiveness. For example, in this study a net vaccination rate of 17.6 % (rather than 11.4 % found in Year 2) would lead to SLV-I cost estimates being lower than those in medical practices.