Comment on: ‘Clinical and Economic Impact of a Potential Switch from 13-Valent to 10-Valent Pneumococcal Conjugate Infant Vaccination in Canada’, Wilson et al., 22 June 2018

In a recent article, Wilson et al. estimated the health and economic impact of switching the pediatric immunization program from the 13-valent (PCV-13) to the 10-valent (PHiD-CV) pneumococcal conjugate vaccine in Canada [1]. In this letter, we would like to highlight a number of methodologic issues that hamper the results that were not included in the original article.

Wilson and colleagues utilized regional pneumococcal incidence data from the Toronto Invasive Bacterial Diseases Network (TIBDN) as the basis of their analysis. The question is whether this is representative for the whole of Canada and why none of the available national data sets were used that would have provided a more robust foundation for the analysis. These include the Canadian Notifiable Disease Surveillance System (CNDSS), the Immunization Monitoring Program, ACTive (IMPACT) network, and the National Enhanced Invasive Pneumococcal Disease Surveillance System (eIPD) pilot [2]. The Public Health Agency of Canada also publishes annual reports of the National Laboratory Surveillance from invasive pneumococcal diseases (IPD) in Canada [3]. These are all sources of data with broader national reach. Differences in serotype distribution as well as immunization rates vary between provinces and would potentially affect serotype response [4]. Furthermore, the incidence figures presented in Wilson et al. are not representative of the most recent IPD incidence rates in the National Laboratory Surveillance [3], suggesting that such arbitrary use of regional data is not appropriate for a Canada-wide analysis.

The regression-based decision-analytic modeling approach used by Wilson and colleagues is not an approach recommended by the International Society for Pharmacoeconomics and Outcomes Research (ISPOR) or the Society for Medical Decision Making (SMDM) [5]. ISPOR and SMDM drafted a series of articles on best practices in the conduct and reporting of modeling studies to ensure reliable and valid results [5]. State-transition models, discrete event simulations, and dynamic transmission models were identified as best-practice modeling techniques subject to the original decision problem. The outcomes when extrapolated over an additional 10-year period may result in uncertainties being grossly magnified; such extrapolation distortions compromise data integrity.

Wilson and colleagues performed age- and serotype-specific regression analyses, resulting in many underlying regression outcomes for which the fit cannot be evaluated. While the authors provided readers with the R² values for each regression equation [ranging from very weak (0.0091) to good (0.9990)], there was no validation of the predicted versus the observed historical data. Additionally, R² values alone cannot determine whether the predictions are biased nor can they indicate the fit of a model (for this, one must consult a goodness-of-fit statistic). Thus, the underlying analysis is built upon multiple fitted parameters for which one cannot determine the fit or bias. Using this approach as a basis for an economic evaluation can never result in confidence of the outcomes. ISPOR guidelines also state that vital model parameters be tested in uncertainty analyses [6]. It is concerning that the authors have not included the slope parameters of any of the covered/uncovered serotypes into any of the sensitivity analyses. As can be seen from the scenario analysis results (Table 4), the source of the data used for the trend analyses significantly impacts the resulting incremental cost and quality-adjusted life-years (QALYs).

Wilson and colleagues have drawn conclusions from this study that are inconsistent with those drawn by the Pan American Health Organization and the World Health Organization on the comparability of the different pneumococcal conjugate vaccine (PCV) products [7, 8]. Particularly, studies have shown that PCVs of differing valencies afford similar levels of protection against all-cause pneumonia and IPD. Neglecting this body of evidence, the authors have applied the same previously explained incorrectly assumed yearly impact (rate reduction) on IPD to the all-cause pneumonia and acute otitis media (AOM). The incidences of these two diseases are orders of magnitude higher than IPD, and thus the use of this approach vastly distorts the potential impact of any switch.

In Canada, PHiD-CV is indicated for the prevention of diseases caused by Streptococcus pneumoniae serotypes 1, 4, 5, 6B, 7F, 9V, 14, 18C, 19F, 23F, and cross-reactive 19A [9], based on post-marketing effectiveness studies conducted in Quebec, Brazil, and Finland. In these studies, the vaccine effectiveness (VE) for PHiD-CV against 19A IPD cases was 71, 82, and 62%, respectively [10‐12]. However, the approach used by Wilson et al. did not account for this cross-reactivity against serotype 19A. According to the Canadian product monograph for PHiD-CV, the VE for AOM cases caused by nontypeable Haemophilus influenzae (NTHi) was 15% [9]. The protection against AOM cases caused by NTHi was not included in the analysis by Wilson et al., even though previous cost-effectiveness analyses have shown that when the NTHi effect is included, PHiD-CV is more cost-effective than PCV-13 [13, 14]. By not including the totality of the published literature, the authors have introduced bias into their study leading to a distortion of the impact of a switch from PCV-13 to PHiD-CV in Canada. The results are also inconsistent with health economic analyses conducted by the Comité sur l’immunisation du Québec (CIQ) demonstrating that PHiD-CV was more cost-effective compared with PCV-13 in Quebec [15].

While the results of Wilson et al. suggest that switching from PCV-13 to PHiD-CV would result in increased costs of $500 million over 10 years, the methodology used and the data on which the analysis is based are prone to bias and likely to distort the resulting conclusions. Caution should be used when interpreting the results of this study as the totality of the evidence was not included, the underlying data set was regionally specific and not nationally representative, and the methodology is prone to bias.