Evaluation of a phased pneumococcal conjugate vaccine introduction in Mongolia using enhanced pneumonia surveillance and community carriage surveys: a study protocol for a prospective observational study and lessons learned

Streptococcus pneumoniae (the pneumococcus) is a major cause of illness and mortality among children worldwide, causing otitis media, meningitis, and pneumonia [1]. Pneumococcal conjugate vaccines (PCV) are widely used in high-income countries, and in recent years there has been increased introduction among low- and middle-income countries (LMICs).

With any new vaccine introduction, countries seek evidence of PCV impact in their own population to guide decision-making and allocation of limited health resources. In assessing the impact of PCV, many countries have used invasive pneumococcal disease (IPD) as the primary endpoint [2‐7], however this endpoint can be relatively rare. In such settings, large impact studies (usually in the tens of thousands) are needed to account for year-to-year variation in case numbers and changes in serotype distribution. IPD relies on robust sample collection (prior to antibiotic use) and diagnostic clinical microbiology capacity, which is often not available in resource-limited settings. Moreover, restricting the evaluation of PCV impact to IPD misses the opportunity to measure vaccine impact on non-bacteraemic pneumonia, one of the most common and significant illnesses caused by the pneumococcus.

Pneumonia is the most common cause of childhood death globally. Pneumococcal pneumonia contributes substantially to the most severe and often fatal forms of pneumonia [8]. There are few data on the impact of PCV on pneumonia in LMICs, due in part to inherent challenges in diagnosing pneumonia and establishing a cause-specific aetiology.

Nasopharyngeal pneumococcal carriage is a precursor to pneumococcal disease [9]. PCVs reduce vaccine-type (VT) serotype carriage while overall pneumococcal carriage is only slightly decreased or unchanged due to serotype replacement. Serotype replacement has been documented following the introduction of PCV [10, 11] and it is critical to consider and monitor as part of long-term vaccine impact. Isolating a pneumococcus from a nasopharyngeal swab in a child with pneumonia does not necessarily indicate a pneumococcal aetiology. However a reduction in the detection of VT serotypes over time will reflect vaccine effectiveness in preventing pneumococcal pneumonia episodes due to those serotypes [12].

Mongolia is a land-locked country in Central Asia. Nearly half of its 3 million people live in the capital, Ulaanbaatar, with the remainder being dispersed over a vast geographic area. In Mongolia, pneumonia is one of the most common causes of mortality among children, accounting for 25% of deaths among children under the age of 5 outside of the neonatal period [13]. The Government of Mongolia (GoM) decided to introduce the 13-valent PCV (PCV13) and to monitor its impact. A collaboration was established between the GoM, the World Health Organization (WHO) and the Murdoch Children’s Research Institute (MCRI) to evaluate the impact of PCV introduction in Mongolia. In the context of relatively low rates of culture-confirmed IPD, and an existing hospital-based surveillance system enrolling cases of pneumonia, we selected pneumonia as the primary endpoint. Here we describe the methodological approach taken to develop prospective hospital-based surveillance for measurement of vaccine impact on pneumonia, the use of community carriage surveys and some of the key challenges and lessons learnt in this ongoing surveillance programme. We believe that this information will benefit other resource-limited countries seeking to establish pneumonia surveillance for the measurement of vaccine impact.

An evaluation plan was developed to use enhanced hospital-based pneumonia surveillance to document the impact of PCV on hospitalised pneumonia in children aged 2–59 months, and cross-sectional carriage surveys of healthy children to measure vaccine impact on community carriage of pneumococci. Additionally, we will evaluate the PCV impact on severe and very severe lower respiratory tract infection (LRTI) cases associated with respiratory syncytial virus (RSV) and influenza virus in a subset of children aged 2–23 months.

This evaluation has been approved by the Human Research Ethics committees of the Mongolian Ministry of Health, the Royal Children’s Hospital (Melbourne) and the World Health Organization Western Pacific Regional Office.

Overall, there are six study objectives:

Mongolia has a very well-structured health system with accurate data on the number of under-5 children in every district. The vast majority of healthcare for children is provided through the public health system with few children accessing the private sector. Most pneumonia cases are therefore captured by the public health services in district and tertiary care facilities. Hospital care for children is geographically and financially accessible in Ulaanbaatar, with inpatient care provided free of charge for children. As such, the vast majority of cases of severe pneumonia among age-eligible children residing in the evaluation districts will be captured by the surveillance system. The surveillance system may thus be considered population-based, permitting the calculation of incidence rates.

The GoM is introducing PCV13 in a phased manner. This approach is consistent with the standard approach taken to new vaccine implementation in the country, and also facilitates rigorous evaluation of impact by allowing for contemporaneous comparison of vaccinated and unvaccinated populations, rather than relying on before and after comparisons alone. The first phase of PCV13 introduction in two of the evaluation districts (Songinokhairkhan and Sukhbaatar, termed “Phase 1 districts”) commenced at the start of June 2016 (Table 1), followed by stepped introduction in the remaining evaluation districts (“Phase 2 districts”). PCV13 was introduced into one Phase 2 district (Bayanzurkh) in June 2017, with introduction into the second Phase 2 district (Chingeltei) and other districts of Ulaanbaatar from March 2018.

PCV13 was implemented as a three-dose schedule, with doses given to infants at two, four and 9 months of age. There was a three-month catch-up campaign at the time of vaccine introduction in 2016 and 2017 in each of the evaluation districts for children up to 23 months of age, who were given two doses of PCV13 (2 months apart) to maximise vaccine impact, alongside vaccine introduction in 2016 and 2017. In 2018 vaccine introduction into Chingeltei and the remaining districts of Ulaanbaatar was done without a catch-up component.

Establishing a robust surveillance system is an important component of monitoring the impact of PCV within a country. The enhanced surveillance system in Mongolia will facilitate the assessment of PCV13 impact on pneumonia, with radiological confirmed disease as the primary outcome. Establishing the surveillance system was not without its challenges, but the lessons learned in addressing these challenges may be of relevance to other countries seeking to establish surveillance for pneumonia.

There have been four key lessons learned thus far. The first lesson has been the importance of establishing a core group of hospital staff at each site to be responsible for the surveillance activities, and for the PCV impact evaluation team to work closely and collaboratively with each hospital surveillance team. The hospital teams should be active partners in the evaluation activity. Such partnership supports the collection of high quality data and ensures that any problems can be identified and addressed rapidly. The winter of 2015/16 in Mongolia was very severe, and hospitals were finding it difficult to manage surveillance enrolment with such a high patient load. In some hospitals, staff from other wards were brought in to relieve the burden in the admissions and paediatric wards, but several of these staff had not been trained on how to enrol patients into the surveillance system. It was not until later, when we reviewed the declining enrolment rate, that these issues were identified. We have since increased our focus on having more regular engagement with hospital teams and actively encourage teams to share with us their experiences in implementing the surveillance.

The second lesson is that a number of external factors can potentially influence disease burden estimates, by influencing the volume of patients presenting to hospital and their severity of disease. It is necessary to maintain awareness of such factors, and to implement strategies to be able to quantify any impact on surveillance-related outcomes. A significant external factor we have identified in this study is a change in referral practice. When our enhanced surveillance commenced, antibiotics included in the IMCI algorithms were available free-of-charge for inpatients only. In 2016, these antibiotics became available at no cost for outpatients presenting to Family Health Centres, which almost certainly would have resulted in a larger proportion of patients being treated as outpatients rather than being referred to hospital. Similarly, the 2013 revision of the WHO IMCI guidelines for the management of acute respiratory infection (removing chest indrawing as a sign of severe pneumonia requiring hospitalisation) was formally adopted in Mongolia in 2015. The adoption of these revised guidelines during the evaluation period would likely result in a reduction in the number of patients being referred for hospital care. The effect of these changes will be reviewed. Having mechanisms that enable regular communication and cooperation with key agencies, both within and outside of health, can help to ensure that potential external factors are identified.

The third lesson is that flexibility in data collection and processes is also necessary, in order to respond to changing circumstances. During 2015–2016 there was a large outbreak of measles in Mongolia. Cases emerged from March 2015, and by the end of that year it appeared that the epidemic was ending; however, in early 2016 there was a major resurgence that continued until July. As measles substantially increases the risk of pneumonia [41], there is potential for this outbreak to result in an increase in pneumonia cases, thus complicating our measurement of PCV13 impact. To be able to quantify the number of potentially measles-associated pneumonia cases being captured in the surveillance system, it was necessary to modify the CRFs and conduct a training update with hospital staff in order to collect data on the measles history of each surveillance case.

The fourth lesson is to ensure a consistent application of the pneumonia surveillance case definition. It is challenging to develop and apply a clear case definition that achieves a balance between capturing all cases that are likely true pneumonia, without overloading doctors with enrolment of all patients with acute respiratory infection, the most common cause of hospital admission for children in Mongolia and much of the world. A review of the existing IB-VPD surveillance system in early 2015, just prior to the commencement of enhanced surveillance, identified the problem of under-enrolment. Consultation with hospital staff identified a lack of clarity in the surveillance case definitions. In some hospitals, suspected pneumonia cases reporting prior antibiotic use (e.g. at home) prior to admission were also being excluded unnecessarily from the surveillance system (prior antibiotic use is recorded on the CRF, but not an exclusion criterion). In liaison with the Ministry of Health and WHO, the surveillance case definition was updated to be more specific (thus reducing the number of eligible cases and consequently the case load for admitting doctors). While this was an important step to ensure all cases could be captured, a shortcoming of this change to the definition is that it will be more difficult to compare results of impact on clinical pneumonia with those from the earlier clinical trials. To ensure that all hospital staff were using the correct case definition and maintain consistency in enrolment from the start of the enhanced surveillance, refresher training in the case definition was conducted in all hospitals. A number of printed resources were produced to assist hospital staff in correct application of the simplified surveillance case definition and to ensure that all necessary surveillance data are collected. This included a wall poster to be placed in the relevant areas in the hospitals, and easy-reference pocket guides to be carried by the doctors. The weekly enrolment rate monitoring system described above was also established. Frequent staff turnover in hospitals presents the need for frequent re-training, both in the case definition and in surveillance procedures. Strategies that we have implemented to meet this need for ongoing training include the train-the-trainer approach and production of short instructional videos.

Given the concern about correct application of the case definition and cases being missed, to ensure complete ascertainment of eligible cases for the first year of enhanced surveillance, a medical record review of all age-eligible pneumonia admissions to the 5 participating hospitals has been undertaken for the first year of enhanced surveillance. The culture of detailed medical record keeping in Mongolia has allowed us to retrospectively capture the majority of data from missed cases through review of their patient file, though unfortunately with the notable absence of radiology data. In addition a retrospective review will also be done for all pneumonia cases in children < 5 years from April 2013 to March 2015, and all empyema cases in children < 5 years from April 2010 to March 2015.

Radiological pneumonia has been shown to be the best pneumonia endpoint for pneumococcal disease [15‐18]. Radiology is therefore central to this evaluation, but it is an area in which many resource-constrained settings have limited scope to produce reliable, high quality data. In establishing the data collection systems for this evaluation, the PCV impact evaluation team worked very closely with hospital radiology departments to strengthen and support their systems, equipment and capacity. A comprehensive audit of equipment and processes as well as training was done prior to start-up of enrolment. These actions and the ongoing weekly quality assurance checks, has resulted in the production of high quality radiologic films for the study. Fewer than 3% of the 3100 films that had been re-read by the end of 2016 were uninterpretable for endpoint consolidation. This is a lower proportion than that reported in the PERCH study (10%) [42]. Ensuring adequate resources are available for ongoing radiology equipment maintenance and repair should be considered in developing evaluation budgets.

Virology testing is a novel aspect of this evaluation project because the clinical evidence in the context of pre- and post-PCV introduction is still limited. The information provided from this study will be critical not only for Mongolia but for many other countries as RSV is a leading pathogen in paediatric LRTI worldwide. These data will also provide a valuable baseline of seasonal, epidemiological burden of RSV and influenza infections in children less than 2 years of age with severe pneumonia. This will complement data collected by the Ministry of Health and WHO RSV surveillance system, which began in May 2017 as part of the influenza-like illness programme and detects milder cases of RSV in all age groups. Moreover, laboratory capacity building on RSV and influenza testing derived from this project has provided the NCCD Virology Laboratory with additional RNA extraction and qPCR machines and a higher capacity to deal with the high volume of samples derived from the project, as well as routine testing, and from the RSV and influenza surveillance. Following each qPCR run, results are loaded into a cloud account that can be accessed by all collaborators. The surveillance may have missed some children with RSV as the enhanced surveillance system did not enrol children with bronchiolitis and the oxygen cut-off was lower than that used for RSV [29].

While pneumococcal nasopharyngeal carriage is a precursor to pneumococcal disease, the interplay between the presence of pneumococci in the nasopharynx and pneumonia is poorly understood. High pneumococcal density (8000 copies/ml) from nasopharyngeal swabs has been shown to have high diagnostic accuracy in diagnosing pneumococcal pneumonia in adults [43]. In children, pneumococcal density in the nasopharynx is higher in children with radiological pneumonia compared to healthy controls or children with milder respiratory tract infection [44]. Nasopharyngeal swab analysis of pneumonia cases provide important information on likely pneumonia aetiology as well as additional information on the circulating serotypes in a population, and can assist in monitoring the direct and indirect effects of PCV13 introduction. To assess pneumococcal carriage we have applied gold-standard microbiological approaches. However, this strategy was also selected as the laboratory flow is relatively ‘modular’ facilitating strengthening of key assays such as lytA qPCR to support on-going surveillance, using the same DNA extraction and real-time PCR platforms as the virology assays. A phased strategy for long-term technology transfer has been developed with the project partners, and will continue to evolve as the project progresses. To date, activities have included a joint visit from the MCRI and IB-VPD microbiologist, an MCRI only visit and a laboratory training visit at MCRI for two NCCD laboratory personnel.

Finally, it is important to put mechanisms in place to ensure that the expanded surveillance and key laboratory capacity provided through the funded project can be sustained. For example, in the case of radiology, equipment was provided to some sites to enable the digitisation of x-rays. This is essential for the evaluation, as it enables the electronic archiving and transfer of images for re-reading and for archiving project data. In sites where this equipment could not be supplied, the project meets the cost of the x-ray film and printing reagents and an x-ray scanner is used to produce an electronic copy. However, in most hospitals there is no internal network that allows doctors to view the image on a computer elsewhere in the hospital, so it is impractical to only have an electronic copy of the radiograph. Without this internal network capability, and with very limited funds for printing x-rays, there is a risk that hospitals will revert to the earlier practice of fluoroscopy, the equipment provided will fall into disuse and the capacity that has been built among the local radiology teams to use the equipment will be lost. However, with the strengthened capacity for clinical research and surveillance created in country, due to exposure and participation of hospital staff, as well as the buy-in of the Ministry of Health to continue with the surveillance following the donor funding period, there is commitment to ensure that the programme is maintained.

Our study has a number of strengths. Firstly the methods used are consistent with other international research activities studying nasopharyngeal carriage, including studies undertaken by our group in Lao PDR and Fiji [45‐47]. Secondly we used sensitive gold-standard molecular methods to detect pneumococcus in the nasopharynx. Our method of microarray with a culture amplification step was the top-performing method in a large, international multi-centre study exploring the best pneumococcal serotyping methods for carriage studies [31], and will provide density data to the level of serotype. Thirdly, the identification of radiological pneumonia cases that have an elevated CRP [48], and have either heavy nasopharyngeal carriage, or carriage of pneumonia-associated serotypes, will identify a subgroup that is more likely to be pneumococcal in origin and may thus be expected to decline in incidence more sharply than the total radiological incidence with the introduction of an effective PCV. Fourthly, our community carriage surveys include children too young to be vaccinated (5–8 weeks of age). This allows us to document the herd immunity in very young infants who are at high risk of pneumococcal disease, especially meningitis.

The PCV impact evaluation in Mongolia will provide data to help guide national vaccine policy, and contribute to the limited evidence base on the burden of pneumococcal pneumonia in LMICs, particularly in Asia. In addition, it provides one of the first experiences of implementing prospective, population-based pneumonia surveillance to evaluate the impact of PCV in a resource-limited setting. With increasing introduction of PCV in these settings, the methods, experiences and lessons learned from this activity may be used to guide the development of such systems in other areas.