Evaluation of case definitions to detect respiratory syncytial virus infection in hospitalized children below 5 years in Rural Western Kenya, 2009–2013

Respiratory syncytial virus (RSV) is a leading viral cause of lower respiratory tract infection in infants and young children, and an important cause of neonatal death in Kenya [1‐9] and worldwide [10‐12]. Testing for RSV is not routinely performed in resource-poor settings, and the utility of clinical case definitions for measuring RSV disease in these contexts is not well described. In Kenya, case definitions designed for other purposes, such as for influenza surveillance, have been used to estimate RSV burden [7, 9]. However, the most sensitive and or specific clinical case definitions for detection of RSV among patients with respiratory illness remain underexplored. In addition, with candidate RSV vaccines in development [2, 13], studies are needed to fully assess RSV disease burden and standardized RSV case definitions will be needed to evaluate vaccine efficacy and impact. An understanding of the performance of different clinical case definitions for identifying RSV infection can guide research and surveillance methodology.

We used data collected between September 2009 - August 2013 from children aged <5 years hospitalized with any respiratory illness at Siaya County Referral Hospital. We evaluated the sensitivity, specificity, positive predictive value (PPV) and negative predictive value (NPV) of individual respiratory signs, symptoms and case definitions used for respiratory disease surveillance, including severe acute respiratory infection (SARI), Integrated Management of Childhood Illness (IMCI) non-severe, severe and very severe pneumonia and a hospitalized influenza like illness (hILI) case definition to detect laboratory-confirmed RSV infections.

During September 2009 − August 2013, a total of 4714 children aged <5 years were hospitalized with an ARI. Of these, 3878 (82.3 %) had NP/OP swabs collected; reasons for non-collection (n = 836) included refusal (n = 464, 55.5 %), death before sample collection (n = 156, 18.7 %), transfer to other facilities before sample collection (n = 30, 3.6 %), too ill to have sample collected (n = 22, 2.6 %), and other (n = 164, 19.6 %). Among the 3878 with swabs collected, 3810 (98.3 %) had RSV test results available; the remaining 68 (1.7 %) were not tested in the laboratory because the vials were either open or the transport media volume did not meet the required threshold. RSV was detected in 470 (12.3 %) of the children tested.

The mean age and oxygen saturation was similar between those tested and untested for RSV, (mean age 1.4 years vs. 1.4 years, p = 0.99) and (mean oxygen saturation 93.7 vs. 93.4, p = 0.26) respectively. Of those tested for RSV, 54.0 % were males compared to 58.0 % of the untested (p = 0.03). Approximately 3 % (103/3810) of those tested for RSV died compared to 19.6 % (177/904) of ARI cases not tested for RSV (p < 0.01), (Table 2).

Slightly more than half 238 (50.6 %) of RSV-positive cases were aged <1 year, (Table 3) and more than three-quarters of all cases were aged <2 years, 360 (76.6 %). Cough, difficulty breathing, night sweats, hypoxia, wheezing, stridor, nasal flaring and chest wall in-drawing were significantly associated with having a laboratory-confirmed RSV infection, (Table 3). The symptom/sign most strongly associated with RSV was cough (RR, 2.56; 95 % CI, 1.60-4.10), (Table 3). The presence of fever (measured or reported) was not associated with RSV infection.

Among the respiratory signs and symptoms examined, cough alone had the highest sensitivity for laboratory confirmed RSV (96.4 %, 95 % CI 94.7-98.1), followed by night sweats, difficulty breathing and tachypnea; (71.7 %, 95 % CI 67.6-75.8), (61.9 %, 95 % CI 57.5-66.3) and (61.3 %, 95 % CI 56.9-65.7) respectively. The other individual signs and symptoms evaluated had sensitivities that ranged from 8–31 %, (Table 4). The presence of hypoxia, wheezing, stridor, nasal flaring and chest wall in-drawing all had specificities >77 % for laboratory confirmed RSV (Table 4).

The standard respiratory case definition with the highest sensitivity for detecting RSV infection was SARI (82.8 %, 95 % CI 79.4-86.2), followed by IMCI very severe pneumonia (50.2 %, 95 % CI 45.7-54.7), (Table 4. Fig. 1). The sensitivities for IMCI non-severe pneumonia, IMCI severe pneumonia and hILI were lower; (20.2 %, 95 % CI 16.6-23.8), (15.1 %, 95 % CI 11.9-18.3) and (23.8 %, 95 % CI 20.0-27.7) respectively. The highest specificity was observed in IMCI severe pneumonia (90.7 %, 95 % CI 89.7-91.7), closely followed by hILI and IMCI non-severe pneumonia definitions; (79.7 %, 95 % CI 78.3-81.1) and (76.7 %, 95 % CI 75.3-78.2) respectively. SARI and IMCI very severe pneumonia had specificities of 22.8 % (95 % CI 21.4-24.2) and 48.5 % (95 % CI 46.8-50.2), respectively. The alternative case definition developed through logistic regression; cough or difficulty breathing plus hypoxia, chest in-drawing, or wheeze had a sensitivity and specificity of (54.7 %, 95 % CI 50.2-59.2) and (60.1 %, 95 % CI 58.9-62.3) respectively. We plotted a receiver operator characteristics chart for these case definitions, (Fig. 1). The PPV for all case definitions, signs and symptoms assessed ranged between 11–20 % whereas the negative predictive values were >87 %, (Table 4).

When stratified by age group, SARI continued to be the most sensitive case definition (79.4 % sensitive in infants aged <1 year and 86.2 % sensitive in children aged 1- < 5 years), and IMCI severe pneumonia was the most specific (88.5 % in infants aged <1 year and 92.6 % in children aged 1- < 5 years). Comparing across age groups, IMCI severe pneumonia was significantly more sensitive (19.3 % vs. 10.8 %, p = 0.01) and less specific (88.5 % vs. 92.6 %, p < 0.01) among those aged <1 year compared to children aged 1- < 5 years (Table 4). Cough or difficulty breathing plus hypoxia, chest in-drawing or wheeze was also significantly more sensitive (66.0 % vs. 43.1 %, p < 0.01) and less specific (54.0 % vs. 66.1 %, p < 0.01) among the younger age group. Difficulty breathing was also significantly more sensitive (69.8 % vs. 53.9 %, p < 0.01) and less specific (43.2 % vs. 53.2 %, p < 0.01) among the younger age group. IMCI non-severe pneumonia was significantly less sensitive (14.3 % vs. 26.3 %, p < 0.01) and more specific (79.5 % vs. 74.5 %, p < 0.01) among infants aged <1 year compared to children aged 1- < 5 years, (Table 4). There was no significant difference in sensitivity, specificity and predictive values for the other case definitions between the <1 and 1- < 5 age groups. When we stratified by season, we observed significant difference in the PPVs for all signs, symptoms and case definitions evaluated, with significantly higher PPVs in the RSV peak season, (Table 5). There was no statistically significant difference in the sensitivities and specificities of the case definitions by number of days from symptom onset, (Figs. 2 and 3) except for SARI where specificity decreased from day 1 to 3 (46.2–25.4 %, p < 0.01) then stabilized, (Fig. 3).

We observed wide variations in the sensitivity and specificity of the signs, symptoms and case definitions evaluated. SARI, a case definition requiring cough and either reported or measured fever and developed for global influenza surveillance was the most sensitive case definition for RSV detection with sensitivity of 83 %. The most specific case definition was IMCI severe pneumonia (cough or difficulty breathing plus chest in-drawing). Generally there was a trade-off between sensitivity and specificity. Cough or difficulty breathing plus hypoxia, chest in-drawing or wheeze provided the best balance; it was the only case definition for which both sensitivity and specificity exceeded 50 %. The predictive values were more consistent across case definitions, with all having relatively low PPV (11–20 %) and high NPV (87–90 %).

In determining the optimal case definition, consideration should be given to the context in which the case definition applies and the objectives for which it is being used. If the primary objective of the study is to estimate the burden of RSV disease then it is important to use a case definition offering high sensitivity. We found that cough alone had the highest sensitivity (96 %) to detect RSV. If the goal was to detect the maximum number of RSV cases then cough alone might be the best definition to use. However, using cough alone would also give very many false positives (PPV 13 %). Of the case definitions evaluated we found that the 2013 WHO SARI case definition was most sensitive for RSV detection. Nonetheless, the SARI case definition may miss up to 17 % of RSV cases. Much of what is known about RSV globally comes from surveillance systems that utilize case definitions intended for other diseases (e.g. influenza, pneumonia) that are not optimized for detection of RSV, the potential for underestimation should therefore be recognized in interpreting data on RSV burden, particularly if using a case definition that requires measured fever (e.g. hILI) or includes only pneumonia without danger signs (e.g. IMCI non-severe or severe pneumonia).

Sometimes maximum specificity is desired for RSV detection. If RSV vaccines currently under development are eventually introduced into routine immunization programs there will be a need to evaluate the impact and effectiveness of the vaccine. Given that RSV testing is not widely available in resource poor settings, a clinical case definition may be most practical for use in vaccine evaluations such as case–control vaccine effectiveness studies or analyses of trends in disease burden. For this purpose, a highly specific case definition is preferable [21, 22]. We found that the presence of stridor offered the highest specificity (96 %). However, it was present in < 8 % of cases, and therefore not easily embraceable as a case definition. Among the case definitions evaluated, we found that IMCI severe pneumonia was the most specific (91 %), while hILI and non-severe pneumonia case definitions also provided reasonable specificity (77–80 %). Measuring vaccine impact and effectiveness using the IMCI severe pneumonia case definitions may be more feasible than using less specific case definitions such as SARI. However, even if the most specific case definition were used, most cases identified may not be RSV-infected because all the case definitions were found to have poor PPV.

We observed positive predictive values (PPVs) ranging from 11–20 % for all signs, symptoms and case definitions, though the PPVs were significantly higher during periods of RSV circulation (cold months of the year in Western Kenya; April through July). A case definition with a high PPV is useful to investigators aiming to maximize the detection of laboratory confirmed cases among those tested. For example, studies focused on subtyping or sequencing for which the positive RSV isolate is essential may benefit from a case definition with optimal PPV. In this analysis, the case definition with the highest PPV for detection of laboratory confirmed RSV was IMCI severe pneumonia. However, even during peak RSV season, roughly one in three children with IMCI severe pneumonia will have a positive RSV test result.

Although the prevalence of RSV infection is generally higher among infants hospitalized with respiratory disease compared with older children hospitalized with respiratory disease [1], we did not observe significant differences in PPV when stratifying by age group. Difficulty breathing, severe pneumonia and the alternative definition were found to be more sensitive among the infants aged <1 year, while IMCI non-severe pneumonia was more sensitive among those aged 1- < 5 years. Older children with RSV are likely to be experiencing a reinfection rather than a primary infection, and therefore may present with milder illness [4]. These findings highlight that choice of case definition for detecting RSV cases should take into account the age group under surveillance.

We found that sensitivity, specificity and predictive values for the case definitions to detect RSV infections did not vary significantly with respect to duration from illness onset to sample collection, suggesting that these case definitions can be used within the 14-day period from symptom onset to detect RSV infection without concern about loss of sensitivity or specificity.

The presence of wheezing, hypoxia, stridor, nasal flaring and chest wall in-drawing were each found to be significantly associated with RSV infection. This finding is similar to findings from Rarieda and Kilifi districts in Kenya and Ballabgarh in northern India [1, 6, 9, 23] and suggest that some of the individual IMCI danger signs might be used to identify subsets of children more likely to have RSV virus infections. It should however be noted that there is some variation in the literature regarding the association of hypoxia with RSV infections. Hypoxia was found to be negatively associated with RSV infection in one study conducted in Kilifi District, Kenya [1], with another study reporting no significant association from the same site [6]. We found that neither measured nor reported fever were associated with RSV infection, however history of fever was a common symptom observed in nearly 90 % of children infected with RSV. Fever was a common symptom in many RSV infected patients in other studies as well [23‐25] with one study [6] observing that fever was significantly more common among the RSV-infected children compared to children with respiratory disease that tested negative for RSV, RR = 1.54(95 % CI, 1.33–1.80). The high prevalence of malaria in our study site [14, 15] may impact the observed association between fever and RSV.

Interestingly, the performance of various case definitions for detecting RSV infection that we observed in this study differed substantially from the findings of a study in Ballabgarh, northern India, for which all hospitalized children aged <5 years were tested for RSV [23]. While we found the SARI case definition to be the most sensitive, and severe IMCI pneumonia to be most specific, that study reported SARI to have a low sensitivity and IMCI pneumonia or severe pneumonia (combined) to have a low specificity. Differences in epidemiologic contexts (e.g. malaria or HIV prevalence) may explain some of the variation in results. The precise definitions used for SARI and IMCI pneumonia also differed across the two studies. Furthermore, the study from India included all hospitalized children (regardless of clinical signs), while our study included only children hospitalized with cough or difficulty breathing or chest pain. Of note, the Indian study found that restricting to even a very broad respiratory case definition (acute onset cough or sore throat or shortness of breath or coryza or clinician judgement that illness due to infection) would fail to detect 13.5 % (10/74) of RSV cases; thus our analysis likely failed to include some children hospitalized with RSV infection.

In interpreting our results, the following additional limitations should be taken into consideration. This analysis was restricted to inpatients; the findings therefore may not be relevant for outpatient respiratory disease surveillance. Furthermore, because non-severe IMCI pneumonia is typically managed in the outpatient setting in Kenya, those children with non-severe pneumonia included in this analysis may be a somewhat biased group. In addition, a substantial number of severe cases were not swabbed and consequently excluded from this analysis; therefore these findings may not be representative of the most severe spectrum of cases with RSV infection. Lastly, RT-PCR was the only method used for RSV detection in our surveillance system; therefore sensitivities and specificities could vary in studies where other methods such as culture or serology are used.

The optimal case definition for use in detecting RSV varies depending upon context. Researchers must take into account their objectives, available resources, and the study population. For estimating the burden of RSV in a context similar to Western Kenya, the WHO SARI case definition could be used, particularly if it is already being implemented in a sentinel influenza surveillance system. However more specific case definitions may be required for evaluating future RSV vaccines in settings where diagnostic testing is not readily available. Additional research to assess the performance of various case definitions for detecting RSV infections across a broad range of epidemiologic contexts is needed.