Skip to main content
Advertisement
Browse Subject Areas
?

Click through the PLOS taxonomy to find articles in your field.

For more information about PLOS Subject Areas, click here.

  • Loading metrics

Host biomarkers and combinatorial scores for the detection of serious and invasive bacterial infection in pediatric patients with fever without source

  • Laurence Lacroix ,

    Roles Conceptualization, Data curation, Formal analysis, Investigation, Methodology, Project administration, Resources, Supervision, Validation, Visualization, Writing – original draft, Writing – review & editing

    laurence.lacroix@hcuge.ch

    Affiliations Pediatric Emergency Department, Children’s Hospital, University Hospitals of Geneva, Geneva, Switzerland, Faculty of Medicine, University of Geneva, Geneva, Switzerland

  • Sebastien Papis,

    Roles Data curation, Formal analysis, Investigation, Writing – review & editing

    Affiliation Department of General Pediatrics, Children’s Hospital, University Hospitals of Geneva, Geneva, Switzerland

  • Chiara Mardegan,

    Roles Data curation, Investigation, Writing – review & editing

    Affiliation Department of General Pediatrics, Children’s Hospital, University Hospitals of Geneva, Geneva, Switzerland

  • Fanny Luterbacher,

    Roles Data curation, Investigation, Writing – review & editing

    Affiliation Department of General Pediatrics, Children’s Hospital, University Hospitals of Geneva, Geneva, Switzerland

  • Arnaud L’Huillier,

    Roles Conceptualization, Data curation, Formal analysis, Funding acquisition, Investigation, Methodology, Project administration, Resources, Supervision, Validation, Visualization, Writing – review & editing

    Affiliations Faculty of Medicine, University of Geneva, Geneva, Switzerland, Department of Pediatric Infectious Diseases, Children’s Hospital, University Hospitals of Geneva, Geneva, Switzerland

  • Cyril Sahyoun,

    Roles Formal analysis, Investigation, Writing – review & editing

    Affiliations Pediatric Emergency Department, Children’s Hospital, University Hospitals of Geneva, Geneva, Switzerland, Faculty of Medicine, University of Geneva, Geneva, Switzerland

  • Kristina Keitel,

    Roles Formal analysis, Investigation, Writing – review & editing

    Affiliation Pediatric Emergency Department, Inselspital, Bern, Switzerland

  • Niv Mastboim,

    Roles Formal analysis, Investigation, Writing – review & editing

    Affiliation MeMed, Tirat Carmel, Israel

  • Liat Etshtein,

    Roles Formal analysis, Investigation, Writing – review & editing

    Affiliation MeMed, Tirat Carmel, Israel

  • Liran Shani,

    Roles Formal analysis, Investigation, Writing – review & editing

    Affiliation MeMed, Tirat Carmel, Israel

  • Einav Simon,

    Roles Formal analysis, Investigation, Writing – review & editing

    Affiliation MeMed, Tirat Carmel, Israel

  • Eran Barash,

    Roles Formal analysis, Investigation, Writing – review & editing

    Affiliation MeMed, Tirat Carmel, Israel

  • Roy Navon,

    Roles Formal analysis, Investigation, Writing – review & editing

    Affiliation MeMed, Tirat Carmel, Israel

  • Tanya M. Gottlieb,

    Roles Formal analysis, Investigation, Writing – review & editing

    Affiliation MeMed, Tirat Carmel, Israel

  • Kfir Oved,

    Roles Formal analysis, Investigation, Writing – review & editing

    Affiliation MeMed, Tirat Carmel, Israel

  • Eran Eden,

    Roles Formal analysis, Investigation, Writing – review & editing

    Affiliation MeMed, Tirat Carmel, Israel

  • Christophe Combescure,

    Roles Conceptualization, Formal analysis, Methodology, Writing – original draft, Writing – review & editing

    Affiliation Department of Clinical Epidemiology, University Hospitals of Geneva, Geneva, Switzerland

  • Annick Galetto-Lacour ,

    Roles Conceptualization, Data curation, Formal analysis, Funding acquisition, Investigation, Methodology, Project administration, Resources, Supervision, Validation, Visualization, Writing – original draft, Writing – review & editing

    ‡ AGL and AG are contributed equally to this work and joint senior authors.

    Affiliations Pediatric Emergency Department, Children’s Hospital, University Hospitals of Geneva, Geneva, Switzerland, Faculty of Medicine, University of Geneva, Geneva, Switzerland

  •  [ ... ],
  • Alain Gervaix

    Roles Conceptualization, Formal analysis, Funding acquisition, Investigation, Methodology, Project administration, Resources, Supervision, Validation, Visualization, Writing – original draft, Writing – review & editing

    ‡ AGL and AG are contributed equally to this work and joint senior authors.

    Affiliations Pediatric Emergency Department, Children’s Hospital, University Hospitals of Geneva, Geneva, Switzerland, Faculty of Medicine, University of Geneva, Geneva, Switzerland

  • [ view all ]
  • [ view less ]

Abstract

Background

Improved tools are required to detect bacterial infection in children with fever without source (FWS), especially when younger than 3 years old. The aim of the present study was to investigate the diagnostic accuracy of a host signature combining for the first time two viral-induced biomarkers, tumor necrosis factor-related apoptosis-inducing ligand (TRAIL) and interferon γ-induced protein-10 (IP-10), with a bacterial-induced one, C-reactive protein (CRP), to reliably predict bacterial infection in children with fever without source (FWS) and to compare its performance to routine individual biomarkers (CRP, procalcitonin (PCT), white blood cell and absolute neutrophil counts, TRAIL, and IP-10) and to the Labscore.

Methods

This was a prospective diagnostic accuracy study conducted in a single tertiary center in children aged less than 3 years old presenting with FWS. Reference standard etiology (bacterial or viral) was assigned by a panel of three independent experts. Diagnostic accuracy (AUC, sensitivity, specificity) of host individual biomarkers and combinatorial scores was evaluated in comparison to reference standard outcomes (expert panel adjudication and microbiological diagnosis).

Results

241 patients were included. 68 of them (28%) were diagnosed with a bacterial infection and 5 (2%) with invasive bacterial infection (IBI). Labscore, ImmunoXpert, and CRP attained the highest AUC values for the detection of bacterial infection, respectively 0.854 (0.804–0.905), 0.827 (0.764–0.890), and 0.807 (0.744–0.869). Labscore and ImmunoXpert outperformed the other single biomarkers with higher sensitivity and/or specificity and showed comparable performance to one another although slightly reduced sensitivity in children < 90 days of age.

Conclusion

Labscore and ImmunoXpert demonstrate high diagnostic accuracy for safely discriminating bacterial infection in children with FWS aged under and over 90 days, supporting their adoption in the assessment of febrile patients.

Introduction

Fever without source (FWS) is a common presenting symptom in children less than 3 years old [1]. It is defined as fever 38.0°C/100.4°F and above for less than 7 days with no underlying cause despite a thorough history and physical exam [2, 3]. Being able to differentiate serious bacterial infections (SBI) or invasive bacterial infections (IBI) necessitating immediate antibiotic treatment from focal bacterial infections or viral infections remains crucial although often challenging, because of lack of specificity in symptoms and signs [4, 5]. Diagnosis is therefore frequently supported by biological exams, such as determination of C-reactive protein (CRP) and procalcitonin (PCT) values. The Labscore takes into account three variables which are independently associated with SBI, weighed according to the odds ratio in the univariate analysis in the original derivation study [6]: 1) PCT (≥2 ng/mL: 4 points, 0.5–2.0 ng/mL: 2 points, <0.5 ng/mL: 0 point) 2) CRP (≥100 mg/L: 4 points, 40–99 mg/L: 2 points, < 40 mg/L: 0 point) and 3) urine dipstick (positive, ie, positive leukocyte esterase and/or positive nitrate: 1 point, negative: 0 point). Consequently, Labscore values range from 0 to 9 points. A cutoff point > = 3 was identified as the best Labscore value for SBI prediction, even when applied to a large external cohorts of children in the same age range [7, 8].

Whereas most biomarkers or combinations of biomarkers are responsive to bacterial infections, a new assay (ImmunoXpert) intended for children aged 90 days and more and adults has been recently developed. It combines for the first time two viral-induced biomarkers, tumor necrosis factor-related apoptosis-inducing ligand (TRAIL) and interferon γ-induced protein-10 (IP-10), with a bacterial-induced one, C-reactive protein (CRP) [9]. It displays three possible outcomes: (1) Viral infection (or non-bacterial etiology): ImmunoXpert score < 35; (2) Equivocal: 35 ≤ ImmunoXpert score ≤ 65; and (3) Bacterial infection (including mixed bacterial and viral co-infection): ImmunoXpert score > 65. This signature has already shown high diagnostic accuracy in a heterogeneous study population of febrile inpatients and emergency department arrivals, both children and adults, presenting with diverse clinical syndromes and pathogens [913]. Biomarkers or combination of biomarkers including the recently described ImmunoXpert have been frequently studied for their clinical utility to accurately guide antibiotic treatment in patients with acute respiratory tract infections [14].

The potential applicability of the TRAIL/IP-10/CRP assay and its comparison with other common biomarkers and the Labscore has never been studied prospectively in a population of children with FWS. Moreover, limited data were available in the fragile subpopulation of patients aged under 90 days in whom the need is serious for improved diagnostic tools to drive decision-making [15, 16]. The objective of the present study was to assess the diagnostic accuracy of various individual and combined bacterial- and viral-specific biomarkers, notably the ImmunoXpert and the Labscore, in children and infants under and over 90 days old presenting with FWS.

Material and methods

Study design

Prospective diagnostic accuracy study in children with FWS aged less than 3 years old (consecutive sample), performed in the pediatric emergency department (PED) of a single Swiss tertiary center from November 2015 to January 2018, using expert panel adjudication and microbiological diagnosis as diagnostic reference standards.

Primary objective.

To prospectively investigate the diagnostic accuracy of a host signature (TRAIL, IP-10, and CRP) to reliably predict bacterial infections and to compare its performance to routine individual biomarkers (CRP, PCT, WBC, ANC, TRAIL, and IP-10) and to the Labscore.

Secondary objectives.

To compare the diagnostic accuracy of the ImmunoXpert, the Labscore, and individual biomarkers to reliably predict IBI.

Participants

Patients aged less than 3 years old presenting to the PED with FWS after a thorough history and physical exam were eligible. Exclusion criteria were comorbidities predisposing to infections such as cancer, primary or secondary immunodeficiency, and iatrogenic immunosuppression.

Study procedure

After informed consent was obtained, patients’ characteristics were recorded (age, sex, delay between fever onset and presentation, maximum temperature before presentation). Patients were assessed with white blood cell count (WBC), absolute neutrophil count (ANC), band count, CRP, PCT, urinary dipstick and culture, and blood culture. Additional testing (CSF culture, chest X-ray, synovial fluid or stool culture etc.) was optional. An additional 0.6 mL sample was drawn, dedicated to the determination of CRP, TRAIL and IP-10 and the ImmunoXpert score. One of the objectives in our study was to assess the differential expression of biomarkers and combinational scores between patient with FWS and children with no fever and no sign for any current infection. Facing the ethical challenge of performing blood puncture in healthy control children with no fever and no current infection, relevant biomarkers from control patients were tested on preexisting blood samples. We did not find healthy pediatric control samples in Switzerland, but an existing healthy Canadian cohort could meet this specific objective. A group of 50 healthy Canadian children from trauma and dental clinics served as control patients. The full study protocol is available as online supplemental material (S1 File). The study was approved by the Institutional Ethics Committee (Cantonal Committee on Ethics in Scientific Research of Geneva, CCER), adheres to STARD 2015 guidelines (S1 Table) [17], and is registered under ClinicalTrials.gov (NCT03224026). No investigation was performed before signature of a written informed consent. The use of sera from control patients was approved by Toronto’s Hospital for Sick Children Ethics Committee.

Reference standard

Reference standard was based on expert panel adjudication, which is a common approach to assigning a final diagnosis in fields where gold standard is lacking, specifically in patients with FWS [18, 19]. On termination of the recruitment phase, the experts (one senior pediatric infectious-disease specialist and two senior pediatric emergency physicians) independently performed an initial etiologic classification (viral, bacterial, or indeterminate). Then, they were asked to independently classify patients with indeterminate diagnosis into a dichotomous outcome (viral or bacterial infection), as recommended in most studies using this kind of reference standard [18]. A “majority” reference standard was established: the reference standard etiology (viral or bacterial infection) was the etiology label assigned by minimum 2 of the 3 experts.

Other definitions of reference standard were also investigated to assess the robustness of the findings. First microbiological diagnosis was established: (1) bacteriologically proven SBI or IBI or (2) no proof of bacterial infection according to predefined criteria. SBI was defined as isolation of a bacterial pathogen from any urine, synovial fluid, bone, or stool specimen [18]. Microbiological urinary tract infection (UTI) was defined according to the AAP Subcommittee on Urinary Tract Infection and Steering Committee on Quality Improvement and Management [20], on a urine specimen obtained through urinary tract catheterization or clean catch of mid-stream urine: (1) patients > 60 days old: abnormal urinalysis defined by the presence of positive leukocyte esterase, nitrite or pyuria (>5 white blood cells (WBCs) per high-power field) and culture growth of at least 50’000 colony-forming units (cfu) per mL of a uropathogen and (2) patients ≤ 60 days old (as recently described after adaptation from the American Academy of Pediatrics practice parameters [21]): growth of 50’000 cfu/mL or more of a uropathogen, or growth of 10’000 cfu/mL or more of a single uropathogen in association with an abnormal urinalysis. IBI was defined as isolation of a compatible bacterial pathogen in blood or cerebrospinal fluid culture. Second, a “unanimous” reference standard was considered: the reference standard etiology (viral or bacterial) was determined only if the same etiology label was assigned by all 3 experts. When analyzing this reference standard, children with an indeterminate etiology or showing equivocal ImmunoXpert score (35 ≤ ImmunoXpert score ≤ 65) were excluded from the subgroup analysis.

Blinding procedure

Experts had access to full medical records but were blinded to the diagnosis of their peers and to TRAIL, IP-10 and ImmunoXpert results; CRP, PCT and urinary dipstick data were available to the experts. The ImmunoXpert test was performed on anonymized samples, and performers/readers of the index tests had no access to clinical information and reference standard results. The index test and panel expert reference standard outcomes were locked prior to unblinding at the end of the recruitment phase.

Sample size

The sample size of this ancillary study was driven by a main study on viremia in patients with FWS, in which a sample of 50 viremic patients was planned [22]. To reach this target, assuming a 10–20% prevalence of viremia, the enrolment of 400 patients was anticipated. A power calculation for the ancillary study was conducted to determine the detectable difference in area under the ROC curve (of any biomarker compared with the ImmunoXpert) with a power of 80% under various sample sizes (0.08 detectable difference with a sample size of 100 to 0.10 with a sample size of 264 patients). The enrolment of patients terminated when the number of patients with viremia corresponded to the target. The power to detect a difference in areas under ROC curve was acceptable for the evaluation of biomarker accuracy.

Concerning power calculation, we assumed (1) a two-sided risk alpha of 0.05, (2) a correlation between ImmunoXpert and the other marker of 0.5 both in patients with a bacterial infection and in those with a viral infection and (3) a viral infection rate 3 times higher than that of bacterial infection. Using the method of sample size calculation for comparison of paired binormal ROC curves proposed by Obuchowski et al [23], the needed sample size was 264 patients (66 with a bacterial infection and 198 with a viral infection).

Statistical analysis

Markers were described in patients with bacterial and viral infections by median and interquartile intervals. Mann-Whitney test was used to compare these groups. Diagnostic performances of biological scores and markers in identifying bacterial infections were assessed by non-parametric ROC curves. The areas under the ROC curves were assessed and compared using the non-parametric approach proposed by Delong et al [24]. This approach accounted for paired data when two compared markers were measured in the same person. Sensitivities and specificities were reported with the Clopper-Pearson exact 95% confidence intervals. The two-sided risk alpha was 0.05 in all comparisons. Software used for statistical analyses was R version 3.5.2 (R Core Team (2018). R: A language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria. URL https://www.R-project.org/).

Results

Characteristics of the study population

Overall, 58120 patients attended our PED during the study period. Among 313 potentially eligible participants, 72 were excluded and 241 were eligible for testing. Reasons for exclusion, stratification by age groups and diagnostic classification by the panel experts are indicated in Fig 1.

thumbnail
Fig 1. Flowchart of the study.

Study flowchart indicating eligible patients, reasons for exclusion, stratification according to age groups and etiology diagnosis assigned by the expert panel.

https://doi.org/10.1371/journal.pone.0294032.g001

Table 1 reports descriptive statistics for the main demographic and baseline epidemiologic variables. Two-thirds of patients were less than 90 days old (n = 155, 64%). They were more frequently male, showed a lower maximal temperature before presentation and presented more frequently in the first 12 hours from fever onset than older patients. The most frequent bacterial infection was urinary tract infection (UTI) (n = 56, 82%) (Table 1). Bacterial prevalence was 29% and 27% among children aged under and ≥ 90 days respectively.

thumbnail
Table 1. Baseline and diagnostic characteristics of study population.

https://doi.org/10.1371/journal.pone.0294032.t001

Fifty healthy patients were included as a control group. Controls were older than patients in the study group, with a median age of 18.1 months (IQR: 13.0 to 25.6) versus 2 months (IQR: 1.0 to 4.9) (p<0.0001).

Differential expression of biomarkers and combinational scores

Biomarker levels were assessed in children with FWS and in control patients for the ImmunoXpert and its components (Fig 2). CRP, PCT, WBC, ANC, Labscore and ImmunoXpert values were significantly higher in children across both age groups with bacterial infections compared to children with viral infections or control patients. TRAIL was significantly higher in patients with viral infections, but lower in children with bacterial infections than in control patients. IP-10 was higher in patients with viral infections and in patients with bacterial infections than in controls. CRP, TRAIL, and IP-10 were expressed differently in children < 90 and ≥ 90 days (S1 Fig).

thumbnail
Fig 2. Distribution of biomarker concentrations according to the final diagnosis.

Distribution of biomarker concentrations in patients with fever without source according to the final diagnosis (viral or bacterial infection) set by expert panel adjudication (and in control patients for ImmunoXpert and its components). Box plots represent median value and interquartile range of the biomarker under study in each group. When comparing groups two by two, p value represents the probability that random chance could produce the observed results when the null hypothesis is true (i.e. when biomarker concentration is equivalent between the two groups under study).

https://doi.org/10.1371/journal.pone.0294032.g002

Diagnostic performance of biomarkers and combinatorial scores

The discriminatory potential of the individual biomarkers and combinatorial scores was assessed using Receiver Operating Characteristic (ROC) analysis (Fig 3). Labscore, ImmunoXpert and CRP attained the highest area under the curve (AUC) values across both age groups (Fig 3). This finding was consistent across the unanimous reference standard subgroup although AUC values were generally lower in the former (S2 Table). The ImmunoXpert interestingly showed a statistically significant difference in its AUC value when compared to that of PCT, ANC, WBC, bands, IP-10, and TRAIL, but not when tested against the Labscore and CRP (Fig 3). Data for the unanimous reference standard subgroup show an identical trend (S2 Table).

thumbnail
Fig 3. ROC curves and AUC for the ability of main parameters and biomarkers to detect bacterial infections according to expert panel diagnoses (majority reference standard).

(A) Biological scores: Labscore and ImmunoXpert, (B) Biomarkers for bacterial infections: CRP, PCT, ANC, WBC, and bands, (C) Biomarkers for viral infections: TRAIL, IP-10 as an inverse relationship. The dashed lines represent a non-informative diagnostic biomarker. The specificity at a cut-off is the probability that a viral infection diagnosed by the expert panel is correctly identified by the biomarker and the sensitivity is the probability that a bacterial infection diagnosed by the expert panel is correctly identified by the biomarker. For Labscore, ImmunoXpert, CRP, PCT, ANC, WBC, and bands (resp. IP10 and TRAIL), the area under the ROC curve (D) is an estimate of the probability (95% CI). a P value for testing the hypothesis that the area under the ROC curve is the same with the ImmunoXpert (all patients), b P value for testing the hypothesis that the area under the ROC curve is equal in patients less than 90 days and in patients more than 90 days.

https://doi.org/10.1371/journal.pone.0294032.g003

Pre-defined single cut-offs were applied to biomarkers and dual cut-offs to scores to assess their sensitivity and specificity for bacterial infection across age groups (Table 2). Labscore and ImmunoXpert outperformed single biomarkers with higher sensitivity and/or specificity and exhibited comparable performance to one another. No adverse event was recorded from performing the tests.

thumbnail
Table 2. Diagnostic performances of individual biomarkers and biological scores compared to expert panel adjudication and to microbiological diagnosis as reference standards.

https://doi.org/10.1371/journal.pone.0294032.t002

Of the 142 patients (68% of total patients) identified with an ImmunoXpert value < 35 at low risk for bacterial infection, 19 of them (13%) were classified as bacterial infections according to the diagnosis set by the panel expert and 10 (7%) of them according to microbiological definitions. Of the 19 patients classified as bacterial infections by the expert panel, 8 patients were diagnosed with fever without source, 9 with urinary tract infection, 1 with otitis media, and 1 with both enteroviral meningitis and E. Coli urinary tract infection.

When microbiological diagnosis served as reference standard, trends in AUC were similar when comparing the scores and biomarkers under study, although each individual biological score or biomarker showed a slightly lower AUC value (S3 Table). This trend was similar in patients more and less than 90 days of age.

Specific performances in the detection of IBI

Five patients were diagnosed with IBI: three with bacterial meningitis (S. pneumoniae, H. influenzae and P. aeruginosa) and/or two with bacteremia (E. coli and S. mitis). They would all have been detected using an ImmunoXpert ≥ 35, a Labscore threshold of 1 or 3, or PCT ≥ 0.5 ng/mL, whereas WBC ≥ 15 G/L or CRP ≥ 40 mg/L would have detected respectively only two and three of these infections (S4 Table). Out of the four children with IBI tested for the ImmunoXpert, three attained scores ≥ 90. One of them would not have been detected if a threshold of 65 had been applied. However, the ImmunoXpert performed in this 28-day-old patient was exploratory since age was out of the recommended age range of the assay. Among patients with ImmunoXpert values < 35, none of them suffered from an underlying IBI.

Discussion

The present study assessed the differential expression of individual biomarkers and combinatorial scores and compared their diagnostic performances to reliably predict bacterial infection in children with FWS. For children aged under and older than 90 days, the ImmunoXpert and the Labscore exhibited the highest performance. These findings were consistent whether the reference standard was based on majority adjudication (etiology assigned to every eligible case), unanimous reference standard subgroup (indeterminate cases excluded) or microbiological diagnosis.

The results support that the ImmunoXpert is a diagnostically accurate tool in the evaluation of patients with FWS. This assay had been derived from a population of febrile children aged more than 3 months old and adults with various underlying etiologies [9]. Replacing the classically used CRP with this combination assay improved the diagnostic accuracy to predict pneumonia or other SBIs [25]. However, it had never been tested specifically in the subgroup of pediatric patients suffering from FWS. Data in children less than 3 months were also limited [16]. Previous evaluation of the diagnostic performances of the ImmunoXpert in children had shown 93.8% sensitivity and 89.8% specificity for the diagnosis of bacterial infection. However, this validation study assessed a truncated sample of febrile children, after exclusion of patients with indeterminate diagnoses or those with equivocal ImmunoXpert scores since equivocal score should not be used in decision making [11]. To stay as close as possible to real conditions, patients with indeterminate diagnosis, who represented a non-negligible proportion of patients (28%) and patients with equivocal ImmunoXpert scores were not excluded in our study. Findings support that the ImmunoXpert sensitivity is in line with previous studies in the ≥3 months indication for use population (77% for any score ≥35 and 85% in the subgroup of patients evaluated against unanimous reference standard) [11, 16]. The lower-than-expected specificity could be explained by patients with indeterminate assay results. It is however more important to rule out than in any severe condition. Although outside the indication for use population, the AUC for the ImmunoXpert in patients less than 3 months is comparable to that of older children. It is somewhat difficult to contextualize results on the ImmunoXpert in pediatric patients with FWS with the existing literature since this is the first study to assess the assay in this context. The assay has been derived from a population of febrile children aged more than 3 months old and adults with various underlying etiologies. It had never been tested specifically in the subgroup of pediatric patients suffering from FWS only. Data in patients less than 3 months with a definite infection were also limited. TRAIL, IP-10, and CRP exhibit different responses to infection in children under and over 90 days, prompting the need for further studies focusing on the derivation of a new algorithm intended for this specific population. In the present population of children presenting with FWS, the Labscore demonstrated the highest AUC for the diagnosis of bacterial infection, no matter the age group, showing similar trends in patients < and ≥ 90 days old. A Labscore ≥ 3, originally described as the recommended cut-off point, permitted the detection of 100% of patients with underlying IBI. However, a Lab-score ≥ 3 demonstrated 67% sensitivity, 88% specificity and 87% negative predictive value (NPV) to reliably predict SBI and/or IBI in febrile children when tested against the panel expert reference standard, with better discriminative ability in patients aged 90 days old and more. Better results were achieved when tested against microbiological reference standard, with 78% sensitivity, 83% specificity and 95% NPV. The diagnostic accuracy of this biomarker to detect underlying bacterial infection in febrile children with FWS is lower than in the original derivation and validation retrospective studies [6, 7] but in concordance with the results from prospective studies [8, 26, 27]. As in previous studies on FWS, the proportion of patients with UTIs is high. These patients were classified as bacterial infections by the experts. In the lack of kidney dimercaptosuccinic acid (DMSA) scan to confirm or exclude true renal involvement, asymptomatic bacteriuria or febrile cystitis could account for false negative cases with no or only mild increase in inflammatory biomarkers.

Most of the literature on the value of biomarkers in febrile children less than 3 months of age has focused on WBC or ANC count in the past, showing no added value of this parameter in detecting bacterial infections [2830]. Our results confirm insufficient sensitivity of these parameters.

In the present study, PCT shows better diagnostic performances in patients older than 90 days than in younger patients. The superiority of PCT and CRP over WBC and ANC in the identification of IBIs in children was recently published [31]. The results in our study do not support the previously described superiority of PCT over CRP in either age group [32]. This could be explained by different PCT thresholds and reference standard outcomes.

Our study has limitations. First, test accuracy may suffer from biased estimate of sensitivity and specificity and reduced predictive values when forcing expert panels to make a dichotomous decision on target disease classification in the presence of uncertainty [33]. Second, the delay between onset of fever and presentation, which could modify any biological value, was highly variable. Last, CRP and PCT which are included in the biological scores under study, were available to experts assigning the diagnosis. A potential overestimation of the accuracy of tests including any of the above-mentioned biomarkers may have occurred. Interestingly, expert panel diagnosis and microbiological diagnosis show equivalent diagnostic performances with even better accuracy when pure microbiological definitions were applied, showing that this effect is probably minimal.

Conclusion

The Labscore and the ImmunoXpert show better diagnostic performances for safely detecting bacterial infection in children with FWS, compared to individual biomarkers such as CRP, PCT, WBC or ANC. Better performances were achieved in children more than 90 days old than in younger patients. Evaluation of combined biological scores and biomarkers in various age groups should encourage the development of decision-making rules based on the most appropriate cut-offs permitting to rule out SBI and IBI. Furthermore, implementation of appropriate point-of-care devices and medico-economic impacts of these approaches should be evaluated, notably in their usefulness in safely decreasing unnecessary antibiotic prescription and admission rates and improving the care of children with FWS.

Supporting information

S1 Fig. Distribution of biomarker concentrations according to the final diagnosis (unanimous diagnosis reference standard).

Distribution of biomarker concentrations in patients with FWS according to the final diagnosis (viral or bacterial infection) set by expert panel adjudication (and in control patients for ImmunoXpert and its components) in the subgroup analyzed under the unanimous reference standard (exclusion of children showing non concordant etiology labels assigned by the 3 experts, with an indeterminate etiology or showing equivocal ImmunoXpert score 35 ≤ ImmunoXpert score ≤ 65).

https://doi.org/10.1371/journal.pone.0294032.s001

(TIF)

S2 Table. Diagnostic performance of individual biomarkers (A) and combinatorial scores (B) compared to unanimous reference standard.

a The ImmunoXpert is intended for patients ≥ 90 days. The analysis of the ImmunoXpert performance in patients < 90 days is exploratory.

https://doi.org/10.1371/journal.pone.0294032.s003

(DOCX)

S3 Table. Areas under the ROC curves (95% CI) for individual biomarkers and combinatorial scores for the detection of bacterial infection according to microbiological diagnosis.

a P value for testing the hypothesis that the area under the ROC curve is equal in patients less than 90 days and in patients more than 90 days. For Labscore, ImmunoXpert, CRP, PCT, ANC, WBC, and bands (resp. IP10 and TRAIL), the area under the ROC curve is an estimate of the probability.

https://doi.org/10.1371/journal.pone.0294032.s004

(DOCX)

S4 Table. Description of patients with microbiologically confirmed invasive bacterial infection.

* Missing data due to insufficient sample.

https://doi.org/10.1371/journal.pone.0294032.s005

(DOCX)

Acknowledgments

We are very grateful to all participating parents and infants, to co-investigators, research nurses, notably Mrs Florence Hugon, and participating clinicians.

References

  1. 1. Greenhow TL, Hung YY and Pantell RH (2016) Management and outcomes of previously healthy, full-term, febrile infants ages 7 to 90 days. Pediatrics 138. pmid:27940667
  2. 2. Esposito S, Rinaldi VE, Argentiero A, Farinelli E, Cofini M, D’Alonzo R, et al. (2018) Approach to neonates and young infants with fever without a source who are at risk for severe bacterial infection. Mediators Inflamm 2018: 4869329. pmid:30581369
  3. 3. Wing R, Dor MR and McQuilkin PA (2013) Fever in the pediatric patient. Emerg Med Clin North Am 31: 1073–1096. pmid:24176480
  4. 4. Ishimine P (2013) Risk stratification and management of the febrile young child. Emerg Med Clin North Am 31: 601–626. pmid:23915596
  5. 5. Baraff LJ (2008) Management of infants and young children with fever without source. Pediatr Ann 37: 673–679. pmid:18972849
  6. 6. Lacour AG, Zamora SA and Gervaix A (2008) A score identifying serious bacterial infections in children with fever without source. Pediatr Infect Dis J 27: 654–656. pmid:18536624
  7. 7. Galetto-Lacour A, Zamora SA, Andreola B, Bressan S, Lacroix L, Da Dalt L, et al. (2010) Validation of a laboratory risk index score for the identification of severe bacterial infection in children with fever without source. Arch Dis Child 95: 968–973. pmid:20515973
  8. 8. Gomez B, Mintegi S, Bressan S, Da Dalt L, Gervaix A, Lacroix L, et al. (2016) Validation of the "Step-by-Step" approach in the management of young febrile infants. Pediatrics 138. pmid:27382134
  9. 9. Oved K, Cohen A, Boico O, Navon R, Friedman T, Etshtein L, et al. (2015) A novel host-proteome signature for distinguishing between acute bacterial and viral infections. PLoS One 10: e0120012. pmid:25785720
  10. 10. Eden E, Srugo I, Gottlieb T, Navon R, Boico O, Cohen A, et al. (2016) Diagnostic accuracy of a TRAIL, IP-10 and CRP combination for discriminating bacterial and viral etiologies at the Emergency Department. J Infect 73: 177–180. pmid:27255416
  11. 11. Srugo I, Klein A, Stein M, Golan-Shany O, Kerem N, Chistyakov I, et al. (2017) Validation of a novel assay to distinguish bacterial and viral infections. Pediatrics 140.
  12. 12. Ashkenazi-Hoffnung L, Oved K, Navon R, Friedman T, Boico O, Paz M, et al. (2018) A host-protein signature is superior to other biomarkers for differentiating between bacterial and viral disease in patients with respiratory infection and fever without source: a prospective observational study. Eur J Clin Microbiol Infect Dis 37: 1361–1371. pmid:29700762
  13. 13. Papan C, Argentiero A, Porwoll M, Hakim U, Farinelli E, Testa I, et al. (2022) A host signature based on TRAIL, IP-10, and CRP for reducing antibiotic overuse in children by differentiating bacterial from viral infections: a prospective, multicentre cohort study. Clin Microbiol Infect 28: 723–730. pmid:34768022
  14. 14. Carlton HC, Savovic J, Dawson S, Mitchelmore PJ and Elwenspoek MMC (2021) Novel point-of-care biomarker combination tests to differentiate acute bacterial from viral respiratory tract infections to guide antibiotic prescribing: a systematic review. Clin Microbiol Infect 27: 1096–1108. pmid:34015531
  15. 15. Kimberlin DW and Poole CL (2017) Assessing the febrile child for serious infection: a step closer to meaningful rapid results. Pediatrics 140. pmid:28904071
  16. 16. van Houten CB, de Groot JAH, Klein A, Srugo I, Chistyakov I, de Waal W, et al. (2017) A host-protein based assay to differentiate between bacterial and viral infections in preschool children (OPPORTUNITY): a double-blind, multicentre, validation study. Lancet Infect Dis 17: 431–440. pmid:28012942
  17. 17. Bossuyt PM, Reitsma JB, Bruns DE, Gatsonis CA, Glasziou PP, Irwig L, et al. (2015) STARD 2015: an updated list of essential items for reporting diagnostic accuracy studies. BMJ 351: h5527. pmid:26511519
  18. 18. Bertens LC, Broekhuizen BD, Naaktgeboren CA, Rutten FH, Hoes AW, van Mourik Y, et al. (2013) Use of expert panels to define the reference standard in diagnostic research: a systematic review of published methods and reporting. PLoS Med 10: e1001531. pmid:24143138
  19. 19. van Houten CB, Naaktgeboren CA, Ashkenazi-Hoffnung L, Ashkenazi S, Avis W, Chistyakov I, et al. (2019) Expert panel diagnosis demonstrated high reproducibility as reference standard in infectious diseases. J Clin Epidemiol 112: 20–27. pmid:30930247
  20. 20. Subcommittee on Urinary Tract Infection SCoQI, Management and Roberts KB (2011) Urinary tract infection: clinical practice guideline for the diagnosis and management of the initial UTI in febrile infants and children 2 to 24 months. Pediatrics 128: 595–610. pmid:21873693
  21. 21. Kuppermann N, Dayan PS, Levine DA, Vitale M, Tzimenatos L, Tunik MG, et al. (2019) A clinical prediction rule to identify febrile infants 60 days and younger at low risk for serious bacterial infections. JAMA Pediatr 173: 342–351. pmid:30776077
  22. 22. L’Huillier AG, Mardegan C, Cordey S, Luterbacher F, Papis S, Hugon F, et al. (2020) Enterovirus, parechovirus, adenovirus and herpes virus type 6 viraemia in fever without source. Arch Dis Child 105: 180–186. pmid:31462437
  23. 23. Xiao-Hua Zhou NAO, Donna K. McClish (2011) Statistical methods in diagnostic medicine. 592 p.
  24. 24. DeLong ER, DeLong DM and Clarke-Pearson DL (1988) Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach. Biometrics 44: 837–845. pmid:3203132
  25. 25. van Houten C, van de Maat JS, Naaktgeboren C, Bont L and Oostenbrink R (2019) Update of a clinical prediction model for serious bacterial infections in preschool children by adding a host-protein-based assay: a diagnostic study. BMJ Paediatr Open 3: e000416. pmid:31646190
  26. 26. Lacroix L, Manzano S, Vandertuin L, Hugon F, Galetto-Lacour A and Gervaix A (2014) Impact of the lab-score on antibiotic prescription rate in children with fever without source: a randomized controlled trial. PLoS One 9: e115061. pmid:25503770
  27. 27. Nijman RG, Moll HA, Smit FJ, Gervaix A, Weerkamp F, Vergouwe Y, et al. (2014) C-reactive protein, procalcitonin and the lab-score for detecting serious bacterial infections in febrile children at the emergency department: a prospective observational study. Pediatr Infect Dis J 33: e273–279. pmid:25093971
  28. 28. Schwartz S, Raveh D, Toker O, Segal G, Godovitch N and Schlesinger Y (2009) A week-by-week analysis of the low-risk criteria for serious bacterial infection in febrile neonates. Arch Dis Child 94: 287–292. pmid:18977786
  29. 29. Bilavsky E, Ashkenazi-Hoffnung L, Yarden-Bilavsky H, Amir J and Livni G (2011) A search for the ’Holy Grail’ in the evaluation of febrile neonates aged 28 days or less: a prospective study. Scand J Infect Dis 43: 264–268. pmid:21171828
  30. 30. Kuppermann N and Walton EA (1999) Immature neutrophils in the blood smears of young febrile children. Archives of Pediatrics & Adolescent Medicine 153: 261–266. pmid:10086403
  31. 31. Hubert-Dibon G, Danjou L, Feildel-Fournial C, Vrignaud B, Masson D, Launay E, et al. (2018) Procalcitonin and C-reactive protein may help to detect invasive bacterial infections in children who have fever without source. Acta Paediatr 107: 1262–1269. pmid:29385638
  32. 32. Milcent K, Faesch S, Gras-Le Guen C, Dubos F, Poulalhon C, Badier I, et al. (2016) Use of procalcitonin assays to predict serious bacterial infection in young febrile infants. JAMA Pediatr 170: 62–69. pmid:26595253
  33. 33. Jenniskens K, Naaktgeboren CA, Reitsma JB, Hooft L, Moons KGM and van Smeden M (2019) Forcing dichotomous disease classification from reference standards leads to bias in diagnostic accuracy estimates: A simulation study. J Clin Epidemiol 111: 1–10. pmid:30904568