Predicting the microbial cause of community-acquired pneumonia: can physicians or a data-driven method differentiate viral from bacterial pneumonia at patient presentation?

The World Health Organization (WHO) estimates that due to antimicrobial resistance, bacterial infections will outcompete any cause of death by 2050 [1], meaning that there is an urgent need for new strategies to improve antibiotic treatments. The Agency for Healthcare Research and Quality (AHRQ) Safety Program for Improving Antibiotic Use recently proposed a structured approach to improve antibiotic decision making by clinicians, which emphasizes the 4 critical time points in antibiotic prescribing [2, 3]. The first time point of this organized approach requires the physician to ask: “Does this patient have an infection that requires antibiotics?”. This question aims to remind the clinician to synthesize all relevant patient information to determine the likelihood of an infection that requires antibiotic therapy. The questionable ability of physicians to answer this first question properly in the context of pneumonia was the impetus for this study.

Community-acquired pneumonia (CAP) is a major global healthcare burden associated with significant morbidity, mortality and costs [4‐9]. Identifying the etiology of CAP is an utmost priority for its management and treatment decisions [10]. Although the range of pathogens that may be involved in these cases is broad, physicians must at least determine whether a bacterial or a viral pathogen (or both) is causing the pneumonia to determine if antibiotic treatment is appropriate. Whether the etiology of CAP is viral or bacterial should be determined based on the patient interview, clinical symptoms and signs, biological findings and radiological data from the very first hours of the patient’s presentation (a time when microbiological findings are typically not yet available). Physicians must use the knowledge obtained from their routine practice and medical education to make sense of these diverse data input streams, triage the resulting complex dataset, and make appropriate decisions. A growing body of research has recently suggested that difficulties in accessing, organizing, and using a substantial amount of data could be significantly ameliorated by use of emerging artificial intelligence (AI)-derived methods, which are nowadays applied in diverse fields including biology, computer science and sociology [11]. AI excels at finding complex relationships in large volumes of data and can rapidly analyze many variables to predict outcomes of interest. In the context of CAP in intensive care units (ICUs), where information are particularly diverse, we wondered if an AI data-driven approach to reducing the medical complexity of a patient could allow us to make a better hypothesis regarding the microbial etiology at the patient’s presentation.

This study was conducted in two steps. First, we performed prospective data collection (step 1); second, we retrospectively assessed the microbial etiology prediction performances of experienced physicians (more than 10 years’ experience) and a computational data-driven approach for this dataset (step 2).

A total of 188 patients diagnosed with CAP were eligible for inclusion over an 18-month period; 153 patients were included; 37% had viral pneumonia, 24% had bacterial pneumonia, 20% had a co-infection and 19% had no identified respiratory pathogen. Finally, we performed the analysis on 93 patients as co-pathogen and no-pathogen cases were excluded. The patient selection flow chart is presented in Fig. 2. The characteristics of the patients according to microbial diagnosis are detailed in Table 2.

Addressing antimicrobial resistance requires investment in several critical areas, the most pressing of which is the ability to make rapid diagnoses to promote appropriate anti-infective therapeutics and limit unnecessary antibiotic use. Here, we set up a pilot study and demonstrated that neither experts nor a mathematical algorithm could accurately predict the microbial etiology of severe CAP within the first 3 hours of hospitalization when there is an urgent need to define the appropriate anti-infective therapeutic strategy.

We encoded all information available in the first 3 hours after admission for a large cohort comparable with other published cohorts in terms of the distribution of causal microbial pathogens, patient characteristics and severity of disease [15‐17]. We demonstrated that experienced clinicians synthesizing all this information failed to adequately answer the question: “is it a viral or a bacterial pneumonia?”, as the discriminant ability between the two diagnoses was considered low. We interpreted our results mainly based on the calculation of likelihood ratios, as recommended for reports of a diagnostic test for an infectious disease [12]. Likelihood ratios incorporate both sensitivity and specificity and, unlike predictive values, do not vary with prevalence, making them good statistical tools to facilitate translation of knowledge from research to clinical practice [12]. In parallel, we designed a data-driven approach. Different AI methods were available; we selected the Random Forest method because it is one of the most efficient strategies for providing a predictive algorithm in this context [18‐21]. Importantly, the final algorithm was tasked with providing predictions for a novel population independent of the dataset used for the algorithm construction. The discriminant abilities of the AI approach restricted to the binary choice “viral” or “bacterial” were superior to those of experts but still considered low or moderate and were ultimately insufficient to provide an indisputable therapeutic decision. It is important to emphasize that we chose a high cutoff value for determining the discriminant ability of the AI approach (LR+ > 10, LR- < 0.1); this choice was made for two reasons. First, in this proof-of-concept study, we did not analyze co-infections and restricted the possible choices to a binary prediction. Because we reduced the complexity of the cases, we expected high predictive performances. Second, the goal of this study was not a prediction of outcomes (e.g., ICU length of stay, mortality), which are informative but do not determine patient management; it was to provide a clear and immediate medical decision: whether or not to prescribe antibiotics. The immediate clinical consequences in this situation demand a high predictive performance. Still, it is important to highlight that the machine learning method we developed achieved an AUC of 0.84, which is superior or at least equal to AUC values usually observed for predictive mathematical models developed for the ICU environment. For instance, the Systemic Inflammatory Response Syndrome (SIRS) criteria, the Simplified Acute Physiology Score II (SAPS II) and the Sequential Organ Failure Assessment (SOFA) have AUC values of 0.61, 0.70 and 0.73, respectively, for identifying sepsis [22]. An AI Sepsis Expert algorithm for early prediction of sepsis has been engineered and achieved AUC values ranging from 0.83–0.85 according to the time of the prediction. An AI method for predicting prolonged mechanical ventilation achieved an AUC of 0.82 [23].

How can it be that AI or machine-learning predictive algorithms that can already automatically drive cars or successfully understand human speech failed to predict the microbial cause of pneumonia accurately? First, having data of excellent quality is critical for the success of AI predictions. The ICU environment is data-rich, providing fertile soil for the development of accurate predictive models [24], but it is also a challenging environment with heterogeneous and complex data. In our study, the data that fueled the AI method were from a real-world data source. It is probably more difficult to create a consistent data format when merging data from interviews, patient examinations, biological and radiological information than when using datasets from the insurance or finance industries. Additionally, data arising from patient examinations and interviews are still strictly dependent on the physician’s skill and experience. Finally, although we hypothesized that the AI capabilities would exceed human skills and make accurate predictions when physicians cannot, we must also consider the null hypothesis: viral and bacterial pneumonias share the same characteristics and cannot be distinguished based on initial clinical, biological or radiological parameters. The dividing lines between the signs and symptoms of a viral versus a bacterial infection could be too blurry to permit the two diagnoses to be discerned without microbial analyses.

Our results emphasize the need to use a rapid turnaround time system for the accurate identification of respiratory pathogens from patient specimens. Utilizing rapid molecular respiratory panel assays may increase the likelihood of optimal treatment of acute respiratory infections [25‐29]. However, antibiotic consumption was not reduced by the use of a molecular point-of-care strategy in adults presenting with acute respiratory illness in a large randomized controlled trial [28]. It seems that we are experiencing a switch in perspectives regarding microbial diagnoses of respiratory infections: physicians are used to dealing with an absence of information, but they will likely be overloaded with information in the near future [30]. The positive detection of respiratory viruses may or may not be useful for the immediate management of a patient [31]. Thus, the development of molecular point-of-care analysis techniques will not lessen the usefulness of our AI strategy. On the contrary, we believe that AI could be a great help in dealing with information overload, which could soon be a common problem. AI methods should not be viewed as ways to replace human expertise but rather as catalysts that accelerate human expertise–based analyses of data. AI methods can assist–rather than replace–in clinical decision-making by transforming complex data into more actionable information. Further studies are needed to assess if AI system integrated with point-of-care rapid molecular respiratory panel assays could be a useful addition for the clinician. Ultimately, randomized controlled trial should determine the effect of this strategy on the decision making regarding antibiotic use.

Neither a panel of experts nor a data-driven approach could accurately distinguish viral from bacterial pneumonia within the first hours of patient admission in ICU for CAP. The heterogeneous and complex data generated in the ICU environment are likely difficult to use to generate an AI algorithm with a high predictive quality. The results of our pilot study at least highlight that we should not treat machine learning and data science as crystal balls for making predictions and automating decision-making; we should rather use these techniques to more critically examine all available information and enhance existing human expertise.