With interest, we read the article by Banoei et al. [1] on machine learning (ML) models to predict mortality among COVID-19 patients. They refer to other studies that failed to predict mortality using ‘conventional statistical analysis’, after which they present a linear ML model as a better suited method for such complex medical problems. We feel such a claim creates an image around ML as an alternative technique that offers solutions where statistical modeling fails. However, ML and statistical modeling are tightly interwoven. Indeed, there is no consensus on whether or how to differentiate between the two [2, 3], e.g., the approach Banoei and colleagues present (Partial Least Squares) could easily be considered a ‘statistical analysis’ as well. A recent systematic review [4] differentiates ML from statistical models based on how ‘automatically’ these models learn and found no difference in discriminative performance. This raises the question why efforts are made to differentiate between statistical modeling and ML, as it does not provide insights into which prediction models work for which kind of problems. We advocate it is more important to demystify ML by emphasizing its connections to statistical models most clinicians are already familiar with, as it may help in setting reasonable expectations for the potential clinical benefit ML could bring. Towards demystifying ML, good reporting of methodology and findings is essential. Due to unclear or incomplete reporting, one may draw wrong conclusions and miss out on opportunities to learn. For instance, Banoei and colleagues are unclear on how frequently measured predictors (like SpO2) were aggregated to one value. We encourage to follow a unified approach for reporting, e.g. the TRIPOD guidelines [5]. Moreover, it is unfortunately still common that the intended use of prediction models is unclear, whereas it has implications for choices in the model development. Likewise, Banoei and colleagues present several models (for mortality prediction and patient clustering) and suggest that these can ‘aid in clinical decision making and resource allocation’, without further specification. We strongly encourage to develop models with an explicit intended use in mind, enabling fair judgement of their clinical relevance. Altogether, we join Banoei and colleagues in their belief that ML models hold a lot of promise as valuable tools in the modern ICU. However, we advise against differentiating them from statistical models and advocate proper reporting about the methodology and intended use.
Authors’ response
Machine learning-based methods vs. conventional statistical methods for studying mortality outcome
(3)
Department of Critical Care Medicine, University of Calgary, Alberta, Canada
(4)
Department of Biological Science, University of Calgary, Alberta, Canada
(5)
Division of Pulmonary and Critical Care Medicine, Johns Hopkins University, Baltimore, MD 21218, USA
(6)
Division of Pulmonary and Critical Care, Miami VA Medical Center, Miami, FL, USA
(7)
Division of Pulmonary and Critical Care and Sleep Medicine, Department of Medicine, College of Medicine, University of Florida, Jacksonville, FL, USA
In response to the Letter to the Editor by J.M. Smit et al., we certainly agree that machine learning (ML)-based methods can be used interchangeably with multivariate data analysis (MVA) and multivariate prediction models (MPMs) [6]. Although ML approaches like partial least square (PLS), statistically inspired modification of PLS (SIMPLS), random forest (RF), Support Vector Machine (SVM), and artificial neural network (ANN) are considered statistical methods, they are notably different from conventional statistical methods (CSM) [7]. Considering the advantages of ML methods, ML approaches have contributed significantly to the early detection, tracing, diagnosis, prognosis and clinical trials of COVID-19 that have been more functional to support researchers in confronting the coronavirus pandemic [8]. Several studies previously have shown that ML can be more appropriate than CSM for the clinical datasets. ML algorithms have proven better able to stratify COVID-19 patients and mortality risk [9], identify high-risk patients with COVID-19 [10].
Anzeige
It is generally believed that there is no single ML method superior to others. The SIMPLS method was the only prediction model used in our current study. SIMPLS has remarkable advantages including a lower risk of overfitting, a high level of interpretation, a high level of variable selection, easy implementation when compared with RF, SVM and ANN methods [6]. SIMPLS can easily account for batch processing and a high degree of correlation (multicollinearity) between and among variables of large datasets. SIMPLS fit the outcome responses with nominal, continuous, and polynomial data type, interaction and categorial effects, and provides strong visualization when compared with other ML and CSM. In our study, SIMPLS was successfully applied to recognize the most differentiating variables involved in the prediction of COVID-19 mortality. Other statistical methods that are not considered prediction models, such as principal component analysis (PCA) and latent class analysis (LCA) were established based on the findings of SIMPLS to identify patients at the highest risk of dying.
In our study, the importance of ML-based model was its ability to predict patient mortality using variables measured at the time of admission, although these variables were frequently measured during the patients’ hospitalization for other purposes.
Our study mainly focused on the importance of ML in clinical application, and the recognition of the most important variables contributing to mortality from COVID-19, instead of describing the details of ML that may be extraneous for clinicians.
Declarations
Consent for publication
Not applicable.
Anzeige
Competing interests
The authors declare that they have no competing interests.
Open AccessThis article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.