Introduction
Acute coronary syndromes (ACS) are the unstable and progressive stages of coronary heart disease, including ST-segment elevation myocardial infarction (STEMI), non-ST-elevation myocardial infarction (NSTEMI), and unstable angina (UA) [
1]. Although advances in early reperfusion therapy and adjuvant drug therapy have improved the prognosis of ACS patients, ACS remains the leading cause of death worldwide [
2]. More than 5% of patients with ACS die in-hospital [
3], even up to 26.7% in some subgroups [
4], and up to 26.5% in long-term follow-up [
5]. Appropriate management can significantly improve the prognosis of patients with ACS; thus, timely and accurate identification of mortality risk and early and appropriate risk stratification are essential.
Traditional risk stratification of ACS patients is based on risk scoring systems, of which the Global Registry of Acute Coronary Events (GRACE) risk score and Thrombolysis in Myocardial Infarction (TIMI) risk score are the most widely used mortality risk prediction tools [
6,
7]. Although these risk scores have been validated and are generally accepted, they have limitations in current clinical practice. First, these risk scores were developed based on data from earlier randomized controlled trials. During that period, contemporary therapies for acute myocardial infarction (AMI) were not widely available, and drug eluting stents and newer generation antiplatelets were not introduced. Therefore, the predictive effect of these risk scores in current practice is questionable [
8]. Second, these risk scores use only selective variables based on traditional statistical methods, inevitably limiting the number of predictors and thus the possibility of missing important information [
9]. In addition, traditional risk scores focus on predicting short-term mortality, such as in-hospital, 14-day, and 30-day mortality, and less on long-term mortality risk [
10]. Therefore, widespread interest has been in exploring more accurate and comprehensive mortality risk prediction models.
Machine learning (ML) is a subdiscipline of artificial intelligence that uses algorithms to identify patterns in large data sets with multiple variables that can be continuously improved with additional data, resulting in pattern algorithms that can predict various outcomes [
11]. It constructs models based on test inputs and correlates all or some predictor variables with the results to make data-driven predictions or decisions [
12]. In recent years, ML has been increasingly used in the medical field, especially in the cardiovascular field, as the availability of medical data continues to increase and computer analysis capabilities continue to improve. Emerging research indicates that the introduction of ML models as a clinical tool to accurately predict the risk of death in ACS patients has the great potential [
13]. However, the performance of different models tends to vary, and it is unclear whether ML models have robust performance in predicting the risk of death in ACS. Therefore, we performed this systematic evaluation and meta-analysis.
Discussion
Our systematic review included 50 original studies and reported 216 ML mortality risk prediction models constructed based on large samples. We found that (1) ML models predicting death in ACS patients at different times showed a relatively excellent prediction with a good composite C-index and accuracy; (2) the specific type of model and the variables included in the model severely affected the prediction of the model, with specific ML models predicting death in ACS patients showing excellent performance; (3) age, sex, systolic blood pressure, serum creatinine, Killip classification, heart rate, diastolic blood pressure, blood glucose, and hemoglobin were commonly used predictors, with age being the most commonly used and important predictor variable described.
ACS is a leading cause of death worldwide, and rapid identification of high-risk patients remains unmet clinical needs. For this purpose, several risk scores have been developed, among which the GRACE risk score is one of the best validated predictive tools. However, this score was developed based on traditional statistical methods that were useful and robust [
13] but had inherent limitations. This limitation limits their ability to handle large data sets with multiple variables and samples [
68]. The chosen predictor variables are run the same way for each individual and uniformly across the range [
13]. There are non-linear relationships and complex interactions between ACS risk factors, and large population-level studies can provide critical insights into ACS risk factors [
69]. Therefore, the inherent characteristics of traditional statistical methods may lead to low model predictive power. ML is an area of artificial intelligence that is part of a broader approach to data analysis [
68]. Unlike traditional predictive models that use selected variables for computation, ML algorithms can easily combine many variables while capturing complex relationships between variables [
69,
70]; and search for the best fit randomly or deterministically according to different algorithms
13 to construct robust prediction models. Most importantly, ML algorithms can better describe the complexity and unpredictability of human physiology in many cases [
70]. These advantages make ML technology suitable for the medical field, especially for outcome prediction. Recent studies have shown that ML algorithms outperform traditional statistical modeling methods [
13]. Our present systematic review supports the above view, showing that the ML algorithm-based prediction model has a more desirable integrated C-index and accuracy.
To visualize the contribution of each characteristic to the risk of occurrence of death in ACS, we calculated the importance of each characteristic. We identified nine variables commonly used to predict short- and long-term mortality in patients with ACS, including five variables from the GRACE score (age, systolic blood pressure, serum creatinine, Killip classification, heart rate) and four new variables (sex, diastolic blood pressure, blood glucose, and hemoglobin). These parameters describe non-modifiable risk factors and different pathophysiological contexts, such as hemodynamics, cardiovascular metabolism, and cardiomyocyte injury. Existing theories and studies also support these risk factors. Age was a well-established predictor and the most commonly used predictor variable found in our current study. Age-related pathological changes in the vascular system play a key role in morbidity and mortality in the elderly. As we age, changes in cardiovascular structure, function, and phenotype accelerate the progression of coronary artery disease, so older patients have more severe lesions and a higher risk of death [
71].
There are gender differences in the outcomes of ACS patients. Studies have consistently shown that women with ACS have poorer short- and long-term outcomes than men, with a disproportionate risk of death [
72,
73]. Female patients with ACS have different risk factor profiles and clinical presentations than male patients. In general, female ACS patients have a higher prevalence of cardiovascular risk factors, such as diabetes, hypertension, and psychosocial risk factors, such as depression [
74]; more atypical symptoms, such as neck pain, fatigue, and dyspnea [
75]; longer duration of ischemia due to pre-hospital delays, evidence-based diagnosis and inadequate treatment [
76]; as well as a high rate of complications during hemodynamic reconstruction [
77]. All of the above contribute to the high risk of death in women with ACS.
Independent of pre-existing metabolic dysregulation in diabetes, hyperglycemia at admission to ACS is associated with poor outcomes, regardless of diabetes status [
78]. The mechanisms underlying the association between hyperglycemia and increased mortality in ACS patients are multifactorial. Increased local and systemic inflammatory responses [
79], altered platelet function and thrombo-fibrinolytic system [
80], increased oxidative stress [
81], endothelial dysfunction [
82], arrhythmic tendencies [
83], and impaired myocardial contractility [
84], all ultimately lead to increased atherosclerotic burden and plaque instability, and an increased risk of death.
We found hemoglobin to be a significant predictor of death in ACS. Recent studies have consistently shown that anemia on admission is always associated with poorer outcomes in ACS, as evidenced by increased mortality at different observed timepoints [
85,
86]. An imbalance in myocardial oxygen supply and demand is necessary to develop ACS. Low hemoglobin levels worsen the myocardial ischemic injury by reducing the oxygen supply to the damaged myocardium. In contrast, increased myocardial oxygen demand exacerbates this imbalance due to the need for higher cardiac output to maintain adequate systemic oxygen supply [
87]. Other plausible explanations are the reduced number of functionally impaired peripheral endothelial progenitor cells and impaired vascular healing capacity in ACS patients with low hemoglobin levels [
88]. In addition, studies have shown that the inflammatory factor C-reactive protein(CRP) is negatively correlated with hemoglobin levels in patients with ACS, which may further increase the risk of death [
89].
The present study found that LR was currently the most widely used modeling method in ACS mortality risk prediction models. It performed better in mortality risk models across time and even sometimes better than others, indicating the importance of valid predictors. Therefore, the development or updating of prediction models should be inclined to incorporate valid, easily collected, minimally invasive predictors.
Limitations and strengths
Although the current results indicate that the predictive ability of ML models appears satisfactory, there are some methodological flaws or limitations to the inclusion of the original study. First, the ML models with different times of death were combined separately in the current study, and their discriminatory performance was assessed according to C-index and accuracy. Still, most risk models were not constructed with overfitting in mind. Second, a large amount of risk model data was derived from retrospective case–control studies and primarily generated training and validation cohorts at a certain ratio (e.g., 7/3) without external validation using an utterly new validation cohort. In addition, the modeling methods of most studied ML models are not clearly described, such as insufficient disclosure of information on hyperparameter tuning and external validation of ML algorithms. The development of predictive models helps in clinical decision-making and resource allocation. However, the risk of bias, reproducibility, and potential usefulness of predictive models can only be fully assessed if the modeling steps of predictive models are adequately and clearly reported [
90]. The transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD) statement presents a list of 22 items, thereby increasing the transparency of predictive modeling studies
91. We strongly recommend detailed and standardized reporting of predictive models according to the TRIPOD statement, which not only helps to improve the quality of ML models but also helps to assess their reliability and increase their credibility.
Although we acknowledge the limitations of the original study, we believe that our meta-analysis still has some merit and clinical relevance. First, this is the first meta-analysis to systematically assess the predictive value of ML models for death at different times in patients with ACS. Second, we ranked the model variables of the original study according to importance, providing the most valuable variables for predicting ACS death, complementing variables not included in the GRACE risk score, which can guide further development of mortality risk models. In addition, the original studies included in our systematic review used different authoritative databases, including the MIMIC-III database and multiple national registry databases for acute myocardial infarction. The extensive use of different databases adds to some extent to the reliability of our results.
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.