Machine learning for subtype definition and risk prediction in heart failure, acute coronary syndromes and atrial fibrillation: systematic review of validity and clinical utility

Of 5918 articles identified by our search, 97 met the inclusion criteria (Additional file_Figure S1).

Studies to date of ML in HF, ACS and AF have been limited by number and type of covariates, population size, geographic location (mainly North American) and clinical setting. As with epidemiologic studies and trials, the generalisability of data and reproducibility of methods [6, 7] are crucial to make findings interpretable and translatable to clinical care. However, the majority of studies to date have not fully considered these factors, resulting in high risk of bias in all studies.

EHR data and advanced data analytics, which have been either under-used or under-reported in studies to date, offer research opportunities across diseases, but ML studies have focused on single diseases, when diseases often co-exist, as for HF, ACS and AF. Outside CVD, perhaps the most promising ML studies to date have been in either large imaging datasets or settings where there is linkage of multimodal clinical data. Whereas the popular concept of ML suggests the use of a wide range of covariates, particularly in clustering, the number and type of variables have been limited. Availability of covariates and quality of recording in EHR determine these limitations rather than research or clinical need. In the future, as more complete and larger EHR datasets become available for research, studies of ML can focus on improved use of limited data in clinical practice, or use of more comprehensive lists of covariates and coexisting diseases for the discovery of new factors in disease definition or prediction. For example, the ESC classification lists 89 causes of HF, which have not been studied together in a single population-based study in EHR or otherwise [3].

The majority of studies have had positive results suggesting publication bias (not formally assessed here), which likely overestimates potential healthcare impact. Lessons must be learned from biomarkers and genomics [149, 150], where lack of standardisation and biased reporting have contributed to lack of translation from science to practice. It is concerning that a significant proportion of studies ML are being developed with questions unrelated to patient benefit, or with data unsuitable to answer them. “Data-driven care” is important in both personalised and precision medicine [151], both of which can benefit from advances in the use of ML. However, a “data-centred” or “data-driven” agenda must remain “patient-centred” rather than “technology-centred” (or in this case, “ML-centred”) [152]. Our framework can be used to plan studies, facilitating clinical relevance and reducing bias.

In order to improve data-driven characterisation of CVD and influence clinical decision-making, ML studies for subtyping and prediction should be larger-scale, across diseases, with standardised reporting and proven external validity. To date, the degree of external validation across subtype definition and risk prediction ML studies has been disappointing, making ML difficult to implement in routine care. External validation will help to understand which clustering and prediction tools are of greatest use, and greater availability of electronic health record data should facilitate this step in the pathway to implementation of ML.

There are major gaps at the phases of clinical utility, particularly open availability of methods and interpretability of methods and results by clinicians. A limitation of our study is that we did not conduct meta-analysis for studies of either subtype classification or risk prediction because it was out of scope for our review. However, variation and lack of standardisation in methods and reporting make meta-analysis challenging and potentially unrepresentative. These issues can be addressed at design and implementation phases, but have been relatively neglected. Importantly, in applications of ML in CVD to date, studies of effectiveness and cost-effectiveness are lacking. Without these evaluations, ML cannot be implemented safely or effectively.

Figure 1 shows deficiencies at development, validation and impact stages and that ML research is not necessarily being conducted in a “sequential” manner, e.g. to ensure that the evaluation of impact is in validated ML models, perhaps reflecting lack or under-use of consensus guidelines. Particularly in CVD, with high disease burden globally and in low- and middle-income countries, the impact of ML has to be considered through a global lens. The fact that the majority of the literature regarding ML in CVD is from the North American context does raise concern that AI could broaden research and clinical inequalities both within and across countries, beyond current debates about the inequalities which may be inherent in algorithms. If individuals are not represented in the data feeding into the algorithms, then the AI cannot benefit them. These inequalities still exist in other domains, such as pharmaceutical trials and genomics, but the situation is improving, e.g. the proportion of RCTs recruiting in low- to middle-income countries has increased in recent years. As EHR and digital healthcare become global phenomena, there is scope to use ML in diverse data and settings. Validation of ML applications in clustering and risk prediction across countries and settings, even between high-income countries, are urgently needed to advance this agenda. We have proposed steps to improve the development and validation of ML in clinical datasets (Additional file_Web table 5).