Major adverse cardiovascular event definitions used in observational analysis of administrative databases: a systematic review

Cardiovascular disease is the leading cause of death in the United States, making it a common target for interventional research [1, 2]. Due to this, the composite endpoint of “major adverse cardiovascular events” (MACE) is an increasingly common primary outcome of interest. In 2008, the United States Food and Drug Administration (FDA), followed by the European Medicines Agency (EMA) in 2012, provided guidance on utilizing a three-point MACE outcome, which includes acute myocardial infarction (AMI), stroke, and cardiovascular mortality in all trials evaluating the cardiovascular safety of diabetic agents [3]. Some trials have utilized a four-point MACE as well, by including hospitalization for unstable angina or revascularization procedures [3, 4]. Five-point MACE further expands on this with the inclusion of heart failure (HF). While MACE is now a better-defined and more ubiquitous outcome among RCTs, its use in observational studies to assess the safety and real-world effectiveness of therapies remains less clear.

Observational studies using large administrative databases can evaluate population-level use and outcomes of therapies in an efficient and cost-effective way [5]. However, several common issues with observational studies limit their reproducibility and comparability to RCTs, thus limiting the utility of MACE as a composite outcome. First, in comparison to RCTs, observational studies often do not consistently report their protocols, including outcome definitions [6]. To allow for useful comparisons between observational studies and RCTs, improved standardization and transparency is needed [7, 8]. Even when MACE components are well-defined, such as AMI or stroke, another challenge encountered in observational studies is how to define outcomes using diagnosis codes available from administrative data. For example, the International Classification of Diseases (ICD) is a commonly used coding system for medical reimbursement and is one of the most frequent sources of information available in administrative databases [9]. Unfortunately, ICD codes can be prone to errors and diagnosis misclassifications, as they are primarily collected for reimbursement purposes and rely on clinical documentation that can vary across settings and providers [5]. Though several studies have attempted to validate the diagnosis codes for commonly used MACE components (e.g., AMI, stroke, and HF), with positive predictive values of upwards of 80–90%, it is unclear how these codes have been taken up in MACE composite outcome definitions [10‐12]. For these reasons, the current use of MACE in observational studies warrants further investigation.

Thus, the purpose of this review was to systematically determine the most common definitions of MACE employed in observational studies using administrative data. With that, our objectives were: i) assess each study’s definition of MACE components (e.g., AMI, stroke), ii) assess the diagnostic criteria used for outcome ascertainment such as codes used and position of codes, and iii) assess whether outcomes had been validated. We hypothesized that, across observational studies, there is great variability in the definitions of MACE used and minimal alignment with the classic three, four, or five-point MACE outcomes. Our hope is that this work will promote a standard approach to the definition of MACE in future studies, allowing for the improved transparency and reproducibility of observational studies.

In our systematic review, we found substantial heterogeneity for MACE composite endpoints used in the literature. The two most common composite MACE definitions were “AMI and stroke”, and “AMI, stroke, and all-cause death,” but they only made up 15.5 and 13.8% of studies, respectively. Compared to MACE definitions used in RCTs, only 8.6% of included observational studies had definitions aligned with three-point MACE and none match four-point or five-point MACE. A large majority of studies included AMI and stroke but there was a lack of consensus with other components included in composite MACE endpoints. For instance, over half of included publications defined MACE using a definition that only was concordant with up to one other study. This diversity makes it challenging to compare findings across studies or to aggregate multiple study results for meta-analyses or systematic reviews when considering different treatment or research questions. Addressing the heterogeneity of MACE definitions used in practice requires attention due to the advantages of the MACE endpoint. Utilizing MACE as a composite outcome can potentially reduce the number of patients that need to be enrolled or identified in a retrospective cohort study, and reduce the follow-up time necessary to observe differences between different treatment groups [99]. These benefits not only potentially reduce research costs but can more expediently answer clinical questions, leading to improved patient care. Therefore, given that MACE endpoints will likely see increasing use as time goes on, there should be a continued effort to standardize and transparently report MACE definitions used in observational studies.

In 2007, findings similar to our study were observed when comparing prospective trials conducted for percutaneous coronary interventions [100]. However, although the FDA in 2008 and the EMA in 2012 attempted to standardize the MACE endpoint definition for RCTs, we found a continued discordance between how observational studies define MACE. Furthermore, we found that the majority of studies did not include mention of the diagnosis position used to define outcomes. Lack of this information prevents the ability to distinguish between incident and prevalent outcomes. While the primary position has been historically used in observational studies to define incident outcomes, the use of only primary diagnosis positions, compared to using secondary positions, may underestimate the rates of MACE for prevalent conditions [101]. The lack of reporting of diagnosis position prevents the ability to make this distinction and prohibits the full interpretation of findings.

Heterogeneity not only existed in defining MACE components but also in which ICD-9-CM and ICD-10-CM codes were used to define each individual MACE component. In our study, none of the top five components of MACE had a consensus ICD-9-CM or ICD-10-CM code that was used among all of the studies. These discrepancies contribute additional variability to our findings, as outcome classification, assuming a standard definition, inherently relies on the providers and medical billers accurately coding diseases. For example, in a study conducted in 11 Canadian emergency departments, the authors reported about 82–86% of agreement with regards to coding hospital conditions [102]. Additionally, a study looking at stroke data collected through the United States Paul Coverdell National Acute Stroke Program found that discordance existed with how strokes were coded, especially at smaller hospitals (< 200 beds) and when differentiating between ischemic strokes, transient ischemic attacks, and subarachnoid and intracerebral hemorrhages [103].

There have been many attempts to validate diagnosis or procedure codes for various MACE component outcomes [32, 104]. Research has predominantly focused on the older ICD-9-CM codes, particularly studies using U.S. administrative data, but studies utilizing ICD-10-CM have more recently been published. Using data from the Centers for Medicare and Medicaid Services, a 2018 study found that the most accurate ICD-9-CM code for coding AMI was 410.xx, with a positive predictive value of 67% [104]. Additionally, the U.S. FDA commissioned the Mini-Sentinel pilot program to systematically assess the validation of common healthcare outcomes in administrative databases [32]. These studies were important steps to identify standard outcome definitions for use across disparate data sources and populations. As part of the program, systematic literature reviews from the U.S. and Canada found that the most accurate ICD-9-CM code for heart failure was 428.xx, with a positive predictive value of 84–100%, while the most accurate ICD-9-CM codes for stroke were 430.x, 431.x and 434.x, all with separate positive predictive values > 80%, and 436.x, which had a positive predictive value > 70% [32, 33]. The Mini-Sentinel program also assessed the ICD-9-CM codes, 410.× 0 or 410.× 1, for AMI and found a positive predictive value of 76.3–94.3% [105]. Fewer validations exist for ICD-10-CM codes. One study conducted in Japan attempted to validate the ICD-10-CM code I21.x for AMI and found a positive predictive value of 82.5% [106]. Another study in Canada found the positive predictive value for correctly coding stroke was 92% for the ICD-10-CM codes, I60.x-I61.x, I63.x-I64.x, H34.1 and G45.x [69]. Based on our findings, the most common ICD-9-CM and 10-CM codes used for AMI were also the most validated, 410.xx and I21.xx, respectively. The most validated and most common ICD-9-CM code used for heart failure was 428.xx. On the other hand, stroke had some discrepancies when comparing the most validated versus the most commonly used ICD codes. This discordance likely existed because of the differences with which stroke can be defined, particularly whether to include acute ischemic strokes with transient ischemic attacks, intracerebral hemorrhages or subarachnoid hemorrhages [69].

There are several limitations to our study. We excluded papers published before 2010, prior to the first published reviews of the Mini-Sentinel program, in order to present a contemporary review of the subject [32]. However, this cutoff likely skewed our results towards more modern MACE definitions and excluded different definitions that were potentially used for older publications. Older studies likely exhibited greater variety in the outcome definitions used. Additionally, because the words “major adverse cardiovascular events” were rarely explicitly written, we had to include any composite definitions that were thought to represent MACE but were listed using another composite name, such “acute cardiovascular events,” based on the discretion of the reviewer. This method potentially excluded studies that used variations of MACE endpoints, but did not conform to similar naming conventions.

Significant heterogeneity exists in how MACE is defined and which ICD-9-CM and 10-CM codes are used to represent each respective MACE outcome in observational studies using administrative databases. The utility of future studies will improve with the use of validated definitions of AMI, stroke, cardiovascular mortality, unstable angina, and HF for the evaluation of MACE outcomes. Further, investigators should ensure that both the ICD diagnosis codes and the code positions are reported in a transparent way. Given the significant heterogeneity that already exists across administrative databases, we recommend the use of more standardized MACE definitions and corresponding ICD-9-CM and ICD-10-CM codes. This practice will allow researchers to more accurately compare findings across studies improve the reproducibility of observational studies, and decrease the potential for misleading conclusions.