Accounting for the mortality benefit of drug-eluting stents in percutaneous coronary intervention: a comparison of methods in a retrospective cohort study

The comparison of alternative treatments has long been a primary mission of both randomized trials and observational studies. With the commitment of $1.1 billion in support of comparative effectiveness research in the American Recovery and Reinvestment Act of 2009, the number of studies comparing different drugs, devices, techniques and systems will undoubtedly increase dramatically [1]. While randomized clinical trials are likely to remain the gold standard for comparing alternative treatments, observational studies should continue to have significant, if not leading, roles in comparative effectiveness research moving forward, particularly in light of recommendations to prioritize assessments of community-based interventions within populations traditionally underrepresented in clinical trials [2].

However, observational studies are subject to a number of limitations, foremost among them the potential for unmeasured variables that confound results. While a number of methods directed at assessing causal effects and eliminating confounding have been developed, few clinical studies describe reasons for the specific choice of method used, and fewer present multiple methods to help corroborate findings[3, 4].

Observational studies comparing drug-eluting stents (DES) and bare metal stents (BMS) for percutaneous coronary intervention (PCI) have consistently shown lower mortality and myocardial infarction associated with DES [5‐20], findings not seen in randomized clinical trials [21‐27]. We applied three common methods to compare DES to BMS within a large observational study population to: 1) determine whether mortality benefit for DES was observed in our study population and 2) to identify potential challenges to the application of these methods to compare treatments in the presence of strong treatment selection.

We conducted a retrospective dynamic cohort study within Kaiser Permanente of Northern California (KPNC), a large integrated healthcare delivery system caring for > 3.2 million individuals that are broadly representative of the local surrounding and statewide population[28]. All health plan members aged 30 years and older between January 1998 and the end of December 2007 were considered eligible. The study was reviewed by the institutional review board of the Kaiser Division of Research, and requirement for informed consent was waived due to the nature of the study. From this cohort, we identified all PCI procedures using either DES or BMS based on relevant International Classification of Diseases, Ninth Revision, Clinical Modification (ICD-9-CM) and Current Procedural Terminology (CPT) codes that occurred within health plan and non-health plan hospitals [29]. Consecutive procedures occurring within 7 days of one another were considered part of the same clinical episode. Procedures in which both DES and BMS were used were excluded from the analysis.

A total of 35,438 PCIs with stent placement between January 1998 and December 2007 were included. Of these, 19,103 (53.9%) utilized DES and 16,335 (46.1%) utilized BMS. Patients receiving DES were significantly different than those receiving BMS in nearly all baseline characteristics, including having significantly higher rates of hypertensions, diabetes, and congestive heart failure (Table 1). DES patients also had lower rates of presentation with ST-elevation myocardial infarction within 30 days prior to PCI, and were more likely to be receiving outpatient cardioprotective medications including statins, beta-blockers and thienopyridines. Crude-mortality was significantly higher at thirty days (2.2% versus 1.6%, P < .001) and one year (5.1% versus 4.6%, P = 0.04) after PCI with BMS compared to DES. The rate of myocardial infarction at one year was 4.7% after BMS and 4.3% after DES (P = 0.06).

Patients receiving BMS after April 2003 were markedly different than those receiving BMS before April 2003, having higher rates of hypertension (65.0% versus 46.8%, P < 0.001), congestive heart failure (4.3% versus 3.2%, P = 0.008), prior PCI (9.3% versus 5.6%, P < 0.001), systemic malignancy (9.3% versus 5.2%, P < 0.001) and prior gastrointestinal bleeding requiring hospitalization (3.0% versus 1.7%, P < 0.001). Crude-mortality at thirty days and one year and myocardial infarction at one year after PCI were significantly higher in patients receiving BMS after April 2003 compared those receiving BMS before 2003 (Figure 2).

We have shown that after the introduction of drug-eluting stents, bare metal stents were reserved for patients with significant comorbidities and high mortality rates. Commonly used propensity-score methods continued to suggest a mortality benefit to DES, while a stent era comparison showed no significant differences in all outcomes for DES and BMS, similar to results from randomized clinical trials.

A number of prior observational studies comparing DES to BMS have suggested reductions in mortality and myocardial infarction associated with DES, the majority of which have utilized propensity-score methods to mitigate potential confounding [5‐20]. However, data from randomized trials have not corroborated these findings [21‐27]. In a meta-analysis of 22 randomized trials and 34 observational studies, Kirtane et al found no differences in mortality or myocardial infarction in randomized trials, but consistent reductions for both outcomes in observational studies, and attributed these differences, at least in part, to residual confounding by unmeasured differences between patients receiving BMS and DES [23]. Our results are consistent with this conclusion. In our cohort, patients who received BMS after April 2003 had significantly greater comorbidities and unadjusted death and myocardial infarction rates compared to patients who received BMS before this date, suggesting that physicians selectively reserved BMS for a sicker population of patients after DES was introduced. While propensity-score adjustment and matching both attempt to account for such differences, they only account for variables that are assessed. In the presence of such a high degree of patient selection, any residual confounding in this case is likely to lead to the appearance of improved outcomes with DES, as was seen in this study. While propensity scores have been put forward as a method which overcomes some of the limitations of traditional logistic regression, they do not address a primary threat to the validity of such studies - unmeasured confounding.

A comparison of the BMS era (100% BMS) to the DES era (88.3% DES) as a surrogate for a BMS-DES comparison eliminates the unwanted influence of patient selection [45]. Consistent with this, the stent era comparison showed no differences in mortality or myocardial infarction between BMS and DES, most closely approximating results seen in randomized trials. However, because such an analysis compares patients undergoing PCI at two different times, important secular trends may have influenced the results. For example, comparisons of stent eras in the Medicare population and in the New York PCI Registry both suggested lower rates of myocardial infarction in the DES era compared to BMS era [36, 44]. However, neither study adjusted for differences in the use of outpatient medications such as statins and beta-blockers, which have increasingly been used in patients with coronary disease over time [46]. In this study, we found significantly higher rates of use of beta blockers, statins, angiotensin-converting enzyme inhibitors, and thienopyridines in the DES era. After accounting for these and other secular trends, no differences in myocardial infarction were observed between DES and BMS patients.

A number of circumstances specific to stent use allowed for the critical examination of these analytical methods. First, the rapid uptake of DES over BMS created a natural experiment allowing use of a stent era comparison. Second, because a large number of patients from randomized clinical trials comparing DES with BMS have been performed, these studies help provide an estimate of the 'truth.' However, in other examples in which the questions of interest may be less well studied, there could be much greater difficulty interpreting disparate results.

In summary, we have compared outcomes after PCI with either DES or BMS, using three commonly used methods of adjustment in observational studies. Large baseline differences in BMS and DES groups suggested strong treatment selection, which were incompletely adjusted for by propensity-score methods. In the presence of such treatment selection, alternative methods of analysis of observational data should be considered.

No additional acknowledgements.

This work was funded by institutional funds from the Massachusetts General Hospital and Kaiser Permanente of Northern California.