Identifying Outcome Measures for Coronary Artery Disease Value-Based Contracting Using the Delphi Method

P2Y12 inhibitors are a mainstay of therapy for patients with coronary artery disease (CAD) and have been shown to reduce recurrent major adverse cardiac events including heart attacks and cardiovascular death [1]. Newer P2Y12 inhibitors prasugrel and ticagrelor have demonstrated improved clinical outcomes compared to clopidogrel but are associated with higher costs and lower adherence [2]. In response to the rising costs of pharmaceuticals, there has been an increased interest in the development of value-based contracts for P2Y12 inhibitors that aim to increase the value and lower the price of medications by aligning incentives and sharing risk between payers and manufacturers [3]. In contrast to traditional volume-based pharmaceutical payment models, value-based contracts link drug payments to real-world, disease-related performance metrics [3]. Because there are numerous potential clinical outcomes in CAD, uncertainty remains regarding which CAD outcomes are most meaningful to all stakeholders—including patients and providers—and should be included in value-based contracts.

There have been three publicly disclosed value-based contracts for antiplatelet medications in the United States to date [4]. Two of the three base drug pricing on reduced hospitalizations due to cardiovascular events, while the third ensures shared coverage of treatment costs if the rate of heart attacks in patients taking the antiplatelet exceeds a certain threshold [5‐7]. It is unclear how these outcome measures were chosen, and whether they are meaningful to all stakeholders affected by value-based contracts. While the Delphi method has been used in previous studies to reach provider expert consensus on clinical CAD indicators [8], to our knowledge, this method has not been used to identify meaningful CAD outcomes among all relevant stakeholders for the specific purpose of value-based contract development.

We therefore conducted a modified Delphi study incorporating multiple stakeholder perspectives, including those of patients, cardiologists, primary care physicians (PCPs), payers, pharmacy benefits manager (PBM) representatives, and pharmaceutical company representatives, to identify the most meaningful CAD outcomes to inform value-based contracts for antiplatelet medications.

We administered a modified Delphi survey using well-established methods to assess the value of various CAD indicators to diverse stakeholders [9, 10]. The Delphi technique employs anonymous, iterative questionnaires to reach expert consensus, allowing all participants to weigh in equally without dominance of the discussion by a small subset of individuals [9]. Between survey rounds, group responses are aggregated and incorporated into the next round’s questionnaire, where participants are given the option to reconsider and revise their previous selections. This process continues until consensus is reached. Unlike a traditional Delphi in which the first round consists of an open-ended solicitation of ideas [10], this modified Delphi utilized a list of CAD indicators established in advance of study onset, as discussed below. This study was deemed exempt by the University of Pittsburgh Institutional Review Board. All procedures performed were in accordance with the ethical standards of the institutional research committee and with the 1964 Helsinki Declaration and its later amendments.

Thirty (30) individuals, including ten (33.3%) patients, ten (33.3%) providers, four (13.3%) payers, four (13.3%) PBM representatives, and two (6.7%) pharmaceutical company representatives expressed interest in participating in the study and were sent the first-round questionnaire. Of these, 27 (90%) completed the Round 1 survey, and 25 of those 27 (93%) completed Round 2. The final Delphi panel consisted of nine (36%) patients, nine (36%) providers, two (8%) payers, three (12%) PBM representatives, and two (8%) pharmaceutical company representatives. Provider experts included five PCPs and four cardiologists, each with at least 5 years of clinical CAD experience. Of the nine patient participants, four indicated that they had a previous heart stent procedure but not a previous heart surgery, one had a previous heart surgery but not a heart stent procedure, three had both a stent procedure and heart surgery, and one had neither procedure. None of the panelists submitted additional indicators to include. “Reducing depression symptoms,” “preventing missed days of work,” and “reducing medication complications like minor bleeding” were ranked the lowest in importance in Round 1 and were removed from the Round 2 survey.

After two Delphi rounds, 11 of 13 outcomes were rated as “important” or “very important” (4 or 5 on the Likert scale) by over 75% of panelists (Table 1). Two of these outcomes (“preventing heart attacks” and “preventing need for a repeat heart procedure”) received a score of 4 or 5 by all stakeholders in Round 2 (100% agreement; Table 1). When asked to select only the top three most meaningful outcomes, 80% of participants chose “preventing heart attacks” and 76% chose “preventing death,” both reaching consensus after the second round (Table 2, last column). Agreement increased between Round 1 and Round 2 for these top two measures, corresponding to decreased selection of several other measures, most notably “reducing medication complications like major bleeding” and “preventing need for a repeat heart procedure” (Table 2).

A two-sample test of proportions identified statistically significant differences between patient and non-patient stakeholders in the ranking of two outcomes as most meaningful. Specifically, “reducing chest pain” and “reducing shortness of breath” were each selected as a most meaningful outcome by 22.2% of patients but by 0% of non-patient stakeholders in Round 2 (p = 0.049, Table 2).

To our knowledge, this is the first Delphi-based study performed among a diverse panel of expert stakeholders to identify meaningful CAD outcome measures for the specific purpose of informing value-based pharmaceutical contract development. After two survey rounds, participants reached consensus in ranking “preventing heart attacks” and “preventing death” as the two most meaningful outcomes.

For value-based contracting to evolve and deliver on the promise of improved care at lower costs, there needs to be greater process transparency, including in the selection of meaningful disease outcomes. Previous Delphi studies have been conducted to reach expert consensus on clinical CAD indicators, including an initiative by the nonprofit International Consortium for Health Outcomes Measurement (ICHOM) that sought to define a consensus standard set of measures for tracking, comparing, and improving the outcomes of CAD care [8]. A working group of 17 CAD experts and patients identified a core set of 13 CAD outcomes that they recommend should be clinically monitored in a standardized manner, which included occurrence of major adverse cardiac events such as myocardial infarction, heart failure, stroke, renal failure, and death, as well as patient-reported outcomes such as quality of life, shortness of breath, chest pain, depression, and functional status [8]. While monitoring this comprehensive list of indicators is beneficial in a clinical setting, measuring 13 outcomes would be onerous and impractical in the context of value-based contracting between payers and pharmaceutical manufacturers. Notably, the ICHOM Delphi panel did not include key value-based contract stakeholders (i.e., PBM representatives, pharmaceutical company representatives, or payers), and thus does not address the question of which measures are considered most meaningful for value-based contract development. Nevertheless, the ICHOM core outcome set substantially informed the development of our Delphi questionnaire.

Heart attack, stroke, and cardiovascular death, as well as major and minor bleeding events, are endpoints typically used in modern clinical trials for antiplatelet therapy [13, 14]. While it is unclear how previous value-based contracts for antiplatelet medications selected their outcomes, their foci on the incidence of heart attacks and hospitalizations due to cardiovascular events suggest the perceived value of these indicators in assessing drug effectiveness in a manner that is feasible to collect through administrative claims data [5‐7]. Our finding that “preventing heart attacks” ranks as the most meaningful outcome among our diverse stakeholder group verifies and supports the utility of this measure in future value-based contracts for CAD medications.

While it may seem commonsensical that patients would find these clinical outcomes to be of high importance, it has been shown that what patients value can differ from conventional clinical endpoints [12]. Performing a Delphi-based assessment that incorporates multiple stakeholder opinions can shed light on such differences. Interestingly, a greater proportion of patients in our study chose “reducing chest pain” and “reducing shortness of breath” as most meaningful outcomes compared to non-patient panelists. While these indicators were selected as most meaningful by only a small minority of patients, this divergence highlights the possibility that what patients may perceive as desired or expected disease outcomes may be overlooked by non-patient stakeholders when defining how the value of medications is evaluated. While the incorporation of patient-reported outcomes in value-based pharmaceutical contracts may present unique challenges to payers and manufacturers, it is important that decision-makers take into consideration the inclusion of such measures as indicators of drug value when they are identified as meaningful.

Our study has several limitations. First, because all patient panelists were recruited from our health plan’s internal Pharmacy Care Management Program, selection bias may have been introduced. Since this program outreaches to patients who have experienced myocardial infarction, coronary artery bypass graft surgery, and/or automatic implantable cardioverter-defibrillator placement and who have been referred for physical or behavioral health intervention, this cohort may have poorer health than the general CAD population. Conversely, patients who participate in the program and who replied to our questionnaires may be more engaged than others. Therefore, the directional effect of this potential selection bias is unclear. Nonetheless, we made a purposeful effort to include a diverse patient group by recruiting patients with different cardiovascular procedure histories, years living with CAD, and insurance products. Another limitation was that all provider participants are located in southwestern Pennsylvania, and therefore their input may not be representative of a broader provider population. Although geographic diversity was not represented among providers, both cardiologists and PCPs across two health systems were included on the panel to capture potential differences in opinion between generalists and specialists with expertise in CAD care. While it was beyond the scope of this study, the conduct of a multi-center, multi-regional Delphi might illuminate potential differences in opinion related to varied local and professional cultural perspectives. In addition, not all individuals who originally expressed interest in participating responded to both survey rounds, and consequently a subset of viewpoints were potentially not represented in the final results. Notably, our response rates of 90% in Round 1 and 93% in Round 2 are consistent with what is typically reported for Delphi studies [9], and participation was sufficient to achieve our study goals. Lastly, while our stakeholder panel was similarly sized if not larger than those described in the literature [9], it was comprised of a relatively small number of non-provider, non-patient representatives. Including a larger sample of individuals in each stakeholder group may have resulted in a different outcome, however, we purposefully over-sampled patients and providers proportional to other stakeholder groups because patient and provider viewpoints are often underrepresented in typical value-based contracts between payers and manufacturers [16].

Our finding that “preventing heart attacks” ranks as the most meaningful outcome among our diverse stakeholder panel verifies and supports the utility of this widely used clinical measure in future value-based contracts for CAD medications.