Coronary calcium score improves the estimation for pretest probability of obstructive coronary artery disease and avoids unnecessary testing in individuals at low extreme of traditional risk factor burden: validation and comparison of CONFIRM score and genders extended model

Recent studies have indicated that clinical value of a test for the diagnosis of obstructive coronary artery disease (CAD) depended on the pretest probability (PTP) [1‐3]. Considering this, current guidelines regard the estimation of PTP as an initial and important step in the evaluation of a symptomatic individual with suspected CAD [4, 5]. Updated Diamond-Forrester method (UDFM), a traditional age, sex and chest pain typicality-based approach to the PTP of obstructive CAD on invasive coronary angiography [6], is currently recommended by the European Society of Cardiology (ESC) [4]. However, several studies determined that UDFM seemed to overestimate the PTP of obstructive CAD, especially in low risk populations [7‐9].

With modern statistical methods and multicenter data from populations who underwent coronary computed tomographic angiography (CCTA), new models, e.g. CONFIRM score [10] and Genders extended model (GEM) [11], were developed and what’s more, the addition of coronary calcium score (CCS) in GEM dramatically improved the estimation of PTP [7, 8]. However, neither CONFIRM score nor GEM has been systematacially validated in symptomatic individuals at low extreme of traditional risk factor (RF) burden, for whom the selection of an appropriate diagnostic strategy is important but difficult [12].

Thus, we aim to validate and compare the two proposed models and investigated whether or not the addition of CCS would avoid unnecessary testing among symptomatic individuals with 0 or 1 RF from a cohort of Chinese patients who underwent CCTA.

Table 1 shows the baseline characteristics of the study cohort by RF burden and presence of obstructive CAD on CCTA. There were 1201 individuals with 0 RF, of whom 363 (30%) were found to have obstructive CAD on CCTA. The mean age was 56.26 years and 425 (35%) were males. Except family history of CAD, all variables were significantly associated to the presence of obstructive CAD. Among 2415 individuals with 1 RF, 654 (27%) had obstructive CAD and these individuals were older and had a higher proportion of men, angina and CCS > 0.

Comparison of discrimination using AUC and IDI is shown in Table 2. The AUC for GEM was significantly larger than that for CONFIRM score, no matter in individuals with 0 (0.843 v.s. 0.762, p < 0.0001) or 1 (0.823 v.s. 0.752, p < 0.0001) RF. Compared to CONFIRM score, GEM demonstrated a positive IDI in individuals with 0 RF (5%, p < 0.0001) and individuals with 1 RF (8%, p < 0.0001), respectively.

During a median follow-up of 17 months (interquartile range, 9–23 months), 137 (3.8%) individuals were lost on follow-up. MACEs occurred in 126 individuals (3.5%), including 4 (0.1%) cardiovascular deaths, 11 (0.3%) nonfatal MIs, 46 (1.3%) unstable angina, and 65 (1.8%) late revascularizations. In individuals with 0 RF, GEM had a significantly better discriminatory ability for MACEs than CONFIRM score (AUC for GEM: 0.785 v.s. AUC for CONFIRM score: 0.703, p < 0.0001). Results were similar among individuals with 1 RF (AUC for GEM: 0.802 v.s. AUC for CONFIRM score: 0.709, p < 0.0001).

Table 3 shows the classification of individuals with 0 RF. Of the 838 negative individuals, by GEM, 269 were correctly reclassified to a lower category, but 72 to a higher category. Of the 363 positive individuals, 64 were correctly reclassified to a higher category but 41 to a lower category. Thus, compared to CONFIRM score, the NRI for GEM was 23.51% in negative, 6.43% in positive, and 29.85% overall (p < 0.0001). Results were similar among individuals with 1 RF (Table 4). The NRI for GEM compared to was as follow: 19.42% for negative, 1.68% for positive and 21.10% overall (p < 0.0001).

CONFIRM score classified 55% (459/838) negative individuals with 0 RF and 58% (1028/1721) negative individuals with 1 RF into medium PTP group, for which noninvasive testing were recommend according to current guidelines. Using GEM instead of CONFIRM score would imply a change for diagnostic strategy in these individuals: 57% (260/459) with 0 RF and 58% (592/1028) with 1 RF into low PTP group, for which no further test was recommend. What’s more, among the 852 individuals, only 8 MACEs occurred (0.9%, no cardiovascular death, 1 nonfatal MI, 3 unstable angina, and 4 late revascularizations).

Comparisons of predicted and observed probabilities of obstructive CAD were made by deciles of PTP in Fig. 1. In individuals with 0 RF, CONFIRM score overestimated the prevalence of obstructive CAD resulting in a poor calibration (H-L χ² = 127.34, p < 0.01). On the contrary, GEM revealed a lower but still significant degree of discordance between observed and predicted probabilities (H-L χ² = 56.17, p < 0.01). Comparably, in individuals with 1 RF, GEM was more well calibrated, whereas calibration for both models was unsatisfactory (CONFIRM score: H-L χ² = 85.31, p < 0.01, GEM: H-L χ² = 38.74, p < 0.01).

This CCTA-based study completed in individuals at low extreme of traditional CAD RF burden (0 or 1 RF) demonstrated that the addition of CCS in GEM provided a more accurate estimation for PTP of obstructive CAD. Compared to CONFIRM score, GEM showed a larger AUC, a positive NRI and less discrepancy between observed and predicted probabilities in individuals with 0 and 1 RF, respectively. What’s more, using GEM instead of CONFIRM score could change diagnostic strategy in these individuals, resulting a decrease in unnecessary testing.

Although ESC guidelines recommend UDFM as the model to estimate PTP of obstructive CAD, it revealed significantly overestimates in several external validation studies completed in CCTA-based cohorts [7‐9]. To address this shortcoming, the medical history-based CONFIRM score was developed from an international cohort of patients undergoing CCTA [10]. The CONFIRM score underwent external validation only once after its publication, showing a positive NRI and less miscalibration, but a similar AUC compared to UDFM. In consideration of the change in the quantitative relationship between CAD and variables in traditional age, sex, chest pain typicality and RF-based approaches [8, 20‐22], many efforts have been made to explore whether newer markers could improve the precision of PTP models. A recent work has emphasized that the incorporation of CCS into Duke clinical score improved the diagnostic accuracy for obstructive CAD compared with Duke clinical score alone [23]. Two external validation study [7, 8] for GEM also demonstrated that the addition of CCS promoted the estimation of PTP in the ability of discrimination, classification and calibration, which was confirmed by the present study in individuals with 0 or 1 RF. What’s more, despite the sub-optimal calibration for both models possibly caused by ethnic variation, our results suggested that GEM including CCS provided a more accurate prediction of obstructive CAD than CONFIRM score in individuals at low extreme of traditional RF burden.

So far as we know, this is the first study that systematically validates and compares PTP models in individuals at low extreme of traditional RF burden. In the contemporary environment of rising healthcare costs, a better strategy to select individuals who might benefit from further testing is needed in daily clinical practice [24]. However, several potential reasons may account for the difficult decision-making of diagnostic strategy for symptomatic individuals at low extreme of traditional RF burden, such as lack of awareness, pursuit of economic benefits and fears about the increase in malpractice liability [25, 26]. Current guidelines recommended noninvasive testing, e.g. CCTA and treadmill exercise testing as the appropriate diagnostic test for individuals with medium PTP [4, 5]. Unfortunately, several large and real-world trials which were completed in symptomatic individuals with low-to- medium risk revealed low rates of cardiovascular event and positive noninvasive testing [22, 27‐29]. In conformity with this, according to the reclassification table in our study, CONFIRM score classified more than half of the negative individuals into medium PTP group, which may cause overuse of noninvasive testing. Conversely, GEM classified most negative individuals into low PTP group, resulting in a positive NRI and IDI. Although IDI of GEM over CONFIRM was modest, even small improvements can become significant when applied to the large number of low risk individuals evaluated for suspected CAD in everyday practice. What’s more, the rate of MACEs in individuals reclassified into low PTP group was extremely low. Thus, the addition of CCS into PTP model could change the diagnostic strategy safely and effectively, leading to an evident decrease of unnecessary testing.

There were several limitations that warrant acknowledgement. First, this cohort is a subset of a retrospective and single-center study. In real clinical practice, a substantial proportion of patients with stable chest pain are directly referred for other testing based on individual physician decision. Second, CCTA oftentimes overestimates the severity of calcified plaques because of the high-density artifacts [30]. We defined unassessable segments due to severe calcification as positive, so that if these segments were assessable and taken into account, any overestimation would increase further. Thus, this hypothesis would not qualitatively change the conclusions in this analysis. Last, in the future, the conclusions of this study need to be validated and confirmed in comparative cost-effectiveness analyses with long-term outcome data.

In individuals with 0 or 1 RF, the addition of CCS in GEM provided a more accurate estimation for PTP of obstructive CAD, due to the improvement in discrimination, classification and calibration compared to CONFIRM score. The application of GEM instead of CONFIRM score could change the diagnostic strategy and avoid unnecessary noninvasive testing in individuals at low extreme of traditional RF burden of CAD.