A prediction model for renal artery stenosis using carotid ultrasonography measurements in patients undergoing coronary angiography

From January to December 2011, consecutive patients undergoing elective CAG at Hanyang University Guri Hospital were prospectively included in this study. Patients with end-stage renal disease, patients undergoing emergency percutaneous coronary intervention (PCI), and patients with a history of renal artery intervention were excluded. Written informed consent was obtained from all patients at the time of enrollment in the study. All patients underwent simultaneous coronary and renal angiography. They also completed physical examinations including measurement of blood pressure, body weight and height, and laboratory tests including serum creatinine level, lipid profiles, hemoglobin A1c level and urinalysis. Body mass index (BMI) was calculated as (weight)/(height)² (kg/m²). Proteinuria was defined as a random urine protein/creatinine ratio of >300 mg/g. The estimated glomerular filtration rate (eGFR) was calculated using the Modification of Diet in Renal Diseases equation. The classification of chronic kidney disease (CKD) stages stated in Kidney Disease Outcomes Quality Initiative guidelines was used to define renal function impairment [12]. Carotid ultrasonography was performed to measure CIMT and CAP. The institutional review board of Hanyang University Guri Hospital approved the study design and procedures.

CIMT was measured from the lower arterial wall on longitudinal views of each distal common carotid artery during end-systole. The measurement of CIMT was achieved using automated software (Philips Healthcare, Andover, MA, USA). The average value of the right- and left-sided CIMTs was used for analysis. CAP was defined as the presence of an area with a ≥50% increase in the intima-media thickness compared to that of the neighboring vessel wall. CAP was sought and identified in both common carotid arteries on transverse views, and was measured on longitudinal views.

The standard approach for CAG in the hospital where this study was performed is femoral, and right radial approach is used in a minority of patients if the femoral approach is not available or the patients specially request the radial approach. The femoral artery approach was used for both the coronary and simultaneous renal angiography in this study. 5-Fr Judkins left and right diagnostic catheters (Cordis, Bridgewater, NJ, USA) were used for left and right coronary angiography, respectively. Renal angiography was performed using a 5-Fr Judkins right diagnostic catheter engaged in or directed to the renal artery ostium, with contrast medium flowing back from the renal artery. Both renal arteries were visualized in anterior-posterior projections. The degree of stenosis was measured using quantitative coronary angiography software (Siemens, Philadelphia, PA, USA). Significant CAD was defined as stenosis ≥70% in at least one coronary artery. Significant RAS was defined as stenosis ≥50% in at least one side.

The subjects were divided into two groups according to the presence or absence of significant RAS. The student’s t test was used to compare continuous variables such as age, BMI, total cholesterol level and eGFR between the two groups. Variables with skewed distributions such as triglyceride level, high density lipoprotein (HDL) cholesterol level and CIMT were compared using Mann–Whitney U test. The χ ² test was used to compare binary variables such as hypertension, gender, diabetes, smoking, significant CAD, presence of CAP and CKD stage ≥3.

The model for predicting significant RAS was developed as following; the best-fit model predicting for significant RAS was determined using multiple logistic regression analysis, the model was validated to identify the degrees of optimism and variance, and finally, a scoring system for predicting significant RAS was developed using the coefficients from the regression formula of the best-fit model.

In order to find the best-fit model, we transformed the continuous variables into binary variables using the Youden index-J of the receiver operating characteristic (ROC) curve to maximize their discriminative powers of the best-fit model (e.g., age ≥67 years, BMI <22 kg/m²). Although dichotomization of continuous variables may cause bias and weaken the discriminative power of the model, we used this method because a model using binary or categorical variables would be more accessible than a model using continuous variables. The extent of CAP was defined as a 3-category variable (none, unilateral, bilateral). Then, multiple logistic regression analysis was performed with all binary variables available and the extent of CAP as a categorical variable, to reduce biases in variable selection. Backward variable selection using Wald statistics was performed to identify significant variables and reduce the best-fit model to acceptable events per variable [13]. The exclusion criterion for the backward selection process was set at p ≥ 0.10, to avoid excluding modestly significant variables.

Next, validation and calibration of the best-fit model were performed using bootstrapping methods. Bootstrap re-sampling is an effective technique for internal validation and calibration of a prediction model [14]. Validation using bootstrap re-sampling would estimate the likely performance of the model on a new sample of patients from a same clinical setting. Calibration would measure the degree of error between the predictive probability and observed probability of the model. The validation was performed with 1,000 re-samples drawn using the 0.632 bootstrap technique [15]. We programed the statistical software to calculate new optimal cut-off points for continuous variables, generate binary variables using the new cut-off values in each bootstrap sample and run a multiple logistic regression analysis using the backward selection method with all the new binary variables. Bootstrap models were built from each bootstrap sample. The estimate of optimism in the best-fit model was calculated from the following formula:

where O is the estimate of optimism, C_{original sample} is the area under the curve (AUC) of the ROC curve of the bootstrap model in the original data, C_{bootstrap sample} is the AUC of the ROC curve of the bootstrap model in each bootstrap sample and M is the number of bootstrap samples.

Finally, using the best-fit model, we developed a risk scoring system for predicting significant RAS. Using the following formula for logistic regression analysis of the best-fit model:

we assigned the integer closest to β_n/β₁ as the score for the variable X_n. Then, we summated the integers to generate a risk score for predicting significant RAS in each patient, if variable X_n was present in the patient. ROC curve analysis was performed to evaluate the discriminative power and optimal cut-off point of the scores for predicting significant RAS. The goodness-of-fit of the risk scoring system for significant RAS was assessed using the Hosmer-Lemeshow test and Levenburg-Marquardt nonlinear regression analysis. The predicted probability of significant RAS was calculated from the following equation.

(when L = α + coefficient β × (risk score) in a logistic regression analysis)

All statistical analyses were performed with statistical software, R-3.0.1 for Windows. The rms package was used for logistic regression analysis and the ROCR, pROC and Epi packages were used to identify the optimal cut-off points for continuous variables and automatically transform the continuous variables into binary variables.

Of the 1141 patients who underwent CAG during the period of study, 641 patients remained for the final analysis (Figure 1). The mean age was 61.2 ± 12.5 years and 49% of the patients were male. Hypertension was present in 61.0% of patients, diabetes in 27.8%, and smoking in 27% of the patients. The number of anti-hypertensive medications (AHM) taken by a patient was 1.6 ± 1.0. The mean serum creatinine was 0.87 ± 0.37 mg/dl, and CKD stage ≥3 was present in 45 patients (7%). Total cholesterol level was 175.8 ± 41.1 mg/dl, HDL cholesterol level 47.3 ± 12.5 mg/dl and triglyceride level 148.6 ± 120.9 mg/dl. The mean CIMT was 0.89 ± 0.26 mm and CAP was present in 271 patients (42%). Significant CAD was present in 168 patients (26%) and significant RAS in 60 patients (9.4%). The baseline characteristics of patients with and without significant RAS are shown in Table 1. The mean age, number of AHM and CIMT were higher in patients with significant RAS than those without significant RAS, whereas BMI, total cholesterol level, HDL cholesterol level and eGFR were lower in patients with significant RAS than those without significant RAS. Triglyceride level was not different between the two groups. Hypertension, diabetes, CKD stage ≥3 and proteinuria were more prevalent in patients with significant RAS than those without significant RAS, whereas the proportions of males and current smokers were not different between the two groups. Significant CAD and the presence of CAP were more prevalent in patients with significant RAS than those without significant RAS.

The optimal cut-off values for continuous variables obtained by ROC curve analysis were CIMT 1.0 mm (AUC = 0.683, p = 0.002), age 67 years (AUC = 0.726, p <0.001), BMI 22 kg/m² (AUC = 0.608, p = 0.003), total cholesterol level 158 mg/dl (AUC = 0.618, p = 0.002), HDL cholesterol level 47 mg/dl (AUC = 0.577, p = 0.128) and triglyceride level 119 mg/dl (AUC = 0.537, p = 0.323). Multiple logistic regression analysis including with all the binary and categorical variables identified significant CAD (odds ratio (OR) 5.8, p <0.001), CKD stage ≥3 (OR 4.3, p = 0.002), four or more AHM (OR 3.6, p = 0.039), BMI ≥22 kg/m² (OR 2.6, p = 0.014), CIMT ≥1.0 mm (OR 2.2, p = 0.020), unilateral CAP (OR 2.9, p = 0.039) and bilateral CAP (OR 5.5, p <0.001) as significant predictors of significant RAS (Figure 2). Among the significant predictors, significant CAD, CKD stage ≥3, four or more AHM and bilateral CAP were stronger predictors of significant RAS. Age ≥67 years and HDL cholesterol level ≥47 mg/dl were marginally significant predictors of significant RAS. Total cholesterol level ≥158 mg/dl, hypertension, male gender, current smoking, diabetes, proteinuria and triglyceride level ≥119 mg/dl were not significant predictors of significant RAS. The average variable inflation factor of all the variables included in the multiple logistic regression analysis was 1.19 and no variable inflation factor of any variable exceeded 1.4. In multiple logistic regression analysis with backward selection, significant CAD, unilateral or bilateral CAP, CKD stage ≥3, four or more AHM, CIMT ≥1.0 mm, age ≥67 years and BMI < 22 kg/m² remained as predictors of significant RAS (Table 2).

Using the results of the multiple logistic regression analysis with backward selection, we developed a scoring system for predicting significant RAS (Table 3). To produce the scoring system, we assigned the simplest integers proportional to the coefficient β of each predictor. The smallest coefficient β was 0.831 for age ≥67 years and the largest one was 1.724 for significant CAD. The ratios of the coefficients of predictors to the smallest coefficient were therefore all between 1 and 2. We assigned a score of 1 to unilateral CAP, CIMT ≥1.0 mm, age ≥67 years and BMI <22 kg/m², and a score of 2 to significant CAD, bilateral CAP, CKD stage ≥3 and four or more AHM. The total scores ranged from 0 to 11. In ROC curve analysis, the scoring system for significant RAS showed an AUC of 0.896 (95% confidence interval 0.869 - 0.918), which was not significantly different from the AUC of the best-fit model (AUC = 0.898, difference = 0.002, p = 0.69 using the DeLong method). The scoring system showed sensitivity of 83.3% and specificity of 81.6% at a cut-off point of ≥4 Using the same cut-off point and a prevalence of significant RAS 9.4%, the positive predictive value was 31.8%, and the negative predictive value was 97.7% (Figure 3). The predicted frequency of significant RAS using the scoring system agreed well with the observed frequency of significant RAS. The Hosmer-Lemeshow test for the goodness-of-fit of the scoring system showed p = 0.881, and the coefficient of determination between the predicted frequency and observed frequency using Levenburg-Marquardt non-linear regression analysis was R ²  = 0.957 (Table 4).

The apparent AUC of the best-fit model derived from multiple logistic regression analysis with backward selection was 0.898 (95% confidence interval 0.872-0.918). Validation of the best-fit model with 1000 bootstrap re-samples showed that the average optimism was 0.023 and the adjusted AUC of the best-fit model was 0.875. The calibration plots of the best-fit model and the bootstrap model are shown in Figure 4. The estimates of both models were slightly non-linear, with the bootstrap model being slightly more non-linear than the best-fit model. Both models agreed well with the ideal line when the predicted probability of significant RAS was low, but the disagreement of the bootstrap model increased with increasing the predicted probability of significant RAS. However, the mean absolute error and 0.9 quantile absolute error of the predicted probability were 0.028 and 0.047 respectively, suggesting only a small degree of bias from overfitting in the best-fit model.

We found that the extents of CAP and CIMT were independent predictors of significant RAS in patients undergoing CAG. The prevalence of significant RAS increased with the presence of CIMT and the extent of CAP. The model for predicting significant RAS which included significant CAD, CKD stage ≥3, four or more AHM, age ≥67 years, BMI <22 kg/m2, CIMT ≥1.0 mm, and the extent of CAP showed a high discrimination power and a small degree of bias. The predictive frequency of the scoring system agreed well with the observed frequency of significant RAS.

The prevalence of significant RAS was 9.4%, which is similar to the findings of previous studies [3, 5, 9, 11, 16, 17]. However, the actual prevalence of significant RAS may be higher, because we excluded patients undergoing emergency PCI and those with a history of renal artery intervention.

Hypertension and diabetes, classic risk factors for atherosclerosis, were not strongly associated with significant RAS in this study because they were already reflected by other predictors, namely significant CAD, CIMT, extent of CAP, old age, four or more AHM. The high prevalence of hypertension among patients undergoing CAG may also have contributed to the weak association between significant RAS and hypertension. Instead, four or more AHM indicating severe, uncontrolled hypertension was a strong predictor of significant RAS. The BMI and total cholesterol levels were paradoxically lower in patients with significant RAS than those without significant RAS. This may be because the BMI and total cholesterol level also partially reflect muscle mass and nutritional status [18, 19]. Patients with significant RAS often have multiple co-morbidities and poor general health. Prezewlocki et al. [10] also reported that BMI <25 kg/m² was a predictor of significant RAS.

Carotid ultrasonography parameters, such as CAP and CIMT, are known as the predictors of cardiovascular disease [7, 8, 20], and the relationship between RAS and systemic atherosclerosis is well established [16]. The association between CIMT and RAS, however, has only been reported in a few studies [21, 22], and the value of the extent of CAP for predicting RAS has not been reported until now. This is the first study using a multiple logistic regression model to report CIMT and the extent of CAP measured by carotid ultrasonography as independent predictors of significant RAS in patients undergoing CAG.

RAS contributes to severe but correctable, hypertension and to left ventricular hypertrophy. Although RAS is an independent risk factor for cardiovascular mortality, screening for RAS in asymptomatic patients is currently controversial, because none of the large randomized trials to date have shown clinical benefits associated with renal artery stenting compared with medical therapy in patients with RAS [23, 24]. Routine “drive-by” renal angiography in all patients undergoing elective CAG is especially difficult to support because of the lack of evidence of benefit and the low prevalence of RAS. A distinction should be made, however, between identifying patients with significant RAS and selecting patients for renal artery intervention. It is important to identify patients with significant RAS who are at increased risk of cardiovascular events and require close observation [25]. RAS is an independent predictor of mortality in patients with cardiovascular disease [1, 2, 25]. From this point of view, it may be useful to determine indications for performing renal angiography at the time of CAG in selected patients undergoing CAG.

The American Heart Association advises renal angiography at the time of CAG in patient with multi-vessel CAD or PAD [6]. The reported prevalence of significant RAS ranges from 10% to 36% in patients with triple-vessel CAD [3, 9‐11, 22] and from 21% to 55% in patients with PAD [9, 11, 25]. Models for predicting significant RAS that include multiple clinical predictors may enable more accurate selection of patients for renal angiography. Several studies have proposed prediction models for significant RAS in patients undergoing CAG [9‐11]. Clinical predictors including age, hypertension, BMI, number of AHM, CAD, PAD, serum creatinine level and eGFR have been used in these models. However, these prediction models were either too complicated or did not provide a sufficiently predictive performance to be applied to clinical practice.

Duplex ultrasonography is a safe and acute non-invasive screening tool for RAS. The sensitivity and specificity of duplex ultrasonography were already 92.5% and 95.7%, respectively, in a 1997 study of patients suspected to have RAS [26]. Duplex ultrasonography will be a superior screening modality to any clinical predictors or scoring methods, if a patient is suspected to have RAS. Nevertheless, a scoring system for predicting RAS can still be a useful tool for clinicians to estimate the risk of RAS in asymptomatic patients undergoing CAG.

Carotid ultrasonography is a simple, non-invasive tool frequently used in current clinical practice to evaluate cardiovascular risk [27]. Our scoring system for predicting significant RAS included CIMT and CAP measured by carotid ultrasonography, and showed better sensitivity and specificity than scoring systems in the previous studies. The high negative predictive value and moderate positive predictive value may enable use of the scoring system as a quick decision-making tool for a physician to undertake a definite diagnostic testing for significant RAS. The goodness-of-fit between the predicted and observed frequency of significant RAS was high in our model. The scoring system was also made as simple as possible, so that it could easily be applied to clinical practice. All the items of the scoring system were assigned in the simplest integers, 1 or 2, and the score range was 0 to11.

We also validated our model with bootstrap re-sampling technique. Validation and calibration procedures are required for a prediction model to be useful in clinical practice. Przewlocki et al. [10] performed validation of their model by random splitting of the data. However, other previous studies that reported models for predicting RAS did not perform validation procedures. Bootstrap re-sampling is a more effective technique for validating a prediction model than data splitting [14], and the 0.632 bootstrap technique used in our validation procedures is a variant bootstrap method that can provide very similar validation to that obtained using an independent data set [26]. A small degree of optimism was observed, and the disagreement in validation and calibration between the predicted probability and actual probability increased with increasing predicted probability. However, the bias from overfitting was acceptable in this study. We believe that our prediction model can be a useful tool for evaluating the risk of significant RAS in patients undergoing CAG.

Our study was performed in a single center and may therefore contain referral bias. Although the patients were included consecutively in the study, the exclusion of patients because of radial approach, emergency PCI, end-stage renal disease, previous renal artery intervention, or inadequately performed renal angiography may have affected the recorded prevalence of significant RAS. Carotid ultrasonography is a simple and non-invasive tool for assessing the extent of atherosclerotic diseases, but, is still not performed routinely in patients undergoing CAG. Although the scoring system can be used for pre-procedural estimation of the probability of RAS, the indications for renal artery intervention in asymptomatic patients still need to be established for the prediction model to be relevant to improving clinical outcomes. Finally, although we validated our model, internally with bootstrap re-sampling technique, the proposed scoring system should be externally validated before being used in routine clinical practice.