Importance of operator training and rest perfusion on the diagnostic accuracy of stress perfusion cardiovascular magnetic resonance

CMR images were acquired using a 3T scanner (Achieva, Philips Healthcare, Best, The Netherlands) equipped with 32-channel phased-array cardiac coil. The protocol included functional assessment, adenosine stress and rest first pass perfusion imaging, and LGE. The images were acquired using standard acquisition protocols and in end-expiratory breath-hold. For stress imaging, 140 μg/kg/min of adenosine was administered. Imaging commenced at least 3 minutes after infusion initiation. A dual bolus (equal volumes of 0.0075 mmol/kg followed by 0.075 mmol/kg after a 20-s pause) of contrast agent (gadobutrol/Gadovist, Schering, Germany) was injected at 4 ml/s by a power injector [13]. For perfusion, a saturation recovery prepared gradient echo pulse sequence accelerated with k–t sensitivity encoding acceleration with 11 training profiles was used. Typical imaging parameters were: 3 short-axis slices covering standard American Heart Association (AHA) segments [14], 120 acquired dynamics/slice, flip angle 20°, TR 2.5 ms, TE 1.25 ms, saturation pre-pulse recovery time 100 ms, pixel size 1.9 × 1.9 mm, slice thickness 10 mm.

Typical imaging parameters for LGE imaging were: long and short axis to fully cover the left ventricle, inversion recovery turbo field echo, flip angle 25°, TR 6 ms, TE 3 ms, pixel size 0.7 × 0.7 mm, slice thickness 10 mm.

Nine operators were chosen amongst the physicians working in our unit and in other European institutions, on the basis of their level of competency, according to the European Society of Cardiology (ESC)/European Association of Cardiovascular Imaging (EACVI) training guidelines [12]. A total of 9 operators, 3 for each competency level, were chosen; all operators had recently obtained the ESC/EACVI certification (within 2 months) for the appropriate level. In brief, level-1 competency ESC certification requires 20 continuous medical education (CME) hours, involvement in 50 CMR cases and 1-month fellowship; level-2 requires at least 50 CME hours, involvement in 150 clinical cases of which 25 must be perfusion studies, a minimum of 3-months fellowship and the European CMR exam; level-3 requires at least 50 CME hours, involvement in 300 clinical cases of which a minimum of 50 must be perfusion studies, at least 12-months training and the European CMR exam. Level-1 competency reflects core CMR training, level-2 is required to report CMR studies with support from a Level-3 operator and Level-3 is required to perform, interpret and report CMR studies fully independently [12].

Each operator was asked to report each of the 53 scans twice over a 4-week period, with a minimum interval of 2 weeks between first and second read. The scans were anonymized and presented to the operator as a full dataset, including stress and rest perfusion and LGE, or as reduced datasets, including stress perfusion and LGE only. The full and reduced datasets were analysed blinded to clinical and angiographic data and in a randomized order on different days. The study flowchart can be seen in Fig. 1.

Visual assessment of adenosine stress perfusion CMR and LGE images, displayed side-by-side, was performed as per clinical practice, in accordance with standardized CMR protocols [15]. A perfusion defect was defined as a regional reduction in myocardial signal during LV first-pass of contrast agent, not related to artefacts and not corresponding to an area of scar on LGE images.

Operators were asked to fill an on-line standardized form and to identify segments with inducible ischemia, to identify the presence and transmurality of LGE [16], to identify the most likely culprit coronary artery based on the standard AHA segmentation [14], and to grade their confidence in the diagnosis and the perceived image quality.

The confidence was graded as: 0- very unconfident, 1- unconfident, 2- confident, 3- very confident. The perceived image quality was graded as: 0- poor, 1- moderate, 2- good, 3- excellent.

Coronary angiography results have been used as reference standard. The threshold for coronary artery lumen stenosis was 70% diameter stenosis for epicardial vessels. All invasive angiographic images have been reviewed by consensus of expert operators.

A different operator, blinded to results of visual perfusion assessment and other clinical/angiographic data, performed the segmentation of the images for semi-automated quantitative analysis using software and methods previously developed and validated by our group. Respiratory motion was corrected using affine image registration by maximization of the joint correlation between consecutive dynamics within an automatically determined region of interest [17]. A temporal maximum intensity projection was calculated to serve as a feature image for automatic contour delineation method. The operator then manually optimized the automatically generated contours to avoid partial volume effects at the endocardial and epicardial borders [17]. The intervention of the operator was limited to image segmentation. Quantitative perfusion analysis was then automatically performed by Fermi-constrained deconvolution according to the methods described by Wilke et al. [18] and Jerosch-Herold et al. [19], optimised for high-resolution pixel-wise analysis [20, 21]. Myocardial perfusion reserve (MPR) was calculated as the ratio between stress and rest myocardial blood flow (MBF) estimates. Ischemia was defined as segments with MPR < 1.5, according to previously validated criteria [22, 23].

Continuous variables are presented as mean ± standard deviation for normally distributed variables and as median with interquartile range for non-parametric data. Normality was assessed with Q-Q plots and the Kolmogorov-Smirnov test. Continuous variables were compared using an unpaired Student t test or the Wilcoxon rank-sum test, as appropriate, and categorical data were compared between groups using the Fisher exact test and Pearson chi-square test. The McNemar test was used for paired dichotomous data. Two-tailed values of p < 0.05 were considered to be statistically significant. One-way ANOVA was used to determine differences between multiple groups. Bonferroni correction was used to account for multiple testing.

The mean age of the population (n = 53) was 60.6 ± 12.7 years. Demographic data are shown in Table 1. The prevalence of CAD in the group of patients included in the analysis was 30.2%, with 16/53 patients positive for CAD on invasive coronary angiography. Left anterior descending (LAD) lesions were identified in 9 (17%) of the cases, left circumflex (LCX) lesions in 8 (15.1%) of the cases, and right coronary artery (RCA) in 13 (24.5%) of the cases. Within the group of patients with CAD, 8 patients had 1-vessel disease (50%), 5 patients 2-vessel disease (31.3%) and 3 patients 3-vessel disease (18.8%).

There was a significant correlation between an operator’s training level and the rate of correct identification of CAD on a per patient level on visual assessment. The diagnosis of Level-3 operators agreed with invasive coronary angiography in 83.6 ± 2.3% of the cases, while this percentage dropped to 65.7 ± 4.3% for Level-2 operators and to 55.7 ± 5.3% for Level-1 operators (p < 0.001 between the 3 groups) (Fig. 2). A significant difference in the agreement with angiography between different levels of training was also observed in a sub-analysis per coronary territory (p < 0.001) (Fig. 3). When different perfusion territories were compared, the agreement between CMR and coronary angiography was higher for the LAD territory, followed by the LCX and by the RCA territories. The same trend was observed in all groups of operators, regardless of the level of training (p < 0.001).

The sensitivity and specificity for operators of different levels of training are reported in Fig. 4. Level-1 operators showed high sensitivity (86.5 ± 6.1%) and low specificity (41.9 ± 10.9%). Level-2 operators had a sensitivity of 57.3 ± 4.7% and a specificity of 69.4 ± 9.9%. Level-3 operators showed a sensitivity of 71.9 ± 13% and a specificity of 88.7 ± 6.7% respectively. There was a statistically significant difference for both sensitivity and specificity between different levels of training (p < 0.001) (Fig. 4).

When rest images were available, there was no statistically significant difference at all levels of training (Fig. 5) and in the overall analysis (69.6 ± 14.3% vs 67.1 ± 13.1%; p = 0.34). However, when rest images were available, a significantly higher level of confidence was reported by the operators (p = 0.022) and subjective image quality was scored at a higher level (p = 0.012).

Figure 6 shows a comparison between the extent of CAD identified by the operators on CMR images in comparison with invasive coronary angiography. An overestimation of the severity of CAD was observed in Level-1 operators, regardless of the number of vessels with CAD. Despite being more accurate, Level-2 and Level-3 operators significantly underestimated the number of positive perfusion territories in patients with multi-vessel CAD.

Quantitative analysis was successfully performed in 51 patients. In 2 cases of patients without CAD, the automated algorithms failed and no results could be calculated. In both cases, this was due to the low quality of the diluted pre-bolus used for the estimation of the arterial input function. Level-3 visual assessment of the 2 cases where quantification failed yielded the correct diagnosis in both cases when both stress and rest images were made available to the readers, and in 66% of interpretations when only stress perfusion was made available to the readers. Quantitative stress perfusion CMR analysis agreed with the results of invasive angiography in 86.3% of the cases, performing significantly better than Level-1 and Level-2 operators (p < 0.001). Level-3 visual assessment and quantitative analysis were not significantly different (p = 0.56) (Fig. 2). Quantitative analysis had a sensitivity of 68.8% and specificity of 94.3%. When the 2 cases in which quantitative analysis failed are considered as a missed diagnosis, the concordance of quantitative analysis with invasive angiography was 83%, with a sensitivity of 68.8% and a specificity of 89.2%.

This study included a selected population with suspected CAD and we excluded patients with primary cardiomyopathy. Thus, our results on diagnostic accuracy do not include other patterns of perfusion abnormalities, which may require even more experience to discern (e.g., microvascular dysfunction).

Moreover, we used an anatomical reference standard (invasive coronary angiography) to compare operators’ performances in interpreting a functional test, while a functional reference standard (e.g., fractional flow reserve) may be more appropriate.

Our results demonstrate that similarly accurate detection of CAD can be achieved by Level-3 operators and by automated perfusion quantification. Although our study was not powered to demonstrate the superiority of quantitative analysis, this has been the subject of a recent study which has reported very similar findings [30]. The non-inferiority of automated quantification to expert visual reads, in combination with the prognostic value of quantitative analysis [23] will facilitate more widespread adoption of stress perfusion CMR by less experienced readers.

Finally, all stress perfusion CMR were acquired in a single center, using a 3 T Philips scanner and a high-resolution k-t sequence. This may not reflect the standard clinical acquisition in other centres.