Automated SPECT analysis compared with expert visual scoring for the detection of FFR-defined coronary artery disease

The study population comprised 206 patients from the PACIFIC study (Comparison of Coronary CT Angiography, Myocardial Perfusion SPECT, PET, and Hybrid Imaging for Diagnosis of Ischemic Heart Disease: Prospective Cohort Study Using Fractional Flow Reserve to Determine Functional Severity of Coronary Stenoses; NCT01521468), who underwent ECG-gated SPECT/CT and invasive coronary angiography (ICA) with routine interrogation by FFR [13]. Enrolled patients were suspected of stable CAD with an intermediate pre-test likelihood and a normal left ventricular function. Exclusion criteria were a documented history of CAD, signs of prior myocardial infarction, contraindication to adenosine, atrial fibrillation, glomerular filtration rate < 45 mL∙min⁻¹, and pregnancy.

Patients were instructed to refrain from intake of products containing caffeine or xanthine 24 h prior to the scans, next to at least 4 h of fasting. A 2-day stress-rest ^99mTc-tetrofosmin protocol was performed in all patients. During continuous infusion of adenosine (140 μg∙kg⁻¹∙min⁻¹), a weight-adjusted dose of 370 to 550 MBq ^99mTc-tetrofosmin was injected. The adenosine infusion was terminated 3 min after tracer injection. Following a delay of 60 min, ECG-gated stress SPECT images were acquired. Rest SPECT imaging was performed on the same day as ICA. Images were acquired on a dual-head hybrid SPECT/CT scanner (Symbia T2, Siemens Medical Solutions, Erlangen, Germany). Emission data were acquired using a parallel-hole, low energy, high resolution collimator with a 20% symmetric window centred at 140 keV, where the two detector heads were positioned at an angle of 90°. The camera heads performed a 180° rotation with, in total, 64 rotational steps of 40 s per projection. ECG-gating was performed with an electrocardiogram R-wave detector with acquisition of 8 frames per cardiac cycle. Images were reconstructed in both a static and gated manner. Characteristics of the embedded two-slice CT component were as follows: slice width 5.0 mm; pitch 1.5; 130 kV, 17 mA; rotation time 0.8 s. SPECT acquisition was followed immediately by a low-dose CT scan during normal breathing and without ECG-gating to correct for attenuation using 130 keV, 20 mAs, a computed tomography dose index of 2.2, and a dose length product of 40. CT images were reconstructed with a 128 × 128 matrix and a slice thickness of 5 mm.

Visual interpretation was performed by a core laboratory. A highly experienced observer (SRU, > 30 years of experience in nuclear cardiology) was blinded to other imaging and angiographic findings, but limited clinical information was available (patient's sex, age, body mass index, type of chest pain, and the presence of a left bundle branch block) because of the direct effects on scan interpretation. MPI images were interpreted based on a 17-segment model [14]. Each segment was scored using a 5-point scoring system (0, normal; 1, mildly decreased; 2, moderately decreased; 3, severely decreased; and 4, absence of segmental uptake). Summed rest scores (SRS), summed stress scores (SSS), and summed difference scores (SDS) were calculated from the segmental scores, with SSS ≥ 4 and SDS ≥ 2 considered abnormal [15, 16]. The expert reader was able to take into account additional information such as raw projections, ECG-gated LV functional information, as well as non-corrected (NC) and AC reconstructions in order to maximize recognition imaging artifacts. Visual interpretation was conclusively classified as normal or abnormal on a per patient basis.

Perfusion parameters were derived in an entirely automated fashion using commercially available software (Cedars-Sinai Quantitative Perfusion SPECT [QPS]) [2, 17, 18]. Each scoring parameter was derived from images with and without AC, representing both the extent and severity of myocardial hypoperfusion. These parameters comprise both the aforementioned scores based on the average defect severity per segment (SSS and SDS), as well as the pixel-wise total perfusion deficit (TPD) during stress (S-TPD) and the ischemic TPD (I-TPD), defined as the difference between stress and rest TPD [18]. SSS ≥ 4, SDS ≥ 2, S-TPD ≥ 5%, and I-TPD ≥ 3% were considered abnormal [15, 16, 19, 20].

Potential enhancement of the diagnostic performance of automated quantitative scoring was also explored. For this purpose, the total study database was consecutively divided into two subgroups. The optimization process comprised two components. First, an institutional normal database was created using data from the first subgroup, the derivation cohort (n = 103). The normal database was developed with SPECT images derived from patients with both normal angiographic findings as well as normal myocardial perfusion using [¹⁵O]H₂O positron emission tomography (PET) imaging, which was additionally performed in the context of the PACIFIC trial [13]. By doing so, only SPECT images derived from patients without CAD were selected. The institutional database could then be generated within the commercially available software package. Second, optimal thresholds were obtained from the derivation cohort for each grading parameter with and without AC. Automated scoring was subsequently performed in the validation cohort (n = 103) with the use of the new normal database and optimized thresholds for abnormal scans.

ICA imaging was performed using a standard protocol in at least two orthogonal directions per evaluated coronary artery segment. In order to induce epicardial coronary vasodilation, 0.2 mL of nitroglycerin was administered intracoronary ahead of contrast injection. All major coronary arteries were interrogated routinely by FFR, regardless of stenosis severity, except for occluded or subtotal lesions of more than 90%. FFR was measured using a 0.014-inch sensor tipped guide wire (Volcano Corporation, Rancho Cordova, CA, USA), which was introduced through a 5- or 6-F guiding catheter, calibrated and advanced into the coronary artery. Intracoronary (150 μg) or intravenous (140 μg∙kg⁻¹∙min⁻¹) adenosine infusion was used to induce maximal coronary hyperaemia. FFR was calculated as the ratio of mean distal intracoronary pressure, and mean arterial pressure. A coronary lesion was considered hemodynamically significant in case of FFR ≤ 0.80, or stenosis severity >90% obtained with quantitative coronary angiography (QCA) if FFR was missing. A stenosis with an FFR > 0.80, or a stenosis severity <30% (obtained with QCA) in the absence of FFR measurements, was not considered to be functionally relevant. Secondary analyses were performed using QCA stenosis severity as a reference with ≥70% stenosis considered obstructive. All images and FFR signals were interpreted by two experienced interventional cardiologists blinded to the SPECT result.

Table 2 shows diagnostic performance in terms of sensitivity, specificity, and diagnostic accuracy of expert visual reading and multiple automated measurements for the detection of hemodynamically significant CAD. These parameters were scored using normal scan databases incorporated in the commercially available software and accepted thresholds of abnormality. Among all automatically assessed scores, only SDS with and without AC (86.5% and 80.0%, respectively, p < 0.001 for both) showed a significantly higher sensitivity than expert reading (56.5%). Sensitivity of the other parameters, including SSS, S-TPD and I-TPD, did not statistically differ from visual analysis, even though a trend was visible in favor of automated analysis. In contrast, specificity of expert reading (93.9%) was significantly higher than that of each of the automatically derived scores. In terms of diagnostic accuracy, automated assessment performed more poorly than visual reading (77.2%), except for SSS AC (69.8%, p = 0.063) and S-TPD AC (71.2%, p = 0.134).

After dividing the total group of patients into derivation and validation cohorts (n = 103 for both), 51 normal SPECT images (including 30 female patients images) were used for the development of a new institutional database. As listed in Table 1, there were no differences in baseline characteristics between the derivation and validation cohorts. Figure 2 shows the average polar maps for the normal database, incorporated in the software package by the vendor, next to the average polar maps derived from the newly generated institutional normal database. Cases selected for the new normal database (normal FFR and normal PET perfusion) showed an SSS = 0 in 7 (14%) cases and an abnormal SSS ≥ 4 in 15 (29%) with the use of the original database. Mean SSS values decreased implementing the institutional database towards SSS = 0 in 34 (67%) cases and SSS ≥ 4 in 1 (2%) case. Based on the derivation cohort and the new normal databases, optimal thresholds were set at SSS ≥ 3, SDS ≥ 2, S-TPD ≥ 5 and I-TPD ≥ 2 for images without AC (AUC: 0.83, 0.84, 0.87, and 0.79, respectively), and at SSS ≥ 2, SDS ≥ 2, S-TPD ≥ 2 and I-TPD ≥ 1 for images with AC (AUC 0.82, 0.83, 0.85, and 0.76, respectively; Fig. 3).

Table 3 presents the diagnostic performance of expert visual reading and automated assessment in the validation cohort using the institutional database and optimal thresholds. The sensitivities for NC images were consistently low and did not significantly differ from visual reads. Automated scoring with AC images provided higher sensitivities, although this difference was significant only for S-TPD and I-TPD (p = 0.001 and p = 0.008, respectively). In contrast, the visually obtained high specificity remained significantly different from all automated AC scores, whereas NC images did not show a significant difference. Consequently, diagnostic accuracy for each automated assessment did not significantly differ from expert visual analysis. Explicitly, the highest accuracies were found for SSS AC (72.5%), SDS AC (72.0%), and S-TPD AC (73.5%) and paralleled expert analysis (73.8%).

Using QCA instead of FFR as a reference, the performance of the expert reader was enhanced, mainly due to a higher sensitivity (online Tables 1 and 2). In general, sensitivity also increased for automated analysis while specificity generally decreased, yielding a heterogeneous change in performance in terms of diagnostic accuracy. Nevertheless, diagnostic accuracy was significantly lower for all automated scoring variables with the use of standard software (online table 1). After the software optimization, NC parameters were comparable to visual reading with regard to diagnostic accuracy, except for I-TPD which was significantly lower. Among the AC parameters, SSS and SDS showed comparable accuracy, whereas S-TPD and I-TPD revealed significantly lower accuracy (online table 2). AUCs of the automated parameters did not significantly differ, but in general, a trend was observed for the numerical smaller AUCs for I-TPD with and without AC (online Figs. 1 and 2).

An accurate assessment of the extent and severity of hypoperfused myocardium with SPECT is important for diagnostic and prognostic purposes [17, 19, 21], but remains subjective when determined visually. Therefore, software tools for automatic quantification have been developed, and subsequently it has been shown that these are more reproducible than visual assessment, even if the latter is performed by highly experienced readers [9, 22]. Arsanjani et al. showed that diagnostic performance of automated analysis did not significantly differ from that of visual assessment [7]. Although these results were promising, the study was hampered by the fact that a significant portion of the study population was expected to be free of CAD based on low risk profiles without confirmation through ICA. Another limitation of several diagnostic studies is that when study patients did undergo ICA, visual estimation of stenosis severity was used as the reference standard, but frequent disagreement between angiographic visual stenosis severity and functional severity is increasingly recognized [11]. As a sub-analysis from the PACIFIC trial, the present study was not hampered by these limitations, as all subjects underwent invasive coronary angiography with routine FFR measurements [13].

The present results from automated analysis using standard software and thresholds, in general, revealed a lower diagnostic accuracy than visual expert reading except for SSS and S-TPD with AC. Furthermore, automated analysis showed higher sensitivity, but lower specificity, than visual reading. This implies that when automated analysis is used for diagnostic purposes, AC is warranted and stress-only protocols are sufficient. Nevertheless, it should be realized that these results hold true only for this particular study population of patients with a normal left ventricular function without prior history of CAD or myocardial infarction. In addition, the slightly higher sensitivity of automated analysis in trade off of a lower specificity could be more favorable as the number of ‘unnecessary’ invasive angiograms might outweigh missing obstructive CAD due to a false negative SPECT. Of note, it seems important to see that the use of FFR instead of the more traditionally used QCA as a reference changed the results to some extent (online table 1). One explanation for the improved performance of visual reading using QCA as a reference, might be the reader’s experience and prior feedback based on QCA rather than FFR (i.e. the readers ‘internal normal database’).

The technique for automated quantification of myocardial perfusion relies on the analysis of tracer distribution within one patient, which then is compared with a database of normal perfusion scans. This so called normal database is usually based on perfusion images from the USA, since most software packages are developed there. Given possible differences in patient habitus and imaging protocols (including tracer doses and scanning settings), this database may not be optimal for other regions [23]. An interesting study from Nakajima et al. showed a significant diagnostic improvement with the use of a region specific normal database in Japan [24]. In contrast, a similar study with a French population did not show a clear benefit compared with the normal database supplied by the vendor [25]. The present study demonstrates the feasibility of creating an institution normal database with only a limited number of normal perfusion studies. However, reviewing Fig. 2 reveals merely minor differences for average polar maps between the present institutional database and the vendor supplied database. Nonetheless, the normalcy rate improved from 71% to 98% and the majority of diagnostic accuracies for automated analyses in the validation cohort directly increased after implementing the institutional database (online Fig. 2). The most pronounced differences (according to average segmental counts) seem to be located in the anterior and basal inferior regions for NC images, probably as a result of attenuation differences due to surrounding soft tissue, such as breast and abdomen. Of interest, the normal tracer distribution of AC images, on average segmental counts maps and standard deviation maps, appears to be very similar for both gender averages as well as for vendor and institution databases. This suggests the possibility to easily exchange AC image normal databases [26]. A limitation of common normal databases is the typical use of SPECT images obtained from patients with a low pretest likelihood for CAD, who did not undergo ICA to confirm. Even though differences might be small, the current database is unique because normal perfusion was guaranteed through the confirmation with ICA and [¹⁵O]H₂O PET imaging in prospectively enrolled patients.

The main purpose of generating an institutional normal database was to improve performance of the automated analysis. Based on the derivation cohort, optimal thresholds were slightly lower than the traditional cut-off values, particularly for AC thresholds. The diagnostic accuracy of the newly set thresholds was higher for attenuation corrected images, confirming the benefit of attenuation correction when automated analysis is used. Although diagnostic accuracy was consistently higher with AC, the question remains whether this justifies the additional radiation burden for patients, or costs and time for imaging laboratories. Furthermore, derivation cohort AC images did not show an improved accuracy when using AUCs, which provide a comprehensive and likely a more adequate evaluation of the diagnostic performance than the dichotomized accuracy (Fig. 3). The validation cohort revealed that implementation of a normal database and optimized thresholds now resulted in an equivalent diagnostic accuracy of automated analysis as compared with visual expert grading for all investigated parameters (SSS, SDS, S-TPD and I-TPD). Using QCA as a reference instead of FFR, diagnostic accuracies of most optimized automated analysis parameters were not significantly different from visual reading (online table 2). In general, however, these numerical differences were somewhat more pronounced than for the comparisons referenced by FFR. The fact that the institutional normal database was created with the use of FFR rather than QCA might have played a role.

Recently, several developments have been implemented clinically in order to improve diagnostic performance of conventional SPECT imaging. For instance, ECG-gated acquisitions provide additional diagnostic and prognostic information such as end-diastolic volume, ejection fraction and transient ischemic dilatation [27‐29]. These functional parameters were not included in the present automated analysis, but hold great potential for further improvement of the automatic SPECT interpretation, for example using machine learning programs [30].

Visual analysis of SPECT imaging slightly outperforms automated analysis with standard software in the detection of FFR-defined significant CAD. After optimization with an institutional normal database and thresholds, however, diagnostic accuracy of automated analysis equalled expert visual analysis without the need for comprehensive reading experience. Therefore, automatic assessment has the potential to simplify the diagnostic process using SPECT, particularly in conjunction with CT-based AC.