Background
Myocardial perfusion scintigraphy (MPS) is widely regarded as a clinically useful non-invasive imaging method for the diagnosis of suspected coronary artery disease, identification of culprit lesions and risk assessment [
1‐
4]. Visual interpretation of MPS studies is dependent on the knowledge of the physician, and subject to inter- and intra-observer variability. Software packages for automated quantification of MPS, such as ejection fraction and summed stress score (SSS), have been developed in order to make the interpretations more standardized.
Recent studies have shown that patients with significant ischemia and without extensive scar were likely to realize a survival benefit from early revascularization [
3,
5]. In contrast, the survival of patients with minimal ischemia was superior with medical therapy without early revascularization. Therefore, determination of the amount of ischemia is essential for patient management. In the studies, the percent ischemic myocardium was calculated using 5 point/20-segment MPS scoring, i.e. taking into account both the extent and severity of the ischemic myocardium. The results from the studies were recently incorporated into guidelines on revascularization. The European Society of Cardiology recommends that patients with stable angina or silent ischemia with “proven large area of ischemia (>10%)” should receive revascularization (class 1B) [
6].
Several studies have demonstrated that the variability between different software packages to calculate scoring values are considerable [
7‐
9], which will affect the calculated percent ischemic myocardium.
It has previously been shown that referring physicians to MPS tend to underestimate the extent of ischemia and infarction reported by the physician in nuclear medicine [
10]. Because of the new recommendations, it is important to report the amount of the ischemic myocardium, preferably as a percentage of the left ventricle. It is also important that there is a consensus among physicians interpreting MPS images regarding the amount of the ischemic myocardium.
In this study we wanted to investigate 1) the inter-observer variability for the extent of reversible perfusion defects reported by different physicians in nuclear medicine, 2) the variability for the reversible perfusion defects obtained from different software tools, 3) the differences between the assessments made by the physicians and the different software packages and 4) if the inter-observer variability is reduced when the physicians are provided with a computerized suggestion of the delineation of the reversible perfusion defects.
Methods
Patients and MPS protocol
The study was approved by the local research ethics committee at Lund University and complies with the Declaration of Helsinki.
Twenty-five patients who underwent MPS in Malmö, Sweden January-February 2006 were included. The patients were chosen in a consecutive manner based on the final report, and only patients who where regarded as ischemic were included. 16 of the patients were male; 9 were stressed using maximal exercise test and 16 using Adenosine; mean age was 71 ± 8.5 years; mean ejection fraction 57 ± 11%; mean end-diastolic volume 139 ± 48 ml; 9 patients got the diagnosis acute coronary syndrome after the MPS (follow-up until 2010). According to the final report, 22 patients had single-vessel disease, 2 had multi-vessel disease and 1 had suspected multi-vessel disease.
The MPS studies were performed according to clinical routine, using a 2-day gated stress/gated rest 99mTc-tetrofosmin protocol, starting with an injection of 600 MBq 99mTc-tetrofosmin at stress. Patients were stressed using either maximal exercise on an ergometer or a pharmacological test with adenosine. The exercise test was continued for at least 1 min after the injection of the tracer and the adenosine infusion for at least 2 min after the injection of the tracer. This examination was followed by a rest study with injection of 600 MBq 99mTc-tetrofosmin.
Stress and rest acquisition began 60 min after the end of the injection of 99mTc-tetrofosmin. Images were obtained according to established clinical protocols, using single photon emission computed tomography over 180° elliptical, autocontour rotations from the 45° right anterior oblique position, with a dual-head gamma camera, e.cam (Siemens AG Medical Solutions, Erlangen, Germany). Patients were imaged in the supine position. A low energy high-resolution collimator and a zoom factor of 1.0 were used. We obtained 64 (32 views per camera) projections in a 128 × 128 matrix, with an acquisition time of 20 s per projection. Stress images were gated to the electrocardiogram using 8 frames per cardiac cycle. No automatic motion-correction program was applied; instead the acquisition was repeated if motion was detected. Tomographic reconstruction and calculation of short and long axis slice images were performed using e.soft (Siemens AG Medical Solutions, Erlangen, Germany). Images were reconstructed with filtered back-projection. A 2D Butterworth pre-reconstruction filter was used with cut-off frequency of 0.45 Nyquist frequency, order 5.
Evaluations made by physicians
Eleven physicians in nuclear medicine from 8 different hospitals in 4 countries (Sweden, Denmark, Germany, and Japan) were invited to participate. Two of the physicians had 5 and 7 years of experience with interpreting MPS, and the other physicians had 15-40 years of experience. All cases were classified visually, and a custom display software was developed for this purpose, allowing the physicians to view non-attenuation corrected slice images (short axis, horizontal and vertical long axis) of the rest and stress studies, polar plots (rest, stress, rest-stress differences, stress/rest ratio) and 3D-images. Colour scales and contrast levels were adjustable. No quantitative results from the software package were available during the evaluation. The physicians were instructed to delineate the ischemic area in the polar plot. They could add as many ischemic areas as they wanted. The physicians were informed that the 25 cases had the diagnosis “reversible ischemia” according to the clinical report (in which all available clinical data, stress test, MPS study, etc. were considered) and the gender of the patient. No other information was given to the physicians (i.e. no information about known cardiac diseases, symptoms, electrocardiogram, stress test results, weight, etc.).
After at least two weeks, all physicians delineated the reversible defects again. This time they were provided a suggestion of the defect delineation by EXINI HeartTM (EXINI Diagnostics AB, Sweden; EXINI), but otherwise the layout of the program was the same as before. The physicians were able to adjust the delineation suggested by EXINI, or they could remove the suggested delineation and create a new delineation. The studies were presented in a different randomized order, compared with the order in the first evaluation. Originally, none of the software packages had the feature for the physician to easily adjust a proposed delineation of a perfusion defect. We therefore developed that on the platform of EXINI and choose to use EXINI as reference method in the paper.
Software packages
Four different software packages were compared; (EXINI version 5.0beta [
11], Emory Cardiac Toolbox version 3.0 (Emory University; ECT [
12]), Quantitative Perfusion SPECT version 4.0 (Cedars Sinai; QPS [
13]), 4D-MSPECT version 4.0 (Invia Medical Solutions; 4DM [
14]). Summed difference scores (SDS) were obtained from all 4 programs and compared to the total possible score from 17 segments (% ischemic myocardium); called EXINI summed difference % (SD%), ECT (SD%), QPS (SD%) and 4DM (SD%) in this article. This approach was included based on the study by Hachamovitch et al [
3] and this measurement takes into account both the extent and the severity of the defects. The extent of the ischemic defects were also obtained from the programs; called EXINI (extent), ECT (extent), QPS (extent) and 4DM (extent) in this article. These measurements also reflect the percentage ischemic myocardium (extent/area), but are calculated using other methods compared to the SD% method. This approach was included due to the recommendations in revascularization guidelines [
6].
Automatic delineation of ischemic regions by EXINI
The automatic delineation of defects in EXINI proceeds in two stages. In stage one, defects in the stress image are delineated. From a number of seed points extracted from local intensity minima, deformable models of circular topology based on the active contour framework [
15] are fit to the image based on edge and intensity information. This produces a set of “stress defects” which may be either reversible and/or irreversible. In a second phase, each stress defect is analysed in order to find strictly reversible sub-regions based on information from the difference image. Regions which are contained inside the stress defects and which exhibit high difference intensities are delineated using a simple smoothing and thresholding approach, producing a set of reversible “difference defects”. This paper concerns difference defects only.
Statistical analysis
Descriptive statistics for continuous variables are given as mean and standard deviation (SD) or median and range. Inter-observer variability is expressed as intra-class correlation (ICC) with 95% confidence intervals (CI) between observers as estimated from linear mixed model analysis where patients and observers were entered as random effects. For comparison between the different programs and observers, linear mixed models were used where program (one at a time) and observers were entered as fixed effects and patients as random effects. For a comparison within observers’ mean values between the two occasions without and with EXINI, Wilcoxon’s signed rank test was used. The level of statistical significance was set at p < 0.05. Bland-Altman plots were created for analysis of relationship between the evaluations made by EXINI and the physicians, where EXINI was used as reference method [
16].
Discussion
In this study, we found that the inter-observer variability for the delineation of ischemia was considerable when both regarding different physicians and different software packages. We also found that the inter-observer variability decreased when the physicians were provided with a suggested delineation of the ischemic defects by EXINI.
The results are of value when considering the new recommendation made by the European Society of Cardiology regarding revascularization: patients with stable angina and ischemic amount of myocardium >10% should receive revascularization, whereas patients with ischemic myocardium <10% should receive only medical treatment [
6]. Therefore, quantitative analysis of ischemic area is needed to determine the therapeutic strategy for patients with ischemic heart disease. But how can the physician interpreting the MPS images decide on the amount of ischemic myocardium when inter-observer variability among physicians and software packages is large? The recommendation was based on studies by Hachamovitch et al [
3,
5], where the authors found that in the setting of no or mild amounts of inducible ischemia, patients undergoing medical therapy had a survival advantage over patients undergoing revascularization. These two lines intersected at a value of ≈ 10% to 12.5% myocardium ischemic, above which the survival benefit for revascularization over medical therapy increased as a function of increasing amounts of inducible ischemia. As seen in Figure
2, the variability among the physicians varied for small, medium-sized and large ischemic areas.
It is well known that SSS, summed rest scores (SRS) and SDS differ between different programs. Guner et al [
7] compared the diagnostic performances of three different software packages in detecting coronary artery disease by
201Tl-MPS. They found that the diagnostic performances of the programs to detect coronary artery disease were similar. However, there were differences in the quantitative values produced by the programs. The differences in SDS between different programs were significant (p < 0.0001). The highest correlation was between 4DM and QPS (r = 0.68), and the lowest was between ECT and QPS (r = 0.41). Wolak et al [
8] concluded that there were differences in myocardial perfusion quantification, diagnostic performance, and degree or automation of three software packages with correlations between SSS for the three different software packages evaluated ranging from r = 0.68 to r = 0.84. Svensson et al [
9] also investigated the quantification of reversible perfusion defects by three different software packages. They found that widely used software packages commonly differ in their quantification of myocardial perfusion defects and conclude that the interpreting physician should be aware of these differences when using scoring systems.
Not only do the scoring values differ between different software tools, there is also the risk of obtaining scores in areas that are not believed to be ischemic (for example can a patient get many scores of 1 in several areas of the myocardium). The method of delineation of defects by EXINI mimics the physicians’ way to delineate. It is also easier to show the referring physician (often a cardiologist) the extent of the defects rather than scoring values. One should remember, however, that the recommendation of 10% is based on studies using scoring values and not defect extents. Such studies are needed before incorporating the method into clinical routine.
Despite the possible obstacles to provide a percentage of ischemic myocardium to the referring cardiologist, our study has also shown a possible solution. When the physicians were provided with a suggested delineation of the defect, the inter-observer variability decreased. Thus, a program where the physicians easily can adjust the suggested delineation could serve as a tool to make the estimated ischemic amount more standardized. Such a program could, beside standardization, also be used for the training of young doctors.
One advantage for the present study is the relatively large number of physicians involved as well as that the physicians were from several hospitals and countries. Typically, studies involving inter-observer variability include only a few physicians, often from the same department, which tend to decrease inter-observer variability. Thus, the present study shows a “truer” variability in evaluations made by different physicians.
The findings of the present study should be considered in the light of some limitations. In order to find enough physicians willing to participate in the study, a limited number of patients were included. However, it is of greater importance to include several physicians and from different departments/countries. Also, we were not able to verify the evaluations made by the physicians and software packages with an independent reference technique. The physicians did not have any background data regarding the patients, no stress test result, and no information about ejection fraction, left ventricular volumes or transient ischemic dilation. It is possible that the ischemic delineations differ from the ones reported here if such information would have been provided.
Competing interests
LE, KNm and KS are employed by EXINI Diagnostics AB.
Authors’ contributions
LE and ET were the principal investigators, participated in its design and coordination and drafted the manuscript. PHd participated in the design of the study, performed the statistical analysis and helped drafting the manuscript. SF, PHk, AJ, AK, OL, ML, SM, KNa, EO, SS, LJ and HW collected the data, helped in the design of the study and revised the manuscript. KNm and KS made the custom display of EXINI, participated in the design and coordination and revised the manuscript. All authors read and approved the final manuscript.