Diagnostic performance of the specific uptake size index for semi-quantitative analysis of I-123-FP-CIT SPECT: harmonized multi-center research setting versus typical clinical single-camera setting

Single-photon emission computed tomography (SPECT) with the I-123 labeled dopamine transporter (DAT) ligand FP-CIT is widely used for detection (or exclusion) of nigrostriatal degeneration in patients with clinically uncertain parkinsonian syndrome (PS) [1]. Semi-quantitative analysis of striatal FP-CIT uptake has the potential to support visual interpretation of the SPECT images [2, 3].

Semi-quantitative analysis in radionuclide imaging often aims at estimating the binding potential (BP), first defined by Mintun and co-workers as BP = Bmax / KD, where Bmax is the density of available binding sites (here DAT) and 1/KD is the affinity of the tracer (here FP-CIT) for the binding site [4]. The binding potential is a measure of the capacity of the region of interest (ROI) for specific binding of the tracer, that is, binding to the binding site of interest. More generally, a binding potential can be defined as the equilibrium concentration of specific binding to some other reference concentration [5]. The most widely used binding potential is the non-displaceable binding potential BPnd based on the concentration of the non-displaceable (by blocking of the binding site) tracer in the ROI as reference, i.e.,

where Cs and Cnd are the equilibrium concentration of specifically bound and non-displaceable tracer in the ROI, respectively. Determination of BPnd requires dynamic imaging and arterial blood sampling for full tracer kinetic modeling. This is not feasible in clinical routine so that a number of methods have been developed to estimate BPnd from a single static scan during equilibrium (for FP-CIT approximately given between 3 and 6 h after i.v. injection [6]). The most widely used among these methods is the specific binding ratio (SBR) defined by the formula [7]

where C is the total count concentration in the striatal ROI, and CR is the total count concentration in a reference region (almost) void of DAT. The SBR according to formula (2) is based on the assumption that CR approximates the count concentration originating from non-displaceable FP-CIT binding in the striatal ROI (the difference C – CR then approximates the specific count density originating from FP-CIT bound to DAT in the striatal ROI). The striatal ROI for computation of the SBR usually anatomically delineates the striatum or the striatal subregion of interest such as caudate nucleus or putamen [8, 9]. A major limitation of the SBR according to formula (2) is the strong underestimation of the striatal FP-CIT concentration C in the SPECT images due to partial volume effects caused by limited spatial resolution of SPECT. The recovery of actual striatal FP-CIT concentration in SPECT images typically is only about 50% if no correction for partial volume and other degrading effects is performed [10‐12]. In addition to strong underestimation of the SBR, partial volume effects cause additional variability. This is due to the fact that the magnitude of the partial volume effect strongly depends on the size and shape of the striatum and the spatial resolution in the reconstructed SPECT image. Spatial resolution varies between SPECT cameras depending on both, hardware and acquisition/reconstruction protocol. The additional variability of the SBR associated with the use of different SPECT cameras negatively impacts the utility of the SBR, particularly in multi-site and in single-site/multi-camera settings or when normal values and cut-offs obtained from healthy subjects at one site are to be used at other sites [13, 14]. Only in an ideal situation in which all main causes of inaccuracy and imprecision of SPECT (attenuation, scatter, partial volume effects, statistical noise) were properly dealt with, the true SBR value would be obtained with any SPECT camera and reduction of inter-camera variability would naturally follow from accuracy.

The specific uptake size index (SUSI) has been proposed to improve accuracy of semi-quantitative analysis in FP-CIT SPECT in practice [7, 15, 16]. It has been thoroughly validated in phantom studies where true count densities are known [15, 17]. The SUSI eliminates partial volume effects by replacing count density in the striatal ROI by a measure of total counts in the ROI. More precisely, the SUSI is defined as

where T is the total number of counts originating from a large striatal ROI, V is the volume of the striatal ROI, and CR is the count concentration in the reference region.

The striatal ROI for the computation of the SUSI is chosen sufficiently large around the striatum to guarantee that all counts originating from the striatum are detected within this ROI. Assuming that image reconstruction is “activity conserving” (that is, counts are neither lost nor artificially produced by the reconstruction, although limited spatial resolution can cause counts originating from the striatum to be localized outside of the striatum), summing the activity over all voxels within the large striatum ROI collects all counts originating from the striatum. This holds true independently of the spatial resolution in the reconstructed SPECT image as long as the ROI is large enough to encompass all counts originating from the striatum. Thus, the SUSI is independent of spatial resolution [15]. The SUSI approach can be seen as a method for partial volume correction of the SBR. This is due to the fact that SBR = SUSI / Vs, where Vs is the actual volume of the striatum, as is easily derived from Eqs. (2) and (3) [15].

The SUSI has been used successfully for the quantitative characterization of age effects on DAT availability in healthy subjects from the European normal control database of FP-CIT SPECT (ENC-DAT) [18]. A total of 13 sites using ten different SPECT camera models contributed to the ENC-DAT study.

The SUSI is typically computed in the original SPECT images in patient space after manual reorientation [7, 15, 16]. A limitation of the SUSI computed in patient space is that it also depends on the size of the striatum, not via partial volume effects as the SBR, but via the total amount of specifically bound FP-CIT that increases with increasing size of the striatum (assuming constant DAT concentration). Thus, inter-subject variability of striatum size causes additional variability of the SUSI that might limit its power to detect nigrostriatal degeneration. The impact of inter-subject variability of striatum size on the SUSI might be eliminated by concentration-preserving stereotactical normalization of the SPECT images into an anatomical standard space prior to ROI analysis. Inter-subject variability of partial volume effects cannot be eliminated this way.

A recent study suggested the SUSI to be more sensitive than the SBR to errors of the estimate of non-displaceable count density in the striatum by the count density CR in the reference region [19]. In both methods, SBR and SUSI, the reference region is used to correct the striatal signal for non-displaceable “background” in the striatum ROI that dilutes the effect of nigrostriatal degeneration in these measures if not corrected for. It is evident that the background contribution increases when the ROI size increases beyond the anatomical boundaries of the striatum. Thus, the relative magnitude of the correction for non-displaceable background increases with increasing ROI size. As a consequence, the computed measure of specific striatal binding (SBR or SUSI) might be more sensitive to errors (including statistical noise) of the estimate of non-displaceable FP-CIT binding the larger the striatal ROI.

Aim of the present study was to compare the SUSI and the SBR with respect to detection of nigrostriatal degeneration in (i) a sample of patients with PS in whom FP-CIT SPECT has been performed with the same SPECT camera using the same acquisition protocol with radius of rotation within very tight limits [20] and the same reconstruction protocol (mono-site/mono-camera (MONO) setting) and (ii) in the multi-site/multi-camera (MULTI) setting of the Parkinson Progression Markers Initiative (PPMI) [21]. Inter-subject variability of spatial resolution in the SPECT images most likely was larger in the MULTI setting compared to the MONO setting (due to camera-specific variability of spatial resolution in the MULTI setting). The primary hypothesis put to test was that the diagnostic performance of the SUSI relative to the SBR is better in the MULTI setting compared to the MONO setting (due to its stability with respect to varying spatial resolution in the SPECT images), particularly when the SUSI is computed after stereotactical normalization (in order to eliminate inter-subject variability of striatum size) rather than in raw images in patient space [7]. The secondary hypothesis put to test was that the variability of the count density in the reference region impacts the diagnostic performance of both SUSI and SBR, and that the effect is larger for the SUSI than for the SBR.

Fully automatic affine transformation of FP-CIT SPECT from patient space into the anatomical MNI template space or vice versa worked properly in all cases according to visual inspection. Reliable spatial transformation is required for automatic semi-quantitative analysis with standard ROIs predefined in template space.

SUSI and SBR showed better classification performance with the putamen as region of interest compared to the whole striatum. The difference was highly significant for both SUSI and SBR (e.g., multi-site/multi-camera setting: putaminal SUSI = 0.993, striatal SUSI = 0.978, p < 0.001, Table 1). The fraction of misclassified cases was almost twice as large for the SUSI of the whole striatum compared to the putaminal SUSI (6.7% versus 3.6% in the MULTI setting, Table 1). Thus, the SUSI should be used with the putamen as striatal region of interest. This might appear in conflict with the rationale of the SUSI at first sight, because there is some loss of putaminal counts and some contamination by counts from the caudate at the boundary between putamen and caudate ROI. This probably explains that all previous studies used the SUSI with the whole striatum as region of interest [7, 15, 16, 18, 19]. Furthermore, the putaminal SUSI depends on the definition of the putamen ROI, in contrast to the striatal SUSI that is rather independent of the striatum ROI as long as it is large enough [15]. However, the SBR has the same limitations: the SBR of the (whole) putamen depends on the putamen ROI, and it is affected by spill-out and spill-in at the boundary between putamen and caudate ROI. These limitations, therefore, should not prevent use of the putaminal SUSI, considering the relevant improvement in diagnostic accuracy it provides compared to the conventional SUSI of the whole striatum. The large putamen ROI used for SUSI analysis in the present study excluded part of the anterior putamen in order to reduce contamination of the putaminal SUSI by counts from the caudate (Fig. 1c). As a result, the contribution of the posterior putamen was pronounced in the putaminal SUSI. This probably contributed to the excellent performance of the putaminal SUSI in the multi-site/multi-camera setting.

In the MONO setting, putaminal SUSI computed in patient space performed slightly worse than the SUSI computed in MNI space (AUC = 0.961 versus 0.969, p = 0.129, total accuracy = 90.2% versus 93.4%, Table 1). Putamen volume was smaller in the two false positive cases compared to the 36 true negative cases based on putaminal SUSI in patient space; it was larger in the ten false negative cases compared to the 74 true positive cases (Fig. 4). This is in line with the fact that the putaminal SUSI increases with increasing putamen size so that the SUSI tends to overestimate putaminal DAT concentration in patients with large putamen and to underestimate putaminal DAT density in patients with small putamen. This can be avoided by scaling the putaminal SUSI to individual putamen size estimated from individual high resolution MRI which, however, is not always available in clinical routine. Alternatively, inter-subject variability of putamen size can be reduced by concentration-preserving stereotactical normalization, that is, stereotactical normalization without modulation to account for the amount of local expansion/contraction that typically is used in MRI-based morphometry to guarantee that regional brain volume is the same in anatomical standard space as in the original image in patient space. Based on the findings of this study, we recommend stereotactical normalization prior to SUSI analysis, although the effects of inter-subject variability of putamen size on diagnostic performance of the SUSI computed in patient space did not reach the level of statistical significance (possibly due to the rather small number of incorrectly classified cases).

Concerning the primary hypothesis put to test in this study (the classification performance of the SUSI relative to the SBR is better in the MULTI setting than in the MONO setting), the putaminal SUSI (computed in MNI space) performed slightly worse than the putaminal SBR in the MONO setting (AUC = 0.969 versus 0.976) and slightly better in the MULTI setting (AUC = 0.993 versus 0.991). This is in line with the primary hypothesis, although neither the difference in the MONO setting nor the difference in the MULTI setting reached statistical significance (p = 0.259 and 0.207, respectively). However, direct statistical comparison of the AUC difference between the SUSI and the SBR between the MULTI setting and the MONO setting showed a trend toward larger AUC difference in the MULTI setting compared to the MONO setting (p = 0.073, Fig. 6), supporting the primary hypothesis. Thus, we recommend the use of the SUSI in multi-site settings and in mono-site settings with more than one SPECT camera.

The fact that the SUSI did not outperform the SBR more clearly in the MULTI setting might be explained by two factors. First, very successful harmonization of SPECT image quality (including spatial resolution) in the PPMI by careful adaption of the acquisition protocol for each single camera and central image reconstruction at a core imaging center. Second, the relative IQR of the voxel intensity in the reference region was significantly larger in cases that were incorrectly classified by the putaminal SUSI compared to those that were correctly classified. The same was observed for the putaminal SBR. This demonstrates that the diagnostic performance of both, SUSI and SBR, is affected by uncertainty of the estimate of non-displaceable striatal FP-CIT count density by the count density in the reference region. The difference of the relative IQR between cases that were incorrectly classified by the putaminal SUSI and those that were correctly classified by the putaminal SUSI was larger in cases correctly classified by the putamen SBR compared to those incorrectly classified by the putamen SBR (Fig. 5). This is in line with the hypothesis that the impact of uncertainty in the estimate of non-displaceable count density is larger for the SUSI than for the SBR. However, the difference did not reach statistical significance. The small number of incorrectly classified cases might have contributed to the lack of significance.

Goethals and co-workers compared the SUSI computed in patient space and SBR computed in template space for detection of nigrostriatal degeneration in a mono-camera setting very similar to the mono-camera setting in the present study [16]. The AUC under the ROC curve for identification of neurodegenerative PS was highest for the SUSI of the whole striatum (minimum over left and right hemisphere, AUC = 0.859, 95% CI 0.766–0.952). The highest AUC achieved among several variants of the SBR was for the minimum over bilateral caudate and bilateral putamen SBR (AUC = 0.830, 95% CI 0.727–0.932). The minimum of putaminal SBR over both hemispheres was not considered in this study. The different ranking of SUSI and SBR in the study by Goethals and co-workers (striatal SUSI better than subregional SBR) compared to the present study (putaminal SBR better than striatal SUSI) most likely is due to methodological differences in the computation of the SBR.

The following limitations of the present study should be noted. First, the classification performance of SUSI and SBR in FP-CIT SPECT not only depends on the setting (multi-site/multi-camera versus mono-site/mono-camera). Classification performance of both semi-quantitative measures also depends on the subjects referred to FP-CIT SPECT.

The PPMI PD sample differs from the group of patients with neurodegenerative PS in the clinical mono-site/mono-camera sample in several respects, particularly by inclusion of patients with atypical neurodegenerative PS (MSA, PSP, CBD) in the clinical mono-site/mono-camera sample and the exclusion of subjects without evidence of dopaminergic deficit (SWEDD) from the PPMI PD sample [25]. There is also a relevant difference between the PPMI group of healthy control subjects and the non-neurodegenerative subgroup of the clinical mono-site/mono-camera sample: the latter included etiologies (e.g., vascular parkinsonism) that are associated with variable reduction of striatal DAT availability in a considerable fraction of patients. The differences between the PPMI multi-site/multi-camera research sample and the clinical mono-site/mono-camera sample most likely explain the fact that all tested semi-quantitative measures performed better in the PPMI sample than in the clinical sample despite the additional variability due to camera-specific effects in the PPMI sample.

Second, the excellent performance of both SUSI and SBR in the PPMI sample limits the power to detect performance differences (ceiling effect). Good diagnostic performance of FP-CIT SPECT independent of the analysis method in general makes it difficult to demonstrate differences in the diagnostic performance between methods and to reliably identify factors that might affect performance [2].

Third, classification performance of semi-quantitative measures also depends on the SPECT cameras, the acquisition protocol (including radius of rotation and delay of acquisition after tracer administration), the reconstruction algorithm, and, in the multi-site/multi-camera setting, the extent of harmonization of acquisition protocol and reconstruction algorithm. These factors not only affect the absolute classification performance of both SUSI and SBR, they also might affect the performance difference between SUSI and SBR. This may affect the generalizability of the results of the present study. In particular, it is to be expected that the SUSI more clearly outperforms the SBR in multi-site/multi-camera settings with lower degree of harmonization than in the PPMI.

The specific uptake size index (SUSI) of striatal FP-CIT uptake is particularly appropriate for semi-quantitative analysis in multi-site/multi-camera settings in which camera-specific effects increase inter-subject variability of spatial resolution. SPECT images should be stereotactically normalized prior to SUSI analysis in order to reduce the impact of inter-subject variability of striatum size. The SUSI of the putamen provides better diagnostic performance than the SUSI of the whole striatum. In mono-camera settings without camera-specific inter-subject variability of spatial resolution, the specific binding ratio (SBR) provides similar performance as the SUSI. Errors of the estimate of non-displaceable count density can cause misclassification by both, SUSI and SBR, particularly in borderline cases. We recommend to check for high uncertainty associated with high variability of voxel intensity in the reference ROI in addition to verification of proper placement of the ROI.