Comparison of low-contrast detectability between uniform and anatomically realistic phantoms—influences on CT image quality assessment

Neck-shaped phantoms with uniform and anatomical texture and hypodense lesions of 10 mm diameter and 4 to 38 HU contrast were imaged with two dose levels. Images were reconstructed with filtered back projection (FBP) and adaptive iterative dose reduction 3D (AIDR 3D). Lesion detectability was assessed by seven radiologists and compared between background types, dose levels, and reconstruction methods.

Two phantom types, which were previously introduced for low-contrast detectability experiments, were used for this study: a uniform type consisting of polymethyl methacrylate with the shape of a patient’s neck and a 3D printed, anatomically realistic type of identical shape [10, 11]. All phantoms had the same dimension of 15.4 cm (length) × 10.6 cm (width). Six different versions of the uniform phantom type and five versions of the anatomical type were used. One version of each type did not contain any lesion. The other versions each contained a single low-contrast lesion of 10 mm diameter in the left parapharyngeal space. The lesion was in the same position in all phantoms. Lesion contrasts were 4, 9, 18, 30, and 38 HU (uniform phantom) and 10, 18, 30, and 38 HU (anatomical phantom). The lesion contrasts were validated in previous studies by HU measurement in 2700 images acquired with six different dose levels (uniform phantom) and in 2808 images acquired with twenty-seven different dose levels (anatomical phantom) [10, 11]. In these validation experiments, lesion contrast was calculated as HU difference between regions of interest (ROIs) of 0.5 cm² inside the lesions and six ROIs of 4.9 cm² (uniform phantom) and one ROI of 3 cm² (anatomical phantom) surrounding the lesions. The lesions were rod-shaped, and the phantoms were constructed in such a way that multiple adjacent images displaying the same lesion and phantom background could be extracted per CT acquisition. Figure 1 shows a CT image of each phantom type and indicates the lesion position. Details on phantom construction, acquisitions, and measurements performed for evaluating lesion contrasts can be found elsewhere [10, 11].

CT images of the uniform phantom originated from a previous study [11] and were acquired on a Canon Aquilion Prime CT scanner (Canon Medical Systems). CT images of the anatomical phantoms were acquired on the same system using identical parameters: helical mode, tube voltage of 120 kVp, fixed collimation of 80 × 0.5 mm, rotation time of 0.5 s, 0.813 pitch, and a 280 mm diameter field of view. A 30- and 120-mA tube currents were used, corresponding to CTDIvol values of 1.4 and 5.6 mGy. Five acquisitions were performed per tube current. Images were reconstructed with 0.5-mm slice thickness and a soft tissue kernel (FC08) using FBP and AIDR 3D. For the subsequent detectability experiment, four CT images were extracted per acquisition of the lesion-bearing phantoms with 9, 18, 30, and 38 HU lesion contrast (uniform phantom) and 10, 18, 30, and 38 HU lesion contrast (anatomical phantom). Thus, a total of 640 lesion-bearing images were extracted (2 phantom types × 4 lesion contrasts × 2 tube currents × 2 reconstruction methods × 5 repeated acquisitions × 4 images).

Each lesion-bearing image was paired with three non-lesion-bearing images of the corresponding phantom type (uniform or anatomical), which were acquired and reconstructed with identical parameters. Each of the resulting 640 image quartets was presented to seven radiologists in a 4-alternative forced choice (4-AFC) experiment. Readers were asked to select the image containing a lesion and to indicate their confidence using a five-step scale ranging from 1 = not confident to 5 = confident. Readings were performed using in-house developed software on diagnostic screens (Eizo RadiForce RX250, Eizo Corporation). In addition to the reading results obtained here, results from a previous reading experiment performed with images of the uniform phantom and 4 HU lesion contrast were included in the analysis [11]. Image acquisitions and readings in that previous study were performed in the same way as in the present study (i.e., the same CT system, acquisition and reconstruction parameters, 4-AFC methodology, and readers were involved). The results were included to complement the current data used to analyze dose and image reconstruction effects in uniform phantoms.

The standard deviation (SD) of pixel values and the noise power spectrum (NPS) were measured using 200 images per phantom type, tube current, and reconstruction method. In each image, a square ROI of 32 × 32 pixels (17.5 × 17.5 mm) was placed in the same location in the parapharyngeal space adjacent to the lesion. The ROI position was selected to include a fairly homogeneous area of the anatomical phantoms. A larger ROI size or multiple ROIs would have led to the inclusion of largely inhomogeneous areas of the anatomical phantoms such as the mandibula or vascular structures. Also, ROI placement inside the lesions was not possible because the lesion size was too small to perform NPS measurement. The 2D NPS was calculated using the following Eq. (1):

where b_x and b_y are the pixel sizes (0.546 mm) in the x- and y-direction, respectively, and L_x and L_y are the ROI lengths (17.5 mm) in the x- and y-direction, respectively. FFT_2D is the 2D fast Fourier transform. ROI_Background is the background noise in ROI(x,y) measured using second-order polynomial fitting by minimizing the residual sum of squares [12]. N_ROI is the number of ROIs (200) per phantom type, tube current, and image reconstruction that was used to average the squared amplitude of the fast Fourier transform.

Figure 2 shows a comparison of detection accuracy and reader confidence between uniform and anatomical phantoms. Averaged results across all readers, dose levels, and reconstruction methods are presented. Phantom background texture significantly affected detectability at all lesion contrasts. Readings of images of the uniform phantom yielded high detection accuracy already at relatively low lesion contrast of 9 HU (89.5%, 95% CI: 82.9 to 96%), which improved to 99.6% (95% CI: 99.1 to 100.2%) at 18 HU and perfect detection at 30 and 38 HU contrast. Conversely, readings of images of the anatomical phantoms yielded low detection accuracy at 10 HU (52.9%, 95% CI: 44.1 to 61.6%) and 18 HU (55.5%, 95% CI: 47.2 to 63.9%), which improved to 91.1% (95% CI: 85.8 to 96.3%) at 30 HU and 97.5% (95% CI: 95.8 to 99.2%) at 38 HU contrast. Clear differences between uniform and anatomical images were also observed for reader confidence (Fig. 2, suppl. table 1). Similar detection accuracies for the two phantom types were achieved when comparing 9 HU lesion contrast in the uniform phantom and 30 HU contrast in the anatomical phantom (89.5% vs. 91.1%, p = 0.587). Readings of images of the uniform phantom with 4 HU lesion contrast originating from a previous study yielded an average detection accuracy of 62.9% across all readers, dose levels, and reconstruction methods (95% CI: 55.8 to 69.9%) [11].

Figure 3 provides a series of uniform and anatomical phantom images acquired at 1.4 and 5.6 mGy and reconstructed with FBP and AIDR 3D. The figure includes uniform images with 9 HU lesion contrast and anatomical images with 30 HU lesion contrast, which yielded similar overall detection accuracies. Detailed detection accuracy results per dose, reconstruction method, and lesion contrast are presented in Tables 1 and 2.

At 18 HU lesion contrast and above, readings of images of the uniform phantom reached 100% detection accuracy and could therefore not be used for the analysis of dose and image reconstruction effects. Results for 4 and 9 HU lesion contrast are summarized in Table 3 and presented in Fig. 4. Dose reduction from 5.6 to 1.4 mGy decreased lesion detectability in uniform images that were reconstructed with FBP (83.6% vs. 55.4%, p < 0.001). AIDR 3D maintained detectability (85% vs. 80.7%, p = 0.375) and was superior to FBP at 1.4 mGy (p < 0.001). Analysis of the uniform phantom thus showed strong dose effects on FBP-reconstructed images and superiority of AIDR 3D at 1.4 mGy.

Figure 5 shows the effects of dose and image reconstruction on detection in anatomical phantoms. Numerical results are provided in Table 4. In contrast to the uniform phantom, AIDR 3D did not show clear advantages over FBP at any dose level (73% vs. 68.2%, p = 0.144 at 1.4 mGy and 81.1% vs. 74.6%, p = 0.111 at 5.6 mGy). Moreover, the strong effects of dose reduction on FBP-reconstructed images were not confirmed. Instead, dose reduction moderately affected detectability in a similar manner for both reconstruction methods (p = 0.027 for FBP and p = 0.018 for AIDR 3D). Analysis of the anatomical phantoms thus neither confirmed the superiority of AIDR 3D nor dose effects observed in the uniform phantom.

Figure 6 shows noise and NPS results per phantom type, dose, and image reconstruction. Numerical results are summarized in Table 5. As expected, low-dose (1.4 mGy), FBP-reconstructed images had the highest noise level in both phantom types (p < 0.001). A dose increase to 5.6 mGy reduced the noise (p < 0.001) except for AIDR 3D-reconstructed images of the anatomical phantoms, which had almost identical noise values at low and high doses (p = 0.26). Remarkably, noise was lower in low-dose AIDR 3D-reconstructed images than in high-dose FBP-reconstructed images of the anatomical, but not of the uniform, phantom, indicating that AIDR 3D was more effective in denoising anatomical images. The NPS curves of the uniform phantom showed a shift towards lower spatial frequencies in low-dose AIDR 3D-reconstructed images with a peak NPS at 0.23 mm⁻¹ and a decrease at lower spatial frequencies. Conversely, all images of the anatomical phantoms yielded peak NPS values at a low spatial frequency of 0.12 mm⁻¹ regardless of dose and image reconstruction. FBP-reconstructed images acquired at 1.4 mGy had a second NPS peak at a spatial frequency of 0.23 mm⁻¹, which flattened with FBP reconstruction at 5.6 mGy and in all images reconstructed with AIDR 3D.