Imaging quality of an artificial intelligence denoising algorithm: validation in 68Ga PSMA-11 PET for patients with biochemical recurrence of prostate cancer
verfasst von:
Charles Margail, Charles Merlin, Tommy Billoux, Maxence Wallaert, Hosameldin Otman, Nicolas Sas, Ioana Molnar, Florent Guillemin, Louis Boyer, Laurent Guy, Marion Tempier, Sophie Levesque, Alban Revy, Florent Cachin, Marion Chanchou
68 Ga-PSMA PET is the leading prostate cancer imaging technique, but the image quality remains noisy and could be further improved using an artificial intelligence-based denoising algorithm. To address this issue, we analyzed the overall quality of reprocessed images compared to standard reconstructions. We also analyzed the diagnostic performances of the different sequences and the impact of the algorithm on lesion intensity and background measures.
Methods
We retrospectively included 30 patients with biochemical recurrence of prostate cancer who had undergone 68 Ga-PSMA-11 PET-CT. We simulated images produced using only a quarter, half, three-quarters, or all of the acquired data material reprocessed using the SubtlePET® denoising algorithm. Three physicians with different levels of experience blindly analyzed every sequence and then used a 5-level Likert scale to assess the series. The binary criterion of lesion detectability was compared between series. We also compared lesion SUV, background uptake, and diagnostic performances of the series (sensitivity, specificity, accuracy).
Results
VPFX-derived series were classified differently but better than standard reconstructions (p < 0.001) using half the data. Q.Clear series were not classified differently using half the signal. Some series were noisy but had no significant effect on lesion detectability (p > 0.05). The SubtlePET® algorithm significantly decreased lesion SUV (p < 0.005) and increased liver background (p < 0.005) and had no substantial effect on the diagnostic performance of each reader.
Conclusion
We show that the SubtlePET® can be used for 68 Ga-PSMA scans using half the signal with similar image quality to Q.Clear series and superior quality to VPFX series. However, it significantly modifies quantitative measurements and should not be used for comparative examinations if standard algorithm is applied during follow-up.
Hinweise
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Abkürzungen
ALARA
As low as reasonably achievable
CI
Confidence interval
CNR
Contrast-to-noise ratio
18FDG
18Fluoro-Desoxy-Glucose
MDT
Metastasis-directed therapy
NRMSE
Normalized root-mean-square error
OSEM
Ordered subset expectation maximization
PSA
Prostate-specific antigen
PSF
Point spread function
PSMA
Prostate-specific membrane antigen
PSNR
Peak signal-to-noise ratio
SBR
Signal-to-background ratio
SNR
Signal-to-noise ratio
SSIM
Structural similarity index
SUV
Standardized uptake value
PET
Positron emission tomography
TOF
Time of flight
TRT
Targeted radiotherapy
Introduction
Artificial intelligence (AI) is finding its way into medicine through medical imaging. AI algorithms can be used at two different points in the processing of cross-sectional images, i.e., either during or after iterative image reconstruction [1]. The SubtlePET® algorithm (SubtlePET, Incepto Medical) is a deep learning-based post-reconstruction image denoising algorithm [2] used in PET imaging. The algorithm is a supervised convolutional neuronal network (CNN) based on a 2.5D encoder–decoder with a U-net shape. It has not been trained in our center [3] but has been trained on millions of images through dataset augmentation [4]. The version 1.3 of the algorithm we used in this study was trained only on FDG and Amyloid PET (PET/CT and PET/MRI). No specific hardware is needed to use this algorithm, and the time between the end of examination and the reconstruction is shorter than 10 min. SubtlePET® is a denoising algorithm that focuses on making the images smoother, and not a deblurring algorithm that would sharpen the lesion contours. This algorithm is already validated by the Food and Drug Administration for tracers marked with Fluorine-18 (18FDG [3] and Amyloid PET [5]) and can divide acquisition time by four without losing image quality [3]. It has not been investigated or trained for different isotope such as the gallium-68 encountered in prostate oncology imaging.
Prostate cancer is the most common cancer in men, and the third leading cause of cancer-related mortality [6]. It has a very high relapse rate of about 50% [7], which is explained by a lack of sensitivity of conventional imaging techniques [8]. Prostate cancer does not show particularly high glucose uptake, and this has prompted the development of dedicated tracers, i.e., prostate-specific membrane antigen (PSMA) ligands, which are typically labeled with gallium-68 in current practice. PSMA is predominantly expressed by prostate cells, and its binding intensity increases with aggressiveness of the cancer [9]. PET with PSMA ligands has shown high diagnostic performances in both initial staging [10] and biological recurrence [11‐13]. Furthermore, gallium-68 has to be produced by a generator, which significantly limits the use of 68 Ga-PSMA in current practice [14].
Anzeige
The wider context of growing public health issues and the development of targeted radiotherapy (TRT) are driving a strong increase in demand for examinations. Reducing acquisition time while maintaining optimal or sufficient image quality would help to overcome the challenge of meeting the increasing needs without penalizing performance. AI, and possibly the SubtlePET® algorithm, could provide solutions by maintaining a satisfactory image quality despite a reduced count statistic. This study applies transfer learning [1], as the SubtlePET® algorithm has not yet been tested with gallium-68-labeled tracers.
This study set out to assess the influence of PET images obtained using the SubtlePET® algorithm on 68 Ga-PSMA PET image quality under four levels of information loss compared to images obtained by standard reconstruction. The secondary objective was to analyze the diagnostic performances of the different PET series (post-processed or not by SubtlePET®) and the influence of SubtlePET® on lesion intensity and background measures.
Methods
Patients
We retrospectively included 30 consecutive patients aged over 18 years old presenting with biochemical recurrence of prostate cancer at the Jean Perrin Cancer Center (Clermont-Ferrand, France) for 68 Ga-PSMA PET from July 17 to October 20, 2020. As per French regulations, every patient had already undergone 18F-choline PET/CT that were negative or inconclusive [15]. The data collected was age, weight, Gleason score, ISUP score, prostate cancer treatment, initial prostate-specific antigen (PSA) level, and last known PSA level before the PET scan.
Every patient received an information letter validated by the Jean Perrin Cancer Center data protection department and was free to refuse use of their data throughout the study.
Anzeige
Image acquisition
The tracer used is 68Ga PSMA-11, also called HBED-CC, Glu-urea-Lys(Ahx)-HBED-CC, or PSMA-HBED-CC [8]. Image acquisition usually begins 60 min after IV injection of 68 Ga-PSMA-11 with an average activity of 2.0 MBq/kg [16] (118–453 MBq) with 4 min per scan step from the vertex to the upper third of the femur. Excluding one patient, the range of activity is thinner (118–224 MBq). A low-dose CT scan was performed for attenuation correction and localization.
Fifteen scans were performed on a PMT-based Discovery 710 Optima 660® scanner, and 15 were performed on a SiPM-based Discovery MIDR® scanner (GE Health, Healthcare, Milwaukee, WI). No cross-validation was made across the two scanners. Both machines have a 700-mm field of view, 256 × 256 matrix, and 2.7 × 2.7 × 3.27 mm3 voxel volumes. The CT parameters of the SiPM-based PET were 124 kV of mean tube voltage (range 120–140 kV), 76.2 mA of mean tube current (range 58–159 mA), a pitch of 0,98, a noise index of 28,2, and a percentage of iterative reconstruction of 40%. The CT parameters of the PMT-based PET were 128 kV of mean tube voltage (range 120–140 kV), 76.2 mA of mean tube current (range 55–123 mA), a pitch of 0.98, a noise index of 27.3, and a percentage of iterative reconstruction of 40%. No cross-validation was done across the two scanners. For each scan, two standard images were reconstructed, the first with VPFX (ordered subset expectation maximization (OSEM) + time of flight (TOF)) and the second with Q.Clear (OSEM + TOF + point spread function (PSF)) [17]. We named these standard reconstructions VP4 and QC4, respectively. The VPFX series were performed with two iterations for 24 subsets.
The algorithm uses a residual learning approach optimized for quantitative (L1 norm) as well as structural similarity. It learns to separate and suppress the noise components while preserving and enhancing the structural components. The networks were trained with paired low- and high-count PET series coming from a wide range of clinical indications and patient BMIs and from a large variety of PET/CT and PET/MR devices (10 General Electric, 5 Siemens, and 2 Philips models). The training data included millions of paired image patches derived from hundreds of patient scans with multi-slice PET data and data augmentation. The list mode allows retrospective reconstruction of images by artificially reducing the count statistic by taking only the data acquired in a given time-interval. For each patient, we used on 1.3.0 of the SubtlePET® post-processing denoising algorithm to create 4 image series equivalent to one-minute, two-minute, three-minute and four-minute steps. We named these series SVP1, SVP2, SVP3, and SVP4 for the VPFX-derived series and SQC1, SQC2, SQC3, and SQC4 for the Q.Clear-derived series, respectively. These series were anonymized by a medical physicist blind to the image interpreting team.
Image analysis
Three nuclear physicians blindly interpreted every series using the PET VCAR module bundled with the General Electrics’ ADW® software. These 3 readers had different degrees of experience: One was a PSMA referent (16-year experience), one was a senior physician (4-year experience), and one was a resident nuclear physician (1-year experience). All three readers had experience reading PSMA PET scans in their daily practice. The VP4 and QC4 series analyzed by the most experienced reader served as the gold standard benchmark for assessing the other series.
Data for lesions suspected of malignancy were collected by classifying the lesions according to anatomical location, i.e., prostate bed, pelvic lymph nodes, secondary bone lesions, or secondary extra-osseous. In the event of lymph node or bone involvement, the precise locations were recorded to facilitate comparison between series. Lesions were classified into 5 levels [18]: negative, equivocal negative, equivocal, equivocal positive, or positive. Lesions labeled as equivocal positive or positive were considered significant.
Image quality
We evaluated the quality of each series based on 3 criteria: overall image quality, interpretability, and visualization or not of suspected lesions. For overall image quality, we used a 5-point Likert scale [5, 19] for noise level, contrast, and signal-to-noise ratio estimated visually (1 = uninterpretable, 2 = bad, 3 = correct, 4 = good, 5 = excellent). We retained levels 3, 4, and 5 as usable in daily practice, thus classifying the series as interpretable.
We also evaluated image quality using two binary indexes: series interpretability and lesion detectability.
Quantitative analysis
Quantitative analyses focused on SUVmean and SUVmax measurements, where SUV is ‘standardized uptake value.’ SUVmean measures the mean activity in a volume, whereas SUVmax retains only the value of the hottest pixel. To evaluate the influence of the algorithm on SUV measurements, we recorded the SUVmax of every lesion found on each examination and background measurement performed.
Anzeige
SUVmax was measured by plotting regions of interest on each lesion by semi-automatic method. For homogenization purposes, the analyses for each specific anatomical location only considered the most intense lesions of each patient. SubtlePET®-induced bias in lesion SUVmax measurements was evaluated using the following formula: (study series SUVmax—reference series SUVmax)/reference series SUVmax [1].
Background measurements were performed by the junior reader using a 2-cm-diameter sphere in the right liver and right gluteal muscle and a 1-cm-diameter circle in the aortic arch [20, 21]. We chose to take the gluteal region as background reference as most lesions in this specific application of prostate pathology are located in the pelvic area. The SUVmean and SUVmax of the background uptake in these regions of interest were measured. Signal-to-background ratio (SBR) was defined by the following formula: lesion SUVmax/background uptake SUVmean [22]. We also performed subgroup analysis based on PSA level, weight, and camera type.
Diagnostic performance of PET series using the SubtlePET® denoising algorithm
To evaluate the diagnostic performance of the SubtlePET® series, we performed a series-by-series analysis per reader. Our gold standard benchmarks were our most experienced reader and the routine-process reconstruction series, i.e., not reprocessed by the denoising algorithm.
True positives were defined as lesions classified as positive by the most experienced reader in the usual reconstruction series (QC4 and VP4) as well as by all readers in all other series. True negatives were defined as lesions classified as negative by the most experienced reader in the usual series and negative by all readers in all other series. False positives were defined as lesions classified as negative or not found by the most experienced reader in the usual series but classified as positive in the other series. False negatives were defined as lesions classified as positive by the most experienced reader in the usual reconstruction series but classified as negative in the other series.
We were thus able to calculate the sensitivity, specificity, and accuracy of each of the series processed by SubtlePET® for each reader.
Statistical analysis
To lend clarity to the statistical analysis, we pooled lesions classified as equivocal positive into the positive group and lesions classified as equivocal and equivocal negative into the negative group.
Subgroup analyses were stratified by patient weight, PSA values, and the two different cameras.
Anzeige
Analyses involving image quality (categorical variables) were compared using Cochran’s Q test for differences between all readers and McNemar’s test for pairwise comparison.
Analyses involving quantitative parameters (continuous variables) were compared by the Student’s paired-samples t test and the Wilcoxon signed rank test.
A p value adjustment was performed́ to account for the multiplicity of tests. Statistical analyses were performed with R software version 4.1.0 (R-Project, GNU GPL, http://cran.r-project.org/).
Results
Population
The characteristics of the patient population are summarized in Tables 1 and 2. Our patients had a mean age of 67.6 years, a majority of Gleason 7 prostate cancer (80%), and a median PSA level before PET of 0.682 ng/mL (min 0.23 ng/mL–max 20 ng/mL). Most patients had been initially treated with radical prostatectomy (89.6%). One patient had prostate cancer at initial diagnosis but was not biochemically recurrent. Average injected activity was 2.09 MBq/kg (min 1.51 MBq/kg–max 5.03 MBq/kg) in accordance with European recommendations [16]. One of the patients was injected with higher-than-normal activity, but there was no recorded evidence of toxicity.
Table 1
Characteristics of the patient population
Median
Mean
Min
Max
Age years
67
67.56
55
81
Weight kg
84
83.4
63
116
Injected activity MBq
164
174.1
118
453
Activity/kg MBq/kg
1.97
2.09
1.51
5.03
Initial PSA ng/mL
11
10.7
5.7
17.46
Last PSA before PET ng/mL
0.682
2.356
0.23
20
Table 2
Characteristics of the patients at diagnosis
n
Initial Gleason score
6
1
7
24
3 + 4
12
4 + 3
12
8
3
9
1
Not available
1
Initial ISUP score
1
1
2
12
3
12
4
3
5
1
Not available
1
Initial treatment
Prostatectomy
26
Radiotherapy
3
None
1
Anzeige
Number and localization of lesions
A total of 1919 lesions were found, i.e., an average of 2.13 lesions per series, of which 1329 (i.e., 69.25%) were clinically significant. Suspicious lesions were found in 25 out of 30 patients. The referent reader found 658 lesions, of which 378 were significant. The senior reader found 574 lesions, of which 425 were significant. The junior reader found 687 lesions, of which 526 were significant. The total number of lesions per series is reported in Table 3. Among the significant lesions, 121 were localized in the prostatic bed, 717 in the lymph nodes, 428 in the bones, and 63 in the extra-osseous tissues. Figure 1 plots the number and location of the lesions according to PSA levels. Mean size was 8.6 mm for positive lesions and 5.9 mm for equivocal-positive lesions. Figure 2 plots mean size of lesions compared to mean SUVmax. Readers tended (p > 0.062) to classify lesions more significantly on the gold standard series.
Table 3
Number and classification of lesions stratified by series and reader
QC4
SQC1
SQC2
SQC3
SQC4
VP4
SVP1
SVP2
SVP3
SVP4
Total
REFERENT
67
62
62
64
67
71
65
63
69
68
658
−
9
11
13
11
9
10
9
12
8
8
100
Equivocal −
11
9
5
9
13
13
12
6
16
13
107
Equivocal
6
8
7
7
8
6
7
6
8
10
73
Equivocal +
9
4
6
6
6
11
6
7
6
6
67
+
32
30
31
31
31
31
31
32
31
31
311
SENIOR
57
58
57
58
57
57
59
57
57
57
574
−
1
4
2
2
1
1
4
2
1
1
19
Equivocal −
12
10
11
15
15
10
12
10
12
13
120
Equivocal
1
1
1
1
1
1
1
1
1
1
10
Equivocal +
7
7
7
5
5
6
5
7
8
6
63
+
36
36
36
35
35
39
37
37
35
36
362
JUNIOR
68
68
69
68
68
72
69
70
68
67
687
−
6
7
7
7
7
5
8
8
6
6
67
Equivocal −
5
4
10
3
6
2
6
4
3
1
44
Equivocal
4
2
4
4
5
3
6
6
7
9
50
Equivocal +
13
14
11
14
13
12
11
12
12
12
124
+
40
41
37
40
37
50
38
40
40
39
402
Total
192
188
188
190
192
200
193
190
194
192
1919
Fig. 1
Location of lesions according to PSA level for the gold standard series interpreted by the referent reader. The column with patients whose PSA level was higher than 5 has been omitted, because after excluding patients at initial staging, there was only one patient with lesions left in this subgroup, making it insignificant
Fig. 2
Mean size and mean SUVmax of all lesions
×
×
Image quality analysis
Overall image quality and interpretability
VPFX
We a found a significant difference (p < 0.001) in overall quality of the VP4, SVP1, SVP2, SVP3, and SVP4 series, with, respectively, 65.6%, 36.6%, 85%, 91.1%, and 100% of the series classified as correct or over. The junior reader had more misclassified (p < 0.05) VPFX, SVP2 and SVP3 series compared to the other readers. The senior reader classified the SVP1 series better compared to the other readers (p < 0.001).
Almost all (99.6%) SVP2, SVP3, and SVP4 series were classified as interpretable by all readers, and 65.6% of the SVP1 series and 83.3% of the VP4 series were classified as interpretable by all readers. Thus, in the overall comparison between series, the series processed by SVP2, SVP3, and SVP4 were significantly (p = 0.001) better classified than the gold standard series (VPFX).
Q.Clear
The only statistically significant difference (p < 0.001) found was between the QC4 and SQC1 series, with 91.1% and 42.2% of the series classified as correct or better, respectively. There was no statistically significant difference in quality between the QC4 series, and the SQC2, SQC3, and SQC4 series, of which 93.3%, 93.3%, and 94.4%, respectively, were rated correct or better.
100% of the QC4, SQC4, SQC3, and SQC2 series were interpretable by all readers versus 64.4% of the SQC1 series.
Lesion detectability
Concerning lesion visualization, there was a statistically significant difference between readers (p < 0.001) but not in the between-series analysis. Note that even though some series were classified as uninterpretable, lesion detection was still possible. Figures 3 and 4 give illustrative examples of the image quality achieved (Fig. 4).
Fig. 3
Patient 1: 112 kg, 1.83 MBq/kg, PSA before PET 20 ng/mL. 3D MIP images on the first lines and axial section on the 2nd lines. The arrow shows a prostatic bed lesion. LUT range 0–10 g/mL
Fig. 4
Patient 2. 63 kg, 3.03 MBq/kg, PSA before PET 2 ng/mL. 3D MIP images on the first lines and axial section on the 2nd lines. The arrow shows a retroperitoneal lymph node. LUT range 0–10 g/mL
×
×
Analysis of quantitative parameters
Influence of the denoising algorithm on lesion SUVmax
The mean SUVmax value was 8.91 for QC4 and 7.7 for VP4. There was a statistically significant decrease (p < 0.05) in measured SUVmax values of lesions in all the series reconstructed with SubtlePET® except for SQC1. The less data acquired in the series, the closer the SUVmax values were to the reference value. The results are reported in Table 4. Further analysis of lesion SUVmax values stratified by lesion size revealed that our cohort only had small lesions, which are particularly sensitive to the partial volume effect (see Fig. 2).
Table 4
Differences in SUVmax of lesions between the series treated with SubtlePET® and the standard series
Influence of the denoising algorithm on background uptake SUV (Appendix 1)
For VPFX-derived series, mean SUVmean was 1.34 for the aorta, 5.03 for the liver, and 0.39 for the right gluteal muscle. Vascular background was significantly higher (p < 0.05) for all series using only half the signal. Liver background was significantly higher (p = 0.001) for all series treated with SubtlePET®. Gluteal background was significantly lower (p < 0.05) for all series processed with SubtlePET®. The decrease was only a modest 0.04 at most, which is not clinically significant but could still facilitate the detection of low-intensity lesions, especially in this indication.
For Q.Clear-derived series, mean SUVmean was 1.09 for the aorta, 5.07 for the liver, and 0.39 for the right gluteal muscle. Liver and gluteal background were significantly different (p < 0.05), with a mean increase of 0.23 in liver values and a mean decrease of 0.03 in gluteal values. Other quantitative parameters were not significant.
Results of background SUVmax relative to VPFX and Q.Clear series are shown in Appendix 1.
Influence of the denoising algorithm on signal-to-background ratio (Table 5)
Table 5
Comparison of the signal-to-background ratios (SBR)
Series
Mean
Difference SBR
CI 95% of difference
p adjusted
Bias (%)
Signal/Background = SUVmax lesion/SUVmean aorta
QC4
5.28
SQC1
4.87
− 0.52
(− 1.22–0.3)
0.183
− 7.8
SQC2
4.37
− 0.94
(− 1.51–0.3)
0.009
− 17.2
SQC3
4.37
− 1.11
(− 1.58–0.62)
0.001
− 17.2
SQC4
4.28
− 1.16
(− 1.64–0.74)
< 0.001
− 18.9
VP4
5.19
SVP1
4.21
− 1.08
(− 2.22–0.07)
0.054
− 18.9
SVP2
4.11
− 1.29
(− 2.09–0.58)
0.007
− 20.8
SVP3
4.02
− 1.66
(− 2.13–0.56)
0.006
− 22.5
SVP4
4.17
− 1.27
(− 2.47–0.8)
0.002
− 19.6
Signal/Background = SUVmax lesion/SUVmean liver
QC4
1.13
SQC1
1.04
− 0.14
(− 0.29–0)
0.061
− 7.9
SQC2
0.92
− 0.19
(− 0.29–0.08)
0.002
− 18.5
SQC3
0.92
− 0.22
(− 0.36–0.14)
< 0.001
− 18.5
SQC4
0.88
− 0.29
(− 0.42–0.19)
< 0.001
− 22.1
VP4
1.07
SVP1
0.89
− 0.25
(− 0.52–0.05)
0.02
− 16.8
SVP2
0.85
− 0.22
(− 0.35–0.1)
0.007
− 20.6
SVP3
0.84
− 0.25
(− 0.47–0.16)
0.004
− 21.5
SVP4
0.85
− 0.28
(− 0.44–0.2)
0.001
− 20.6
Signal/Background = SUVmax lesion/SUVmean gluteal
QC4
14.89
SQC1
15.36
0.80
(2.06–2.94)
0.456
3.2
SQC2
13.60
− 1.01
(− 2.74–1.12)
0.247
− 8.7
SQC3
13.60
− 1.37
(− 2.34–0.2)
0.42
− 8.7
SQC4
13.29
− 2.07
(− 3.22–0.85)
0.13
− 10.7
VP4
13.96
SVP1
12.53
− 1.81
(− 3.58–0.92)
0.267
− 10.2
SVP2
12.91
− 0.88
(− 2.51–0.91)
0.214
− 7.5
SVP3
12.63
− 1.66
(− 2.65–0.36)
0.084
− 9.5
SVP4
12.67
− 1.76
(− 2.47–0.8)
0.17
− 9.2
Mean SBR with the gluteal reference was 14.89 for QC4 and 13.89 for VP4. No significant difference was found for the SBR with the right gluteal region as reference. This result confirms the stability of lesion detectability even if the series were not considered visually interpretable.
Taking the hepatic background as reference, a statistically significant difference (p < 0.05) was demonstrated for all series except for SQC1. We thus showed a very modest decrease in the signal-to-background ratio (between − 0.28 and − 0.14 depending on the series).
There was a modest but significant (p < 0.05) decrease in signal-to-background ratios with the vascular region as the reference for the series computed from half the signal.
Subgroup analysis
Subgroup analysis by PSA on PET scan
In the subgroup analysis on PSA on PET, we excluded the patient at initial staging. This left only one patient in the subgroup with a PSA level above 5 ng/mL, and so we did not perform analysis on this subgroup. The results are illustrated in Fig. 5. SUVmax values increased with PSA level except for patients with a PSA level lower than 1.
Fig. 5
Mean SUVmax of lesions per series as a function of PSA
×
Subgroup analysis by weight
Despite our small number of patients, we performed a subgroup analysis by weight. The difference was only statistically significant (p < 0.05) for patients with a weight higher than median in SVP2, SVP3, SVP4, SQC2, SQC3, SQC4 and in patients lower than median SQC3, SQC4. Results were expressed with the definition of the signal-to-background ratio as (SUVmax lesion/SUVmean liver) and reported in Appendix 2. SubtlePET® tended to lower the signal-to-background ratios more often within heavier patients in the VPFX-derived series.
Subgroup analysis by camera
We performed a subgroup analysis to see the impact of the camera on the level of background in each series. There was a slight decrease in SUVmean that was only statistically significant in the gluteal region and the aortic region for VP4 and the Q.Clear-derived series on the newest camera. Results are shown in Appendix 3. Furthermore, there was no statistically significant difference with in the le level of lesion SUVmax or in SBR.
Diagnosis accuracy was statistically compared, respectively, between algorithm models and readers. Few differences were shown between images examined by the same reader. Accuracy was superior to 0.98 whatever the algorithm used by the referent. However, analysis was statistically different between senor or junior readers and referent.
×
×
Discussion
The SubtlePET® algorithm significantly improved (p < 0.001) the quality of VPFX-derived images computed from half the data (p < 0.001). For Q.Clear series, there was no statistically significant difference between series using at least half of the signal, knowing that Q.Clear already optimizes the images. Note that this analysis of image quality remains subjective. Even if some series were of insufficient quality, lesion detectability did not differ (p < 0.27) for all series. Regarding the influence of SubtlePET® on SUV measurements, the algorithm significantly lowered the lesion values (p < 0.005). The liver background measured by SUVmean increased significantly in both types of reconstruction, but this change was not clinically significant. Background SUVmax measurements followed a more contrasted pattern, with an increase in the Q.Clear-derived series but a decrease in the SVP4 series. Moreover, the SUVmax measurement of background regions was not very relevant because not including in the SBR. There was a slight decrease in the SUVmean of the aorta or gluteal on the PMT-based PET scan. Even if this decrease was light (< 0.18 g/mL), it might change the significant of small lesions when interpreting according to the PROMISE criteria [23]. The algorithm did not influence the diagnostic performance of each reader.
Strengths
We were able to establish external validation of using the algorithm, as it was not trained or supervised at our center [3]. Moreover, as the SubtlePET® algorithm is a post-reconstruction image processing tool, it is independent of machine brand or model and therefore allows flexibility. In addition, two recent French studies [22, 24] found similar results on the main objective of image quality with only half the activity injected. One of these studies [22] used the Philips technology. Our results on SBR, lesion intensity, and level of background activity were also concordant even though we used a different radiopharmaceutical tracer.
The differences in image quality achieved for the VPFX series were very encouraging. For the series derived from the Q.Clear algorithm, which has already been optimized, our study suggests an equivalent quality of images produced when using only half of the data.
Signal-to-noise ratio correlates with detectability [18, 25]. We thus find a correlation in our study between analysis and detectability which remains stable despite a decrease in image quality. This improves readability from the series using only half the signal.
Perspectives
AI holds exciting prospects for nuclear medicine. Given how image quality is shaped by the relationship between injected activity and acquisition duration [26], our results raise several prospects.
First, a decrease in activity would limit the amount of irradiation of patients and staff. With an effective dose of 0.02 mSv per MBq [16], the average irradiation of patients in our cohort was 3.5 mSv, and so halving the activity would reduce the dose to 1.7 mSv. These activities remain very low and can even be considered negligible given the age of the patients and the cancer context.
The results of the VISION [27] study on 177Lu-PSMA internal radiation therapy have triggered a paradigm shift in prostate cancer treatment. This new and very promising therapy requires an imaging examination with PSMA before treatment initiation. In this context, demand for 68 Ga-PSMA PET scans is constantly increasing. AI is a good tool to help address this increasing demand for imaging. The germanium/gallium generators are scarce, which severely limits the availability of 68 Ga-PSMA, and so a decrease in dose per patient would allow a larger population to benefit from this technique. The current generation of gallium generators can prepare doses for 2–3 patients per day before dose reduction. From a technical point of view, it is possible to synthesize the radiopharmaceutical for twice as many patients by decreasing the doses, but a cost study would be needed to evaluate the feasibility of preparing 5–6 doses per day. However, given the radioactive decay during scanning time, it would increase but not double the number of doses with only one scanner available.
On the other hand, a decrease in acquisition times would serve to minimize motion artifacts and machine occupancy times while improving patient comfort. In addition, some studies [10, 21] have shown that the best delay after injection for the best contrast between tumor and non-tumor tissues is three hours. This three-hour imaging protocol for injections is, however, more restrictive in nuclear medicine departments, whereas most other tracers require a delay of only one hour. A decrease in acquisition time could thus facilitate the implementation of such a protocol.
Finally, the therapeutic impact of PSMA PET scans in this oligometastasis population is to guide stereotactic radiotherapy [18, 23, 28]. In this context, further studies are needed to evaluate the influence of SubtlePET® on fixation volumes.
Limitations
First, the cohort included a small number of patients, making the localizations found according to PSA levels not similar to larger study [12]. Nevertheless, the increase of SUVmax in our cohort according to PSA level is consistent with the data from the literature [9].
Second, one of the biases of the study was that the junior reader only interpreted scans on Q.Clear-series reconstructions and not on VPFX-series reconstructions, as Q.Clear had been routinized in the department before his arrival. This explains why this reader qualitatively misclassified the VPFX series, which introduced a bias in the analysis of the overall quality and interpretability of the VPFX-derived series. Furthermore, PSMA-11 is not as specific as one might expect [11, 23, 29], and so this tracer comes with a learning curve. However, the three readers who participated in this study were familiar with interpreting these examinations in routine practice.
Third, for practical and especially ethical reasons, we do not have histological confirmation of lesions suspected of malignancy [30].
Fourth, this was a retrospective study that used the data available for reprocessing by SubtlePET®. Regarding routine feasibility, a prospective study [24] showed that the time lag between the end of the examination and the possibility of reading the SubtlePET® images was less than 10 min.
Fifth, we favored a subjective analysis of image quality over an objective analysis based on the use of computerized image comparison tools. Many studies [31‐35] have used tools such as NRMSE (normalized root-mean-square error), PSNR (peak signal-to-noise ratio), or SSIM (structural similarity index), but all of these tools are tedious to set up and their clinical value remains limited [31].
Sixth, for data analysis, we did not use the PROMISE criteria [23] because at the time of the study, this classification was not currently used in the department.
Seventh, a bias of our study is the need of negative or inconclusive F-18-Choline PET/CT before giving access to 68-Ga-PSMA. This could explain why in our cohort, we found very small lesions.
Eighth, the choice of gold standard based on the referent was due to his experiment of reviewing almost all PSMA PET/CT during multi-disciplinary staff. For next studies, we shall use a consensus reading for gold standard.
Ninth, the height of patients were not available, so we could not do a subgroup analysis on BMI. Moreover, a study [36] showed that the BMI did not impact the liver SNR for Q.Clear. Also, no cross-validation was done between the two cameras but their CT reconstruction parameters were very similar.
Finally, the decrease in lesion SUVmax values makes it impossible to compare two successive scans if one of them is not processed with SubtlePET®. Moreover, as SubtlePET® decreases liver SBR, algorithm diagnostic value has to be evaluated in 177-Lu-PSMA population where patients are selected according metastasis vs liver uptake comparison [37]. In addition, almost all the lesions found were smaller than a centimeter and so were potentially influenced by the partial volume effect. This could explain the decrease in SUVmax values. However, a recent study [22] found the same results with a decrease of less than 10% in SUVmax value with 18FDG. The authors suggested that this decrease was mainly in small and moderate uptake. Further research would be needed to evaluate the biases introduced by this algorithm on lesions with much higher fixation intensities, such as in patients with high tumor burden.
Conclusion
SubtlePET® maintains image quality and detectability of 68-Ga-PSMA PET using only half of the signal. Note, however, that the SubtlePET® algorithm significantly modifies the SUVmax values of the lesions and should therefore not be used for re-evaluation if the previous examination was reconstructed with a standard algorithm. This study shows that the SubtlePET®, which has already validated for 18FDG and amyloid PET, could also be used for 68 Ga-PSMA examinations.
Acknowledgements
We thank Valentina Garibotto, Jean-Pierre Willi and Ismini Mainta for their valuable input.
Declarations
Ethics approval and consent to participate
This study was performed in compliance with European law No. 2016/679 and French law No. 78–17 of January 6, 1978. The study was approved by Approval was granted by the Centre Jean Perrin institutional review board and registered on 19/12/2019 with the French Health Data Hub under No. FOV/2018/0053. We certify that the study was performed in accordance with the ethical standards as laid down in the 1964 Declaration of Helsinki and its later amendments or comparable ethical standards. A written informed consent was obtained from all participants.
Consent for publication
Informed consent was obtained from each participant included in the study. The authors affirm that human research participants provided informed consent for publication of the images featuring in Figs. 3 and 4.
Competing interests
The authors declare no competing interests.
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Imaging quality of an artificial intelligence denoising algorithm: validation in 68Ga PSMA-11 PET for patients with biochemical recurrence of prostate cancer
verfasst von
Charles Margail Charles Merlin Tommy Billoux Maxence Wallaert Hosameldin Otman Nicolas Sas Ioana Molnar Florent Guillemin Louis Boyer Laurent Guy Marion Tempier Sophie Levesque Alban Revy Florent Cachin Marion Chanchou
