The role of DSC MR perfusion in predicting IDH mutation and 1p19q codeletion status in gliomas: meta-analysis and technical considerations

The research question was “what is the current performance of DSC MR perfusion in identifying IDH mutation and 1p19q codeletion status in glioma patients?”. To answer this question, a systematic literature search was performed on PubMed, Medline, and Embase to find relevant publications between 01/01/2000 and 01/01/2023. The patient/intervention/comparator/outcomes (PICO) framework was used to define the search items: (P) = glioma, glioblastoma, brain tumor/tumor, brain neoplasia/neoplasm; (I) = dynamic susceptibility contrast, DSC, MR perfusion, brain perfusion; (C) = biopsy, histopathology, molecular profile, isocitrate dehydrogenase/IDH, 1p19q codeletion (and common variations such as 1p/19q, 1p-19q, 1p 19q, codeletion, codeleted); and (O) = predict, identify, classify, stratify, subtype, categorize, characterize, assign, cluster, distinguish. The PICO framework categories were combined using “AND,” while variations within categories were combined via “OR.” The same search items combinations were used for all databases. Reference lists were also reviewed to identify further eligible publications.

The study selection process was conducted in EndNote X9. Eligibility criteria included peer-reviewed, English-written publications available through electronic indexation satisfying the following conditions: (1) DSC was used as an index test to determine IDH mutation and/or 1p19q codeletion status; (2) histopathology with molecular profiling was employed as the reference standard; (3) all patients received both the index test and reference standard within a reasonable timeframe with blinding of the evaluators.

Exclusion criteria were (1) studies employing multiparametric imaging, (2) the use of regression models incorporating additional covariates (e.g., sex, age) to predict IDH and 1p19q status, (3) studies using machine learning, (4) very small studies (< 10 glioma patients), (5) pediatric studies, and (6) non-original research (e.g., case-reports, reviews, abstracts, conference presentations, book chapters).

LS and CT independently extracted data from both the manuscripts as well as the supplementary materials of the included studies using a standardized Microsoft Excel 2016 spreadsheet previously agreed by all authors. We extracted data on (1) study design (retrospective vs. prospective), (2) participant characteristics (i.e., mean age, sex ratio, and ethnical breakdown), (3) glioma characteristics (i.e., grade and stage), (4) DSC imaging acquisition parameters (e.g., time to echo [TE], repetition time [TR], pulse sequence, field of view, slice thickness, slice gap, use of contrast preload, contrast dose), and (5) prefusion map post-processing (e.g., region of interest [ROI] selection, leakage correction, standardization/normalization methodology). In addition, we extracted DSC metrics (e.g., relative cerebral blood volume [rCBV]) in IDHwt, IDHm, IDHm with 1p19q codeletion and IDHm without 1p19q codeletion groups. Finally, we recorded the diagnostic accuracy for the prediction of IDH and 1p19q status (e.g., sensitivity, specificity, positive predictive value, negative predictive value). In general, there was a good agreement between LS and CT for both categorical (Cohen’s Kappa 0.88) and continuous variables (intraclass correlation coefficient: 0.92). The input of a senior reviewer (SB) was sought to resolve discrepancies.

This systematic review’s quality was assured by the Quality Assessment of Diagnostic Accuracy Studies 2 (QUADAS-2) questionnaire in its original format [17]. Patient selection, study conduct, and interpretation of IDH and 1p19q status by DSC and histopathological analysis with molecular profiling were appraised in line with the review question exploring applicability concerns and risks of bias.

Heterogeneity between studies was assessed using Cochran’s Q and Higgins I² statistics. A Cochran’s Q test with a p-value < 0.05 or I² > 50% were interpreted as potentially indicative of the presence of heterogeneity [18]. Regardless of the degree of heterogeneity, a random-effects model was used for all meta-analyses. In addition, meta-regression was conducted to explore sources of heterogeneity in any meta-analysis including at least 4 studies. Evaluated moderator variables included continuous (contrast dose, TE, TR, slice thickness, slice gap, acquisition time) and categorical variables (contrast type, pulse sequence, ROI selection, DSC standardization/normalization, analysis software, arterial input function [AIF] and the use of leakage correction).

Publication bias was assessed via the visual inspection of contour-enhanced funnel and Egger’s test. Asymmetrical funnel plots or Egger’s test p < 0.05 were interpreted as potentially indicating the possibility of publication bias [19].

We performed comparisons between IDHm and IDHwt for different combinations of WHO grades including grade 2, grade 3, grade 4, and across all WHO grades. In addition, we compared IDHm with 1p19q codeletion vs. IDHm without 1p19q codeletion. Firstly, we extracted the mean, standard deviation (SD), and the number of glioma patients in each of the groups described above for each available DSC metric. As the measurement scales differed between the studies, we performed the comparisons by calculating the SMDs and their 95% confidence intervals [CIs] [20].

In this analysis, we included all studies which reported enough metrics enabling the computation of true positives (TP), true negatives (TN), false positives (FP), and false negatives (FN).

We used validated diagnostic test accuracy meta-analysis methodology to calculate the pooled sensitivity and specificity [21]. Given the inter-dependence of sensitivity and specificity, a bivariate model was used to generate the hierarchical summary receiver operating characteristic (HSROC) curve with its 95% confidence region and to compute the area under the ROC curve (AUC) [22].

In addition, we computed the pooled diagnostic odds ratios (DORs), positive likelihood ratios (PLRs), and negative LRs (NLRs) and their 95% CIs. DOR represents the odds of the IDH mutation being correctly identified by DSC. PLR captures the ratio of the probability that DSC predicts an IDH mutation in IDHm vs IDHwt (i.e., LR for positive results). In contrast, NLR represents the LR for negative results.

A PRISMA flow chart of the database search process is presented in Fig. 1. Briefly, the initial search identified 352 potential articles. After removing duplicates and screening abstracts against our research question, we further considered 102 papers. Following the application of our inclusion and exclusion criteria, 16 manuscripts were included in the quantitative analysis [23‐38]. The characteristics of these studies were extracted and tabulated in Table 1.

Overall, the 16 studies included in the quantitative analysis were of a high methodological quality.

The QUADAS-2 spreadsheet summarizing our evaluation for each questionnaire item for each of the included studies is provided in Supplementary Material 1. The risk of bias was evaluated in the patient selection, index test, reference standard, and flow and timing domains according to the QUADAS-2 questionnaire (Fig. 2). In the patient selection domain, 13% of studies did not provide enough information on their exclusion criteria and 13% might have made inappropriate exclusions (e.g., excluding individuals for whom a full molecular profile besides IDH status was not available). The risk of bias in the index test domain was assessed as unclear in 44% of studies, which did not explicitly state that neuroradiologists were blinded to IDH mutation status. Although 75% of studies did not specify the interval between the MR perfusion and biopsy, the risk of bias was considered low given that IDH mutation status is unlikely to change in a relatively short timeframe. Applicability concerns were also evaluated in the patient selection, index test, and reference standard domains. The were no applicability concerns by design as such studies would be filtered by our exclusion criteria.

The most commonly used DSC metric when comparing IDHm vs. IDHwt was rCBV_mean. The SMD meta-analysis results are presented in Table 2, while the diagnostic accuracy meta-analysis results are summarized in Table 3.

When attempting to differentiate IDHm from IDHwt, the highest pooled sensitivity was achieved by rCBV 10^th percentile (92%, 95% CI: [86, 93]) (Table 3), and the highest pooled specificity by rCBVmean (82% [72, 89]) (Fig. 3). Although the highest diagnostic performance was observed for rCBV 10^th percentile (DOR = 20.96 [2.34–187.95], AUC = 0.91), this analysis included only 2 studies. In general, the heterogeneity between the studies was substantial (i.e., I² > 50%, Cochran’s Q p < 0.05) except for rCBV 75^th percentile sensitivity (I² = 42%, p = 0.068) and specificity (I² = 50%, p = 0.318), rCBV 25^th percentile specificity (I² = 0%, p = 0.552) and rCBV 10^th percentile sensitivity (I² = 0%, p = 0.423) analyses.

According to the SMD meta-analysis, IDHm consistently had higher rCBV values across all DSC metrics compared to their IDHwt counterparts except for rCBV 10^th percentile where the difference was not statistically significant. The highest SMDs were observed for rCBV_mean (SMD =  − 0.8 [− 1.1, − 0.5]) (Fig. 3), rCBV_max (SMD =  − 0.8 [− 1.2, − 0.5]), and rCBV 75^th percentile (SMD =  − 0.8 [− 1.2, − 0.5]) (Table 2). In contrast to the sensitivity and specificity meta-analyses, heterogeneity was generally low except for the rCBV_mean (I² = 78%, p < 0.001) and rCBV 10^th percentile (I² = 84%, p = 0.013) analyses. In general, the Egger tests and the visual inspection of the contour-enhanced funnel plots suggested a low possibility of publication bias (Figs. 4 and 5).

The SMD meta-analysis results are presented in Table 2, and the diagnostic accuracy meta-analysis results are summarized in Table 3.rCBV_mean was lower in grade 2 IDHm compared to grade 2 IDHwt tumors (SMD =  − 1.1 [− 2.0, − 0.2]), however with considerable heterogeneity (I² = 72%, p = 0.027) and possibility of publication bias (Egger test p-value = 0.017).

In grade 3 gliomas, rCBV_mean was lower in IDHm (SMD =  − 0.5 [− 0.2, − 0.8]). For predicting IDH status, rCBV_mean yielded a pooled sensitivity of 83% [60, 94], a pooled specificity of 95% [74, 99], and an AUC of 0.88. The heterogeneity and possibility of publication bias were low.

In grade 4 gliomas, rCBV_mean differentiated IDHm from IDHwt with a pooled sensitivity of 82% [55, 95] and pooled specificity of 96% [69, 100] yielding an AUC = 0.84. Indeed, rCBV_mean was lower in IDHm vs IDHwt (SMD =  − 0.5 [− 0.9, − 0.1]). The heterogeneity and possibility of publication bias were also low.

The SMD meta-analysis results are presented in Table 2. A diagnostic accuracy meta-analysis was not performed due to a lack of studies. IDHm with 1p19q codeletion demonstrated higher rCBV_mean compared to IDHm without 1p19q codeletion (SMD = 0.9 [0.2, 1.5]). A higher rCBV 90^th percentile was also observed in IDHm with 1p19q codeletion (SMD = 0.9 [0.1, 1.7]). Heterogeneity between the studies was high (all I² > 50%, all p < 0.05) albeit the possibility of publication bias was low (Egger test p-value = 0.633).

The meta-regression results for the SMD analysis are presented in Supplementary Table S1, and for the diagnostic accuracy analysis in Supplementary Table S2 from Supplementary Material 2.

Shorter TEs were associated with higher absolute SMDs for rCBV_mean (i.e., greater absolute difference between IDHm and IDHwt) which translated into higher pooled sensitivities (for a 1 ms decrease in TE, the sensitivity increased by 0.1) in the bivariate sensitivity-specificity model. Similarly, shorter TRs increased the absolute SMDs between IDHm and IDHwt for rCBV_mean (for a 1 ms increase in TR, the absolute value of SMD increased by 0.001), however without significant increase in diagnostic accuracy (both p > 0.100). Smaller slice thicknesses were associated with higher absolute SMDs for rCBV_median, while smaller slice gaps were associated with higher pooled sensitivities and specificities for rCBV_mean.

In general, DSC metrics were usually calculated either based on manual hotspot or manual total enhancing tumor area ROIs. Studies using hotspot ROIs obtained a higher sensitivity (98% [87, 100]) at the cost of lower specificity (63% [49, 75]) for distinguishing IDHm from IDHwt using rCBV_mean.

No other moderator was associated with study heterogeneity in any of the remaining meta-regressions.

Our study highlights the lack of DSC MR perfusion protocol standardization and suggests specific technical parameters contributing to heterogeneity. Specifically, shorter TEs and smaller slice gaps were associated with higher sensitivities in the bivariate sensitivity-specificity model, while shorter TEs, shorter TRs, and smaller slice thicknesses were linked to higher absolute SMDs between IDHm and IDHwt. The need for standardization of DSC MR perfusion is widely recognized, and our results highlight important technical considerations for improved diagnostic performance. Across both the SMD and diagnostic performance meta-regressions, TE was the only consistent metric associated with heterogeneity. The effects of TE in the quality of quantitative analysis of DSC perfusion have been previously described. In accordance with our findings, previous studies highlighted the limitations of long TEs likely due to saturation of the arterial input signal. To overcome this limitation, dual-echo and more recently multi-echo techniques have been proposed for the optimization of DSC perfusion [51, 52]. However, TE influences DSC perfusion in conjunction with multiple parameters, including flip angle, slice thickness, and contrast pre-load [52‐54]. Using DSC perfusion simulations, Leu et al. [54] examined the effects of such acquisition parameters to the “faithful” estimation of rCBV. In an exploratory manner, we assessed the rCBV estimation error of the included papers with adequate protocol information, based on the published heat map diagram [54]. Based on this predictive model, many of the included studies would yield higher rCBV estimation errors compared to the optimized acquisition parameters [23, 25, 32, 34, 38]. In contrast, studies with optimized parameters would be expected to yield lower rCBV estimation errors and, interestingly, predominantly reported higher classification performance [26, 28, 30, 35]. For example, the study by Kickingereder et al. [28], predicted to yield the minimum rCBV estimation error, correctly classified 88% of tumors using rCBV 90^th percentile (accuracy range: 82–88% for all percentiles). Such an assessment of methodological quality is limited since it requires complete protocol and post-processing data which are not available for all studies and thus does not allow appropriate comparisons. However, it can be postulated that errors in estimating rCBV may influence the classification performance of DSC perfusion and this insight can highlight areas for optimization of the technique. In addition, when using a shorter TR, the T₁ of blood is minimized allowing for more accurate measurements of rCBV which may better expose the subtle differences in perfusion between IDHm and IDHwt. Lastly, smaller slice thicknesses and slice gaps would increase the image resolution. This would magnify any existing differences in glioma vascular signatures between IDHm vs. IDHwt leading to a better discriminative power of DSC. However, this obviously comes at the expense of longer acquisition times.

A wide range of DSC metrics were reported by the included studies (Tables 2 and 3). Although rCBV_mean was the most widely employed, it may not be the optimal metric for capturing the vascular signatures of gliomas. Indeed, Choi et al. [24] suggested that specific segments of the signal intensity time curve may exhibit a better predictive performance for IDH mutation status. More specifically, IDHwt demonstrated a steeper downslope of signal intensity in the initial parts of the curve and less steep upslope in the region of signal recovery. Such minute temporal discriminatory features may be lost when using aggregate metrics such as rCBV_mean. A novel metric might be the percentage signal recovery ratio (PSR) which better characterizes the DSC signal intensity curve and was successfully employed for IDHm status prediction [25]. Similarly, M-enhanced analysis of the DSC signal intensity curve is also an exciting prospect [9]. The low number of studies employing such metrics or methodologies prevented us from conducting meta-analyses. However, these innovations warrant further research and external validation.

Studies employing histogram analysis of DSC perfusion maps reveal histogram features as potential predictors of IDHm status [8, 27, 29]. Despite the smaller sample sizes, our findings highlight the discriminative value of the extreme rCBV percentiles for IDH prediction. When distinguishing IDHm from IDHwt, the highest sensitivities were observed for rCBV 10^th (i.e., 92% [86, 96]), 75^th (i.e., 91% [80, 96]) and 90^th percentiles (i.e., 88% [62, 97]). Thus, the use of histogram features for predicting IDH status warrants further investigation.

Radiogenomics introduce further capabilities for non-invasive glioma genotyping. This is highlighted by a recent systematic review which reported a high diagnostic performance (i.e., pooled sensitivities > 90% and pooled specificities > 87%) for both the prediction of IDH and 1p19q status [13]. By applying advanced machine learning models to imaging techniques such as DSC perfusion, there is a potential for improved performance in genotype status prediction [8]. However, benefits may extend beyond glioma genotyping to precision treatment selection and patient surveillance through the identification of independent vascular profiles. Recent studies demonstrate this capability, with promising outcomes for metrics beyond rCBVmean [24‐26, 28]. Such applications would further benefit from improved DSC acquisition protocols as optimized parameters associating with a higher discriminative performance may have synergistic effects with the machine learning models. This highlights the importance of this study, as we characterize acquisition parameter trends associated with a higher diagnostic performance in distinguishing IDHm from IDHwt.

Despite utilizing robust methodologies and performing stratified random-effects meta-analyses, our study has several limitations. Firstly, we did not prospectively register the study protocol in a systematic review database such as the International Prospective Register of Systematic Reviews (PROSPERO). Prior registration is considered to increase transparency and reduce the likelihood of bias. Also, assessment of the performance of DSC perfusion in grading gliomas posed inherent limitations due to the inclusion of studies prior the transition to the current 2021 WHO classification. Although we believe our results on glioma grading provide insights in the value of DSC perfusion in predicting perfusion phenotypes related to microvascular proliferation and necrosis, they are insufficient to assess grading in accordance with current criteria, which remains to be investigated. Furthermore, we did not include studies employing machine learning–enhanced DSC perfusion since their methodology would not allow the assessment of DSC in isolation. Similarly, we did not explore the role of other MR perfusion techniques such as DCE or arterial spin labeling which might be better suited for non-invasive glioma genotyping. Considerable inter-study heterogeneity was identified by our analysis, and this prevented us from providing generalizable cut-off values to distinguish IDHm from IDHwt, and IDHm with 1p19q codeletion and IDHm without 1pq19q codeletion. To explore sources of heterogeneity, we performed meta-regression which highlighted moderator variables associated with a higher discriminatory performance for meta-analyses including at least 4 studies. However, the number of studies in most meta-analyses and meta-regressions was small. The latter limits the generalizability of our findings and warrants further investigation. Similarly, limitations in the number of studies precluded a diagnostic accuracy meta-analysis for the prediction of 1p19q codeletion status in IDHm, which remains a relevant research question.

Our study indicates a promising diagnostic performance of DSC perfusion for the non-invasive prediction of IDH mutation and 1p19q codeletion status, and subsequently for glioma grading based on the current WHO classification. We propose that limitations in diagnostic performance may not reflect inherent inadequacies of DSC perfusion. Rather, this is potentially attributed to the heterogeneity in acquisition protocols and post-processing as well as to limitations of commonly used perfusion metrics which fail to capture the dynamic properties of the signal-intensity curve. Research on optimization DSC perfusion protocols and on alternative novel perfusion metrics beyond rCBV may open new horizons for the clinical applications of DSC. This may not be limited to pre-treatment subtyping of gliomas, but could extend to personalized treatment selection and post-treatment surveillance. Machine learning can further enhance such capabilities and, in addition, allow the incorporation of multiparametric data from complementary advanced techniques (e.g., MRS).