Advertisement
Browse Subject Areas
?

Click through the PLOS taxonomy to find articles in your field.

For more information about PLOS Subject Areas, click here.

  • Loading metrics

Open Access

Peer-reviewed

Research Article

Semi-Quantitative Parameter Analysis of DCE-MRI Revisited: Monte-Carlo Simulation, Clinical Comparisons, and Clinical Validation of Measurement Errors in Patients with Type 2 Neurofibromatosis

Abstract

Purpose

To compare semi-quantitative (SQ) and pharmacokinetic (PK) parameters for analysis of dynamic contrast enhanced MR data (DCE-MRI) and investigate error-propagation in SQ parameters.

Methods

Clinical data was collected from five patients with type 2-neurofibromatosis (NF2) receiving anti-angiogenic therapy for rapidly growing vestibular schwannoma (VS). There were 7 VS and 5 meningiomas. Patients were scanned prior to therapy and at days 3 and 90 of treatment. Data was collected using a dual injection technique to permit direct comparison of SQ and PK parameters. Monte Carlo modeling was performed to assess potential measurement errors in SQ parameters in persistent, washout, and weakly enhancing tissues. The simulation predictions for five semi-quantitative parameters were tested using the clinical DCE-MRI data.

Results

In VS, SQ parameters and Ktrans showed close correlation and demonstrated similar therapy induced reductions. In meningioma, only the denoised Signal Enhancement Ratio (Rse1/se2(DN)) showed a significant therapy induced reduction (p<0.05). Simulation demonstrated: 1) Precision of SQ metrics normalized to the pre-contrast-baseline values (MSErel and ∑MSErel) is improved by use of an averaged value from multiple baseline scans; 2) signal enhancement ratio Rmse1/mse2 shows considerable susceptibility to noise; 3) removal of outlier values to produce a new parameter, Rmse1/mse2(DN), improves precision and sensitivity to therapy induced changes. Direct comparison of in-vivo analysis with Monte Carlo simulation supported the simulation predicted error distributions of semi-quantitative metrics.

Conclusion

PK and SQ parameters showed similar sensitivity to anti-angiogenic therapy induced changes in VS. Modeling studies confirmed the benefits of averaging baseline signal from multiple images for normalized SQ metrics and demonstrated poor noise tolerance in the widely used signal enhancement ratio, which is corrected by removal of outlier values.

Citation: Jackson A, Li K-L, Zhu X (2014) Semi-Quantitative Parameter Analysis of DCE-MRI Revisited: Monte-Carlo Simulation, Clinical Comparisons, and Clinical Validation of Measurement Errors in Patients with Type 2 Neurofibromatosis. PLoS ONE 9(3): e90300. https://doi.org/10.1371/journal.pone.0090300

Editor: Wooin Lee, University of Kentucky, United States of America

Received: July 3, 2013; Accepted: February 3, 2014; Published: March 4, 2014

Copyright: © 2014 Jackson et al. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.

Funding: This study was locally funded by the Manchester Cancer Research Centre. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

Competing interests: The authors have declared that no competing interests exist.

Introduction

Analysis of dynamic contrast enhanced MRI (DCE-MRI) data is commonly performed by applying pharmacokinetic (PK) models to changes in contrast agent concentration derived from observed changes in signal intensity (SI). A simpler approach is to perform direct analysis of changes in SI using one or more of a range of established semi-quantitative (SQ) descriptors.

Almost all early DCE-MRI studies employed simple SQ metrics derived by mathematical analysis of observed SI-time course data (SI-TC). With the growth of PK approaches the field has become dichotomized with consensus groups recommending PK analysis [1], [2], [3], [4] whilst, at the same time clinical radiologists are far more likely to use SQ metrics which have become essential clinical tools across a range of oncological applications [5], [6], [7], [8], [9], [10], [11], [12], [13], [14], [15], [16], [17], [18], [19], [20], [21], [22], [23], [24], [25], [26], [27]. This widespread adoption of SQ metrics reflects the simplicity of the approach, the ability to use straightforward, often slow, dynamic acquisition sequences with good spatial resolution and coverage, the wide availability of clinical analysis software and, most importantly, clear clinical evidence that the techniques are beneficial.

Despite widespread clinical adoption SQ parameters are commonly avoided for clinical trial applications in the belief that they are less biologically specific and more prone to variability than parameters derived from PK modeling, [2], [4] although, no detailed study of the behavior of SQ parameters or direct comparison of SQ and PK parameters has been presented. Many studies have however shown significant problems associated with PK derived parameters arising from the need for high temporal resolution sampling, accurate arterial input function definition and problems associated with curve fitting based analysis approaches which are limiting widespread implementation of DCE-MRI, particularly into multi-center studies [4], [28], [29], [30].

The signal enhancement ratio (Rmse1/mse2) is particularly attractive SQ metric which is little affected by variation in tissue T10 values and has been shown to correlate closely with the redistribution rate constant (kep), a commonly analyzed PK parameter [31]. This behavior identifies Rmse1/mse2 as a potential simple and attractive surrogate of PK analyses, which has led to widespread adoption, particularly in breast cancer studies [9], [10], [11], [12], [13], [32], [33], [34], [35], [36].

Anxieties around the use of SQ parameters for quantitative studies have little or no substantive evidence base [37]. Multicenter clinical applications are common and the development of “normalized” SQ metrics represents an attempt to minimize variation across multi-center/multivendor acquisitions. Wider use of SQ parameters for clinical trials and multicenter studies would simplify data analysis and potentially improve implementation of DCE-MRI. However, before this could occur it is essential that we have a greater understanding of their ability to identify therapy induced changes, the comparative performance of SQ and PK derived biomarkers and their variability in the setting of multicenter studies.

Direct comparison of SQ and PK parameters is complicated by the differing acquisition approaches employed. SQ parameters are typically used in low temporal resolution data allowing the use of high spatial resolution and large volume coverage. In contrast, PK parameters depend on high temporal resolution to allow accurate fitting of the analytical model, which produces compromises in both spatial resolution and coverage. We have recently described a dual injection technique, ICR-DICE (Improved Contrast and spatial Resolution using Dual Injection Conrast Enhanced MRI) [38]. This uses two separate contrast enhanced dynamic acquisitions to provide a high temporal resolution dataset, ideal for identification of an arterial input function for PK mapping, and a high spatial resolution dataset for measurement of tissue residue functions allowing direct, pixel by pixel comparison of SQ and PK metrics in the same dataset.

This study uses a combination of Monte Carlo modeling and clinical ICR-DICE data from patients with type 2 Neurofibromatosis and was designed to address five basic issues:

  1. What is the predicted effect of variations in signal intensity curve shape and signal to noise ratio on errors in the estimates of SQ parameters?
  2. Do observations from the clinical DCE-MRI data support Monte-Carlo predictions?
  3. Does the behavior of the signal enhancement ratio (Rmse1/mse2) provide a satisfactory surrogate metric for formal PK based analyses?
  4. What is the optimal number of samples to define pre-enhancement signal intensity on estimates of SQ parameters?
  5. How do SQ and PK derived metrics compare in the detection of anti-angiogenic therapy induced changes in a single centre setting?

Methods

Choice of Semi-Quantitative Parameters

Candidate SQ parameters were categorized into 5 groups, each characterized by a similar mechanism of error propagation during parameter calculation. An additional novel parameter, designed to improve computational stability, was subsequently added. The final 6 SQ parameters were:

  1. Absolute MRI signal enhancement (MSE) [39], (e.g., MSE1min = SI1min,post−SIpre), where SI1min,post is signal intensity at one minute post contrast agent (CA) injection and SIpre is SI pre-injection. One commonly used example of this category is the maximum intensity change per time interval ratio (MITR) [40].
  2. Relative signal enhancement (MSErel) [14], [22], [41] which uses a baseline value for normalization in order to reduce the dependence on biological and imaging system variables, e.g., MSErel, 1min = (SI1min post−SIpre)/SIpre, where MSErel, 1min is MSErel at one minute post CA injection.
  3. The sum of MSE over a fixed post-injection duration, (∑MSE). which can also be defined as the area under the enhancement curve (AUC) [1].
  4. The sum of MSErel over a fixed post-injection duration, (∑MSErel), which can also be defined as baseline-normalized AUC [1].
  5. Signal enhancement ratio, (Rmse1/mse2) commonly defined as the ratio of early to late contrast enhancement ratio [32], e.g., Rmse1/mse2 = (SI1min post−SIpre)/(SI5min post−SIpre); Alternatively, defined as the ratio of late to early contrast enhancement, e.g., Rmse2/mse1 = (SI5min post−SIpre)/(SI1min post−SIpre), which was also coined the name of SER (Signal Enhancement Ratio).
  6. Denoised Signal enhancement ratio (Rmse1/mse2(DN) Comparison of values in mse1, mse2, and Rmse1/mse2 maps demonstrated that 3–4% of voxels have spurious values resulting from noise-induced near-zero values in mse2 maps whereas mse1≠0. To remove these outliers, thresholds of 0<Rmse1/mse2<1.45 were used to exclude voxels outside the 95th percentile and Rmse1/mse2 calculated as described above. Note that a median filter could be used to filter outliers, but may also affect high frequency components of the SER maps [42].

Monte-Carlo Modeling of Measurement Errors

Generation of typical signal intensity-time course curves.

Three patterns of signal intensity time course curves (SI-TC) were chosen for Monte-Carlo analysis [19]: 1) persistent enhancement, 2) contrast washout, and 3) weak contrast enhancement. These represent the three principle variants that have biological value in published studies of SQ parameters. Examples of each type were identified in high-spatial resolution DCE-MRI data of a vestibular schwannoma (VS) in one patient (vide infra). These exemplar signal-time course curves were used to generate a tissue model by:

  1. Converting SI-TC to contrast concentration time course curves (CC-TC).
  2. Fitting with the modified Tofts model [43], [44].
  3. Generating synthetic SI-time curves from the derived fitting parameters using measured baseline SI and pre-contrast T1 relaxation time (T10), and a literature value of the longitudinal relaxivity of the contrast medium (4.39 mM−1 sec−1) [45] (Figs 1a–c).
  4. Sampling the synthetic time course data with a 1-minute temporal resolution for use as “ground truth” signal intensity-time course curves (SI-TC) in subsequent Monte Carlo analyses (Figs 1 d–f).
thumbnail
Download:
Figure 1. Shows examples of the three typical kinetic enhancement curves used for modelling.

curve I (persistent type), curve II (washout type), and curve III (weak enhancement). (a–c) demonstrate the [Gd] curves fitted with the modified Tofts model using the ICR-DICE method. The fitted kinetic parameter values are given above each curve. The small panel at the up-right corner of (c) shows the SI curves from the corresponding region in the high-temporal resolution (Δt = 1 s) pre-bolus data set of the ICR-DICE acquisition. (d–f) demonstrate the SI curves converted from the fitted [Gd] curves. A 1-minute temporal resolution was used to extract data from the theoretical SI curves (showed as asterisk points), which are used as the ‘true’ SI enhancing curves in the following Monte Carlo simulations.

https://doi.org/10.1371/journal.pone.0090300.g001

Monte Carlo simulation for error analysis.

MSE1min, MSErel,1min, ∑MSE, ∑MSErel and Rmse1/mse2, were calculated for each of the reconstructed SI-time curves to produce ‘true’ values (where mse1 = MSE1min, mse2 = MSE5min and the sum of MSE and MSErel were performed over a 5 minute duration). Rician white noise with noise levels ( =  standard deviation/mean baseline signal) of 5%, 10%, and 15% was subsequently added to the simulated SI-TC and SQ parameters calculated to produce ‘measured’ values. A total of 106 repetitions were performed for each given condition (i.e., a specific SI-TC type), and a given noise level. Percent deviations (PD) of the ‘measured’ from ‘true’ values were calculated. Frequency distributions of the percentage deviations were displayed as histograms and distribution characteristics including measures of distribution position (the mean and the median), dispersion (the range, percentiles, and SD), skewness, and kurtosis were calculated. Rank statistics (the range, the 5th and 95th percentiles, and the median) of the PD distributions were displayed using box-and-whisker plots.

Monte Carlo simulation for optimal number of pre-contrast time points.

The use of a mean value of multiple pre-contrast time points as SIpre for calculation of SQ parameters will reduce the effects of noise and improve reliability [46]. To determine the optimal number of the pre-contrast time points, the above Monte Carlo simulation was repeated but with varying number of pre-contrast time points based on 15% noise level (a typical noise level in our data). 104 repetitions were performed for each given condition (i.e., a specific SI-TC type), and a given number of pre-contrast time points.

Clinical Studies

Ethical statement.

The clinical study received approval from the NHS Health Research Authority National Research Ethics Service, North West Committee, Greater Manchester Central, Rec Reference 13/NW/0131 and all patients gave written informed consent for inclusion in the study. All imaging data is archived within the CRUK-EPSRC cancer imaging centre in Cambridge and Manchester archival database and is available to external investigators in anonymized form.

Clinical data acquisition.

DCE-MRI data were collected in five patients with type 2 neurofibromatosis (NF2), with a total of 12 tumors (7 vestibular schwannoma (VS) and 5 meningiomas). Patients were treated with the anti-vascular endothelial growth factor antibody bevacizumab (5 mg/kg fortnightly, Avastin, Hoffman La-Roche, CH) and were imaged on 3 occasions: pre-treatment (day 0), 3 days (day 3), and 3 months (day 90) following treatment.

DCE-MRI data were collected as described previously [38] using a dual injection technique with an initial high-temporal resolution (1 s), low-spatial resolution acquisition for measurement of the arterial input function followed by a low-temporal, high-spatial resolution (voxel size  = 1×1×2 mm) acquisition for measurement of the tissue residue function. Contrast agent (CA; gadoterate meglumine; Dotarem, Geurbet S.A.) was administered by power injector as an intravenous bolus at a rate of 3 ml/s, followed by a chaser of 20 mls of 0.9% saline administered at the same rate. A low dose of CA (0.017 mmol/kg) was used for the first, high temporal resolution acquisition. For the second, high spatial resolution acquisition a standard dose (0.1 mmol/kg) was administered synchronized with 7th frame of the dynamic acquision yielding six pre-contrast time points in each SI(t) curve.

Validation of Monte Carlo Error Predictions

Single pixel DCE-MRI data from pre-treatment scans of all tumors (7 VS and 5 meningiomas) was pooled in order to test the predictions of the Monte-Carlo modeling process. Surrogate “true values” of SQ parameters were developed using the assumption that: where CC-TC data shows good agreement with the modified Toft's model, the resulting fitted function represents the true underlying CC-TC free of noise effects. For this purpose the residual autocorrelation function and fraction of modeling information (FMI) [47] is a proper quality control for defining the denoised SI curves.

We therefore use the following approach:

  1. Data for all tumor voxels (n = 117,527) was fitted using a modified Toft's model and estimated values of Ktrans,ve and vp were derived.
  2. Voxels where the total error is dominated by modeling error effects were excluded by examining the FMI (Voxels with FMI≤0.995 were excluded).
  3. The majority of tumor voxels under this study had a noise level of 0.08 to 0.17, corresponding to the simulated data of 0.10–0.15 noise levels. Voxels with noise level outside the range of 0.08–0.17 were therefore excluded.
  4. Voxels with SI(t) curves resembling the typical persistent enhancement (0.03<Ktrans<0.09 min−1, 0.55<ve<0.90, and 0.001<vp<0.07) and typical washout patterns (0.3<Ktrans<1.0 min−1, 0.20<ve<0.45, and vp<0.15) were identified. The Ranges for values of Ktrans, ve and vp for the persistent and washout curves were setup with the consideration of obtaining enough in vivo curves for the validation while keeping the typical persistent or washout type.
  5. Theoretical SI-TC were generated for the selected voxels using the measured PK parameters and T10 values of the corresponding voxels.
  6. SQ parameters were calculated from the theoretical SI-TC to serve as “true values”.

Figure 2 shows examples of curves used in the PD distribution analysis (left column) and of those excluded (right column) based on the temporal autocorrelation analysis.

thumbnail
Download:
Figure 2. Shows fitting results for Experimental and modelled signal enhancement data SI(t).

Experimental data are shown with symbols of “+”. Modeled data are shown as a solid line. The good fits (left column) are characterized by a lower (near zero) ‘Mean Ar2–30’ and a higher fraction of modeling information (FMI) [47], where Ar is the autocorrelation of the residual, and the ‘Mean Ar2–30’ was the mean Ar over the time lags 2 to 30. The poor fits with considerable modeling error (right column) are characterized by a higher ‘Mean Ar2–30’ and a lower FMI.

https://doi.org/10.1371/journal.pone.0090300.g002

These voxel selection criteria, outlined above, were used to select a subgroup of voxels whose enhancement curves corresponded very closely to the three principal subtypes identified in the Monte Carlo simulation. In addition, although most of the voxels in the tumors could be fitted with a scaled fitting error (SFE) around 15% or lower, only around one third of the typical persistent or typical washout curves could be fitted with a fraction of modeling information (FMI) equal to, or higher than, 0.997 and these voxels were used in the validation. This resulted in the exclusion of the majority of voxels that were represented by mixed curve types leaving 8291 voxels with persistent and 448 with washout type curves. This selection process is necessitated to support direct comparison with the three main curve types selected for the Monte Carlo simulation. We therefore divided the 106 Monte Carlo simulated persistent type curves into 100 groups (each has 10,000 samples), and the 106 simulated washout type curves into 2000 groups (each has 500 samples). PD distributions were calculated for each of the subgroups. A range of values for each of the descriptive statistics of the PD distribution were produced and used for comparison to the in vivo data.

Comparing therapy-induced changes in PK and SQ parameters.

DCE-MRI datasets from pre-treatment, 3 days and 3 months post treatment were analysed for each patient. Tumors were automatically segmented using high-spatial resolution 3D Bayesian probability maps [48]. 3D Parametric maps of the five SQ parameters and of Ktrans, vp, ve, and rate constant kep (≡Ktrans/ve), calculated using the modified Tofts model [43], [44], were generated from each data set. Calculations of SQ parameters used a pre-contrast measurement consisting of an average of 5 pre-contrast baseline measurements.

Pre- and post-treatment differences in SQ parameters were tested using two samples Wilcoxon rank-sum test for VS and meningiomas respectively. Spearman's rank order correlation was used to analyze the relationship between ktrans and each of four SQ parameters (MSE, MSErel, ∑MSE, ∑MSErel) across two tumor groups, i.e. VS and meningioma for each visit.

Since a close correlation between Rmse1/mse2 and kep has been reported by previous workers [12], [31], we used Spearman's rank order correlation to analyze the relationship between Rse1/se2 and kep across the two tumor groups for each visit and also performed a pixel-by-pixel comparison of the pre- and post-treatment values of Rmse1/mse2 and kep using scatter plots and linear regression analyses.

Results

Monte Carlo Simulation

1. What is the predicted effect of variations in signal intensity curve shape and signal to noise ratio on errors in the estimates of SQ parameters?

Measures for distribution characteristics of PD values from the 106 Monte Carlo repetitions are shown in Table 1 and Figure 3. Table 2 compares the descriptive statistics of PD distributions from Monte Carlo simulations and in vivo data. The in vivo values for all descriptive statistics, in general, lie within the range (minimum and maximum) given by the Monte Carlo simulated PD distributions.

thumbnail
Download:
Figure 3. Box-and-whisker plots showing the PD distributions of the five empirical parameters.

The range between 5th–95th percentiles is shown by the box. Extreme values are shown by whiskers. Median is shown by a dot within the box. The PD distributions calculated from 1000000 Monte Carlo repetitions. ENR for each simulated condition are annotated. Formula for calculation of ENR can be found in Table 1.

https://doi.org/10.1371/journal.pone.0090300.g003

thumbnail
Download:
Table 1. Descriptive statistics for PD distributions calculated from 106 Monte Carlo repetitions for each of the five SQ parameters under varying noise and pharmacokinetic conditions.

https://doi.org/10.1371/journal.pone.0090300.t001

thumbnail
Download:
Table 2. Comparison of the SQ parameter PD distributions calculated from Monte Carlo simulations and the in vivo data.

https://doi.org/10.1371/journal.pone.0090300.t002

2. Do observations from the clinical DCE-MRI data support Monte-Carlo predictions?

Simulation and in vivo data demonstrate close agreement and show the following:

  1. Parameters “normalized” using pre-contrast signal intensities show poorer precision than non-normalized metrics. SD values of PD distributions of normalized metrics were 37%−94% greater than their non-normalized counterparts at noise level of 5% and 46%−106% greater at noise level of 15%. In vivo data showed that the 90% confidence ranges (range 5th to 95th percentiles) of the PD for normalized metrics were 22%−85% greater than their non-normalized counterparts and 82%−309% greater in the extreme range (range between the minimum and maximum values) (Table 2) (23).
  2. ∑MSE and ∑MSErel show greater precision for persistent and lower precision for washout type curves when compared to MSE1min and MSE1min, rel. SD values of PD distributions of ∑MSE and ∑MSErel were 50% and 30% less than for MSE1min and MSE1min, rel for persistent but 19%−20% greater than their non-sum counterparts for washout type curves (Table 1). In vivo data confirmed these predictions showing that the SD of PD for ∑MSE and ∑MSErel were 57% and 38% less than MSE1min and MSE1min, rel for persistent but 5% and 18% greater for washout type curves in the 90% confidence ranges of the PD (Table 2).
  3. Both simulation and in vivo data showed that: (1) PDs in MSE and ∑MSE are normally distributed (K-S test, p>0.05); (2) PD distributions normalized MSE and ∑MSE show a right skew although the K-S test was not significant, 3) the PD distribution in Rmse1/mse2 is not normal with fat tails in the PD distributions [49] (K-S test (p<0.001).

3. Does the behavior of the signal enhancement ratio (Rmse1/mse2) provide a satisfactory surrogate metric for formal PK based analyses?

Previous studies have recommended the use of Rmse1/mse2 because it shows correlation with kep [12], [31]. In the current study Rmse1/mse2 was significantly correlated with kep on day 0 (p = 0.026) and day 90 (p = 0.004), although this relationship was not observed on day 3 (p>0.05). Figure 4a shows longitudinal coregistered Rmse1/mse2 and kep maps on day 0 and day 3. Scatter plots of kep pixel values from day 0 and day 3 show correlation (Fig. 4b: R2VS = 0.38; R2meningioma = 0.57) with a clear change in the slope of the regression line reflecting treatment induced change in VS (kep, post = 0.48, kep, pre+0.05) but no change in meningioma (kep, post = 0.94, kep, pre+0.00). Despite the expected correlations between the two parameters the scatterplot for Rmse1/mse2 shows weak correlations (Fig. 4: R2VS = 0.19; R2meningioma = 0.23) with reduced separation between VS and meningioma.

thumbnail
Download:
Figure 4. Demonstrating direct comparisons between Rmse1/mse2 and kep in response to treatment in a single patient.

(a) Longitudinal spatially co-registered maps of Rmse1/mse2 (top) and kep (bottom) before (left) and 3 days after bevacizumab treatment (right) from a central slice of the tumors. (b) Pixel-by-pixel scatter plots of the parametric values (3 days post-treatment against the pre-treatment) produced from the right vestibular schwannoma and the meningioma on the same slice as in (a). The values of Rmse1/mse2 were calculated with an average of the five pre-contrast baseline measurements as SIpre, MSE1 = MSE1min and MSE2 = MSE5min. Mean kep of VS was 0.30±0.10 on day 0 and 0.19±0.08 on day 3. Mean kep of meningioma was 0.38±0.09 on day 0 and 0.36±0.09 on day 3. Mean Rmse1/mse2 of VS was 0.82±0.22 on day 0 and 0.68±0.19 on day 3. Mean Rmse1/mse2 of meningioma was 0.94±.27 on day 0 and 0.94±0.18 on day 3.

https://doi.org/10.1371/journal.pone.0090300.g004

In modeling studies Rmse1/mse2 showed the greatest variability in response to noise and PK conditions. Rmse1/mse2 was much more robust for persistent than for washout type curves. Simulations showed that SD values of the PD distributions for washout were 16%, 263% and 2E4% greater than those of persistent type curves for noise levels of 5%, 10%, and 15% respectively (Table 1, Fig 3). In vivo data confirmed this behavior showing SD values for washout 93% greater than for persistent type curves in the 90% confidence ranges of the PD, and 897% greater in the extreme range (Table 2).

Comparison of values in mse1, mse2, and Rmse1/mse2 maps demonstrated that 3−4% of voxels have spurious values resulting from noise-induced near-zero denominator (se2) values in the ration calculation. To remove these outliers, a thresholds of 0<Rmse1/mse2<1.45 was used to exclude voxels outside the 95th percentile. The denoised Rmse1/mse2 (Rmse1/mse2 (DN)) demonstrates treatment–induced changes in VS similar to those seen with other SQ metrics. Post-treatment Rmse1/mse2 (DN) of VS 3 and 90 days after therapy were significantly smaller than pre-treatment (p = 0.017; p = 0.026), whilst post-treatment Rmse1/mse2 (DN) of meningiomas at day 3 showed no change (p>0.05). Post treatment Rmse1/mse2(DN) at day 90 of meningiomas were significant smaller than pre-treatment (P = 0.008). Spearman's rank correlations showed Rmse1/mse2 (DN) showed close positive correlation with kep for all three visits.

4. What is the optimal number of samples to define pre-enhancement signal intensity on estimates of SQ parameters?

The Monte-Carlo simulation showed that, with a 15% noise level (a typical noise level in our data), two pre-contrast time frames improved the precision of SQ parameter estimation with a resulting reduction in the standard deviation of PDs of 13%, 24%, 25%, and 31% for MSE, MSErel, ∑MSE, and ∑MSErel respectively for persistent type curves; 13%, 26%, 25%, and 31% for washout type curves. The improvement started from 2 and came to a plateau at time frame 6. The PD distributions of normalized metrics were more affected with reduction of positive skewness and kurtosis. The precision of Rmse1/mse2 was least affected by increasing number of pre-contrast time points for persistent type curves, but most affected for washout type curves.

5. How do SQ and PK derived metrics compare in the detection of anti-angiogenic therapy induced changes in a single centre setting?

Parametric maps of both PK and SQ parameters were of high quality (Fig 5). Table 3, and Figure 6 show mean values of PK and SQ parameters in pre and post-treatment tumors. There were no significant changes of longitudinal relaxation rates R1 (≡1/T1) in VS or meningiomas after treatment. In VS; Ktrans, vp, MSE, MSErel, ΣMSE and ΣMSErel and Rmse1/mse2(DN) measured on day 3 and day 90 were significantly lower than at day 0. From the SQ parameters, only Rmse1/mse2 showed no significant changes in response to therapy. From the PK parameters ve showed no significant treatment related change. In meningiomas; Rmse1/mse2(DN) measured 90 days after treatment was significantly lower than at day 0. Other than this there was no significant post-therapy change of PK (i.e. Ktrans, vp. ve) or SQ (i.e. MSE, MSErel, ΣMSE, ΣMSErel, Rmse1/mse2) parameters in meningiomas.

thumbnail
Download:
Figure 5. Axial view of central slices of 3D longitudinally co-registered kinetic and semi-quantitative parametric maps obtained in 26-year-old woman, who has had a progressive VS (arrow heads) and an occipital located meningioma (arrow) undergoing treatment with bevacizumab.

Note that a ring-like structure on the margin of the meningioma can only be seen on the vp maps (arrow), but not on any of semi-quantitative maps.

https://doi.org/10.1371/journal.pone.0090300.g005

thumbnail
Download:
Figure 6. Plots of the changes in each of the SQ and PK parameters for VS during treatment.

Error bars represent 95% confidence limits. Significant change compared to pre-treatment (paired comparisons) is indicated by stars, * = p<0.05, ** = p<0.01.

https://doi.org/10.1371/journal.pone.0090300.g006

thumbnail
Download:
Table 3. Comparison of mean values of DCE parameters in pre- and post-treatment VS and Meningiomas.

https://doi.org/10.1371/journal.pone.0090300.t003

Table 4 lists correlation coefficients and its significance level where the relationships between Ktrans and SQ parameters, i.e. MSE, MSErel, ΣMSE and ΣMSErel, were tested using Spearman's rank order test. Mean values of MSE, MSErel, ΣMSE and ΣMSErel in VS and Meningiomas were all significantly correlated to mean values of Ktrans on day 0, day 3 and day 90 (p = 0.002−0.042) with the exception of ΣMSErel measured on day 90, which showed only a trend of relationship towards Ktrans (p = 0.076).

thumbnail
Download:
Table 4. Spearman's rank order correlation between Ktrans and SQ parameters, i.e. MSE, MSErel, ΣMSE and ΣMSErel.

https://doi.org/10.1371/journal.pone.0090300.t004

Discussion

DCE-MRI has become one of the most widely used techniques for the characterization of tissue microvasculature with particularly wide uptake in oncology [3], [4], [50], [51]. Despite this there remain significant problems with application and dissemination of the technique, largely resulting from variability between imaging platforms. In clinical trials this has led to a dominance of PK based analysis techniques since it is reasoned, probably correctly, that the use of calculated contrast concentration in place of SI data reduces a major aspect of variation [3], [4]. However the use of PK metrics introduces additional problems including the need to define an accurate arterial input function (causing competing demands for temporal and spatial resolution), difficulties with measurement of pre- and post- treatment T1 values required for calculation of contrast concentration, the choice of appropriate pharmacokinetic models and the need for curve fitting analyses. The PK approach is therefore complex and, until recently, has been largely limited to individual centre studies [52]. As a consequence there is wide variation in acquisition and analysis approaches [51]. It is surprising that, 20 years after the introduction of DCE-MRI there is no standardization of acquisition sequences or analytical software for clinical or research use despite extensive investment in time and resources by academic groups, government and the pharmaceutical industry [1], [3], [4], [37], [53], [54], [55].

Previous investigators showed the diagnostic importance of the shape of time–signal intensity curves [56] in differentiating benign and malignant enhancing lesions in breast [19], [57], [58], hepatic Lesion [59], and lesions of bone [60], brain [27], colorectal [61] and prostate [62]. Some studies analyzed correlation of CA enhancement patterns on MR images with histopathological findings and tumor angiogenesis [63], [64], and with PET imaging[65]. This study analyzed error propagation of SQ parameters in three types of SI-time courses, selected on the basis of these published findings, to investigate these curve shapes' preference for robust calculation of SQ parameters.

SQ parameters “normalized” using pre-contrast signal intensities are thought to be necessary to reduce the dependency on both biological and imaging variables such as coil filling factor, transmitter and receiver gain etc., which vary from scan to scan and/or patient to patient [18]. As expected, the normalized metrics (MSErel and ∑MSErel) were found to be more susceptible to noise than their non-normalized counterparts (MSE and ∑MSE) and we have confirmed suggestions by previous workers that the use of multiple pre-contrast time points to obtain a mean value of SIpre, significantly improves estimation errors [46]. We have also shown that this benefit is significant when only 2 pre-contrast data points are collected and plateaus at 6. In the therapy setting ∑MSE, MSE, ∑MSErel, and MSErel, correlated with and showed similar behavior to Ktrans, [66], [67]. The predicted and observed error distributions in SQ parameters described here could be considered a disadvantage to their use in clinical trials. However, similar Monte-Carlo modeling analyses of PK analytical approaches have demonstrated not only noise related errors, equal to or greater than those described here, but also systematic bias resulting in very poor precision in Ktrans at low values, particularly the presence of low SNR [29].

The signal enhancement ratio (Rmse1/mse2) is little affected by variation in tissue T10 values and has been shown to correlate closely with the redistribution rate constant (kep), a commonly analyzed PK parameter [31]. This makes Rmse1/mse2 a potentially attractive and very widely used metric, particularly in breast cancer studies [9], [10], [11], [12], [13], [32], [33], [34], [35], [36]. However, in the majority of these studies Rmse1/mse2 was measured on an averaged SI curve from a region of interest producing high SNR [12]. We have shown that single pixel mapping is associated with poor SNR and considerable heterogeneity in data quality. We demonstrated that Rmse1/mse22 has poor tolerance of low SNR, which is particularly severe when it is calculated from washout type curves. The outliers identified from both simulation and in vivo Rmse1/mse2 lead to high kurtosis and skewness in the PD distribution function. It is clear that these outliers should be carefully treated prior to statistical analysis. It is encouraging that, as shown in this study, the method using 95th percentile as a threshold for the removal of outliers has demonstrated efficacy in restoring the power of Rmse1/mse2, in detecting the therapy-induced changes and resulted in the demonstration of therapy induced changes in meningiomas which were not detected by other SQ or PK parameters. Use of an Rmse1/mse2 histogram based volume analysis is an alternative approach that is also less affected by outliers [11].

One of the major stated benefits of the PK analytical approach is the physiological specificity of the metrics. When DCE-MRI studies of antiangiogenic therapies became common, Ktrans was widely used as an indicator of changes in endothelial permeability. In fact it represents a compound metric affected by both blood flow and the product of the surface area and permeability of the capillary endothelial membrane. Nonetheless, the use of Ktrans as a biomarker effectively removes confounding effects due to variations in vp and ve. The statistical power of any given metric to detect significant change is dependent both on the change in mean/median value and, more importantly on the shape and width of the data distribution. In this study, the group coefficients of variation for SQ metrics in VS ranged from 18–24%. However, the CoV for Ktrans was considerably less both at day 0 and at 3 days of treatment (COV 8.3 and 11% respectively). This was associated with very high COV for vp (40 and 33% respectively). This narrow distribution resulted in improved statistical power for Ktrans compared to SQ metrics such that a 25% reduction in Ktrans at day 90 produced similar significance values to 37%, 37%, 42% and 43% decreases in MSE, MSErel, ∑MSE and ∑MSErel respectively. The reasons for this difference in COV is not clear, however it seems likely that the systematic removal of the contributions of intravascular contrast (vp) removed a significant source of variation from the estimated mean tumor values. Interestingly, the denoised signal enhancement ratio (Rmse1/mse2(DN)) also demonstrated low COV (13.7%, 8.0% and 6.4% at days 0,3 and 90) and showed treatment induced changes of similar significance to Ktrans despite a reduction of only 15.1% in mean values. In meningiomas pre-treatment Ktrans had the largest COV of any metric other than vp (40% and 60% respectively). Rmse1/mse2 (DN) showed the lowest COV (6.1%, 9.6%, and 3.6% at days 0, 3 and 90) and was the only metric to demonstrate significant therapy induced changes with a reduction in mean values of 14% compared for example to non-significant reduction in mean Kep of 13%. The Rmse1/mse2(DN) parameter is also designed to remove confounding numerical effects from estimates of the mean value. These results demonstrate the importance of variation in the mean estimated values of each parameter across the tumor population and favour the use of parameters where variation is minimized by choice of the appropriate metric, whether SQ or PK. Unfortunately the difference in behavior between meningiomas and VS shows that the potential variation in individual metrics may be tumor specific and cannot be predicted during trial design. These observations also support the suggestions of previous workers that analysis of DCE-MRI data should ideally be performed on a pixel by pixel basis [2], [3] and that the effects of spatial heterogeneity should be specifically included into the analytical approach [68].

In conclusion, this study begins to challenge the commonly expressed criticisms of SQ metrics for pixel-by-pixel parametric mapping and clinical trials. The SQ parameters in this study show relatively high tolerance to poor SNR compared to previous studies of PK metrics and demonstrated equivalent therapy induced changes. This paper is deliberately controversial and the majority of reviews and consensus workshop reports on DCE-MRI favor PK over SQ metrics for the reasons discussed above. However we believe that the results presented here demonstrates some clear advantages of SQ metrics and supports a re-evaluation of the utility of these metrics, which must include comparative studies of baseline reproducibility and multi-vendor/multicenter implementations.

Acknowledgments

This is a contribution from the CRUK-EPSRC Cancer Imaging Centre in Cambridge and Manchester. We thank Dr. Neil Thacker for discussions on error propagation, Dr. Sha Zhao at University of Manchester for discussions on calibration and QA, Dr. Daniel Balvay for discussion on identifying modeling errors.

Author Contributions

Conceived and designed the experiments: KL AJ XZ. Performed the experiments: AJ KL XZ. Analyzed the data: AJ KL XZ. Contributed reagents/materials/analysis tools: XZ KL. Wrote the paper: AJ KL XZ.

References

  1. 1. Evelhoch JL (1999) Key factors in the acquisition of contrast kinetic data for oncology. J Magn Reson Imaging 10: 254–259.
  2. 2. Leach MO, Brindle KM, Evelhoch JL, Griffiths JR, Horsman MR, et al. (2003) Assessment of antiangiogenic and antivascular therapeutics using MRI: recommendations for appropriate methodology for clinical trials. Br J Radiol 76 Suppl 1S87–91.
  3. 3. Leach MO, Brindle KM, Evelhoch JL, Griffiths JR, Horsman MR, et al. (2005) The assessment of antiangiogenic and antivascular therapies in early-stage clinical trials using magnetic resonance imaging: issues and recommendations. British journal of cancer 92: 1599–1610.
  4. 4. Leach MO, Morgan B, Tofts PS, Buckley DL, Huang W, et al. (2012) Imaging vascular function for early stage clinical trials using dynamic contrast-enhanced magnetic resonance imaging. European radiology 22: 1451–1464.
  5. 5. Engelbrecht MR, Huisman HJ, Laheij RJ, Jager GJ, van Leenders GJ, et al. (2003) Discrimination of prostate cancer from normal peripheral zone and central gland tissue by using dynamic contrast-enhanced MR imaging. Radiology 229: 248–254.
  6. 6. Padhani AR, Gapinski CJ, Macvicar DA, Parker GJ, Suckling J, et al. (2000) Dynamic contrast enhanced MRI of prostate cancer: correlation with morphology and tumour stage, histological grade and PSA. Clin Radiol 55: 99–109.
  7. 7. Rouviere O, Raudrant A, Ecochard R, Colin-Pangaud C, Pasquiou C, et al. (2003) Characterization of time-enhancement curves of benign and malignant prostate tissue at dynamic MR imaging. Eur Radiol 13: 931–942.
  8. 8. Arasu VA, Chen RC, Newitt DN, Chang CB, Tso H, et al. (2011) Can signal enhancement ratio (SER) reduce the number of recommended biopsies without affecting cancer yield in occult MRI-detected lesions? Acad Radiol 18: 716–721.
  9. 9. Jansen SA, Lin VC, Giger ML, Li H, Karczmar GS, et al. (2011) Normal parenchymal enhancement patterns in women undergoing MR screening of the breast. Eur Radiol 21: 1374–1382.
  10. 10. Jansen SA, Shimauchi A, Zak L, Fan X, Karczmar GS, et al. (2011) The diverse pathology and kinetics of mass, nonmass, and focus enhancement on MR imaging of the breast. J Magn Reson Imaging 33: 1382–1389.
  11. 11. Li KL, Partridge SC, Joe BN, Gibbs JE, Lu Y, et al. (2008) Invasive breast cancer: predicting disease recurrence by using high-spatial-resolution signal enhancement ratio imaging. Radiology 248: 79–87.
  12. 12. Partridge SC, Vanantwerp RK, Doot RK, Chai X, Kurland BF, et al. (2010) Association between serial dynamic contrast-enhanced MRI and dynamic 18F-FDG PET measures in patients undergoing neoadjuvant chemotherapy for locally advanced breast cancer. J Magn Reson Imaging 32: 1124–1131.
  13. 13. Hattangadi J, Park C, Rembert J, Klifa C, Hwang J, et al. (2008) Breast stromal enhancement on MRI is associated with response to neoadjuvant chemotherapy. AJR Am J Roentgenol 190: 1630–1636.
  14. 14. Gribbestad IS, Nilsen G, Fjosne HE, Kvinnsland S, Haugen OA, et al. (1994) Comparative signal intensity measurements in dynamic gadolinium-enhanced MR mammography. JMagn ResonImaging 4: 477–480.
  15. 15. Williams TC, DeMartini WB, Partridge SC, Peacock S, Lehman CD (2007) Breast MR imaging: computer-aided evaluation program for discriminating benign from malignant lesions. Radiology 244: 94–103.
  16. 16. Flickinger FW, Allison JD, Sherry RM, Wright JC (1990) Differentiation of benign from malignant breast masses by time-intensity evaluation of contrast enhanced MRI. Breast disease: tissue characterization with Gd-DTPA enhancement profiles. Radiology 174: 491–494.
  17. 17. Su MY, Cheung YC, Fruehauf JP, Yu H, Nalcioglu O, et al. (2003) Correlation of dynamic contrast enhancement MRI parameters with microvessel density and VEGF for assessment of angiogenesis in breast cancer. J Magn Reson Imaging 18: 467–477.
  18. 18. Brown J, Buckley D, Coulthard A, Dixon AK, Dixon JM, et al. (2000) Magnetic resonance imaging screening in women at genetic risk of breast cancer: imaging and analysis protocol for the UK multicentre study. UK MRI Breast Screening Study Advisory Group. Magn Reson Imaging 18: 765–776.
  19. 19. Kuhl CK, Mielcareck P, Klaschik S, Leutner C, Wardelmann E, et al. (1999) Dynamic breast MR imaging: are signal intensity time course data useful for differential diagnosis of enhancing lesions? Radiology 211: 101–110.
  20. 20. Akisik MF, Sandrasegaran K, Bu G, Lin C, Hutchins GD, et al. (2010) Pancreatic cancer: utility of dynamic contrast-enhanced MR imaging in assessment of antiangiogenic therapy. Radiology 256: 441–449.
  21. 21. Zahra MA, Tan LT, Priest AN, Graves MJ, Arends M, et al. (2009) Semiquantitative and quantitative dynamic contrast-enhanced magnetic resonance imaging measurements predict radiation response in cervix cancer. Int J Radiat Oncol Biol Phys 74: 766–773.
  22. 22. Florie J, Wasser MN, Arts-Cieslik K, Akkerman EM, Siersema PD, et al. (2006) Dynamic contrast-enhanced MRI of the bowel wall for assessment of disease activity in Crohn's disease. AJR Am J Roentgenol 186: 1384–1392.
  23. 23. de Lussanet QG, Backes WH, Griffioen AW, Padhani AR, Baeten CI, et al. (2005) Dynamic contrast-enhanced magnetic resonance imaging of radiation therapy-induced microcirculation changes in rectal cancer. Int J Radiat Oncol Biol Phys 63: 1309–1315.
  24. 24. Zhang XM, Yu D, Zhang HL, Dai Y, Bi D, et al. (2008) 3D dynamic contrast-enhanced MRI of rectal carcinoma at 3T: correlation with microvascular density and vascular endothelial growth factor markers of tumor angiogenesis. J Magn Reson Imaging 27: 1309–1316.
  25. 25. Dyke JP, Panicek DM, Healey JH, Meyers PA, Huvos AG, et al. (2003) Osteogenic and Ewing sarcomas: estimation of necrotic fraction during induction chemotherapy with dynamic contrast-enhanced MR imaging. Radiology 228: 271–278.
  26. 26. Narang J, Jain R, Arbab AS, Mikkelsen T, Scarpace L, et al. (2011) Differentiating treatment-induced necrosis from recurrent/progressive brain tumor using nonmodel-based semiquantitative indices derived from dynamic contrast-enhanced T1-weighted MR perfusion. Neuro Oncol 13: 1037–1046.
  27. 27. Lavini C, Verhoeff JJ, Majoie CB, Stalpers LJ, Richel DJ, et al. (2011) Model-based, semiquantitative and time intensity curve shape analysis of dynamic contrast-enhanced MRI: a comparison in patients undergoing antiangiogenic treatment for recurrent glioma. J Magn Reson Imaging 34: 1303–1312.
  28. 28. Tofts PS, Collins DJ (2011) Multicentre imaging measurements for oncology and in the brain. The British journal of radiology 84 Spec No 2: S213–226.
  29. 29. Li KL, Jackson A (2003) New hybrid technique for accurate and reproducible quantitation of dynamic contrast-enhanced MRI data. Magn Reson Med 50: 1286–1295.
  30. 30. Buckley DL (2002) Uncertainty in the analysis of tracer kinetics using dynamic contrast-enhanced T(1)-weighted MRI. Magn ResonMed 47: 601–606.
  31. 31. Li KL, Henry RG, Wilmes LJ, Gibbs J, Zhu X, et al. (2007) Kinetic assessment of breast tumors using high spatial resolution signal enhancement ratio (SER) imaging. Magn Reson Med 58: 572–581.
  32. 32. Esserman L, Hylton N, George T, Weidner N (1999) Contrast-enhanced magnetic resonance imaging to assess tumor histopathology and angiogenesis in breast carcinoma. Breast J 5: 13–21.
  33. 33. Furman-Haran E, Degani H (2002) Parametric analysis of breast MRI. J Comput Assist Tomogr 26: 376–386.
  34. 34. Bone B, Szabo BK, Perbeck LG, Veress B, Aspelin P (2003) Can contrast-enhanced MR imaging predict survival in breast cancer? Acta Radiol 44: 373–378.
  35. 35. Levman J, Leung T, Causer P, Plewes D, Martel AL (2008) Classification of dynamic contrast-enhanced magnetic resonance breast lesions by support vector machines. IEEE Trans Med Imaging 27: 688–696.
  36. 36. Jansen SA, Newstead GM, Abe H, Shimauchi A, Schmidt RA, et al. (2007) Pure ductal carcinoma in situ: kinetic and morphologic MR characteristics compared with mammographic appearance and nuclear grade. Radiology 245: 684–691.
  37. 37. Evelhoch J, Garwood M, Vigneron D, Knopp M, Sullivan D, et al. (2005) Expanding the use of magnetic resonance in the assessment of tumor response to therapy: workshop report. Cancer Res 65: 7041–7044.
  38. 38. Li KL, Buonaccorsi G, Thompson G, Cain JR, Watkins A, et al. (2012) An improved coverage and spatial resolution—using dual injection dynamic contrast-enhanced (ICE-DICE) MRI: a novel dynamic contrast-enhanced technique for cerebral tumors. Magn Reson Med 68: 452–462.
  39. 39. Kaiser WA, Zeitler E (1989) MR imaging of the breast: fast imaging sequences with and without Gd-DTPA. Preliminary observations. Radiology 170: 681–686.
  40. 40. Flickinger FW, Allison JD, Sherry RM, Wright JC (1993) Differentiation of benign from malignant breast masses by time-intensity evaluation of contrast enhanced MRI. Magn ResonImaging 11: 617–620.
  41. 41. Ikeda O, Yamashita Y, Morishita S, Kido T, Kitajima M, et al. (1999) Characterization of breast masses by dynamic enhanced MR imaging. A logistic regression analysis. Acta Radiol 40: 585–592.
  42. 42. Karahaliou A, Vassiou K, Arikidis NS, Skiadopoulos S, Kanavou T, et al. Assessing heterogeneity of lesion enhancement kinetics in dynamic contrast-enhanced MRI for breast cancer diagnosis. Br J Radiol 83: 296–309.
  43. 43. Tofts PS (1997) Modeling tracer kinetics in dynamic Gd-DTPA MR imaging. J Magn Reson Imaging 7: 91–101.
  44. 44. Fritz-Hansen T, Rostrup E, Sondergaard L, Ring PB, Amtorp O, et al. (1998) Capillary transfer constant of Gd-DTPA in the myocardium at rest and during vasodilation assessed by MRI. Magn Reson Med 40: 922–929.
  45. 45. Zhu XP, Li KL, Kamaly-Asl ID, Checkley DR, Tessier JJ, et al. (2000) Quantification of endothelial permeability, leakage space, and blood volume in brain tumors using combined T1 and T2* contrast-enhanced dynamic MR imaging. J Magn Reson Imaging 11: 575–585.
  46. 46. Schabel MC, Parker DL (2008) Uncertainty and bias in contrast concentration measurements using spoiled gradient echo pulse sequences. Phys Med Biol 53: 2345–2373.
  47. 47. Balvay D, Frouin F, Calmon G, Bessoud B, Kahn E, et al. (2005) New criteria for assessing fit quality in dynamic contrast-enhanced T1-weighted MRI for perfusion and permeability imaging. Magn Reson Med 54: 868–877.
  48. 48. Vokurka EA, Herwadkar A, Thacker NA, Ramsden RT, Jackson A (2002) Using Bayesian Tissue Classification to Improve the Accuracy of Vestibular Schwannoma Volume and Growth Measurement. AJNR AmJ Neuroradiol 23: 459–467.
  49. 49. Millard S, Neerchal N (2000) Environmental Statistics with S-PLUS: CRC Press.
    • 50. O'Connor JP, Jackson A, Parker GJ, Roberts C, Jayson GC (2012) Dynamic contrast-enhanced MRI in clinical trials of antivascular therapies. Nat Rev Clin Oncol 9: 167–177.
    • 51. O'Connor JP, Tofts PS, Miles KA, Parkes LM, Thompson G, et al. (2011) Dynamic contrast-enhanced imaging techniques: CT and MRI. Br J Radiol 84 Spec No 2: S112–120.
    • 52. Tofts PS, Collins DJ (2011) Multicentre imaging measurements for oncology and in the brain. Br J Radiol 84 Spec No 2: S213–226.
    • 53. Evelhoch JL, Brown T, Chenevert T, Clarke L, Daniel B, et al.. (2000) Consensus recommendation for acquisition of dynamic contrtast-enhanced MRI data in oncology. ISMRM, eighth scientific meeting, Denver: 1439.
      • 54. Waterton JC, Pylkkanen L (2012) Qualification of imaging biomarkers for oncology drug development. Eur J Cancer 48: 409–415.
      • 55. Buckler AJ, Bresolin L, Dunnick NR, Sullivan DC, Aerts HJ, et al. (2011) Quantitative imaging test approval and biomarker qualification: interrelated but distinct activities. Radiology 259: 875–884.
      • 56. Sourbron SP, Buckley DL (2011) On the scope and interpretation of the Tofts models for DCE-MRI. Magn Reson Med 66: 735–745.
      • 57. El Khouli RH, Macura KJ, Jacobs MA, Khalil TH, Kamel IR, et al. (2009) Dynamic contrast-enhanced MRI of the breast: quantitative method for kinetic curve type assessment. AJR Am J Roentgenol 193: W295–300.
      • 58. Warren R, Hayes C, Pointon L, Hoff R, Gilbert FJ, et al. (2006) A test of performance of breast MRI interpretation in a multicentre screening study. Magn Reson Imaging 24: 917–929.
      • 59. Koh TS, Thng CH, Hartono S, Kwek JW, Khoo JB, et al. (2011) Dynamic contrast-enhanced MRI of neuroendocrine hepatic metastases: A feasibility study using a dual-input two-compartment model. Magn Reson Med 65: 250–260.
      • 60. Lavini C, Pikaart BP, de Jonge MC, Schaap GR, Maas M (2009) Region of interest and pixel-by-pixel analysis of dynamic contrast enhanced magnetic resonance imaging parameters and time-intensity curve shapes: a comparison in chondroid tumors. Magn Reson Imaging 27: 62–68.
      • 61. Tuncbilek N, Karakas HM, Altaner S (2004) Dynamic MRI in indirect estimation of microvessel density, histologic grade, and prognosis in colorectal adenocarcinomas. Abdom Imaging 29: 166–172.
      • 62. Zelhof B, Lowry M, Rodrigues G, Kraus S, Turnbull L (2009) Description of magnetic resonance imaging-derived enhancement variables in pathologically confirmed prostate cancer and normal peripheral zone regions. BJU Int 104: 621–627.
      • 63. Buadu LD, Murakami J, Murayama S, Hashiguchi N, Sakai S, et al. (1996) Breast lesions: correlation of contrast medium enhancement patterns on MR images with histopathologic findings and tumor angiogenesis. Radiology 200: 639–649.
      • 64. Kuhl CK, Schild HH (2000) Dynamic image interpretation of MRI of the breast. J Magn Reson Imaging 12: 965–974.
      • 65. Bolouri MS, Elias SG, Wisner DJ, Behr SC, Hawkins RA, et al. (2013) Triple-negative and non-triple-negative invasive breast cancer: association between MR and fluorine 18 fluorodeoxyglucose PET imaging. Radiology 269: 354–361.
      • 66. Galbraith SM, Lodge MA, Taylor NJ, Rustin GJ, Bentzen S, et al. (2002) Reproducibility of dynamic contrast-enhanced MRI in human muscle and tumours: comparison of quantitative and semi-quantitative analysis. NMR Biomed 15: 132–142.
      • 67. Ferl GZ, Xu L, Friesenhahn M, Bernstein LJ, Barboriak DP, et al. (2010) An automated method for nonparametric kinetic analysis of clinical DCE-MRI data: application to glioblastoma treated with bevacizumab. Magn Reson Med 63: 1366–1375.
      • 68. Asselin MC, O'Connor JP, Boellaard R, Thacker NA, Jackson A (2012) Quantifying heterogeneity in human tumours using MRI and PET. Eur J Cancer 48: 447–455.