Introduction
-
Evaluating the agreement between automated and quality-controlled, manual segmentation of articular cartilage as “ground truth”.
-
Testing the accuracy (correlations and systematic offsets) of quantitative cartilage morphometry measures (thickness, volume, surface areas) derived from automated segmentations compared to manual segmentation.
-
Analysis of the longitudinal test–retest reproducibility of quantitative cartilage measures derived from automated vs. manual segmentation over a 1-year period (using year-1 and -2 follow-up data).
-
Comparison of the agreement, accuracy, and longitudinal test–retest reproducibility of the automated segmentation (and quantitative cartilage measures derived therefrom) between two different MRI sequences with different contrasts and orientations.
Materials and methods
Participants and MR imaging
Manual segmentation
Automated, U-Net-based segmentation
Automated post-processing
-
Filling of small gaps by detecting enclosed, unsegmented areas
-
Removal of segmentations in slices not connected to the segmentation in the same or other slices (i.e., segmentations at implausible locations)
-
Removal of spikes (smoothing)
-
Removal of femoral cartilage segmentations outside the (60%) femoral ROI
-
Separation of segmentations into the subchondral bone area (tAB), articular cartilage surface area (AC), and inner cartilage (IC), which was required for the computation of morphological parameters (see below).
Statistical analysis
Results
Training set (n = 50) | Validation set (n = 21) | Test set (n = 21) | ||||
---|---|---|---|---|---|---|
N/Mean | %/SD | N/Mean | %/SD | N/Mean | %/SD | |
Sex | ||||||
Women | 29 | 58 | 13 | 62 | 13 | 62 |
Men | 21 | 42 | 8 | 38 | 8 | 38 |
Age (years) | 54.1 | 7.3 | 53.9 | 8.0 | 56.9 | 7.5 |
BMI (kg/m2) | 24.1 | 3.0 | 24.5 | 3.2 | 25.1 | 3.3 |
Height (m) | 1.68 | 0.09 | 1.67 | 0.09 | 1.67 | 0.09 |
Agreement of the automated U-Net segmentation with manual segmentation
DSC | VOE (%) | HD (mm) | ASSD (mm) | |||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
corFLASH | sagDESS | corFLASH | sagDESS | corFLASH | sagDESS | corFLASH | sagDESS | |||||||||
Mean | SD | Mean | SD | Mean | SD | Mean | SD | Mean | SD | Mean | SD | Mean | SD | Mean | SD | |
Main run: before post-processing | ||||||||||||||||
MT | 0.92 | 0.02 | 0.91 | 0.02 | 15.5 | 3.0 | 17.1 | 2.7 | 8.27 | 13.32 | 2.79 | 1.08 | 0.15 | 0.04 | 0.13 | 0.03 |
cMF | 0.88 | 0.03 | 0.89 | 0.03 | 21.9 | 4.8 | 20.0 | 4.3 | 5.72 | 11.02 | 6.14 | 11.24 | 0.28 | 0.13 | 0.18 | 0.07 |
LT | 0.92 | 0.02 | 0.92 | 0.02 | 14.9 | 3.3 | 15.4 | 2.9 | 8.06 | 8.80 | 5.31 | 8.46 | 0.17 | 0.05 | 0.17 | 0.04 |
cLF | 0.88 | 0.02 | 0.90 | 0.02 | 20.8 | 3.8 | 17.8 | 2.8 | 3.66 | 2.33 | 3.86 | 5.54 | 0.26 | 0.09 | 0.14 | 0.03 |
Main run: after post-processing | ||||||||||||||||
MT | 0.92 | 0.02 | 0.91 | 0.02 | 15.5 | 3.0 | 17.0 | 2.7 | 2.28 | 0.67 | 2.39 | 0.47 | 0.14 | 0.03 | 0.13 | 0.03 |
cMF | 0.91 | 0.03 | 0.89 | 0.03 | 16.4 | 4.6 | 20.1 | 4.4 | 2.60 | 1.21 | 3.36 | 0.72 | 0.13 | 0.08 | 0.17 | 0.06 |
LT | 0.92 | 0.02 | 0.92 | 0.02 | 14.9 | 3.3 | 15.4 | 2.9 | 3.00 | 1.09 | 3.21 | 0.92 | 0.16 | 0.04 | 0.17 | 0.04 |
cLF | 0.91 | 0.02 | 0.90 | 0.02 | 15.8 | 4.0 | 17.9 | 2.8 | 2.08 | 0.56 | 2.65 | 0.55 | 0.13 | 0.06 | 0.14 | 0.03 |
Repeated run: before post-processing | ||||||||||||||||
MT | 0.92 | 0.02 | 0.91 | 0.02 | 15.3 | 2.7 | 16.4 | 2.8 | 6.38 | 8.39 | 2.52 | 0.78 | 0.14 | 0.03 | 0.13 | 0.03 |
cMF | 0.84 | 0.03 | 0.89 | 0.03 | 26.8 | 5.2 | 20.0 | 4.5 | 8.70 | 10.12 | 8.21 | 11.86 | 0.44 | 0.21 | 0.18 | 0.09 |
LT | 0.92 | 0.02 | 0.92 | 0.02 | 14.8 | 3.4 | 15.0 | 2.8 | 5.64 | 7.24 | 3.00 | 1.00 | 0.15 | 0.04 | 0.16 | 0.03 |
cLF | 0.88 | 0.03 | 0.91 | 0.02 | 21.2 | 4.7 | 17.0 | 2.6 | 5.63 | 8.64 | 7.15 | 12.77 | 0.25 | 0.12 | 0.14 | 0.05 |
Repeated run: after post-processing | ||||||||||||||||
MT | 0.92 | 0.02 | 0.91 | 0.02 | 15.2 | 2.7 | 16.4 | 2.8 | 2.21 | 0.68 | 2.30 | 0.62 | 0.14 | 0.03 | 0.13 | 0.03 |
cMF | 0.91 | 0.02 | 0.89 | 0.03 | 17.0 | 3.9 | 20.2 | 4.6 | 2.50 | 1.00 | 3.40 | 0.75 | 0.14 | 0.06 | 0.18 | 0.07 |
LT | 0.92 | 0.02 | 0.92 | 0.02 | 14.7 | 3.4 | 15.0 | 2.8 | 3.02 | 0.93 | 3.00 | 1.00 | 0.15 | 0.04 | 0.17 | 0.03 |
cLF | 0.90 | 0.03 | 0.91 | 0.02 | 17.5 | 4.8 | 17.1 | 2.5 | 2.18 | 0.48 | 2.42 | 0.49 | 0.16 | 0.08 | 0.13 | 0.02 |
Accuracy of cartilage morphometry using automated, U-Net vs. manual segmentation
Manual | U-Net | Manual vs. U-Net | ||||||
---|---|---|---|---|---|---|---|---|
Mean | SD | Mean | SD | Diff (%) | P | r | ||
Cartilage thickness (mm) | ||||||||
MFTC | corFLASH | 3.6 | 0.4 | 3.8 | 0.4 | 5.1 | < 0.001 | 0.96 |
sagDESS | 3.4 | 0.5 | 3.5 | 0.4 | 2.8 | 0.019 | 0.95 | |
MT | corFLASH | 1.7 | 0.2 | 1.8 | 0.2 | 4.7 | < 0.001 | 0.97 |
sagDESS | 1.6 | 0.2 | 1.7 | 0.2 | 3.7 | 0.006 | 0.93 | |
cMF | corFLASH | 1.8 | 0.3 | 1.9 | 0.2 | 5.5 | < 0.001 | 0.93 |
sagDESS | 1.8 | 0.3 | 1.8 | 0.2 | 1.9 | 0.224 | 0.92 | |
LFTC | corFLASH | 3.8 | 0.5 | 4.0 | 0.5 | 4.4 | < 0.001 | 0.98 |
sagDESS | 3.8 | 0.5 | 3.9 | 0.5 | 4.1 | < 0.001 | 0.97 | |
LT | corFLASH | 2.1 | 0.3 | 2.2 | 0.3 | 4.6 | < 0.001 | 0.97 |
sagDESS | 2.0 | 0.3 | 2.1 | 0.3 | 3.5 | < 0.001 | 0.97 | |
cLF | corFLASH | 1.7 | 0.3 | 1.8 | 0.2 | 4.2 | < 0.001 | 0.97 |
sagDESS | 1.7 | 0.3 | 1.8 | 0.2 | 4.8 | 0.001 | 0.96 | |
Cartilage volume (mm3) | ||||||||
MFTC | corFLASH | 2947 | 829 | 3218 | 797 | 9.2 | < 0.001 | 0.99 |
sagDESS | 2801 | 794 | 2912 | 737 | 4.0 | 0.005 | 0.98 | |
MT | corFLASH | 1945 | 570 | 2120 | 534 | 9.0 | < 0.001 | 0.99 |
sagDESS | 1703 | 532 | 1848 | 496 | 8.5 | < 0.001 | 0.98 | |
cMF | corFLASH | 1003 | 279 | 1099 | 277 | 9.6 | < 0.001 | 0.96 |
sagDESS | 1098 | 285 | 1064 | 256 | -3.1 | 0.154 | 0.93 | |
LFTC | corFLASH | 3123 | 859 | 3421 | 904 | 9.6 | < 0.001 | 0.99 |
sagDESS | 3291 | 928 | 3433 | 872 | 4.3 | < 0.001 | 0.99 | |
LT | corFLASH | 2083 | 589 | 2294 | 622 | 10.1 | < 0.001 | 0.99 |
sagDESS | 2144 | 646 | 2295 | 618 | 7.0 | < 0.001 | 0.99 | |
cLF | corFLASH | 1039 | 298 | 1128 | 302 | 8.5 | < 0.001 | 0.98 |
sagDESS | 1147 | 311 | 1138 | 264 | -0.8 | 0.623 | 0.97 | |
Total area of subchondral bone (cm2) | ||||||||
MFTC | corFLASH | 16.1 | 2.8 | 16.3 | 2.8 | 1.5 | 0.040 | 0.98 |
sagDESS | 15.3 | 2.7 | 15.7 | 2.7 | 2.5 | < 0.001 | 0.99 | |
MT | corFLASH | 11.0 | 1.9 | 11.2 | 1.8 | 1.7 | 0.027 | 0.98 |
sagDESS | 10.0 | 1.7 | 10.4 | 1.8 | 3.8 | < 0.001 | 0.99 | |
cMF | corFLASH | 5.1 | 1.0 | 5.2 | 1.0 | 1.1 | 0.395 | 0.96 |
sagDESS | 5.3 | 1.0 | 5.3 | 1.0 | 0.0 | 0.999 | 0.95 | |
LFTC | corFLASH | 15.2 | 2.6 | 15.5 | 2.7 | 1.9 | 0.015 | 0.98 |
sagDESS | 15.6 | 2.8 | 15.8 | 2.7 | 1.6 | 0.024 | 0.99 | |
LT | corFLASH | 9.4 | 1.6 | 9.7 | 1.7 | 2.9 | 0.010 | 0.96 |
sagDESS | 9.9 | 1.7 | 10.1 | 1.7 | 2.2 | 0.018 | 0.98 | |
cLF | corFLASH | 5.8 | 1.1 | 5.8 | 1.1 | 0.1 | 0.904 | 0.98 |
sagDESS | 5.7 | 1.1 | 5.7 | 1.0 | 0.5 | 0.485 | 0.99 |
DSC | VOE | HD | ASSD | |
---|---|---|---|---|
corFLASH | ||||
MT | − 0.61 | 0.61 | − 0.15 | 0.44 |
cMF | − 0.49 | 0.50 | − 0.17 | 0.26 |
LT | − 0.50 | 0.50 | − 0.29 | 0.38 |
cLF | − 0.32 | 0.32 | − 0.19 | 0.14 |
sagDESS | ||||
MT | − 0.09 | 0.10 | − 0.02 | − 0.01 |
cMF | − 0.42 | 0.43 | 0.10 | 0.33 |
LT | − 0.58 | 0.58 | − 0.35 | 0.00 |
cLF | − 0.42 | 0.42 | − 0.38 | 0.05 |
Longitudinal test–retest reproducibility
Manual | U-Net | |||||||||
---|---|---|---|---|---|---|---|---|---|---|
MC ± SD | MC (%) | P | RMS SD | RMS CV% | MC ± SD | MC (%) | P | RMS SD | RMS CV% | |
corFLASH | ||||||||||
Cartilage thickness (mm) | ||||||||||
MFTC | 0.00 ± 0.06 | 0.1 | 0.76 | 0.04 | 1.2 | 0.00 ± 0.05 | 0.0 | 0.93 | 0.04 | 1.0 |
MT | 0.00 ± 0.04 | 0.0 | 0.96 | 0.02 | 1.5 | 0.01 ± 0.03 | 0.3 | 0.41 | 0.02 | 1.1 |
cMF | 0.00 ± 0.04 | 0.2 | 0.67 | 0.03 | 1.5 | 0.00 ± 0.04 | − 0.2 | 0.62 | 0.03 | 1.4 |
LFTC | 0.01 ± 0.07 | 0.2 | 0.71 | 0.05 | 1.3 | 0.01 ± 0.09 | 0.2 | 0.64 | 0.06 | 1.6 |
LT | 0.00 ± 0.04 | − 0.2 | 0.72 | 0.03 | 1.3 | 0.01 ± 0.07 | 0.3 | 0.65 | 0.05 | 2.1 |
cLF | 0.01 ± 0.04 | 0.5 | 0.38 | 0.03 | 1.9 | 0.00 ± 0.05 | 0.2 | 0.81 | 0.04 | 2.1 |
Cartilage volume (mm3) | ||||||||||
MFTC | 2.6 ± 63.4 | 0.1 | 0.85 | 43.8 | 1.5 | − 0.7 ± 79.3 | 0.0 | 0.97 | 54.7 | 1.7 |
MT | − 2.3 ± 51.6 | − 0.1 | 0.84 | 35.6 | 1.9 | 5.5 ± 66.7 | 0.3 | 0.71 | 46.2 | 2.2 |
cMF | 4.9 ± 30.2 | 0.5 | 0.47 | 21.1 | 2.2 | − 6.3 ± 39.5 | − 0.6 | 0.47 | 27.6 | 2.6 |
LFTC | − 9.2 ± 69.5 | − 0.3 | 0.55 | 48.4 | 1.6 | − 41.8 ± 108.1 | − 1.2 | 0.09 | 80.2 | 2.4 |
LT | − 13.5 ± 58.9 | − 0.6 | 0.31 | 41.8 | 2.0 | − 33.1 ± 80.2 | − 1.5 | 0.07 | 60.1 | 2.7 |
cLF | 4.3 ± 30.4 | 0.4 | 0.52 | 21.2 | 2.1 | − 8.6 ± 55.9 | − 0.8 | 0.49 | 39.1 | 3.5 |
Total area of subchondral bone (cm2) | ||||||||||
MFTC | − 0.01 ± 0.12 | − 0.1 | 0.66 | 0.08 | 0.5 | − 0.03 ± 0.31 | − 0.2 | 0.66 | 0.22 | 1.3 |
MT | − 0.03 ± 0.10 | − 0.3 | 0.20 | 0.07 | 0.6 | 0.00 ± 0.23 | 0.0 | 1.00 | 0.16 | 1.4 |
cMF | 0.02 ± 0.08 | 0.3 | 0.37 | 0.06 | 1.1 | − 0.03 ± 0.17 | − 0.6 | 0.43 | 0.12 | 2.4 |
LFTC | − 0.03 ± 0.12 | − 0.2 | 0.33 | 0.09 | 0.6 | − 0.17 ± 0.45 | − 1.1 | 0.09 | 0.33 | 2.2 |
LT | − 0.02 ± 0.10 | − 0.3 | 0.30 | 0.07 | 0.8 | − 0.10 ± 0.38 | − 1.0 | 0.25 | 0.27 | 2.8 |
cLF | 0.00 ± 0.07 | − 0.1 | 0.86 | 0.05 | 0.9 | − 0.08 ± 0.24 | − 1.3 | 0.16 | 0.17 | 3.0 |
sagDESS | ||||||||||
Cartilage thickness (mm) | ||||||||||
MFTC | 0.03 ± 0.05 | 0.8 | 0.01 | 0.04 | 1.2 | 0.01 ± 0.07 | 0.3 | 0.47 | 0.05 | 1.4 |
MT | 0.02 ± 0.03 | 1.1 | 0.02 | 0.03 | 1.5 | 0.02 ± 0.04 | 1.0 | 0.05 | 0.03 | 1.7 |
cMF | 0.01 ± 0.04 | 0.6 | 0.22 | 0.03 | 1.7 | − 0.01 ± 0.05 | − 0.3 | 0.63 | 0.04 | 2.1 |
LFTC | 0.01 ± 0.07 | 0.4 | 0.39 | 0.05 | 1.4 | − 0.02 ± 0.07 | − 0.4 | 0.33 | 0.05 | 1.3 |
LT | 0.02 ± 0.06 | 0.8 | 0.24 | 0.04 | 2.0 | − 0.01 ± 0.05 | − 0.4 | 0.38 | 0.03 | 1.5 |
cLF | 0.00 ± 0.04 | − 0.1 | 0.90 | 0.03 | 1.6 | − 0.01 ± 0.06 | − 0.4 | 0.61 | 0.04 | 2.2 |
Cartilage volume (mm3) | ||||||||||
MFTC | 23.9 ± 63.8 | 0.9 | 0.10 | 47.1 | 1.7 | − 4.0 ± 79.5 | − 0.1 | 0.82 | 54.9 | 1.9 |
MT | 22.9 ± 44.5 | 1.3 | 0.03 | 34.7 | 2.0 | 3.3 ± 55.6 | 0.2 | 0.79 | 38.4 | 2.1 |
cMF | 1.1 ± 34.3 | 0.1 | 0.89 | 23.7 | 2.2 | − 7.3 ± 47.3 | − 0.7 | 0.49 | 33.0 | 3.3 |
LFTC | 34.3 ± 83.2 | 1.0 | 0.07 | 62.3 | 1.9 | − 29.3 ± 123.5 | − 0.9 | 0.29 | 87.7 | 2.7 |
LT | 29.0 ± 70.6 | 1.4 | 0.07 | 52.9 | 2.5 | − 5.4 ± 84.8 | − 0.2 | 0.77 | 58.7 | 2.6 |
cLF | 5.3 ± 28.8 | 0.5 | 0.41 | 20.2 | 1.7 | − 23.9 ± 63.0 | − 2.3 | 0.10 | 46.6 | 4.5 |
Total area of subchondral bone (cm2) | ||||||||||
MFTC | − 0.02 ± 0.24 | − 0.1 | 0.77 | 0.16 | 1.1 | − 0.13 ± 0.41 | − 0.8 | 0.17 | 0.30 | 1.9 |
MT | 0.04 ± 0.17 | 0.4 | 0.36 | 0.12 | 1.2 | − 0.09 ± 0.30 | − 0.8 | 0.21 | 0.22 | 2.1 |
cMF | − 0.05 ± 0.10 | − 1.0 | 0.03 | 0.08 | 1.4 | − 0.04 ± 0.27 | − 0.8 | 0.48 | 0.19 | 3.6 |
LFTC | 0.06 ± 0.17 | 0.4 | 0.10 | 0.13 | 0.8 | − 0.12 ± 0.45 | − 0.8 | 0.24 | 0.32 | 2.0 |
LT | 0.05 ± 0.13 | 0.5 | 0.08 | 0.09 | 0.9 | 0.00 ± 0.27 | 0.0 | 0.96 | 0.19 | 1.9 |
cLF | 0.02 ± 0.12 | 0.3 | 0.56 | 0.08 | 1.4 | − 0.12 ± 0.25 | − 2.1 | 0.04 | 0.19 | 3.4 |
corFLASH | sagDESS | |||||||
---|---|---|---|---|---|---|---|---|
Manual | U-Net | Manual | U-Net | |||||
SEM | SDC | SEM | SDC | SEM | SDC | SEM | SDC | |
Cartilage thickness (mm) | ||||||||
MFTC | 0.09 | 0.24 | 0.07 | 0.21 | 0.07 | 0.20 | 0.10 | 0.28 |
LFTC | 0.10 | 0.27 | 0.13 | 0.36 | 0.10 | 0.29 | 0.10 | 0.28 |
MT | 0.05 | 0.14 | 0.04 | 0.11 | 0.05 | 0.13 | 0.05 | 0.15 |
cMF | 0.06 | 0.16 | 0.05 | 0.15 | 0.06 | 0.16 | 0.08 | 0.21 |
LT | 0.06 | 0.15 | 0.09 | 0.26 | 0.08 | 0.23 | 0.06 | 0.18 |
cLF | 0.06 | 0.17 | 0.08 | 0.21 | 0.06 | 0.16 | 0.08 | 0.22 |
Cartilage volume (mm3) | ||||||||
MFTC | 90 | 249 | 112 | 311 | 90 | 250 | 112 | 312 |
LFTC | 98 | 273 | 153 | 424 | 118 | 326 | 175 | 484 |
MT | 73 | 202 | 94 | 261 | 63 | 175 | 79 | 218 |
cMF | 43 | 118 | 56 | 155 | 49 | 135 | 67 | 185 |
LT | 83 | 231 | 113 | 314 | 100 | 277 | 120 | 332 |
cLF | 43 | 119 | 79 | 219 | 41 | 113 | 89 | 247 |
Total area of subchondral bone (cm2) | ||||||||
MFTC | 0.17 | 0.46 | 0.44 | 1.22 | 0.34 | 0.93 | 0.59 | 1.63 |
LFTC | 0.18 | 0.49 | 0.64 | 1.77 | 0.25 | 0.68 | 0.63 | 1.75 |
MT | 0.13 | 0.37 | 0.32 | 0.89 | 0.25 | 0.68 | 0.42 | 1.17 |
cMF | 0.12 | 0.32 | 0.25 | 0.68 | 0.14 | 0.38 | 0.38 | 1.06 |
LT | 0.15 | 0.41 | 0.53 | 1.47 | 0.18 | 0.49 | 0.39 | 1.08 |
cLF | 0.10 | 0.29 | 0.33 | 0.93 | 0.16 | 0.45 | 0.36 | 0.99 |