Cone beam computed tomography in the assessment of TMJ deformity in children with JIA: repeatability of a novel scoring system

The CBCT examinations took place at the Regional Competence Centres for Oral Health. They were conducted by experienced radiographers with the participants positioned in the Frankfort plane horizontal with their teeth in maximal intercuspal position. CBCT model and machine settings are outlined in Table 1.

Prior to scoring calibration between readers was performed during a number of face-to-face and online meetings to address volume reorientation, identification of landmarks and multiplanar reconstructions. Further, imaging features and their grading were carefully discussed and standardised based on both single images and complete examinations with a particular focus on discrimination between categories. Based on previous literature five measurements (glenoid fossa depth and length, glenoid fossa/articular eminence inclination angle, and condyle length and width) and 20 imaging features describing anatomy and deformity (overall impression of TMJ deformity, condyle volume and position, joint surface continuity, irregularity and flattening of the condyle and glenoid fossa/articular eminence, apposition, ankylosis, heterotopic bone formation, loose joint body and findings suggestive of osteoarthritis) were identified for the present study [6, 26, 35‐42]. Definitions of volume reorientation, measurements and variables are given in Figs. 1 and 2 and Table 2, respectively.

The examinations were read independently by two radiologists and one dento-maxillofacial radiologist (TAA, OA and CX, with 13, 12 and 14 years of experience in imaging, respectively). The examinations were anonymised for all information except scan date and study site. Image viewing conditions were standardised (diagnostic screens, ambient lighting etc.) and the Digital Imaging and Communications in Medicine (DICOM) image viewers included in the three respective CBCT systems were used (details in Table 1). Adjustment of window-level settings was allowed. After an interval of minimum three weeks the reading was repeated by TAA.

To examine the potential of CBCT to specify the location of pathology, the glenoid fossa and the condyle were scored for irregularities and flattening by segmental scoring and as a whole, in separate sessions. However, preliminary results showed that segmental scoring had very poor agreement [43], and this was therefore omitted from further analysis.

Ordinal data are presented as medians (ranges), dichotomous data as proportions and continuous data as medians (IQR) or means (±SD). For categorical variables, intra- and interobserver agreement was assessed with kappa (κ) coefficients (95% confidence interval). Fleiss’ kappa was applied for three observers and Cohen’s simple or linear weighted kappa for two observers or observations. A κ coefficient of < 0 was considered poor, 0–0.20 slight, 0.21–0.40 fair, 0.41–0.60 moderate, 0.61–0.80 substantial and 0.81–1.00 almost perfect [44]. For continuous variables, intra- and interobserver agreement was assessed by calculating the mean difference and standard deviation (SD) of the differences to establish the 95% limits of agreement (LOA, mean difference ± 1.96*SD) as advised by Bland and Altman [45]. Outliers were removed from final analysis if the value was more than four standard deviations from the mean [46]. Bland–Altman plots are usually informally interpreted, and we set the limit for clinically acceptable agreement (100: sample mean × 95% LOA) at 15%. The mean differences were used as a measure of bias, and considered statistically significant given a p-value < 0.05 (two tailed, one-sample t-test). Proportional bias was assumed if linear regression was statistically significant. To assess the potential impact of different CBCT-systems, intraobserver analyses were repeated stratified by study site for the categorical variables, and a one-way between-groups analysis of variance was conducted for the intraobserver mean differences. All statistical analysis was performed using IBM SPSS version 28 (IBM, Chicago, IL). The level of statistical significance was set at 5% (p-value < 0.05).

The distribution of scores for each of the CBCT-features are given in Fig. 3 (right TMJ, first reading). Six features were not further analysed due to severely skewed distribution. Absolute, intra- and interobserver agreement for the remaining 14 features are detailed in Table 4. Examples of continuity of the articular surface, irregularities and flattening are shown in Figs. 4, 5, 6, and 7.

Assessment of the overall impression of TMJ deformity on a 0–2 scale showed almost perfect agreement for the same reader, with a kappa coefficient of 0.81 (95% CI 0.69–0.92) (Table 4). The interobserver agreement was substantial, with a Fleiss’ kappa coefficient of 0.70 (0.61–0.78). The absolute agreement between three observers was 64 out of 82 (78%) (Table 4).

There was a substantial to moderate agreement for the assessment of flattening of the articular eminence and glenoid fossa, surface irregularity and continuity of the articular eminence and glenoid fossa on a 0–2 scale, with kappa coefficients of 0.77, 0.66 and 0.46, respectively, for the same reader. The inter-reader agreement was moderate to substantial with kappa coefficients between 0.43 and 0.65 (Table 4).

Assessment of condylar flattening, both on a sagittal view, 0–3 scale, and on a coronal view, 0–2 scale, showed almost perfect and substantial agreement for the same reader, with kappa coefficients of 0.82 and 0.71, respectively. The inter-reader agreement was substantial and moderate, with kappa coefficients of 0.76 and 0.60 (Table 4). There was a substantial to moderate intra- and inter-reader agreement for the assessment of condylar surface irregularity, reduced condylar volume and continuity of the condyle, as well as for assessment of secondary degenerative change such as subchondral sclerosis and the presence of osteophytes (Table 4). For assessment of condyle position within the glenoid fossa the interobserver agreement was slight, k = 0.10, and intra-observer agreement fair, k = 0.38. Analysis stratified for study site did not change the results.

Measurements and differences resembled normal distribution patterns. The mean glenoid fossa lengths and depths, mean glenoid fossa/articular eminence inclination angles and the mean condylar diameters are given in Table 5. In the final analysis 24 outliers were removed. Bland–Altman plots of differences in all measurements showed relatively wide 95% limits of agreement, varying from 13.6 to 83.8% of the sample means (Table 5). Measurement variation both within and between the observers is illustrated in Fig. 8.

The mean differences (bias) for the same, and between observers ranged from 0.0 to 0.2 mm and from 0.0 to 0.4 mm for linear measurements. For the angular measurements the corresponding figures were 0.3–1.8 and 0.2–2.6 degrees. The mean differences were significantly different from zero (p < 0.05) in a number of measurements, as noted in Table 5. Furthermore, there was proportional bias in 17 out of 72 measurement pairs, of which 12 had a positive slope. Examples of Bland–Altman plots without and with bias are given in Additional file 2. In the stratified analysis the mean differences were not significantly different between the study sites in 19 out of 20 measurements.

There are limitations to our study. Firstly, there is the subjective nature of developing any scoring system with inherent biases in readers understanding of what to score. To overcome this, several calibration meetings were held and an atlas depicting examples with different scores was established. Secondly, we were unable to include CBCT of healthy volunteers for ethical reasons. Of note approximately half the joints were scored with no or minor pathology. Thirdly, there are limitations to the use of Cohen’s kappa, particularly in datasets with skewed distributions [51]. We therefore present the proportion agreement and the distribution of scores to increase the transparency of the results. Fourthly, we chose to use only two or three observers to assess the scoring system’s potential given optimal conditions, rather than in a clinical setting. Fifthly, we acknowledge that despite efforts to obtain a balanced material, we were unable to properly assess the precision of rarely occurring imaging features of JIA, like subchondral pseudocyst, osteophytes, ankylosis, apposition, heterotopic calcification and loose joint body, due to their very low prevalence and even absence in our material. Finally, the CBCT machines were from different vendors between the sites, and hence the scan parameters had to be adapted locally. The differences in spatial resolution could potentially have influenced the ability to discriminate/detect discrete pathological findings and landmarks, however, this issue was thoroughly addressed during the calibration sessions and stratified analysis showed no relevant differences between study sites.

The strengths of this study include a large sample of children and adolescents with JIA, a meticulous standardization including construction of an atlas, the examination of both precision and repeatability and the use of image volumes instead of single images or stacks of images, thus reflecting the entire process of reading images. We included a breadth of pathology and severity of imaging features which were assessed by a scoring system that tested a wide variety of CBCT features of JIA.