Interobserver reliability of the Tile classification system for pelvic fractures among radiologists and surgeons

Pelvic ring fractures account for approximately 3% of skeletal injuries, with a reported incidence of 23/100,000 persons/year [1, 2]. They result from high-energy impacts and are usually associated with multiple injuries [3, 4]. Due to significant retroperitoneal bleeding or severe extra-pelvic injuries, most often of the chest or the central nervous system, the mortality rate from pelvic ring injuries may reach up to 30%, especially in hemodynamically unstable patients [5]. Potential pelvic bleeding sources include bony fracture surfaces, the disrupted pelvic venous plexus, and arterial bleeding from branches of iliac vessels [6, 7].

With the aim of guiding clinical management and providing a common language for both clinicians and researchers based on the fracture pattern, several classification systems for pelvic ring injuries have been developed [8‐13]. One of the most frequently used is the Tile classification, first proposed in 1980 [14, 15]. It is based on the mode of mechanical pelvic ring instability. Type A fractures do not concern the pelvic ring per se and are stable, type B are rotationally unstable, whereas type C are in addition vertically unstable. Each main fracture type is further subdivided into nine first-order subcategories and a total of 26 second-order subcategories [16]. Although originally designed for use with plain radiographs, the Tile classification is now routinely used based on CT images [17], which allow for a more precise evaluation of the posterior elements of the pelvic ring [18‐20] and can further aid in identifying bleeding from pelvic sources and associated abdominal injuries [21‐23].

For any classification system to be useful, there should be a high interobserver reliability (IOR); otherwise, it may not be possible to correctly interpret study results or justify clinical decisions and management algorithms based on such classification systems. For instance, there is controversy about the clinical usefulness of classification systems in terms of the association of fracture patterns with the risk of significant bleeding and mortality, whether the Tile [24‐30] or other systems [9, 29, 31‐34] are used. So far, four studies have examined the IOR of the Tile and other classification systems, finding IORs ranging from poor to moderate. None of these studies included radiologists as observers [35‐38].

The primary aim of this study was to examine the IOR of the Tile classification based on CT scans read by radiologists and surgeons with varying levels of experience. The secondary aim was to assess whether the classification ratings by different observers influenced the association between the pelvic fracture type and mortality in the present study.

Among the 238 pelvic fractures included for analysis, there were 54 Tile A (23%), 82 Tile B (34%), and 102 (43%) Tile C types. In the present study cohort, the mean ISS was 24 (SD, 13) and the most frequently associated major (AIS ≥ 3) injuries concerned the chest in 98 (41%), the abdomen in 54 (23%), and the head in 53 (22%) patients. Mean patient age was 44 years (SD, 20 years); 158 patients (66%) were males. Surgical stabilization of the pelvis was performed in 52 (22%) of patients and 25 (11%) underwent arterial angio-embolization for active pelvic bleeding. Table 1 summarizes the detailed diagnostic reference values based on the consensual review by the three experts, and Table 2 shows the characteristics of the study population. Figure 1 illustrates examples of the three main categories (A/B/C) of the Tile classification.

For the three main Tile categories (A, B, C), the combined IOR for all six observers was moderate (kappa = 0.44). When analyzed by specialty, IOR was moderate among radiologists (kappa = 0.47) but only fair among surgeons (kappa = 0.34). The combined IOR for the nine first-order subcategories (A1–3, B1–3, C1–3) was fair, overall (kappa = 0.31) and by specialty, yet kappa values were higher among radiologists (kappa = 0.35) than among surgeons (kappa = 0.23). Table 3 summarizes the combined IORs with the corresponding kappa values.

For both main and first-order subcategories, the individual two-rater IORs between each of the six observer ratings and the reference classification were substantial for the two attendings (radiology and surgery; kappa = 0.79 and 0.74, respectively) and the radiology resident (kappa = 0.79). They were moderate for the surgery resident (kappa = 0.58) and interns of both specialties (kappa = 0.43 and 0.52, respectively).

By general level of experience, the two-rater IORs were moderate for attendings (kappa = 0.60 and 0.41) and residents (kappa = 0.55 and 0.42), but only fair for interns (kappa = 0.33 and 0.27), both for the main and first-order subcategories. Kappa values were consistently higher for the main categories.

By level of experience and specialty, the two-rater IORs were only fair whenever one of the raters was an intern, both for the main (kappa = 0.37–0.40) and first-order subcategories (kappa = 0.28–0.37). Again, kappa values were consistently higher for the main categories. For the attending-resident pairs, the IORs were substantial for radiology (kappa = 0.70) and moderate for surgery (kappa = 0.47) for the main categories but decreased to moderate (kappa = 0.56) and fair (kappa = 0.31), respectively, for the first-order subcategories. Table 4 summarizes all two-rater kappa values.

The agreement on the 50 randomly chosen control cases between the designated raters and their peers with a comparable level of expertise ranged from moderate to almost perfect (kappa = 0.57–0.81) (Supplementary Table 2).

The overall 30-day mortality rate of the study cohort was 15% (36/238). Based on the reference classification by the three experts, none of the main Tile categories (A, B, C) of pelvic fractures was associated with mortality (p = 0.06). However, when based on the classification ratings of each of the six observers separately, an association of Tile C fracture types with mortality was observed for two out of six raters. Table 5 summarizes the association of main Tile categories of pelvic fractures with 30-day mortality for the reference and each rater.

The results of this study show that the overall combined IORs for the Tile pelvic fracture classification system using CT alone are moderate for the three main categories and fair for the nine first-order subcategories. Radiologists had a higher combined IOR (moderate) than surgeons (fair), but only for the three main categories. The IORs of the attendings from both specialties and the radiology resident were substantial, but only moderate for the other raters. These findings suggest an improvement of IOR with increasing level of experience and radiological specialization. When taking the classifications by each rater individually and relating them to mortality, a significant association was observed in two out of six cases, showing the potential influence of IOR on study results, conclusions, and potential implications for clinical decision-making, management algorithms, and recommendations.

For any imaging-based classification system to be useful, there should be a high level of inter- and intraobserver agreement. This has important implications for communication in research (comparability of study results) and, in consequence, clinical activities (classification-based outcome prediction or management guidelines). The importance of reliably linking imaging findings with outcomes in musculoskeletal radiology studies has been recently highlighted by Tagliafico et al [44]. Several classification systems for pelvic fractures have been proposed in the past [9, 11‐15, 18]. Only four studies [35‐38] assessing the IORs of the Tile [14, 15] and other classification systems [13, 18] have been published so far. To our knowledge, the present study is the largest to date on the IOR of the Tile pelvic fracture classification, allowing for a statistically sound evaluation of the first-order subcategory. It is also the first to include not only surgeons but also radiologists with different levels of experience among the raters.

Koo et al [35] were the first to evaluate the Tile and the Young-Burgess (YB) [18] classification systems by assessing IORs based on the interpretation of plain radiographs first and then comparing them with IORs based on the ratings of CT scans. All six observers were orthopedic traumatologists with different levels of expertise (pelvic/acetabular specialists, orthopedic traumatologists, and orthopedic trainees). Although only 30 patients were included in their study, subcategories were also assessed and results reported. The IORs for the Tile system pre- (kappa = 0.30) and post-CT (kappa = 0.33) were only fair, mainly due to the poor agreement among raters who were not pelvic/acetabular specialists. In line with our findings, advanced observer experience increased the IOR (from kappa = 0.07 to kappa = 0.84). One of the main clinically relevant findings in their study was that CT, compared with plain radiographs alone, improved the reliability of fracture stability assessment from moderate to almost perfect. Furey et al [36], in addition to IOR, also assessed the intraobserver reliability of both the Tile and YB systems. Their study, conducted at the center where the YB classification was designed, was based on plain radiographs and CTs for each fracture and included 89 patients and 5 observers (all experienced orthopedic trauma surgeons). Only the three main Tile categories were evaluated. Moderate IORs were found for both the Tile (kappa = 0.47) and YB (kappa = 0.46) classifications. Intraobserver reliabilities were only moderate for the Tile (kappa = 0.47), but substantial for the YB (kappa = 0.72) main classifications. Gabbe et al [37] rated 100 pre-interventional (no PCCDs or external fixators placed) pelvic CTs and plain radiographs (anteroposterior view only) by three experienced orthopedic surgeons (over 52 years of combined experience in managing pelvic fractures) from different Australian level 1 trauma centers. The authors found only slight IORs for both the Tile (kappa = 0.10–0.17) and YB (kappa = 0.17–0.19) main classifications, insufficient for clinical or research purposes according to their conclusion. Berger-Groch et al [38] recently published a study including 154 patients with pelvic fractures. Only CT images were interpreted by four observers (two senior orthopedic surgeons, one resident in training, and one medical student). Inter- and intraobserver reliabilities were assessed for the Tile, YB, and Rommens [13] classification systems. The overall IORs, expressed as intraclass correlation coefficients [45], were fair for the Tile (0.55) and the other classification systems and were strongly dependent on rater experience.

There may be several causes for the only slight to moderate overall IORs of the Tile classification system in the present and previous studies. One explanation may be the weight of disagreements among inexperienced raters, decreasing the combined IORs, despite substantial to almost perfect IORs among experienced orthopedic traumatologists. This could be observed in both studies that included raters with different levels of training, showing an improvement of IOR with increasing rater experience. However, one of the two studies with exclusively experienced raters also showed only moderate inter- and intraobserver reliabilities for the Tile classification system. Interestingly, the intraobserver reliability in that study was substantial for the YB system, potentially attributable to the fact that the raters routinely worked with the YB classification, and not with the Tile system [36]. Inter- and intraobserver reliabilities may therefore also depend on the regularity with which any system is applied by a given observer. The second study with only experienced observers found an only slight IOR of the Tile system. The presence of a greater number of more complex Tile type B and C fractures compared with the other previously published studies may explain the only slight IORs in their series [37]. However, the higher, moderate overall IORs in the present patient cohort, despite the highest rate of Type B and C fractures among all studies, do not support this hypothesis.

Our study has several limitations. Firstly, patients with PCCDs in place while undergoing CT (63%) were included in the present study. It has been shown previously that the presence of PCCDs may lead to misinterpretation of the fracture pattern and thus have an influence on fracture classification [46‐48] and assessment of pelvic ring stability [49]. However, given the widespread use of PCCDs in the prehospital setting [50], this situation is now frequently encountered and represents clinical conditions under which fracture classifications have to occur. Secondly, as highlighted by Gabbe et al in their study [37], attributing mortality to any particular pelvic fracture type of the Tile classification based on CT is problematic, since the most severely injured patients presenting with hemodynamic instability and the highest mortality frequently do not undergo CT imaging. Therefore, the association of CT-based pelvic fracture type with mortality must be interpreted with caution. Thirdly, since it is routinely used at our center, we only assessed the Tile and no other classification system, unlike all other studies [35‐38]. As suggested by the results of Furey et al [36], who observed better intraobserver reliabilities for the system routinely used at their hospital, we think there would have been a bias in the results for all not routinely used classification systems. Furthermore, unlike in three of the four existing studies [35, 36, 51], we only assessed IORs for CTs and not for plain radiographs. The current clinical practice no longer includes the three plain radiographical views for which the Tile classification was initially developed [52]. In our center, only plain radiographs (anteroposterior view) of the pelvis are obtained for hemodynamically unstable trauma patients. Hemodynamically stable trauma patients undergo a routine contrast-enhanced whole-body CT.

The present study is the first to include radiologists among the raters, and the largest so far assessing IORs of the Tile classification system, allowing for statistically appropriate evaluation of its first-order subcategories. Second-order subcategories were not assessed for IOR in the present study. A statistically appropriate analysis would have required the inclusion of 1352 cases. Given the decrease in IOR from moderate to fair from the main to the first-order subcategories, we believe that a second-order subcategory IOR below, or at best equal to fair, can be expected. Therefore, the extension of the observation period or inclusion of other centers to reach the required case number was considered excessive.

Similar to the results of previous studies, the overall IOR of the Tile classification system is only fair (nine first-order subcategories) to moderate (three main categories). It depends not only on the level of experience of the observers, but it also seems to be better among radiologists than surgeons. However, even among radiologists, misclassification of pelvic fractures by the least experienced, likely to occur in emergency radiology departments and particularly during night shifts, may have important implications in clinical decision-making and management of severely injured patients. In research settings, misclassification may lead to erroneous study results and conclusions, with subsequent inappropriate translation into clinical management algorithms. This may be avoided by using only classification ratings from experienced observers.