The Amsterdam wrist rules: the multicenter prospective derivation and external validation of a clinical decision rule for the use of radiography in acute wrist trauma

The study protocol has previously been published [15]. We performed a multicenter prospective study that consisted of three components: (1) derivation of a clinical prediction model for detecting wrist fractures in patients following wrist trauma; (2) external validation of the model in a new patient population enrolled in a different setting; and (3) design of a clinical decision rule. The study was conducted in the Emergency Departments of five Dutch hospitals from November 11, 2010 to June 25 2014. The participating hospitals included one academic hospital and four regional teaching hospitals: the Academic Medical Centre, the Tergooi Hospital, the Sint Lucas Andreas Hospital, the Flevo Hospital and Spaarne Hospital. The central Medical Ethical Review Committee of the Academic Medical Center approved the study without the need for informed consent (NL34430.018.10). Additionally, the local Medical Ethical Review Committees of all four participating hospitals approved the study.

The derivation cohort comprised all patients enrolled in the academic hospital. The validation cohort included all patients enrolled in the four other participating hospitals.

We included all consecutive adult patients who presented to the Emergency Department with pain or tenderness secondary to wrist trauma. The wrist was defined as the proximal segment of the hand, including the carpal bones and the associated soft parts; and the distal segment of the ulnar and radial bone. Wrist trauma was defined as any high or low energetic trauma involving the wrist, such as a fall on outstretched hand (FOOSH). We excluded patients whose injury occurred more than 72 h previously or multi trauma patients (Injury Severity Score ≥16). Patients who already had an X-ray made previous to their visit to the Emergency Department (for example requested by their general practitioner or by another hospital) were excluded as well. Additionally, physicians were instructed not to include patients if radiographs had already been ordered and they were aware of the outcome (fracture present or not).

Eligible patients were included upon presentation in the Emergency Department. Data were collected prospectively by the treating physicians on standardized Case Record Forms (CRF). Patients were evaluated for 19 clinical variables including patient characteristics, physical examination and functional testing (Table 1). We based the selection of variables on clinical experience and previous studies [13, 14]. The questions on the Case Record Form (CRF) were presented in a dichotomous nature (yes/no). Eligibility and data collection forms were verified by two authors by cross-checking the medical records of all patients six months after inclusion.

The assessors were all physicians and included consultant emergency medicine physicians; emergency medicine residents; surgical residents; orthopaedic residents and general practice residents. All physicians received regular instructions and training on how to assess the clinical variables in a standardized manner. Additionally, we provided informative pocket cards and posters. In order not to disrupt common practice, referral for radiography and type of treatment were at the discretion of the treating physician. Although the study did not mandate radiographs on all wrist-injured patients, only 5 out of 1019 patients (0.5 %) did not receive an X-ray of the wrist.

The reference standard was the presence of a fracture of the distal radius, ulna or one of the carpal bones, as assessed by the attending radiologist on the X-ray at presentation. A fracture was defined as a disruption of one or more of the cortices. A fissure and an avulsion were recorded as a fracture. The radiologist was blinded to the contents of the Case Record Forms. Radiographic series comprised at least one posterior-anterior (PA) and one lateral view with 90 degrees of elbow flexion; and any further conventional imaging available (for example scaphoid series). We did not take findings on additional Computed Tomography scans or Magnetic Resonance Image scans into account.

A common rule of thumb to determine the sample size of the development of a prediction model is at least ten events (fractures) per variable [16]. Patients were evaluated for 19 variables. Therefore the inclusion of minimum of 190 patients who sustained a fracture was required in the derivation cohort. According to a similar rule of thumb, external validation requires at least 100 patients with an event (fracture) and 100 patients without an event (no fracture) [16]. We continued enrolling patients after the required sample size was achieved to maintain the study infrastructure required for the subsequent implementation study.

For efficient statistical analysis [17‐19], we used imputation techniques to impute the missing values (aregImpute function from the Hmisc library, R, version 3.0.1.) For each variable containing missings, the aregImpute package draws values from a random sample from the non-missing values with replacement. Using this data, aregImpute fits a flexible model that predicts the missing target variable while finding its optimum transformation. Each missing variable is then imputed with the observed value whose predicted transformed value is closest to the predicted transformed value of the missing variable. We considered an imputation model that included all dichotomous variables; prehensile grip strength and the outcome. The set of first imputations was used for the analyses.

We derived two clinical prediction models: one for all wrist fractures (distal ulna, distal radius and carpal bone) and one for distal radius fractures only. Using data on patients enrolled in the academic hospital, multivariate logistic regression models with all 19 potential predictors were fit. These full modes were reduced using a stepwise backward elimination process based on a liberal p-value of 0.2 [20]. To estimate the internal validation of performance we used bootstrapping (500 replications). Bootstrapping provided the shrinkage factor that was used for the regression coefficients [21].

To assess general applicability, we validated the shrunk models in the cohort that included all patients enrolled in the four other participating hospitals. For each patient in the validation cohort, the probability of a wrist fracture or of a distal radius fracture was calculated using the prediction models. The validity of the models was assessed by comparing the predicted probabilities of a fracture with the observed fractures. To estimate the ability of the models to discriminate between patients with and without a fracture, we calculated the Areas under the Receiver Operating Characteristics Curve (AUC). The AUC ranges from 0.5–1, with a higher score indicating more accurate predictions. The models were also evaluated for their agreement between predicted fractures and observed fractures. This is otherwise known as the model calibration and was assessed by plotting the predicted probability of a fracture and the observed frequency of fractures. The ideal slope of such a plot is 1, indicating perfect agreement between observed and predicted risks [20]. As a final step, the models were fit on data from both cohorts combined to obtain robust estimates of the regression coefficients. These final modes were internally validated by bootstrapping as for the initial models.

We enrolled 1019 patients from five participating hospitals. A total of 137 patients (13 %) were excluded patients from the analysis for various reasons (Fig. 1). In total, 882 patients were analyzed (Table 2). In 470 patients (53 %), a fracture of the distal radius, distal ulna or one of the carpal bones was identified on conventional radiographic series. A distal radius fracture was the most common fracture (44 %).

In the derivation cohort, 487 patients were analyzed with a median age of 48 years (interquartile range, 29 - 61) and women were slightly overrepresented (57 %). A fall on outstretched hand was the most common mechanism of injury (66 %). In 251 patients (52 %) in the derivation cohort, a fracture of the distal radius, ulna or one of the carpal bones was identified.

In the validation cohort, 395 patients with similar demographic characteristics were analyzed (Table 2). In 219 of these patients (55 %), a fracture of the distal radius, distal ulna or one of the carpal bones was identified.

In both the derivation and the development cohort, around 80 % of the cases had fully complete Case Record Forms. With the exception of prehensile grip strength, missing values comprised less than 5 % for each variable (see Additional file 1).

A clinical prediction model for all fractures was derived that included eight variables: age; sex (if male), swelling of the wrist; swelling of the anatomical snuffbox, visible deformation; distal radius tender to palpation; pain on radial deviation and painful axial compression of the thumb. The Area Under the Curve (AUC) of this model was 0.84 (95 % CI: 0.81–0.88) and 0.82 (95 % CI: 0.79–0.85) after correcting for model optimism by bootstrapping.

The coefficient of each dichotomous variable reflects the amount of change in the probability of a fracture (Table 3). The presence of a dichotomous variable with a positive coefficient adds to the probability of a fracture. The presence of a dichotomous variable with a negative coefficient decreases the probability. The coefficient of the continuous variable age reflects the amount of change in probability for every ten-year increase in age. Except for painful axial compression of the thumb (coefficient -0.37), the presence of all variables adds to the probability of a fracture. Painful axial compression of the thumb decreases the probability of a fracture.

A clinical prediction model for only distal radius fractures was derived that also included eight variables: age; swelling of the wrist; visible deformation; distal radius tender to palpation; pain on ulnar deviation; palmar flexion, supination and the painful radioulnar ballottement test (Table 4). The presence of all variables except pain on ulnar deviation increases the probability of a distal radius fracture. Pain on ulnar deviation (coefficient -0.67 (95 % CI: -1.35–0.02) decreases the probability of a distal radius fracture. The Area Under the Curve (AUC) of this model was 0.91 (95 % CI: 0.88–0.93) and 0.90 (95 % CI: 0.87–0.92) after optimism correction by bootstrapping.

The external performance of the models was assessed in the 395 patients in the validation cohort. The Area Under the Curve at external validation of the model for all fractures was 0.81 (95 % CI: 0.77–0.85) and the calibration slope was 0.94 (95 % CI: 0.74–1.13). The Area Under the Curve at external validation of the model for only distal radius fractures was 0.86 (95 % CI: 0.82–0.89) and the calibration slope was 1.07 (95 % CI: 0.84–1.29).

The Amsterdam Wrist Rules (AWR) for all wrist fractures showed a sensitivity and specificity of 98 % (95 % CI: 95–99 %) and 21 % (95 % CI: 15–28 %) (Table 5). Its negative predictive value was 90 % (95 % CI: 81–99 %). The sensitivity and specificity for only distal radius fractures were 98 % (95 % CI: 97–100 %) and 25 % (95 % CI: 19–31 %) (Table 5). The AWR was able to rule out 19 % (41 / 219) of the patients without a wrist fracture and 25 % (53 / 211) of the patients without a distal radius fracture. If the AWR had been used for all fractures, an X-ray would have been requested for 89.6 % (354 / 395) of patients instead of 100 %. This is an absolute reduction of 10.4 %. The final formula to calculate the probabilities are depicted in Table 6. The AUC of the final model after bootstrapping was 0.88 (95 % CI: 0.86–0.90)