Identification of patients with suboptimal results after hip arthroplasty: development of a preliminary prediction algorithm

This study entailed a retrospective analysis of longitudinal, prospectively collected data. The methodology adheres to the Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) guidelines for observational cohort studies (Additional file 1).

Our prospective arthroplasty database was consulted in order to identify patients eligible for inclusion in the study. The database contains extensive baseline and follow-up data on patients undergoing hip procedures, including THA and HR. All patients provide informed consent to participate. Independent assessors who are not involved in the medical care of the patients collect the prospective data.

Inclusion criteria were 1) patients undergoing primary unilateral THA or HR, 2) diagnosis of primary hip OA, 3) complete preoperative and one to two-year postoperative self-reported outcome questionnaire responses. The main exclusion criteria were 1) THA or HR of the contralateral hip before the relevant follow-up evaluation, 2) revision of the implant before the one to two-year follow-up, 3) diagnosis of inflammatory hip arthritis, pediatric hip disease, post-traumatic hip or any hip disease other than primary OA. Data on all patients having undergone hip interventions were assessed for inclusion. All patients were initially interviewed just before their intervention. Postoperative outcomes were collected 12 to 24 months after the surgery.

Functional status and pain levels were assessed preoperatively and at follow-up with the Western Ontario and McMaster Universities Osteoarthritis Index (WOMAC) [23]. The WOMAC consists of the following domains: pain (5 items), stiffness (2 items) and functional limitation (17 items). Items are scored on a 5-point Likert scale representing different degrees of intensity (none, mild, moderate, severe or extreme). The scores of each domain as well as the total score were standardized on a 0 to 100 scale, with a greater score indicating more pain, stiffness or functional limitation. The psychometric qualities of the WOMAC, including its responsiveness, convergent construct validity and reliability have been found excellent for evaluating patients with hip OA undergoing hip arthroplasty [24, 25].

At follow-up, self-perceived joint perception was measured by asking the patient a multiple-choice question: “How do you perceive your operated hip?” with the possible responses being ”Like a native or natural joint”, ”Like an artificial joint with no restriction”, ”Like an artificial joint with minimal restriction”, ”Like an artificial joint with major restriction” and ”Like a non-functional joint” [26]. Evaluation of joint perception has been strongly associated with validated clinical scores of patient-reported outcome measures and can be employed as a measure of patient satisfaction [26].

No consensus exists regarding what represents poor outcome following hip arthroplasty. Hence, patients of risk of suboptimal outcomes were defined as the ones in the worst quartile of the total WOMAC score at follow-up (i.e. WOMAC score >11.5) and perceiving their hip ”Like an artificial joint with minimal restriction”, "Like an artificial joint with major restriction” or "Like a non-functional joint”.

Baseline and follow-up mean WOMAC scores along with their standard deviations were calculated. Differences between time points in relation to total WOMAC scores and the respective domains was assessed using paired samples Student-t tests, with a significance level set at 0.05.

The classification and regression tree approach was used to build the PA as it is one of the most effective algorithms of recursive partitioning [27]. It is based on maximizing the within-node homogeneity by evaluating all combinations of potential predictors, thus minimizing the within-node error. The Gini impurity measure was used as a splitting criterion to develop the decision trees [28]. Data for all the patients in the training set was used to develop the PA. Firstly, all the potential predictor variables were employed to develop models using an automated approach. Secondly, a manual approach entailed the development of additional models by inputting independent variables that were judged to be more readily available and easier to employ in a clinical setting. For example, age and gender were favoured over the number of comorbidities and previous hip interventions because the latter two could be affected by a recall bias or would require extensive medical file review. The predictive values of every model were calculated along with their 95 % confidence intervals, namely sensitivity, specificity, positive and negative predictive values as well as positive and negative likelihood ratios [29]. Among all the proposed models, the one that showed the highest level of sensitivity and an acceptable level of specificity and that fit the ease-of-use criterion was selected in order to develop the screening tool. Internal validity of the model was then evaluated by the use of 1,000 bootstrap resamples [30]. All analyses were performed using IBM SPSS Statistics Version 20.0 (SPSS Inc., Chicago).

The research ethics committee of our centre approved the study annually.

Our database yielded 2963 entries with at least some preoperative data on hip arthroplasty procedures performed from October 2004 to February 2014. Out of these, 1207 procedures (40.7 %) fit the inclusion criteria. Incomplete preoperative and/or postoperative data required for the purposes of the current study obliged the exclusion of a further 942 entries. Thus, a total of 265 primary hip arthroplasty interventions (60 classical THAs, 128 large-femoral head diameter THAs, and 77 h) with complete preoperative and postoperative data were included in the study (follow-up mean ± SD: 446.3 ± 171.1 days), representing a participation proportion of 22.0 %.

Table 1 shows selected characteristics of the 265 patients included in the study. The mean age of the participants was 52.0 (SD 9.0) and 67.4 % were male. The mean BMI was 28.2 (SD 5.1) and each patient had on average 0.79 comorbidities (SD 0.96).

Mean follow-up was 446 (SD 171) days and ranged from 253 to 1638 days. Postoperatively, the patients had significantly improved on pain (−44.9, SD 22.6, 95 % CI −42.1 to −47.6), stiffness (−44.6, SD 25.1, 95 % CI −41.6 to −47.7), function (−43.6, SD 21.9, 95 % CI −40.9 to −46.2) as well as total WOMAC score (−43.9, SD 21.1, 95 % CI −41.4 to −46.5) (Table 2). Seventy-six patients (29 %) reported that they perceived their prosthetic joint as artificial with minimal or major restrictions (Table 3).

Out of the 265 patients eligible for inclusion in the study, 40 (15.1 %) had a total WOMAC score > 11.5 and perceived their joint as artificial with minimal or major restrictions. Hence, these patients were considered as having suboptimal surgical outcomes.

After developing several prediction rules, the algorithm with the highest level of sensitivity and an appropriate level of specificity was chosen. It consists of patient gender, age at the time of surgery, body mass index (BMI), and 3 items of the preoperative WOMAC, namely degree of pain with walking on a flat surface and during night and degree of difficulty with putting socks or stockings (Fig. 1). Patients respond sequentially to the questions and their risk status is determined according to the classification algorithm (Fig. 2).

The final PA correctly identified 30 out of the 40 patients considered at risk of suboptimal outcome and 175 patients out of 225 were identified as not at risk of suboptimal outcome (Table 4). Therefore, the PA had a sensitivity of 75.0 % (95 % CI: 59.8–85.8), a specificity of 77.8 % (95 % CI: 71.9–82.7 and a positive likelihood ratio of 3.38 (95 % CI: 2.49–4.57) (Table 5). The other prediction models that were also considered are presented in Additional file 2.

Validation of the rule was established using 1,000 bootstrap re-samples. Table 5 indicates the estimated bootstrap values of the predictive measures being close to the original ones, thus suggesting an appropriate accuracy of the proposed model.

The developed PA is, to our knowledge, the first one of its kind to discriminate THA or HR results based on more than one parameter, namely patients’ functional, stiffness and pain levels as well as their perception of the replaced hip joint. In the context of a lack of an accepted standard of surgical failure, this approach increases the likelihood of the patients thusly classified to truly present subpar outcomes. Moreover, this classification identified 15.1 % of the patients as having unsuccessful outcomes, well in line with the published proportions of what can be considered a suboptimal outcome [8]. Finally, the rigorous statistical analysis employed in the development of the PA underlines the stringency of our approach.

Due to its retrospective design, the study has a certain risk of selection bias. Compared to the subjects excluded due to missing data, the included participants were, on average, younger, had a greater number of comorbidities and a greater proportion were male (p < 0.05, data not shown). Nevertheless, the preoperative baseline status as measured by the WOMAC domains as well as the total WOMAC score was not statistically significantly different between the included and the non-included subjects (p > 0.05, data not shown). Additionally, only 265 out of the 1207 procedures (22.0 %) that were performed during the study time period met the inclusion criteria, therefore potentially limiting the generalizability of the results. Moreover, the population under study was patients undergoing primary unilateral hip replacement procedures, which precludes the utilization of this tool for patients undergoing revision or bilateral interventions. Recent evidence identifies other variables potentially associated with hip arthroplasty outcomes that were not included in our study, thus potentially limiting the pool of candidate predictor variables. Although there is no consensus regarding the optimal sample size for developing models employing recursive partitioning, the progressively smaller number of cases in the leafs as the tree was built may limit the reliability of the findings, prompting further development in a subsequent study. Before employing it in a clinical setting, the decision rule has to be validated in a different sample of patients. Moreover, the performance of the PA has to be compared to clinical judgement alone and its financial impacts require evaluation.