Criteria for failure and worsening after surgery for lumbar disc herniation: a multicenter observational study based on data from the Norwegian Registry for Spine Surgery

In spine surgery, several well-validated patient-reported outcome measures (PROMs) have been recommended, such as the Oswestry Disability Index (ODI) [1], Numerical Rating Scale (NRS) for leg pain and back pain [2], and the EuroQol 5D (EQ-5D) [3]. Still, clinicians are often unfamiliar with their interpretation. In large cohorts, even small and clinically irrelevant PROM changes tend to reach statistical significance [4]. To provide cutoffs on PROM changes that are perceived as meaningful and important by the patients, the “minimal important change” (MIC) has been defined by various methods [5‐7]. A recent review proposed an MIC cutoff for the ODI of ten points, or 30% improvement from baseline [8]. Several studies have identified MIC cutoffs for the NRS back pain and leg pain from 2 to 2.5 [8, 9]. In addition, cutoffs for substantial clinical improvements, such as “success” after lumbar disc surgery, have been reported both for the ODI (20), NRS back pain (2.5), NRS leg pain (3.5), and EQ-5D (0.3) [9‐11]. A large proportion of the patients are difficult to classify as either improved, unchanged, or worse after surgery [12]. Consequently, cutoffs on the PROMs for deterioration and “failure” may be different from those of “non-success”. Previously, authors have used various methods and different concepts for defining cutoffs for clinical meaningful improvements [10, 12], resulting in a diversity of recommended threshold values [8, 21, 22]. This makes it even more difficult to disentangle “failure” from constructs developed to identify improvements. There is clearly a grey zone between “failure” and “non-success” [13], “minimal meaningful improvements”, or a “satisfactory symptom state” [14]. Using an external anchor method to define “failure” more accurately could provide more robust definitions of this outcome category [11]. It is, therefore, important to differentiate between “failure” and “non-success”.

The indication for operative treatment of lumbar disc herniation is relative, and the decision to operate must be based on a trade-off between possible benefits, risks, and costs [15]. In clinical trials, focus is generally placed on improvements such as “success rates”. To enhance quality assessment and shared decision-making, it is crucial to consider the other end of the scale and to discuss the possibility of adverse outcomes with the patients. Avoiding inefficient operations may have a greater impact on treatment outcomes, than improving surgical technique [16]. The first step would be to try to define cutoffs for “failure” and “worsening” on the PROMs. When informing the patient about possible outcomes, we think that it is important to differentiate between being unchanged after surgery, which might be an acceptable risk, and actually getting worse, which might be harmful. Previous studies show that larger cohorts are needed to clearly define clinically meaningful thresholds for such outcomes, especially for worsening [17, 18].

The Norwegian Registry for Spine Surgery (NorSpine) collects clinical data (PROMs) on the majority of patients operated for lumbar disc herniation in Norway. Its purpose is to evaluate treatment outcomes from the “real life” of daily clinical practice and use this information to improve the quality of the health services [19, 20]. The aim of this study was to estimate the most accurate cutoffs for both failure and worsening after surgical treatment of lumbar disc herniation, using data from the large registry cohort of the NORspine. Such benchmark criteria could be used for calculating sample size in research and facilitate shared decision-making among doctors and patients, clinical audit, and comparisons of outcomes across surgical units.

6840 out of 9930 (69%) patients had 12-month follow-up data. Among those lost to follow-up were more smokers, a higher number of sickness benefits recipients, and more patients who had been operated previously (Table 2). Furthermore, they had a lower level of education, and fewer were operated on for paresis. Except for back pain, there was no statistical significant difference in PROMs at baseline. Patients who did not respond to the follow-up scored slightly higher for back pain than those who responded.

During surgery, an operating microscope or loupes were used in 5936 of 6840 (87%) cases. A total of 885 (13%) had a reoperation on the same level, 466 (7%) on a different level, and 66 (1%) on both the same and a different level between L1 and S1. The perioperative complication rate was 169 (3%) with 115 (2%) dural tears, 21 (0.3%) nerve root injuries, 24 (0.4%) hematomas requiring transfusion or reoperation, and 9 (0.1%) cardiorespiratory complications.

Few data points were missing for the baseline PROMs: ODI (13, 0.2%), EQ-5D (252, 3.7%), NRS back pain (170, 2.5%), and NRS leg pain (159, 2.3%). At 12-month follow-up, 40 (0.6%) were missing data on GPE, 11 (0.2%) on ODI, 520 (7.6%) on EQ-5D, 47 (0.7%) on NRS back pain, and 66 (1%) on NRS leg pain. GPE scores for the entire population are shown in Table 3. Mean improvement (95% CI) for each PROM from baseline to 12-month follow-up for the total sample was 28.7 (28.2–29.2) for the ODI, 0.45 (0.44–0.46) for EQ-5D, 3.2 (3.1–3.3) for back pain, and 4.4 (4.3–4.5) for leg pain, p < 0.001.

After evaluating the mean score differences of all PROMs across the categories of the GPE, the study group concluded that the definition of a score range of 4–7 for “failure” and 6–7 for “worsening” was appropriate (Table 3). Figures illustrating these differences are shown in the appendix (Figs. 1x–4x).

For each GPE outcome group, the baseline adjusted mean scores of the PROMs (ANCOVA) after 12 months are shown in Table 3.

We estimated the optimal cutoff values for failure and worsening 12 months after surgery for lumbar disc herniation, using four recommended PROMs. An ODI% improvement of less than 33% was the most accurate measure for identifying patients for whom the surgery had failed. Back pain, both the mean % change, and the final raw score at 12 months, also showed high accuracy for identifying failure. We found no significant difference in outcome scores among patient groups who considered themselves as “unchanged” or “slightly worse”, which is in accordance with a previous study [12]. A final ODI raw score of more than 48 at 12-month follow-up had the highest accuracy for identifying patients reporting worsening, followed by a final raw score of 7.5 for NRS back pain. A potential explanation for this finding might be that those with a final ODI over a threshold value of 48 will tend to consider themselves as worse, irrespective of the amount of change. These patients are exhausted after more than a year with unresolved severe pain and disability, not compatible with a normal life (Fig. 10x, appendix). One previous study also found a high correspondence between the final raw score and the GPE scale as an external anchor [17].

Compared to the GPE, all cutoffs categorized a higher proportion of the outcomes as “failure” or “worsening”. Since the individual PROMs represent different concepts, the variation between the individual outcome measures and GPE scale is to be expected [10, 11]. For instance, even the disease-specific ten item ODI could fail to address issues important to patients. Individuals might also weigh each item differently according to their preferences.

We chose to classify all patients who scored unchanged or worse (GPE > 3), as “failure” and those scoring much worse or worse than ever (GPE 6–7) as “worsening”. These definitions are supported by our data, i.e., differences in mean PROMs between the GPE groups in ANOVA and ANCOVA analyses, as shown in Table 3 and Figs. 1x–5x (appendix). A large group of patients (n = 1676, 24%) classified themselves as “slightly better”, “unchanged”, or “slightly worse” on the GPE, and would be the most susceptible of being misclassified [12]. While it was not possible to separate the “unchanged” from the “slightly worse” based on PROMs, patients defining themselves as “slightly better” (16%) had a mean improvement in the ODI score of 15.1, more than the previously defined cutoff for the Minimal Clinical Important Change (MCIC) [8]. Hence, it is reasonable not to include them in the failure group. While non-success implies a degree of improvement, failure does not, which might be of importance for litigation issues. The distinction between these two concepts could also be used in the development of predictive models in value-based health care [29].

The mean PROM improvements in this study were in line with results from other clinical trials [30‐33]. Failure and success rates, however, are highly dependent on where the cutoff levels are set to classify outcomes, and types of PROMs used [11]. Mean change in NRS back pain showed the highest failure rate (31%) and mean change in NRS leg pain the lowest (20%). Back-pain intensity is not the primary indication for lumbar discectomy without fusion. It could therefore be expected that, for instance, the NRS leg pain classified a lower failure rate [34]. Our findings indicate that patients reporting failure and worsening tend to be concerned about back pain, even though leg pain may have improved. An explanation may be that a large proportion of patients operated for lumbar disc disease will expect a substantial improvement in back pain [35].

We have defined cutoff values with acceptable sensitivity and specificity on validated PROMs to classify outcomes as “failure” and “worsening” 12 months after lumbar disc surgery.