Responsiveness of the Shoulder Pain and Disability Index in patients with adhesive capsulitis

The Shoulder Pain and Disability Index (SPADI) is a self-administered questionnaire consisting of items grouped into pain and disability subscales. Rating is on visual analogue scales, and the means of the two subscales are combined to produce a total score ranging from 0 (best) to 100 (worst). The SPADI was designed to measure the impact of shoulder pathology in terms of pain and disability, for both current status and change over time. The original developers stated the rationale for developing this type of joint-specific instrument: it was expected to measure the impact of specific joint problems more precisely than global health assessment instruments, and also to be better in demonstrating the effect of a treatment directed at one joint only [1]. These properties are closely linked to responsiveness, defined as the ability of an instrument to accurately detect change when it has occurred [2].

Responsiveness of the SPADI has been assessed using retrospective self-assessment of global change as a reference criterion, a study [3] cited by several researchers. Patients were given their baseline responses when follow-up SPADI scores were recorded along with the global rating of change. Then the SPADI change score was compared to the measure of global change, where the patient had rated his shoulder problem as "cured", "improved", "the same" or "worse" compared to his baseline examination. Based on these comparisons, the authors stated that "the SPADI∆ (baseline – follow-up) discriminated accurately between subjects who improved versus those who stayed the same or worsened" [3]. This statement regarding SPADI responsiveness can hardly be expected to apply to settings in clinical trials where more traditional designs are used when gathering follow-up scores. Retrospective methods of computing responsiveness yield little information about the ability of an instrument to detect treatment effects, and they should not be used as a basis for the choice of an instrument for applications to clinical trials [4, 5].

SPADI responsiveness has been compared with the responsiveness of other health assessment scales, both global [6‐10] and shoulder-specific [8‐11]. SPADI is reported to be one of the more responsive scales [12]. It is, however, problematic to make conclusions on responsiveness based on comparisons only with such very similar types of instruments.

A considerable number of shoulder self-report questionnaires have been proposed [13]. The rationale for their employment in research settings must be that there are advantages concerning the properties of the new instruments. This obvious criterion often seems to be ignored. As a consequence, a variety of instruments sometimes makes it a complex task to interpret the results of trials. Furthermore, for most of the shoulder questionnaires, evidence for their validity in various diagnostic groups used in clinical trials is often limited, at best. The SPADI is no exception to this, even though it is one of the shoulder rating instruments that have been most extensively studied [12]. It has also been employed in several clinical trials involving patients with adhesive capsulitis [14‐18].

The objective of this study is to investigate reproducibility and responsiveness of the SPADI when evaluating patients with adhesive capsulitis (See Additional file 1 for the Norwegian version [19] of the SPADI). Reproducibility is assessed with a test-retest of presumably "stable" patients. Responsiveness is investigated by using baseline and follow-up scores from a recently reported clinical trial regarding hydrodilatation and corticosteroid injections in patients with adhesive capsulitis [20]. Subjects included in the clinical trial were outpatients attending the Department of Physical Medicine and Rehabilitation of Ullevål University Hospital in the period Dec. 2003 – June 2005. A hydrodilatation procedure including corticosteroids was compared with the injection of corticosteroids without hydrodilatation. Patients were given three injections with two-week intervals, and all injections were given under fluoroscopic guidance. Seventy-six patients were included and groups were compared six weeks after treatment in order to identify potential treatment effects of hydrodilatation. The main inclusion criteria were shoulder pain and reduction of passive ROM in the affected shoulder of 30° or more for at least two out of three glenohumeral movements (flexion, abduction and external rotation).

There are several aspects of responsiveness [2], reflecting the different ways instruments are used in various settings. "Internal" [21] responsiveness statistics refer to the ability to produce statistically significant changes in scores, dependent on the study population and intervention. Interpretation of SPADI "internal" responsiveness figures is facilitated in this study by reporting corresponding figures for shoulder ROM, thereby allowing for head-to-head comparisons of SPADI and a more traditional outcome measure [22] for shoulder capsulitis.

Responsiveness can also be measured in terms of the strength of the relationship between changes in the outcome measure of interest and changes in some external standard, e.g. important clinical variables. This aspect of responsiveness is called "external" responsiveness [21]. Previous researchers have investigated the relationship between shoulder ROM and shoulder scales in cross-sectional analyses [11, 23‐27], while our aim is to compare change scores. In general, we expect associations of moderate strength. We expect association with SPADI to be stronger for measures of active ROM than for passive ROM, and we expect stronger associations for the disability subscale than for the pain subscale.

The regional ethics committee granted ethical approval for the trial. The procedures followed protocol and complied with the Helsinki Declaration as revised in 1983 and current national ethical standards for such studies.

Seventy-six patients were included in the clinical trial [20]. Fourteen of these patients were not able to take part in the SPADI reproducibility substudy for practical reasons. Furthermore, two patients did not respond to a sufficient number of items at retest. Hence sixty patients were available for the SPADI reproducibility analysis. Details for test/retest scores and reproducibility estimates for subscales and SPADI total are given in Table 1. Distributions of SPADI subscale and total scores on both occasions approximate normal distributions according to plots (not shown). Kolmogorov-Smirnov and Shapiro-Wilk tests were non-significant except for the disability scale at retest. Results indicate a "smallest detectable difference" (SDD) of 17 points for the SPADI total score. This means that for approximately 95% of the pairs of observations, the difference between scores was 17 points or smaller. The intraclass correlation coefficient (ICC) was 0.89.

Plotting SPADI means of the two observations against the test-retest difference for each patient does not give any indication that measurement errors vary systematically over the range of possible scores (Figure 1).

The main finding in this study is that SPADI was more responsive than measurements of shoulder ROM. Comparing responsiveness of a self-evaluation questionnaire like SPADI and an impairment measure like shoulder ROM may seem odd to some readers. Few researchers have compared the responsiveness of shoulder scales with the responsiveness of traditional shoulder outcome measures. However, this type of comparison was a natural choice when investigating SPADI responsiveness in this population. Both SPADI and shoulder ROM have been used as outcome variables in several clinical trials involving patients with adhesive capsulitis. Furthermore, shoulder ROM was employed as a gold standard surrogate when testing criterion validity and responsiveness of SPADI in the original article by Roach et al. [1].

Reliability (reproducibility) is a necessary precondition for the appropriate application of change scores in general [42], but is especially important when interpreting change scores for individual patients [43]. Investigation of SPADI reproducibility with a test-retest design indicated that approximately 95% of the pairs of observations did not differ by more than 17 points. This is slightly better than what has previously been reported for other study populations where a similar design has been used [6, 8, 29].

Studying SPADI reproducibility, we observed a mean score difference between the two administrations of the questionnaire. The finding suggests some form of session bias. One could suspect that ROM measurements on the first test occasion caused temporarily increased pain in some patients. Another possible explanation is that patients were included early in the development of the condition, so that a stable pain situation had not yet been reached. Measurements in this study were analyzed according to a simple one-way ANOVA model, and thus we did not plan to control for a session effect. This means that our results might tend to over-estimate measurement errors. However, we analyzed data by a two-way ANOVA model post-hoc, and results for reproducibility were very similar to those reported in this study from the one-way model analyses.

Estimates for reproducibility were used in subsequent responsiveness analyses to produce a "reliable change proportion" (RCP) figure for SPADI which was compared with corresponding figures for shoulder ROM. A higher RCP was found for SPADI than for the individual movements of shoulder ROM. This could be interpreted either as smaller measurement errors or as a larger "change" for SPADI.

Group-level responsiveness was investigated by the "standardized response mean" (SRM) statistic. Again, the SPADI was more responsive than individual-movement ROM measures. The combined ROM measures were also generally more responsive than the individual-movement measures, the reason for this being the relatively smaller measurement errors for combined movements [32, 44]. From purely statistical and practical points of view, the SPADI appears as a more attractive outcome measure than shoulder ROM in this study. The ratio of sample sizes required to detect a given clinical effect is equal to the square of the ratio of standardized response means [45]. Hence estimates for SRM indicate two- or three-fold increases in necessary sample size if a single ROM movement is chosen instead of SPADI as the primary outcome measure in a trial of this type. Likewise, if SPADI and a ROM measure are both used as outcome measures in a trial, SPADI may have a better chance than the ROM measure to detect a treatment effect. The possibility to detect a significant difference is one thing, however. When comparing responsiveness of ROM and SPADI in this way, we compare the ability to detect change, but we are not comparing the ability to detect the same change. We analyze the change in scales which do not measure the same construct. Clearly, the aim of each separate study must guide in the selection of outcome measures.

Compared to previous studies reporting SRM for SPADI, estimates in this population are high. Five previous studies report much lower SRMs [6, 7, 9, 11, 34], while in a recently published study [10], SRM for SPADI was almost as high as in this study. The researchers also reported responsiveness of the cASES "Motion active" (SRM: 1.54) and "Motion passive" (SRM: 1.47), which are constructs that (roughly) correspond to the constructs C.AROM (SRM: 1.28) and C.PROM (SRM: 1.52) in the present study. Responsiveness for these ROM measures is quite similar in the two studies, and in both studies lower than the SPADI total. However, results for responsiveness are in general specific to the study population, the intervention and the overall design of the study. Comparisons across studies are difficult.

In this study, confidence intervals and test statistics for the SRM and RCP are calculated as if the estimated variability in scores (SD of change and SDD, respectively) represents the underlying "true" variability. This is an approximation. A more exact method would probably result in wider confidence intervals and more conservative test results.

Investigation of "external" responsiveness is sometimes performed in order to demonstrate that a new measure may replace an old one. This was not really our aim: we simply wanted to investigate the relationship between changes in shoulder ROM and SPADI in these patients. Since there is no valid way to measure change directly [42], comparisons were based on the difference between respective measurements for the two different time points. Correlations between improvement in ROM and SPADI were below 0.50. In the original study by Roach et al. [1], associations between SPADI and active ROM change scores were stronger, coefficients ranging from -0.52 to -0.70. Limited information regarding the variability of change scores in that study restricts further comparisons, however.

The ability to discern associations among variables is impaired by a lack of reproducibility in direct proportion to the products of the reliabilities of the measurements involved [46]. Furthermore, one must expect the reliability of a difference between measurements to be lower than the reliability of the separate measurements [44]. Hence there is reason to believe that the association between "true" changes for ROM and SPADI is somewhat stronger than indicated by the correlation coefficients reported in this study.

Of the translation process, the Stage VI (submission of reports to the developers of the original questionnaire) was not performed, and this is a weakness of the study. We were not able to get in contact with the respective researchers.

Results indicate that SPADI is more responsive than shoulder ROM measurements, which have been extensively employed as shoulder outcome variables in these patients. The relationship between changes in shoulder ROM and SPADI suggest that they measure overlapping underlying phenomena. The results in this study support incorporating the SPADI questionnaire in patient evaluation procedures when designing clinical trials where patients with adhesive capsulitis are investigated.