Cross-cultural translation of the Western Ontario Cuff Index in Chinese and its validation in patients with rotator cuff disorders

The translation work of the WORC followed the principles of previously published guidelines [28, 29]. The entire process consisted of 5 steps: (1) Forward translation from English to Chinese by 2 bilingual translators independently (an orthopedic surgeon from our department and a professional translator). (2) For a synthesis of the translations, a discussion was held to integrate the 2 independent forward translation drafts; later we obtained the primary Chinese version of WORC (C-WORC). (3) Backward translation by 2 independent native English translators (OA and JR) who are well conversant in Chinese, the 2 translators have medical backgrounds, but no knowledge of the original WORC. (4) Creation of a pre-final version: a pre-final version was created by an expert committee after comparing the translated versions to the original version. (5) Twenty patients with RC disorders were invited to complete the pre-final C-WORC for assessment, and feedback was collected. A third meeting was conveyed by all research members for final adjustments according to this feedback, and the final version of the C-WORC was obtained.

Patients enrolled in this study were mainly recruited from those with RC disorders admitted to Chengdu Military General Hospital and Changhai Hospital of Shanghai from January 2015 to March 2016. The inclusion criteria were patients > 18 years old with Chinese as their mother tongue; definitely diagnosed as RC by history, physical examination, and appropriate radiological evaluations and scheduled for arthroscopic shoulder surgery. The exclusion criteria were patients with chronic inflammatory diseases or impairments in the cervical spine, elbow, or hand affecting the shoulder function and/or with other systemic diseases. The number of patients enrolled also needed to meet the standard sample size of the health questionnaire proposed by Terwee et al. [30], that is, scale results from at least 100 patients for internal consistency analysis and at least 50 patients were required for the reliability and validity analysis. All patients who participated in the study had carefully read and signed informed consent, and the clinical study was approved by the ethics committee of our hospital (No. CHEC 2015–012).

The patients were asked to provide demographic information such as sex, age and weight on the first day of enrollment, and to independently complete the C-WORC, OSS, SF-36, and C-RC-QOL (for another study) in a quiet meeting room. One week after the first day of enrollment, also the day before the arthroscopic surgery, they completed the C-WORC for the second time to evaluate the test-retest reliability of the scale. Six months after the surgery, when the patients came to our hospital for a regular check, they completed the C-WORC for the third time to help evaluate the responsiveness.

The WORC is a self-assessment scale that was developed to measure the quality of life of patients with RC disorders. It contains 21 items representing 5 subscales: physical symptoms (6 items), sports/recreation (4 items), work (4 items), lifestyle (4 items), and emotions (3 items), which encompass all aspects of heath as defined by the World Health Organization [31]. Each item is answered on a 100 - mm visual analog scale. The scores of 21 items are added to give a total score from 0 to 2100. To make scoring more understandable, the authors of the original version recommend that the data be converted to a percentage score by inverting the raw score and converting it to a score out of 100. A score of 0 is the worst score possible, and a score of 100 implies no reduction in the HRQOL [10]. If a response is lacking in any subscale, the lost item score can be compensated for by the mean of the other items in the subscale. Nevertheless, losses of more than two items in a subscale cannot be compensated, they must be listed as incomplete [32].

The OSS is also a self-assessment scale that was developed for patients with shoulder pain. It assesses the effect of shoulder joint disease on daily living and quality of life with 12 items [33]. Each item of the OSS has 5 answer options, each option corresponds to 0–4 points, and the score for the whole questionnaire ranges from 0 (worst) to 48 (best), indicating that patients with lower scores have worse shoulder joint function [34]. The SF-36 is a generic scale used to evaluate quality of life, and consists of 8 subscales with 36 items. Each subscale of the SF-36 has a special scoring method, and the final score is converted to a percentage (0–100). The lower the SF-36 score, the worse is the quality of life or functional status [35]. Chinese versions of these 2 scales are available that have been proven to have acceptable reliability, validity, and responsiveness [15, 36].

The reliability test of the C-WORC mainly included the evaluation of the test-retest reliability and internal consistency. The test-retest reliability of the scale was evaluated via the comparison of the first 2 rounds results’ of the C-WORC. The intraclass correlation coefficient (ICC), which was derived from a 2-way analysis of variance in a random effect model, was used as the evaluating index. The scale was considered to have good or excellent reliability when the ICC was > 0.8 or 0.9, respectively [37]. With Cronbach’s alpha as an evaluation index of the internal consistency of the C-WORC, the scale was deemed to show acceptable, good, or excellent internal consistency when this index was >0.7, 0.8, or 0.9, respectively [30]. We further depicted Bland-Altman plots to observe for systematic error between the first two rounds of investigations [38].

Then we assessed the validity of the C-WORC by evaluating both the content validity and construct validity. The content validity consists of the assessments of comprehensiveness and the relevance of items [39]. The item response rate, ceiling/floor effects, and patient feedback were the 3 indexes for comprehensiveness assessment. If the response rate for each item in the scale was >95%, the ceiling/floor effects of each subscale were <15%, and there were no difficulties in understanding the items that were fed back from the patients filling in the C-WORC, the scale was considered to have good comprehensiveness [30, 40]. In addition, a rehabilitation medicine expert and 3 orthopedics specialists were invited to help judge whether the items were relevant for the construct to be measured and for the population of patients with RC disorders [39]. Because there is no gold standard for evaluating the validity of the C-WORC, the hypotheses testing method was employed to evaluate the construct validity [39]. In this study, we selected the OSS and SF-36 as the control scales for the C-WORC. On the basis of the content of each scale, we hypothesized that the physical subscales of the C-WORC (physical symptoms, sports/recreation, work, life style) should be well correlated with the OSS and the physical subscales of SF-36 (physical functioning, role physical, bodily pain, general health), but poorly with the mental subscales of SF-36 (vitality, social functioning, role emotional, mental health). Correspondingly, the emotions subscale of the C-WORC should be well correlated with the mental subscales of SF-36, and poorly with the OSS and the physical subscales of SF-36. In addition, because the OSS is specifically designed to assess the patient’s function in the shoulder area, and the SF-36, however, is only a generic scale, we hypothesized that the correlation between the C-WORC and the OSS should be better than that of any subscales of SF-36. On the basis of the above hypotheses, we used the results derived from the first round investigation to calculate the correlation coefficient (r or r _s) of the C-WORC with the subscales of SF-36 and the OSS. In addition, all the scores were tested for normal distribution using the Kolmogorov-Smirnov test, and Pearson’s (r) and Spearman’s (r _s) correlation coefficients were used for parametric and non-parametric score data, respectively. The construct validity of the C-WORC was evaluated by comparing the compatibility of the results with our initial hypotheses. The correlations were judged as poor (r/r _s = 0–0.2), fair (r/r _s = 0.2–0.4), moderate (r/r _s = 0.4–0.6), good (r/r _s = 0.6–0.8), or excellent (r = 0.8–1.0) [41].

Finally, we evaluated the responsiveness of the C-WORC by comparing the scale results before and 6 months after arthroscopic surgery. Effect size (ES) and standardized response mean (SRM) were the 2 indices to evaluate the responsiveness. SRM was defined as the mean change between these time points divided by the SD of this change. The ES was defined as the mean change between the preoperative results and the 6-month postoperative results divided by the SD of the preoperative C-WORC score [42]. The ES and SRM were considered large if >0.80, moderate if between 0.51 and 0.80, and small if lower than 0.50 [43].

Statistical package for the Social Sciences, version 20.0 (SPSS, Chicago, IL, USA) was used for statistical analysis.

Overall, 152 patients (74 male and 78 female) with RC disorders admitted to both of the hospitals from January 2015 to March 2016 met the inclusion/exclusion criteria, out of which 124 patients (81.6% of those invited,69 male and 55 female) finally agreed to participate in the study upon our invitation. All patients completed the scales for the first two rounds, and 16 patients did not visit the hospital again to complete the third round scale 6 months after shoulder arthroscopic surgery. Hence, the sample size for the reliability and validity assessment of the C-WORC was 124, while that for the responsiveness assessment was 108. The patients initially enrolled were an average age of 47.3 (ranging from 20–66), and the duration of pain was 30.7 months on average (ranging from 1–72 months). More detailed demographic information is summarized in Table 1.

The forward and backward translation of the WORC went smoothly. In the process of translation, only item 17 was slightly modified; “roughhousing or horsing around with family or friends” was changed to “roughhousing or horsing around with friends”, which is more suitable for the Chinese culture. Overall, 20 patients (10 male and 10 female) with RC disorders had completed the pre-final version of the C-WORC, and no participant complained of irregularities in the items or difficulties understanding the items. This version was used as the final version in the subsequent validation phase without any further change.

The overall scale of the C-WORC had excellent internal consistency (Cronbach’s alpha = 0.950), and each subscale of the C-WORC showed good or excellent internal consistency (Cronbach’s alpha = 0.872–0.954) (Table 2). Moreover, the global test-retest reliability of the C-WORC was good (ICC = 0.893), and the test-retest reliability of each subscale was also good or excellent (ICC = 0.828–0.961) (Table 3). The Bland-Altman plots showed no systematic error between the results of the first two rounds (Fig. 1), which also confirmed and highlighted the good test-retest agreement of the C-WORC.

In the formal study, there was one item not answered in both the work and emotions subscales (1/124, 0.8%), and the 2 items missed appeared in the same questionnaire (Table 2). The average score of the subscale and each subscale was 30.9–42.9, and the emotions subscale had the highest average score and the work subscale had the lowest score (Table 2). The overall scale and each subscale of the C-WORC did not show a ceiling effect (0–0.8%) and a floor effect (0–1.6%). In addition, after the completion of the C-WORC, no patients reported any difficulties in understanding any items in the scale. Content analysis of the C-WORC was performed by the rehabilitation medical expert and orthopedics specialists and it was agreed that the data derived from each individual item in the C-WORC were sufficient for the HRQOL evaluation of patients with RC disorders. Therefore, no addition or deletion of any items was recommended. On the basis of the above results, we believed that the C-WORC had good content validity.

Data for construct validity assessment were listed in Table 3. The scores of the Sports/Recreation subscale (C-WORC) and all subscales of SF-36 (excluding the GH subscale) were not normally distributed, so Spearman’s (r _s) correlation coefficient was used for these subscales. The correlations between the physical subscales of the C-WORC and the OSS and the physical subscales of SF-36 were at least moderate (r/r _s = 0.472–0.787; P < 0.001), and that with the mental subscales of SF-36 were poor or fair (r/r _s = 0.180–0.368; P = <0.001–0.045). At the meanwhile, the correlations between the emotions subscale of the C-WORC and the mental subscales of SF-36 were at least moderate (r/r _s = 0.520–0.713; P < 0.001), and with the physical subscales of SF-36 and the OSS were poor or fair (r/r _s = 0.126–0.289; P = 0.001–0.045). In addition, the correlations between the physical subscales of the C-WORC and the OSS (r/r _s = 0.672–0.787; P < 0.001) were stronger than that with the physical subscales of SF-36 (r/r _s = 0.472–0.655; P < 0.001). The above results were consistent with our hypothesis, so it could be interpreted that the C-WORC has good construct validity.

Finally, we evaluated the responsiveness of the C-WORC by comparing the scales completed before and after arthroscopic surgery. Relevant data were listed in Table 3. In general, the average scores of the overall scale and other subscales had all increased after the arthroscopic surgery. Both the ES (1.57–2.27) and SRM (1.52–2.28) values exceeded 1.00, suggesting good responsiveness to the C-WORC.