Structural validity of the Boston Carpal Tunnel Questionnaire and its short version, the 6-Item CTS symptoms scale: a Rasch analysis one year after surgery

The most common nerve entrapment is carpal tunnel syndrome (CTS) [1], which is caused by chronic pressure on the median nerve in the carpal tunnel of the wrist. Different etiological and risk factors for carpal tunnel syndrome have been described, including repetitive hand or wrist use, a high body mass index, tenosynovitis of the flexor tendons, and having a close relative with carpal tunnel syndrome [2‐4]. The main symptoms are pain, numbness and tingling in the hand and arm. Carpal tunnel syndrome can cause functional deficits, typically weakness of palmar abduction or even complete inability to oppose the thumb. Rehabilitation using various maneuvers or night splinting of the wrist are used in mild cases, while carpal tunnel release (transection of the transverse carpal ligament) is an established treatment in more severe cases or when non-operative treatments fail to improve the symptoms [4].

Measurement of the treatment effect of carpal tunnel syndrome has shown an increasing shift towards the use of patient-reported outcomes (PROs) [5]. Condition-specific measures can provide valuable patient-centered information on the success of treatment [6]. The Boston Carpal Tunnel Questionnaire (BCTQ) PRO measure was developed in 1993 to assess symptoms and functional impairment caused by carpal tunnel syndrome [7]. The BCTQ has since been extensively tested using classical test theory and longitudinal validation methods [8‐18]. It is assumed that a well-constructed PRO instrument designed to measure one latent trait will display a unidimensional structure. Unidimensionality means that the items in a questionnaire measure only a single underlying construct. This enables more accurate evaluation of longitudinal change in scores on the underlying trait. In clinical practice and research, it is also important that measures that adequately provide the requisite information are as short as possible. For these reasons, Atroshi and colleagues applied exploratory factor analysis and the Item Response Theory (IRT) to the BCTQ and produced a short version of the symptoms scale which they called the CTS-6 [19]. However, the construct validity of the BCTQ or CTS-6 has not been analyzed using the Rasch Measurement Theory (RMT).

RMT analysis provides information on the structural validity of the BCTQ and CTS-6. The RMT model allows investigation of the ability of a scale to measure a latent trait such as symptoms or function. Moreover, item and scale fit in the predefined model and responses to the construct-specific scales can be tested. Furthermore, response bias can be tested for each scale item using differential item functioning (DIF). DIF occurs when different groups within the sample, e.g., men and women, respond differently to an individual item. Whereas classical test theory and several other psychometric methods test how well a model fits the data, the RMT tests how well the data fit a predefined model. The RMT can thus be considered more robust and rigorous for psychometric analyses of construct validity.

This study applied the RMT model to the BCTQ and the CTS-6 to investigate their unidimensionality, item and scale fit, residual correlation, differential item functioning, scale coverage/targeting, and person separation in patients who had undergone carpal tunnel release due to carpal tunnel syndrome.

In total, 259 patients participated in the study (response rate 49%). Of these, 217 had provided adequate data and were included in the analysis. Patients’ sociodemographic and clinical data are presented in Table 1.

Rasch Measurement Theory analysis of the CTS-6 demonstrated a unidimensional structure after some adjustments, acceptable item fit to the model, a few disordered item thresholds and one significant differential item functioning favoring gender. The BCTQ Symptom Severity Scale in turn demonstrated multidimensionality, acceptable item fit to the RMT model, several disordered item thresholds and significant differential item functioning favoring gender and age. The BCTQ Functional Status Scale demonstrated a unidimensional structure, acceptable item fit, and several differential item functioning items favoring gender and age.

In this study, the CTS-6 was not found to have latent trait unidimensionality, raising concern as to whether the resulting score is valid when all items are summed. To solve the problem of multidimensionality, the non-fitting items can either be removed or new testlets created. While the CTS-6 showed good item fit (see below), the creation of a testlet for frequency of symptoms or testlets for pain and numbness were needed to solve the problem. Although the testlet for the frequency of symptoms demonstrated a unidimensional structure, two other testlets for pain and numbness were created, as these are clinically important symptoms and relevant to patients with carpal tunnel syndrome. The items in the testlets for pain and numbness were in a logical clinical relation to each other, and well suited to the formation of two separate subscales. Since both testlets satisfied the assumptions of the unidimensionality, the results of the study indicate that the CTS-6 might perform well when separated into item sets measuring pain or numbness (i.e., items 1, 2, 5 for pain and items 3, 4, 6 for numbness) in patients who have undergone carpal tunnel release.

We could not compare our results with previous studies because of the paucity of CTS-6 structural validity studies using the RMT model in patients with carpal tunnel syndrome. However, Atroshi and colleagues [19], who developed the CTS-6 from the longer version of the BCTQ Symptom Severity Scale by using exploratory factor analysis and IRT, found that in the patients undergoing carpal tunnel release, one dominant factor explained 58% of the variance, and that all 6 items were associated with that factor. Thus, they reasoned that the CTS-6 is unidimensional, as the first factor should be dominant and account for more than 20% of the variability. The difference between our result and that of Atroshi et al. may be explained by the different statistical methods used and patient groups studied (non-operated vs. operated).

In this study, unidimensionality was supported when the items of the BCTQ Symptom Severity Scale were bundled into two polytomous super-items. The BCTQ Functional Status Scale, however, demonstrated unidimensionality, indicating that all eight items in the scale measure the level of hand-related disability.

Item fit to the model was indicated to be good in the CTS-6 as well as in both BCTQ scales. In addition to item fit, person fit was also good at the questionnaire and testlet levels. This may partially be explained by the fact that the present sample of patients studied 1 year after carpal tunnel release surgery was rather homogeneous.

In the CTS-6, only item 6 (“How often did the following symptoms in your hand wake you up at night? - numbness or tingling”) exhibited significant non-uniform differential item functioning favoring gender after Bonferroni correction. The results suggested that women with mild symptoms tend to report waking up more often than male counterparts, whereas men with moderate to severe symptoms tend to report to wake up more often than female counterparts.

One item (“How long on average does an episode of pain last during the daytime?”) on the BCTQ Symptom Severity Scale exhibited non-uniform differential item functioning favoring age. This suggests that subjects under age 60 with moderate symptoms in the middle of the curve (Fig. 6) tended to report longer pain duration than those aged 60 or older. The BCTQ Symptom Severity Scale also contained one uniform differential item functioning favoring gender, viz. item 7 (“Do you have weakness in your hand or wrist?”). This suggests that, on the same level of symptoms, men tended to report more weakness in their hand or wrist than women. This is in line with Atroshi et al. [19] who found significant differential item functioning in item 7, with men showing higher values than women in patients undergoing carpal tunnel release. In the BCTQ Functional Status Scale several significant uniform and non-uniform differential item functioning items favoring gender and age were found, and hence the scale may give biased estimates of hand-related disability at different ages in both women and men. Taken together, the items of the CTS-6 and BCTQ Symptom Severity Scale showed that they are relatively invariant in both genders at different ages, whereas the BCTQ Functional Status Scale contains several invariant items. The BCTQ Functional Status Scale, in particular, could be improved by changing or deleting items in order to generate a differential item functioning-free scale for subjects who have undergone carpal tunnel release.

In their targeting ability, both the CTS-6 and BCTQ Symptom Severity Scale covered patients, as their mean obtained location score was around zero. This indicates a well-targeted measure which is neither too easy nor too hard. A positive mean location score would indicate that the sample as a whole was located at a higher level of symptoms than the mean, while a negative value would indicate the opposite. The results of the CTS-6 and BCTQ Symptom Severity Scale also revealed no differences between the age or gender distributions in the person-item threshold. Instead, the mean location score for the BCTQ Functional Status Scale was negative, indicating that the sample as a whole was located at a lower level of disability than the mean. In practice, this means that the BCTQ Functional Status Scale may have limited ability to detect functional status, or changes in it, in patients who have undergone carpal tunnel release and thus already have lower disability. This may especially be the case in younger subjects, as in this study the patients under age 60 exhibited lower disability values than those aged 60 above. No gender difference was noted in the person-item threshold distribution.

In the RMT model, the person separation index is used instead of reliability indices. However, the person separation index is analogous to a reliability index. The separation index describes the ratio of genuine separation to separation including measurement error. In the present study, the person separation index values of 0.73, 0.86 and 0.77 for the CTS-6, BCTQ Symptom Severity and Functional Status Scales, respectively, demonstrated good reliability for all three measures, as a minimum value of 0.7 is required for group use and 0.85 for individual use [29]. These reliability values are on more or less the same level as observed by Atroshi et al. [19] In their study, the reliability of the person separation index was 0.87 for the Symptom Severity Scale, and 0.88 for the CTS-6, indicating that the performance of the CTS-6 is similar to that of the original 11-item Symptom Severity Scale. In our recent report on a test-retest reliability experiment with most of the same subjects (N=193), we showed, from the perspective of classical test theory, that both of the BCTQ scales had high internal consistency, with a Cronbach’s alpha of 0.93 for both symptoms and function [21]. In the RMT partial credit model analysis, the calculation is equivalent to the Cronbach’s alpha, except that the logit value, as opposed to raw score, is used in the same formula. Both reports, with slightly different sample sizes, confirm that the scales perform well in separating respondents on the latent trait continuum.

Item threshold analysis may show a disordered threshold, if the response option wording is ambiguous, or if respondents find it difficult to discriminate between response options. In the present item threshold analysis of the CTS-6, items 3 (“How severe are the following symptoms in your hand? - numbness or tingling at night”) and 6 (“How often did the following symptoms in your hand wake you up at night? - numbness or tingling”) were disordered. In both items, merging response categories 3 (moderate/2 or 3 times) and 4 (severe/4 or 5 times) in the 5-point scale resulted in disordered response category thresholds in item 5 (“How often did the following symptoms in your hand wake you up at night? –pain”). However, merging response categories 3 and 4 in the 5-point scale produced ordered thresholds for each item and made the response categories work as intended. These results suggest that response options 3 and 4 in certain original items of the CTS-6 may be semantically or numerically too close to one another.

In the item threshold analysis of the BCTQ Symptom Severity Scale, 7 of the 11 items showed ordered categories. In the 4 items which showed disordered thresholds (items 4, 5, 7 and 11), merging response categories 3 (≈ moderate) and 4 (≈ severe) in the 5-point scale produced ordered thresholds. All eight items in the BCTQ Functional Status Scale had ordered thresholds, indicating that patients were able to differentiate between the response options.

The main strength of this study is that it is the first to systematically investigate the structural validity of the BCTQ and 6-item CTS by applying the RMT Model. We also had a sample of well over 200 patients, which may be regarded as of sufficient size for conducting psychometric analyses [20].

A limitation is that we only had patients who had undergone surgery for treatment of carpal tunnel syndrome. Thus, the results are generalizable mainly to patients in that situation. Another limitation relates to the fact that the questionnaires were distributed to individuals 1 year after surgery. It is thus possible that many symptoms will have resolved by this time point. The third limitation concerns the cross-sectional study design, which does not allow the monitoring of the test-retest reliability and responsiveness of the questionnaires. We have, however, previously measured and reported on the test-retest reliability of the BCTQ (Symptom Severity Scale and Functional Status Scale) with the most of the same patients [21]. Both scales showed excellent test-retest reliability, each with an intraclass correlation coefficient greater than 0.8. Nevertheless, the responsiveness of the BCTQ and CTS-6 remains to be studied in heterogeneously treated carpal tunnel syndrome patients. To the best of the authors’ knowledge, this is the first study to apply the RMT to the CTS-6 and the BCTQ. The RMT has advantages in assessing measurement constructs and item response categories. The RMT model utilizes predefined mathematics-based criteria into which the data should fit for successful measurement of a latent trait. The generalizability of the present results to other language versions is strong. The RMT assumes that there is one latent trait in the scale. This has its limitations in the analysis, such as when assessing a scale for symptoms that may have multiple aspects. Further research could investigate how the CTS-6 functions in longitudinal settings both as a 6-item scale and as two distinct scales for pain and numbness. This would provide further knowledge on the signal-to-noise ratio for the CTS-6 index score and its scores measuring for two different latent traits.

By fitting the data to the RMT model, the CTS-6 showed superior psychometric properties compared to its original version, the BCTQ Symptom Severity Scale, in surgically treated carpal tunnel release patients. The CTS-6 might perform more accurately if separated into two sets of items with separate scores measuring pain or numbness as a specific latent trait. The Functional Status Scale of the BCTQ has acceptable structural validity, although several of its items display differential item functioning favoring women or men and different age groups.