Introduction

There are currently several ways to assess the wheelchair skills of people with spinal cord injury (SCI). However, none of the methods available are ideal for the typical hospital setting while also being appropriate for the community.1 Existing assessments of wheelchair skills include; the Wheelchair Skills Test 4.2, Wheelchair Circuit Test, Adapted Wheelchair Circuit Test, Obstacle Couse Assessment of Wheelchair User Performance Test, The Five Additional Mobility and the Locomotor Item Test (which is used in conjunction with the Functional Independence Measure) and the Test of Wheeled Mobility and Short Wheelie Test.2, 3, 4, 5, 6, 7 Each of these assessments have their strengths, however none of the existing assessments are quick to administer while providing a summary of a person’s wheelchair skills that is sensitive to the types of changes seen in people with acute SCI. For example, the Wheelchair Skills Test 4.2, Obstacle Course Assessment of Wheelchair User Performance Test and the Test of Wheeled Mobility and Short Wheelie Test all take at least 30 min to complete.3, 5, 7 The Wheelchair Skills Test 4.2 was designed as a comprehensive checklist to guide wheelchair training programs and is appropriate for this purpose, but it is not a quick assessment that can be used in routine clinical practice to monitor change over time.8 In addition, most existing wheelchair skill tests cannot be easily performed in the typical hospital setting without establishing a dedicated environment that includes access to relatively large open spaces and specialised equipment. Specifically, the Wheelchair Circuit Test requires an adapted treadmill for wheelchairs, and the Test of Wheeled Mobility and Short Wheelie Test require six ramps of varying gradients and four different gutter thresholds.3, 4 The Obstacle Course Assessment of Wheelchair User Performance Test is also difficult to replicate because it requires access to a gravel pathway and a variety of gutters and ramps.7

Another problem with existing assessments is their inclusion of items that are not directly related to wheelchair skills. Unrelated items include those related to either transferring or moving about on a bed, and items related to other skills, such as folding up a wheelchair or opening a door.1, 2, 3, 4, 5, 6, 7 Most of these types of skills are already captured in the Spinal Cord Independence Measure (SCIM) or other equivalent outcome measures.9 Therefore, it is not efficient to duplicate these measures in clinical practice when assessing wheelchair skills. Accordingly, there is a need for a quick and simple test that can be used in conjunction with the SCIM or equivalent measures, and that focuses exclusively on a small set of key wheelchair skills. Therefore the purpose of this study was to develop and examine the reliability and validity of a test constructed to objectively assess the skills of manual wheelchair users with SCI. The test was designed for use in the acute hospital setting as well as in the community without requiring access to extensive or specialised testing equipment. It reflected skills necessary for both indoor and outdoor manual wheelchair use.

Methods

Phase 1: development of the tool

A literature review was conducted and followed by a Delphi panel process to develop the tool. The panel consisted of 12 physiotherapists with extensive clinical expertise across the continuum of care of SCI rehabilitation. Members of the research team facilitated the Delphi panel process (EJG, SMM). During the first Delphi round, the panel identified 31 items from existing wheelchair skill assessments for potential inclusion. In the second Delphi round, panel members were required to comment on whether items reflected competency in wheelchair skills and whether items were of relevance to people with SCI. They were also asked about the feasibility of scoring items in the hospital setting. From this, a prioritised list was determined. To be eligible for inclusion, items needed to capture changes in wheelchair skills across the continuum of care. They also needed to be testable within the acute hospital setting without requiring costly equipment or excessive space. The third Delphi round was held to achieve consensus regarding the final 5 items for inclusion in the instrument, with each item examining a key attribute of wheelchair mobility. The panel members provided confidential feedback to the research team throughout the process either in person or by telephone or email. The feedback was summarised anonymously and distributed in written format to the panel members. Consumer feedback from 2 individuals with acute SCI undergoing primary rehabilitation and 2 individuals living in the community with a SCI was sought. Feedback was requested on each of the selected items. During the fourth and final Delphi round, consensus was reached regarding the scoring needed for each item to ensure it had enough range to capture the varying levels of ability and competency of individuals with SCI who self propel a manual wheelchair. The resulting assessment tool was called the Queensland Evaluation of Wheelchair Skills (QEWS) (Refer to Appendix 1).

Phase 2: reliability study

The intra-rater and inter-rater reliability of the QEWS were determined by testing 100 participants who were recruited in two ways: either on admission to the Princess Alexandra Hospital Australia following an acute SCI, or through outpatient departments and local community organisations via the use of information brochures and posters displayed in waiting rooms. Participants were included if they had sustained a SCI and used a manual wheelchair as their primary form of mobility. Individuals were excluded if they were unable to propel a manual wheelchair independently indoors over flat surfaces or were unable to follow basic instructions. Informed consent was obtained from all participants and all applicable institutional and governmental regulations concerning the ethical use of human volunteers were followed during the course of this research.

Participants were scored three times; once by a first assessor and then twice by a second assessor. The first assessment was done in person. The participants’ performance was video recorded at this time. Using the video recording of the first assessment, a different assessor did the second and third assessments. There was a period of at least 2 weeks between the second and third assessments (Refer to Figure 1). All assessments were completed and scored by qualified physiotherapists. A total of ten physiotherapists were used across the course of the study, and they had a median (IQR) of 15 (9 to 26) years clinical experience in SCI (Refer to Table 1). Their assessment scores were used to determine both the intra-rater reliability and inter-rater reliability

Figure 1
figure 1

Diagram of phase 2: the reliability studies (n=100).

Full size image
Table 1 Demographic details of the raters (n=10)
Full size table

Intra-rater reliability was determined by comparing scores completed by the same rater on two occasions, and inter-rater reliability was determined by comparing scores from the two different raters of the same performance. For both sets of analyses, Cohen’s weighted kappa coefficient was used to evaluate reliability for each of the five items of the QEWS, with linear weighting applied to penalize disagreements of more than one level on ordinal item scales. An intra-class correlation coefficient (ICC) was used to determine reliability for the QEWS total scores. In addition, percent close agreements were calculated and Cronbach’s alpha was used to examine the internal consistency of the five items of the QEWS.

Phase 3: validity study

The third phase of the study was to assess the validity of the QEWS. This was done in the following two ways.

Phase 3a (concurrent validity): This was tested in the 100 participants used for Phase 2. The QEWS total score of the 100 participants was compared with physiotherapists’ global ratings of wheelchair skills, which were completed on the same days as the QEWS. The global rating involved rating overall performance on an 11-point scale that was anchored at the ends by the labels ‘very poor performance’ and ‘very high performance’. Generalized linear modelling was used to examine the association between the QEWS total scores and the physiotherapists’ global ratings of wheelchair skill performance. A model fitting exercise was conducted to determine the most appropriate model parameters for the data, using the Akaike Information Criterion as an indicator of model fit with penalty for model complexity. Gaussian (family) and Identify (link) parameters produced the closest model fit, although the significance of the association was consistent regardless of parameter selection. This association was also represented on a scatterplot with a line of best fit (Refer to Figure 2).

Figure 2
figure 2

QEWS total score vs global rating of wheelchair skills (n=100).

Full size image

Phase 3b (construct validity): This was tested in a subgroup of twenty participants who had recently sustained a SCI and commenced rehabilitation. They were assessed using the QEWS within 2 weeks of commencing rehabilitation and then 10 weeks later. The median (and IQR) change in QEWS total scores between the first and second assessment was calculated. These data were used to give an indication of the responsiveness of the instrument to improvement in wheelchair skills over time.

Results

Phase 1: Development of the QEWS

The QEWS consists of five items. Each item is scored on a 6-point scale ranging from 0 to 5 with a score of 0 indicating poor performance and a score of 5 indicating good performance. The scores for each item are tallied to a total possible score of 25. A brief description of each item is as follows (Refer to Appendix 1 for details).

Item 1: negotiating an indoor circuit

This involves 5 components, namely, propelling forwards, moving over a piece of wood (to represent a doorway threshold), negotiating cones in a figure of eight pattern, turning 180 degrees and propelling backwards. The number of components that the individual can independently perform determines the score.

Item 2: ascending and descending a ramp

This item examines the ability of an individual to ascend and descend a ramp. A 1:14 gradient ramp is required to reflect the ramps commonly used to access homes and community buildings. The score is determined by a combination of assistance required and time taken to ascend and descend the ramp.

Item 3: maintaining balance on back wheels

This item reflects the ability of an individual to maintain balance on the back wheels in a variety of positions; stationary position, moving forwards, turning 90 degrees, turning 360 degrees and descending a 1:14 gradient ramp. The score is determined by the difficulty of the skill performed.

Item 4: ascending and descending a gutter

This item consists of two parts. Firstly, the ability to ascend a 15 cm gutter and secondly, to descend a 15 cm gutter. The 15 cm gutter was selected to represent the height of most community gutters. A hierarchical scoring system is used, with the score reflecting the complexity of the skill that the individual can independently perform.

Item 5: six-minute push test

This item was included to provide an indication of an individual’s fitness and ability to propel a manual wheelchair for an extended period of time.10 The item involves pushing as far as possible in six minutes, and the distance pushed determines the score.

All scoring is done by observing the participant attempting the item. A score of 0 is given if the therapist or participant deems it unsafe to attempt an item.

Phase 2: Reliability Study

The demographic information for participants in the study is provided in Table 2. The results for the intra-rater and inter-rater reliability are shown in Table 3. The weighted Kappa values for each of the five items of the QEWS, for the intra-rater and inter-rater designs, ranged between 0.84 and 0.99, indicating very good reliability. In terms of agreement, scores were within one point of each other between 97 and 100% of the time. The intra-class correlation coefficients (and associated 95% confidence intervals) for the QEWS total score reflecting the intra-rater reliability and inter-rater reliability were 1.00 (0.99 to 1.00) and 0.98 (0.98 to 0.99), respectively, indicating good to perfect reliability.11 These scores agreed within one point of each other 96 to 91% of the time. The internal consistency reliability coefficient (Cronbach’s alpha) of the five items was 0.86.

Table 2 Demographic details of the participants used for Phase 2, 3a and 3b of the study
Full size table
Table 3 Phase 2: Intra-rater and inter-rater reliability of the five items of the QEWS and the QEWS total score (n=100)
Full size table

Phase 3: Validity study

Phase 3a (concurrent validity): A strong linear association (r2=0.93) between the QEWS total score and the concurrent global rating of wheelchair skills is displayed in Figure 2. The generalised linear modelling confirmed this to be a significant association (coefficient=2.30, 95% confidence interval=2.18 to 2.43, P<0.001).

Phase 3b (construct validity): The QEWS total score was higher at the second assessment by a median (IQR) of 4 (1 to 6) points.

Discussion

The QEWS is a quick, reliable and valid assessment of wheelchair skills in people with SCI. The test does not require specialised equipment although it does require a stopwatch, cones and piece of wood (to represent a doorway threshold) as well as access to a 10 and 30 m track, a ramp and a gutter. Each assessment can be completed in less than 20 min and sometimes considerably less depending on the characteristics of the individual. Item 5 is the most time consuming aspect of the QEWS because it includes a six-minute push test.

The intra-rater and inter-rater reliability of the five items of the QEWS and the QEWS total score were found to be very good and sometimes perfect. The use of videos to test reliability had both benefits and drawbacks. On the one hand the videos may have over-stated reliability because they removed some of the inherent variability that is associated with real-life assessments due to small differences in set-up and instructions to participants. On the other hand, the videos sometimes made it more challenging to rate performance because of difficulties in determining the amount of physical assistance required, particularly for item 4 (ascending and descending a gutter). In addition, it was sometimes difficult to determine the time taken to complete item 2 (ascending and descending a ramp) from the video because the start and finish lines were hard to see, where as this was not an issue for the therapists during real life assessments.

The validity of the QEWS was reflected in the high correlation between the QEWS total scores and the physiotherapists’ global ratings of wheelchair skills. Similar to other studies, the global rating of wheelchair skills was used to reflect therapists’ overall impression of the participants’ wheelchair skills.8 However, our estimate of the strength of association between physiotherapist global rating and the QEWS needs verifying in future studies, because the same therapists scored both the QEWS and global ratings, and it would be a stronger test if the two scores were provided by different therapists. Validity was also examined by demonstrating changes in scores over time. There was a median (IQR) of a 4/25 (1 to 6) point change in the QEWS total scores over the 10-week period. This amount of change was anticipated and demonstrates that the QEWS is appropriately sensitive to change in wheelchair mobility skill over time. The amount of change over time was not influenced by level of lesion. The change in the QEWS total score for participants with paraplegia (median, 4; IQR, 2 to 6) was similar to participants with tetraplegia (median, 4; IQR, 1 to 6). There was good spread of data across the QEWS scale (see Figure 2). This indicates that the QEWS does not have floor or ceiling effects.

Another important aspect of this study is that it provides important normative data on the wheelchair skills of 100 people with SCI (Refer to Table 2). To our knowledge, no other study has previously provided this type of information for such a large sample. This information could be used by therapists to gauge the typical level of wheelchair mobility attained by people with different types of SCI.

Conclusion

The QEWS is quick to administer and easy to use in the acute hospital setting. It is also reliable, valid and sufficiently sensitive to detect change in wheelchair mobility skills over a 10-week period. Further work is required to determine the usefulness of the instrument in the community settings.

Data archiving

There were no data to deposit.