Elements virtual rehabilitation improves motor, cognitive, and functional outcomes in adult stroke: evidence from a randomized controlled pilot study

Stroke patients were recruited from the inpatient rehabilitation ward of a large tertiary hospital in Sydney, Australia. All patients had been admitted to the ward to address identified upper extremity dysfunction following a unilateral stroke, confirmed on neuroimaging. Additional inclusion criteria included: (1) ability to communicate in English, and understand and follow oral instructions; and (2) ability to maintain sitting balance unassisted. Exclusion criteria included: (1) a previous history of neurological (other than stroke), psychiatric, or developmental disorder; (2) loss of visual acuity preventing perception of visual material; or (3) under 18 years of age. Rehabilitation staff assisted in the identification of eligible candidates. All participants (or their carers) provided written informed consent prior to participation.

The primary outcome was the Box and Blocks Task (BBT), which measures upper-limb motor skill [40]. The test is comprised of two hinged boxes (each 27 cm × 24 cm, with walls 8.5 cm high), separated by a vertical wooden barrier (15.2 cm high). One box is filled with 150 2.5-cm wooden cubes/blocks. The goal is to move as many blocks as possible over the barrier and into the opposite box in 60 s using one hand at a time. The BBT has been used frequently to assess motor function in patients with stroke, with demonstrated responsiveness to early post-stroke recovery [41] and high predictive validity, correlating in excess of 0.90 with more comprehensive examinations of upper limb motor function such as the Fugl-Meyer Assessment and the Action Research Arm Test [42], while offering the advantage of briefer administration.

Secondary outcomes inluded the Montreal Cognitive Assessment (MoCA), the Groton Maze Learning Task (GMLT) and the Set Shift Task (SST) from the CogState computerized assessment battery, and the Neurobehavioural Functioning Inventory (NFI). The MoCA is a brief (12-item) screening of general intellectual function across six domains: orientation, attention, language, visuospatial, memory, and executive function; scores below 26 out of 30 suggest cognitive impairment [43]. The MoCA has been repeatedly validated for assessing post-stroke cognitive status [44, 45], and its widespread use provides a standardized outcome with established clinical utility. At each assessment time point, a different one of three alternate forms was administered, with the order randomised and counter-balanced between participants.

The Elements tasks require attention, reasoning, and problem solving to discern task demands, and deduce the relationships and interactive principles at play (see descriptions below). Furthermore, Elements training requires participants to flexibly shift their response patterns from one task to another, or to inhibit an overlearned response in one particular task (Task 4: Go/No-Go). The GMLT and SST were therefore administered to measure these aspects of executive control [46]. The GMLT requires participants to deduce through trial-and-error learning a 28-step pathway hidden within a 10 × 10 grid. Participants repeat the same path several times, with the expectation subsequent trials will be completed more efficiently. In the SST participants are required to sort playing cards based on underlying rule sets related to either the color (red or black) or number shown on the card. The examinee must learn test rules through trial-and-error strategies, and flexibly shift sets as the test rules change over time. For both tasks, the outcome of interest was total errors, with lower scores indicative of better performance. Tasks from the CogState Battery were specifically designed to minimize practice effects and to be employed in repeated measures designs [46, 47].

Finally, the NFI is a 76-item patient reported outcome measure (PROM) of functional behaviour and symptoms in everyday life after ABI [48]. A range of symptoms commonly associated with neurological injury are distributed over six subscales: Depression, Somatic, Memory/Attention (aka Cognition), Communication, Aggression and Motor. The Motor sub-scale is not limited to upper-limb performance, but relates to whole-body strength, coordination and mobility, and the ability to complete daily tasks. The Cognition and Communication sub-scales measure the extent to which people lose track of time, forget important information, and have difficulty expressing themselves or comprehending others. The Somatic sub-scale measures physiological symptoms including headaches, nausea, dizziness and fatigue, while the Depression and Aggression sub-scales address emotional and behavioral issues, respectively. NFI total scores range from 70 to 350, with higher values indicating worsening ability to interact purposely with the environment [49].

Patients were stratified by age and type of stroke (ischemic or hemorrhagic), and then randomly allocated to the experimental (VR + TAU) or control (TAU alone) group. Concealed block randomization (1:1) was completed by breaking sequentially numbered opaque envelopes, pre-prepared by the study coordinator (JMR) using a random number generator (https://​www.​sealedenvelope.​com/​simple-randomiser/​v1/​lists). The research team and patients participating in this study were not blinded to assignment. Through medical chart review and patient interview, baseline information on sociodemographic and medical history, and current neurological and radiological data were collected. Assessment of motor, cognitive, and functional outcomes occurred at three time points: prior to Elements training (pre-test); immediately following training (post-test); and, one-month after the completion of training (follow-up).

The experimental and control group both received 3 h of daily conventional occupational and physiotherapy (i.e. TAU), provided by the treating allied health rehabilitation service at the hospital. TAU was individualized on the basis of collaborative care planning goals set by the patient and the treating team. Typical goals focused on range of motion exercises, muscle strengthening and coordination, and re-training of daily living skills (e.g. eating, grooming, toileting, dressing, transfers).

In addition to TAU, participants in the experimental group also received 12 sessions of VR, evenly distributed over four weeks. Elements training has previously been described in detail [37, 39], but briefly consisted of 30–40 min one-on-one sessions administered using a client-centered approach, with the level of individual task difficulty varied according to the participant’s level of performance and progress. Sessions were conducted with a registered psychologist, with training in virtual rehabilitation provided by the senior author, in a private therapy room at the hospital, free of distractions. Using four hand-held objects (i.e., the four “elements” in the shape of a circle, pentagon, triangle, and rectangle), the participant engaged with a virtual environment presented on a 42 in. touchscreen LCD panel (Multitaction™) with inbuilt CPU. Elements tasks included: Task 1 (Bases) consists of the home base and four potential movement targets, all 78 mm in diameter. The circular targets are cued in a fixed order (east, north, west, south) using an illuminated border. Task 2 (Random Bases) has the same configuration of targets, but they are highlighted in random order. Task 3 (Chase Task) begins with a blank screen. A target circle then appears randomly in one of nine locations. These locations are configured along three radials emanating from the home base. Task 4 (Go/No-Go) uses the same target positions as Task 3, however, additional distractor targets (a pentagon, triangle and rectangle) appear. Participants are instructed to place the object on the circular targets only and to resist moving to distractors. Tasks 5, 6 and 7 require participants to explore the virtual environment, by creating various shapes and sounds through movement.

During each session, participants progressed through a series of unimanual, goal-directed tasks (Tasks 1–4), followed by an exploratory task (Tasks 5–7) of their choosing (Fig. 1). The four goal-directed tasks involve movement (lift or slide using one hand) and placement of the circular hand-held object toward select targets; performance metrics (speed and accuracy) for the least and most affected hand are logged at the end of each task, and plots showing progress over time are discussed at the end of each session (i.e. explicit feedback). During the tasks, augmented auditory and visual feedback is presented in real-time, reinforcing movement-related attributes like speed, trajectory and endpoint contact (i.e., implicit feedback). For example, visual AF includes a fading trail of a hand-held object’s path, a waxing luminescence around targets as an object approaches, and ripple effects when an object is placed on a target. Auditory AF includes one tonal source that increases in pitch with greater movement speed, a second tone that increases in pitch as an object approaches a target, and a third tone emitted when an object is placed on a target.

The three exploratory tasks included Mixer (Task 5), Squiggles (Task 6), and Swarm (Task 7). Mixer consists of nine circles in a 3 × 3 grid. Moving the circular hand-held object close to a circle starts to activate its sound and spinning border animation. The pitch and tone of the sound vary according to the hand-held object’s proximity to the circle. Participants can activate different combinations of circles at any time to produce an overall soundscape. Squiggles presents a blank display upon which participants can draw lines and shapes by sliding any of the four different hand-held objects across the screen. As each object is moved, a trail animation is drawn along its path and a musical tone plays. Once the participant lifts the object, the trail animates and moves across the screen. Each object has a unique visual trail and musical tone. Swarm encourages bimanual control to explore the audiovisual relationships between all four hand-held objects. When placed on the screen, multiple colored shapes slowly gravitate toward and swarm around the base of each hand-held object. As each object is moved, its associated swarm follows. The movement, color, size and sound characteristics of each swarm change when the distance between objects is altered and the different swarms overlap/interact.

The Elements tasks were designed to exploit a range of factors known to enhance training intensity and motor learning: concurrent AF [50]; embodied interaction using tangible interfaces [51, 52], and heightened task engagement using a combination of goal-directed and exploratory tasks. These factors are thought to enhance not only the motor control/learning processes that underpin movement skills [53], but also aspects of cognitive control which, together, improve the transfer of skill to everyday behavior [14, 54]. The principled design in VR is important; customized systems like Elements tend to be more effective than off-the-shelf commercial gaming systems [9, 12], which encourage nonspecific movements and may not permit participant-specific settings.

With a desired power of 0.80, and the expectation of a medium treatment effect (Cohen’s d > 0.50) based on reported changes in motor activity (i.e. Box and Blocks test) in comparable VR studies [31, 55, 56], a sample size of 10–12 participants per group was determined as adequate (G*Power 3.1.7 program, http://​www.​gpower.​hhu.​de/​). This calculation was also qualitatively consistent with the sample sizes in methodologically similar proof of concept studies of VR for stroke [57‐59].

All data was checked for normality using Shapiro-Wilk’s and Levene’s tests; where violations were detected the non-parametric alternative Mann-Whitney U test was applied. The analysis of training effects was conducted in three parts. First, scores on each outcome measure at each time point (pre-test, post-test, follow-up) were compared between the two groups using a series of 95% CI’s and independent t-tests. To control the rate of false positives in the planned multiple comparisons, the Benjamini–Hochberg procedure [60], with a false discovery rate of 0.07, was applied to determine the number of significant results in each family of tests (6 motor outcomes, 6 cognitive outcomes, 12 participation outcomes). Second, for each treatment group, the significance of pre-test to post-test and pre-test to follow-up change on each outcome measure was analyzed using dependent t-tests, and the magnitude of each effect reported as Cohen’s d. Effect sizes were interpreted according to the conventions of Cohen [61]: small ≥ 0.2; medium ≥ 0.5; and, large ≥ 0.8. Third, pre-post change scores were applied to calculate the Proportion of Achieved Recovery for each group. Calculated as [100 x (score_post-test – score_pre-test) / (score_maximum – score_pre-test)], proportional recovery normalizes change scores across the severity spectrum for comparing recovery among cohorts with both severe initial impairment (and hence greater room for improvement) and more mild initial impairment [62]. Proportional Recovery Scores were not used to predict recovery. For the MoCA (higher scores denote better outcomes) and the NFI (lower scores denote better outcomes), the maximum scores were 30 and 70, respectively. For the BBT, gender-, hand dominance-, and age-based healthy adult normative data published by the original authors were fitted to each individual participant and used to define the maximum score [63].

We acknowledge the limitations of this research. First, although comparable with other proof of concept studies of VR for stroke [21, 57‐59], the current study was based on a modest sample of 21 participants. While the effect sizes of the experimental group were all large [61], suggesting the study was adequately powered, replication in a bigger sample is encouraged, permitting analysis of potential moderators of the extent of proportional recovery achieved, and enhancing the generalizability of the findings. In particular, the current cohort was predominantly of mild severity (as per the NIHSS), and due to ischemic stroke. These characteristics are representative of the natural incidence of stroke, wherein > 75% of strokes suffered by older adults are ischemic [98], and more than half of all ischemic strokes are classified on the NIHSS as mild [99]. However, it is uncertain how the current findings apply to hemorrhagic stroke and patients with more severe baseline neurological deficits.

Active control group designs are typically preferred for their capability to presumably control for Hawthorne effects and other biases when comparison groups are not balanced in terms of time in therapy. While the design of the current study was informed by our recent meta-analysis of VR for stroke that suggested effect sizes are not over-estimated in studies that lack of an active control group [9], we acknowledge that the intervention group received around 7 h of additional activity over the course of the current study. Although this represented only a small increase (12%) above the approximate 60 h of conventional rehabilitation the control group received, future studies could attempt to minimize any potential confound by balancing the “dose” of training between groups. Furthermore, functional outcomes in the current study were derived from subjective self-report. While PROMs can be more predictive of rehabilitation outcomes than objective performance measures [100], they may be vulnerable to recall and expectancy bias, and post-stroke impairments in self-awareness [101]. Independent corroboration of functional outcomes from a reliable informant is recommended for future studies. Finally, outcomes assessors were not blinded, which can increase the risk of detection bias [102].

Follow-up in the current study occurred only one-month after the completion of training. However, rehabilitation gains realized in the first six-months after stroke may not be maintained [2], and longer follow-up periods are encouraged to provide more robust data on the durability of VR. In addition, the resources required to implement the Elements VR program were admittedly substantial, and a barrier to widespread adoption into routine clinical practice [103]. Development and evaluation of a lower-cost, portable, tablet-based version of the Elements system is therefore underway. Finally, while the validity of predicting recovery after stroke using the Proportion of Achieved Recovery formula has recently been challenged [104, 105], when used simply as a tool for comparing recovery we note only two patients in the experimental group, and none in the control group, achieved what could be described as a complete or near-complete recovery (e.g. achieved ≥70% of their proportional recovery [106]) during the four-week training period. To achieve the dose and intensity of therapy needed for neural reorganization and functional change, exploration of alternatives and adjuncts to clinician-led inpatient rehabilitation are encouraged, such as VR that can also be delivered in the community [107], and that patients can self-administer [108, 109].