Background
In most countries, stroke is among most common causes of death and one of the main causes of acquired adult disability [
1]. Because most patients with stroke survive the initial illness, the greatest impact is usually caused by the long term consequences for patients and their families [
2]. It is estimated that 33 to 42 % of stroke survivors require assistance for daily living activities three to six months post stroke, and of these, 36 % continue to be disabled five years later [
3,
4]. Although remarkable developments have been made in the medical treatment of stroke, it continues to heavily rely on rehabilitation interventions. In addition to motor disabilities, more than 40 % of stroke survivors are left with cognitive impairment after the event and almost two thirds are affected by mild cognitive impairment, and therefore are at risk of developing dementia [
5]. Besides having a direct influence on the quality of life of patients and their caregivers, cognitive impairment after stroke is also associated with higher mortality [
6] and greater rates of institutionalization [
7]. Cognition is important for overall recovery since its impairment reduces a person’s ability to plan and initiate self-directed activities, to solve problems, to sustain and divide attention, to memorize information and to understand task instructions. It has been shown that recovery of cognitive function of stroke patients in inpatient rehabilitation is directly related to their level of participation in rehabilitation activities [
8]. Thus, reducing the impact of post stroke cognitive impairment through appropriate rehabilitation programs is an essential goal.
Current cognitive rehabilitation practice tends to be directed towards isolated cognitive domains including attention (focusing, shifting, dividing or sustaining), executive functions (planning, inhibition, control), visuo-spatial ability (visual search, drawing, construction), memory (recall and recognition of visual and verbal information) and language (expressive and receptive) [
9]. Although there is evidence on the efficacy of current methods [
10], an important concern is how effectively the improvements of these abilities that are trained separately generalize, leading to sustained improvement in everyday functioning [
11,
12]. When we consider the cognitive domains required for activities of daily living (ADL’s) such as a successful meal preparation – the patient must define a menu, identify the needed ingredients, write a shopping list, organize the time for shopping and preparing the meal – we acknowledge that multiple dimensions of cognition are engaged and, thereby, suggesting that need to be rehabilitated as a whole as opposed to independently [
13]. Unfortunately, there is insufficient evidence to determine if and how the ecological validity of current cognitive rehabilitation methods impacts recovery [
14,
15].
Current cognitive rehabilitation methodologies suffer other limitations besides the generalization of improvements to functional activities, social participation and life satisfaction. For instance, it is known that an intensive and individualized training is preferable [
16]. Personalized rehabilitation involves an assessment of each patient’s impairments, a definition of attainable goals for improvement, an intervention to assist in the achievement of goals and, finally, a reassessment to measure improvements [
2]. However, in-depth patient assessment is expensive and time consuming, and currently impracticable due to the scarcity of professionals and resources, resulting in a suboptimal intensity, personalization and duration of rehabilitation interventions [
17]. Further, although there is growing evidence that patients may achieve improvements on functional tasks even many months after having a stroke [
18], most rehabilitation therapies are only guaranteed within three to 6 months post stroke [
19]. Additionally, a James Lind Alliance study [
20] interviewed 799 chronic stroke patients who reported that cognitive problems had not been addressed appropriately, especially when compared with mobility, confirming that it is essential to find adaptable and accessible tools that can be used frequently and intensively by patients at the clinic or at home after discharge, in order to maximize rehabilitation outcomes. Caregivers and health professionals were also interviewed and indicated that investigating ways to improve cognition after stroke should be a research priority [
21].
Virtual Reality (VR) and interactive technologies have emerged as a valuable approach in stroke rehabilitation by providing the opportunity to practice cognitive and motor activities that are not or cannot be usually practiced within the clinical environment, such as training attention abilities in street crossing situations [
22], executive functions by visiting a supermarket [
23], or performing simulations of real-life scenarios and activities in urban virtual environments [
24,
25]. Yet, the advantages of VR to address stroke impairments go beyond ecological validity of training, with a growing body of evidence especially in the motor rehabilitation domain [
26]. Virtual environments are designed to be more enjoyable than conventional rehabilitation methods. The introduction of gaming elements and immediate feedback on performance enhance motivation, thereby encouraging higher numbers of repetitions [
27]. Additionally, it enables the systematic presentation of stimulus and challenges in a hierarchical fashion, which can be varied from simple to complex upon success [
28], making it progressively challenging according to patients abilities. Further, when stroke survivors suffer of hemiparesis in their dominant arm, this interferes with their ability to perform paper-and-pencil tasks, which in turn may impede cognitive training. Thus, another central advantage of VR is the possibility to be integrated with accessible interfaces such as adapted joysticks, natural user interfaces or robotic systems [
29].
Despite important scientific and engineering activity in VR based systems for cognitive and motor rehabilitation, the majority of studies to date have evaluated interventions that were designed to address motor impairments. According to the most recent Cochrane review [
26], there are only few randomized controlled studies that include cognitive rehabilitation and/or cognition assessment. Kim and colleagues [
30] performed a study with USN patients, where 12 experimental group patients received computer-based cognitive rehabilitation, including IREX system® (Vivid group, Toronto, Canada), and 12 control group patients received only computer-based cognitive rehabilitation with ComCog® (Maxmedica Inc., Seoul, Korea). Their results suggested that VR training might be a beneficial therapeutic technique on USN in stroke patients. Kim and colleagues [
31] also investigated the effect of VR on the recovery of cognitive impairment in 28 stroke patients by comparing VR training with the IREX system® to computer-based cognitive rehabilitation with ComCog®. Results showed significant improvements in both groups, with the VR group having greater improvements in the attention domain. A study from Chirivella and colleagues [
32] had 12 stroke patients in a stroke rehabilitation program using Neuro@Home, a cognitive and motor software-based rehabilitation platform. After an intervention of 8 weeks with 60 min sessions focused in attention, working memory, executive functions and visual perception training, patients showed significant improvements in attention, memory or executive functions. More recently and, in a more ADL’s simulation perspective, Gamito and colleagues [
33] tested the effectiveness of a VR application for neuropsychological rehabilitation in a group of 20 stroke patients. Results showed significant improvements in attention and memory functions in the intervention group, but not in the control group, not subject to any intervention. Also in an ADL’s perspective, a pilot study from Rand and colleagues [
34] explored the potential of a virtual supermarket (V-Mall) with 4 stroke patients. The intervention entailed ten 60-min sessions and was focused on improving multitasking while the participant was engaged in a virtual shopping task. Their main results support V-Mall potential as an effective tool for the rehabilitation of post stroke multitasking deficits during the performance of daily tasks. Most of these VR-based interventions do not address cognitive deficits in an integrative manner [
30,
32,
33], or are not ecologically valid [
30,
31]. The ADL’s simulation systems may represent a better real-world transfer rehabilitation, however, these systems lack difficulty customization [
33,
34]. The AGATHE project developed a tool to suppress this demand, offering patients customized rehabilitation sessions through simulated ADL’s [
25], however there are no efficacy clinical trials with this tool. Overall, we can conclude that results are encouraging but further research is needed, especially to clarify if VR, and more concretely training through the simulation of activities of daily living, is equivalent or more effective than conventional cognitive training [
26].
In this paper we present a one-month clinical randomized controlled trial with 18 stroke patients: nine performing a VR-based intervention and nine performing a conventional intervention. The VR-based intervention involves a virtual simulation of a city – the Reh@City – where several activities of daily living are trained. Reh@City enables an integrative and personalized cognitive rehabilitation process, targeting several cognitive domains such as memory, attention, executive functions and visuo-spatial abilities in a more ecologically valid approach. Additionally, Reh@City makes the interaction with the virtual world accesible through its interface, and the complexity of the scenarios is adapted to the patients’ profile.
Results
According to the Kolmogorov-Smirnov (KS) test, data were normally distributed in both groups for age (KS
Experimental = .156,
p = .200; KS
Control = .196,
p = .200) and in the control group for years of schooling (KS
Experimental = .394,
p = .001; KS
Control = .267,
p = .063). Data were not normally distributed for gender, lesion location and months post-stroke. No differences between groups were found with the Mann-Withney test (Table
2).
Table 2
Demographic characteristics (presented as Medians and IQR) of both groups and differences between groups (MW)
Age | 58 (48–71) | 53 (50.5–65.5) | 35.000 | .666 |
Gender | Female = 55.6 %; Male = 44.4 % | Female = 55.6 %; Male = 44.4 % | 40.500 | .100 |
Schooling | 4 (4–10.5) | 9 (4–9) | 46.500 | .605 |
Lesion location | Right = 55.6 %; Left = 44.4 % | Right = 55.6 %; Left = 44.4 % | 36.000 | .730 |
Months post-stroke | 7 (4–49) | 4 (3–11.5) | 23.000 | .136 |
Concerning the neuropsychological assessment measures at baseline, data were normally distributed in both groups for ACE (KSExperimental = .218, p = .200; KSControl = .185, p = .200) and only in the control group for the TMT A time (KSExperimental = .390, p < .001; KSControl = .169, p = .200) and the Picture Arrangement test (KSExperimental = .371, p = .001; KSControl = .240, p = .143). Data were also normally distributed in both groups for the subjective general health status for the memory (KSExperimental = .227, p = .200; KSControl = .122, p = .200), emotion (KSExperimental = .254, p = .096; KSControl = .147, p = .200), communication (KSExperimental = .151, p = .200; KSControl = .175, p = .200), ADL’s (KSExperimental = .159, p = .200; KSControl = .204, p = .200) an overall recovery (KSExperimental = .269, p = .059; KSControl = .264, p = .071) SIS dimensions. Social participation had a normal distribution only in the control group (KSExperimental = .299, p = .020; KSControl = .149, p = .200).
Global cognitive functioning
Table
3 describes the global cognitive functioning, as assessed by the ACE, of both groups in the pre and post intervention assessments. A Wilcoxon test for within-groups differences revealed that only the experimental group presented significant statistical improvements between pre and post assessment moments in both ACE (W
(9) = 44.000, Z = −2.549,
p = .011,
r = .85) and MMSE (W
(9) = 34.000, Z = −2.246,
p = .025,
r = .75). Additionally, we also have found significant improvements in attention (W
(9) = 28.000, Z = −2.375,
p = .018,
r = .79), memory (W
(9) = 28.000, Z = −2.384,
p = .017,
r = .79) and visuo-spacial ability (W
(9) = 28.000, Z = −2.388, =.017,
r = .80) domains only in the experimental group. Concerning the control group, the only significant change was a decline in verbal fluency (W
(9) = 2.500, Z = −2.209,
p = .027,
r = .74).
Table 3
ACE and MMSE scores (presented as Medians and IQR) pre and post intervention with within-groups (W) comparisons and pre to post-intervention difference with between-groups (MW) comparisons
ACE-Total | 72 (61–75.5) | 81 (68–86.5) | 44.000 |
.011
| 66 (54.5–81) | 69 (58–78) | 24.000 | .398 | 13.500 |
.014
|
MMSE | 23 (20.5–26) | 29 (25–29) | 34.000 |
.025
| 23 (20.5–26) | 26 (21–26.5) | 28.500 | .136 | 18.000 |
.050
|
ACE-Attention | 15 (14–16.5) | 18 (16.5–18) | 28.000 |
.018
| 14 (12–16.5) | 16 (12.5–17) | 13.500 | .518 | 17.500 |
.040
|
ACE-Memory | 15 (13–18) | 18 (15–21.5) | 28.000 |
.017
| 18 (11–19.5) | 18 (12.5–21) | 11.000 | .336 | 23.000 | .136 |
ACE-Fluency | 5 (2.5–6) | 6 (4–7.5) | 27.000 | .196 | 6 (4–8) | 5 (2.5–5.5) | 2.500 |
.027
| 13.000 |
.014
|
ACE-Language | 22 (21.5–23) | 24 (21–26) | 33.500 | .191 | 19 (16–22) | 21 (17–24.5) | 22.000 | .168 | 32.500 | .489 |
ACE-Visuo-spatial | 12 (7.5–14.5) | 14 (13–15) | 28.000 |
.017
| 12 (7.5–13.5) | 14 (7–15.5) | 16.000 | .246 | 26.500 | .222 |
A Mann-Whitney test indicated that the experimental group improved, significantly more than the control group, in terms of general cognitive functioning, as assessed by ACE (U = 13.500, Z = −2.388, p = .014, r = .56) and MMSE (U = 18.000, Z = −1.996, p = .050, r = .47). The experimental group presented also significantly higher scores in the attention domain (U = 17.000, Z = −2.066, p = .040, r = .49). We also found significant differences between groups in the fluency task (U = 13.000, Z = −2.487, p = .014, r = .59) with improvements in the experimental group and decline in the control group. There were no differences between groups for memory (U = 23.000, Z = −1.578, p = 136, r = .37), language (U = 32.500, Z = −.713, p = 489, r = .17) and visuo-spatial (U = 26.500, Z = −1.263, p = .222, r = .30) domains.
Attention
Table
4 describes the TMT A and TMT B performance for both groups, in terms of errors and completion time, pre and post intervention. No within group differences were identified by comparing the time to completion of the TMT A test in the experimental (W
(9) = 16.500, Z = −.711,
p = .477,
r = .24) and control (W
(9) = 17.500, Z = −1.153,
p = .249,
r = .38) groups, nor were there differences for the number of errors in the experimental (W
(9) = 1.000, Z = −1.089,
p = .276,
r = .36) and control (W
(9) = 5.000,
p = −1.190,
p = .234,
r = .40) groups. Consistently for the TMT B, there were no differences for the time to completion in the experimental (W
(9) = 5.000, Z = −1.153,
p = .249,
r = .38) and the control (W
(9) = 3.000, Z = −1.572,
p = .116,
r = .52) groups, as well as differences in the number of errors in the experimental group (W
(9) = .000, Z = −1.890,
p = .059,
r = .63). However, we found differences in the control group (W
(9) = .000, Z = −2.060,
p = .039,
r = .69).
Table 4
TMT A, TMT B and Picture Arrangement scores (presented as Medians and IQR) pre and post intervention with within-groups (W) comparisons and pre to post-intervention difference with between-groups (MW) comparisons
A Time (seconds) | 74 (53–160.5) | 67 (60–110) | 16.500 | .477 | 120 (71.5–166) | 97 (80.5–150) | 17.500 | .553 | 42.000 | .931 |
A Errors | 0 (0–3) | 1 (0–1) | 1.000 | .276 | 1 (0–3) | 1 (0–2) | 5.000 | .234 | 40.000 | 1 |
B Time (seconds) | 360 (224–360) | 240 (190–360) | 5.000 | .249 | 360 (334–360) | 296 (226.5–360) | 3.000 | .116 | 43.500 | .796 |
B Errors | 4 (1.50–4) | 3 (0–4) | .000 | .059 | 4 (3–4) | 3 (1.50–3.50) | .000 |
.039
| 35.500 | .666 |
Pic. Arrangement | 2 (0–2) | 4 (1.50–6.50) | 21.000 |
.026
| 2 (1–3.50) | 2 (1–4) | 2.000 | .655 | 43.500 | .063 |
For the TMT A, both groups took less time to complete the post intervention test but with no significant differences between groups (U = 39.000, Z = −.132, p = .931, r = .03). For the TMT B, the experimental group took less time to completion when comparing to the control group, although this difference was not significant. There were no significant between group differences for the number of errors for both TMT A (U = 40.000, Z = .047, p = 1, r = .01) and TMT B (U = 35.500, Z = −.482, p = .666, r = .11).
Executive functions
Table
4 describes the Picture Arrangement test performance for both groups pre and post intervention. In this executive functioning test, we have found significant differences within the experimental (W
(9) = 21.000, Z = −2.232,
p = .026,
r = .74) but not within the control (W
(9) = 2.000, Z = −.447,
p = .655,
r = .15) group. There was a tendency to significance for the experimental group to have better performance, when compared to the control, at the end of the intervention (U = 19.500, Z = −2.042,
p = .063,
r = .24).
Subjective general health status
Table
5 describes the answers of both groups pre and post intervention to the SIS questionnaire. The SIS indicated that both groups perceived themselves as being better after the intervention. Improvements within the experimental group were significant for the physical domain (W
(9) = 43.000, Z = −2.431,
p = .015,
r = .81), namely strength (W
(9) = 28.000, Z = −2.388,
p = .017,
r = .80) and mobility (W
(9) = 36.000, Z = −2.527,
p = .012,
r = .84), memory (W
(9) = 40.000, Z = −2.081,
p = .037,
r = .69), emotion (W
(9) = 40.500, Z = −2.136,
p = .033,
r = .71), social participation (W
(9) = 34.000, Z = −2.240,
p = .025,
r = .75) and overall recovery (W
(9) = 28.000, Z = −2.401,
p = .016,
r = .80); but not for communication (W
(9) = 21.500, Z = −1.279,
p = .201,
r = .43), ADL’s (W
(9) = 38.000, Z = −1.840,
p = .066,
r = .61) and hand function (W
(9) = 23.500, Z = −1.614,
p = .106,
r = .54). The differences within the control group were significant for the physical dimension (W
(9) = 41.000, Z = −2.192,
p = .028,
r = .73), namely for the mobility (W
(9) = 26.000, Z = −2.028,
p = .043,
r = .68), memory (W
(9) = 36.000, Z = −2.524,
p = .012,
r = .84) and social participation (W
(9) = 36.000, Z = −2.521,
p = .012,
r = .84); but not for strength (W
(9) = 25.000, Z = −1.859, p = .063,
r = .62), emotion (W
(9) = 30.000, Z = −1.682,
p = .092,
r = .56), communication (W
(9) = 20.000, Z = −1.014,
p = .310,
r = .34), ADL’s (W
(9) = 38.000, Z = −1.838,
p = .066,
r = .61), hand function (W
(9) = 18.000, Z = −1.594,
p = .111,
r = .53) and overall recovery (W
(9) = 30.500, Z = −1.763,
p = .078,
r = .59). There were no significant differences between groups in the strenght, mobility, hand function, ADL’s, memory, emotion, communication, social participation, and overall recovery dimensions of the SIS.
Table 5
SIS scores (presented as Medians and IQR) pre and post intervention with within-groups (W) comparisons and pre to post-intervention difference with between-groups (MW) comparisons
Physical | 42.6 (35.5–56.9) | 51.6 (37.7–71.7) | 43.000 |
.015
| 39.4 (12.4–46.9) | 38.1 (24.2–58.3) | 41.000 |
.028
| 38.000 | .863 |
Strength | 50 (30–59.4) | 62.5 (36.3–71.9) | 28.000 |
.017
| 37.5 (12.5–53.1) | 43.8 (25–62.5) | 25.000 | .063 | 40.000 | .964 |
Memory | 62.5 (45.3–82.8) | 71.9 (53.1–86.6) | 40.000 |
.037
| 56.3 (32.8–70.3) | 62.5 (46.9–79.7) | 36.000 |
.012
| 30.000 | .387 |
Emotion | 75 (55.5–84.7) | 83.3 (75–87.4) | 40.500 |
.033
| 58.3 (45.8–73.6) | 66.67 ± 27.78 | 30.000 | .092 | 50.500 | .387 |
Communication | 75 (60.7–91.1) | 85.7 (62.5–94.6) | 21.500 | .200 | 67.9 (42.9–80.4) | 67.9 (44.6–83.9) | 20.000 | .310 | 42.500 | .863 |
Mobility | 67.5 (42.5–74.9) | 75 (51.3–86.3) | 36.000 |
.012
| 40 (22.5–53.8) | 52.5 (31.3–58.8) | 26.000 |
.043
| 37.500 | .790 |
Hand Function | 15 (0–40) | 40 (5–55) | 23.500 | .106 | 25 (0–30) | 25 (0–45) | 18.000 | .111 | 37.000 | .752 |
ADL’s | 50 (37.5–80.2) | 56.3 (49–86.5) | 38.000 | .066 | 43.8 (14.6–53.1) | 45.8 (30.2–63.6) | 38.000 | .066 | 38.000 | .863 |
Social | 63.9 (29.2–72.3) | 66.7 (53.5–83.3) | 34.000 |
.025
| 36.1 (29.2–51.4) | 50 (41.7–58.3) | 36.000 |
.012
| 41.000 | 1 |
Recovery | 50 (40–55) | 70 (55–80) | 28.000 |
.016
| 40 (40–55) | 60 (45–75) | 30.500 | .078 | 31.500 | .436 |
Usability
Although only 3 out of 9 participants from the experimental group had previous computer experience, there was a good acceptance of the system with no reported problems in the execution of the VR task. Observational information and subjective statements from the participants were consistent with the SUS scores, which reported good levels of usability and satisfaction for the Reh@City (Mdn = 80/100, IQR = 75–87.5).
Discussion
In the past several VR systems have been developed for brain injury rehabilitation, some of which were developed but not field tested [
24,
25] or have only gone through studies with a small number of participants and/or without control groups [
23,
32,
51]. Most of the existing randomized controlled trials with VR-based cognitive rehabilitation, focus in specific cognitive domains, as memory [
52,
53] and attention [
33], or specific deficits, as USN [
22,
30]. Instead, Reh@City was developed to target the rehabilitation of multiple cognitive domains simultaneously requiring the execution of daily routines in progressive levels of cognitive complexity. Our study, besides its limitations, is the first randomized controlled trial that shows evidence that VR-based cognitive rehabilitation in an ecologically valid context could be more effective than conventional training.
Comparing VR and control interventions, in terms of global cognitive functioning, as assessed with the ACE and the MMSE, only the experimental group improved significantly from pre to post-intervention. These significant improvements were also verified in the between-groups analysis. We have found significant improvements in attention, memory and visuo-spatial abilities for the experimental group. Attention and memory improvements are consistent with a study from Gamito and colleagues [
33], which compared a VR-based intervention (ADL’s simulations targeting attention and memory) with conventional rehabilitation. The visuo-spatial improvements are consistent with Kim and colleagues [
30] study, which compared a VR-based intervention with a computer-based intervention in USN. Considering executive functions, our control group had a significant decline in verbal fluency from pre to post intervention. The Picture Arrangement Test specifically assessed problem resolution and processing speed and its results revealed a pre to post intervention improvement only in the experimental group, which we consider a very promising result for further research.
The assessment of processing speed and attention with the TMT A and B revealed only a significant difference in the reduction of the number of errors, from pre to post intervention in the performance of the TMT B, in the control group. This result is not consistent with the other assessments and with previous studies, which found significant attention improvements, only in the experimental group [
31]. The fact that this test is highly influenced by schooling [
54] and that our sample had few years of education might explain the persistence of low performance in this test from pre to post assessment.
Besides cognition, we assessed the intervention’s impact in the multiple domains of health and life with the SIS 3.0. Self-reported data revealed that the experimental group improved significantly in the physical domain, namely strength and mobility, memory, emotion, social participation and overall recovery. Instead, the control group decreased in the physical domain and only improved in memory, mobility and social participation. Nevertheless, no differences between groups were identified. There are CID’s cut-offs for SIS 3.0 motor dimensions (strength = 9.2; ADL’s = 5.9; mobility = 4.5; hand function = 17.8) [
55] and both groups’ improvements were clinically important for strength, ADL’s and mobility. These findings are especially relevant because our VR intervention targeted cognitive aspects but also improved the physical domain, more specifically motor strength, and the emotional condition of patients, as well as their own perception of overall recovery after stroke. Finally, the interaction with the our system was reported as very positive, with high levels of engagement and motivation, which is important to enhance adherence to treatment. The good usability and satisfaction scores obtained with the SUS confirmed these observations.
Despite the positive impact, some limitations of our study must be considered when interpreting the results. Concerning the sample, eighteen participants can be considered a small number, though it is comparable with previous similar interventions [
31,
33]. In addition, there was heterogeneity between groups, especially related to time post-stroke. Although the experimental group was more chronic than the control, this difference was not statistically significant. The dosing of 4 h was of low intensity, and therefore might have not been sufficient to achieve greater or measurable improvements in both groups. Intervention duration of similar previous studies range from 6 to 18 h distributed in sessions of 30 to 60 min, 3 to 5 times a week [
30‐
34]. Furthermore, the intervention was not blind since the same person performed the assessment and the intervention. Regarding the cognitive assessment, there might have been learning effects of the tools since none of them have parallel versions for multiple assessments. Yet, even if a learning effect existed, this would apply to both intervention and control groups and the comparison would still be valid. Nevertheless there are not established clinically important differences (CID’s) for the cognitive assessment tools, through the improvement scores from pre to post-intervention we can conclude that Reh@City, being it designed to address attention, memory, visuo-spatial abilities and executive functions, revealed to be more effective for cognitive rehabilitation than our control intervention. Although it would be relevant to have complementary information with a real-world assessment in a supermarket, pharmacy, post-office and bank, unfortunately this required logistics that could not be implemented for this study. In addition, the main objective was to clinically assess the impact of the Reh@City as a cognitive rehabilitation tool and not necessarily to assess the extent of transfer from VR to actual ADLs.