What is the impact of user affect on motor learning in virtual environments after stroke? A scoping review

An increasing evidence base supports the use of virtual reality (VR) and active video gaming (AVG) systems to promote motor learning in stroke rehabilitation [1‐4]. However, practical and logistical barriers to VR/AVG implementation in clinical sites have been well described [5‐7]. To support their use, researchers and developers often emphasize the potential advantages of VR/AVG systems over conventional interventions, including that these technologies may enhance a patient’s affective experience in therapy for the purpose of facilitating recovery [8‐11]. Examining the role of affective factors for motor learning is an emerging area of emphasis in rehabilitation [2, 12‐15].

VR/AVG use may enhance patients’ motivation to participate in rehabilitation as well as their engagement in therapeutic tasks. Motivation encourages action toward a goal by eliciting and/or sustaining goal-directed behavior [16]. Motivation can be intrinsic (derived from personal curiosity, importance or relevance of the goal) or extrinsic (elicited via external reward) [17]. Engagement is a cognitive and affective quality or experience of a user during an activity [16]. Many characteristics of VR/AVG play can contribute to user motivation and engagement, such as novelty, salient audiovisual graphics, interactivity, feedback, socialization, optimal challenge [14], extrinsic rewards, intrinsic curiosity or desire to improve in the game, goal-oriented tasks, and meaningful play [18].

Motivation and engagement are hypothesized to support motor learning either indirectly, through increased practice dosage leading to increased repetitive practice, or directly, via enhanced dopaminergic mechanisms influencing motor learning processes [15, 16]. Yet evidence is required to support these claims. A logical first step is to understand how these constructs are being measured within VR/AVG intervention studies. Several studies have used practice dosage or intensity as an indicator of motivation or engagement [19‐21]. To the authors’ knowledge, few have specifically evaluated the indirect mechanistic pathway by correlating measurement of patient motivation or engagement in VR/AVGs with practice dosage or intensity. While participants in VR/AVG studies report higher motivation as compared to conventional interventions [22‐24], conclusions regarding the relationship between motivation and intervention outcomes are limited by lack of consistency and rigour in measurement, including the use of instruments with poor psychometric properties [22, 23].

The body of research exploring the direct effects of engagement or motivation on motor learning is still in its infancy. Lohse et al. [16] were the first to evaluate whether a more audiovisually enriched as compared to more sterile version of a novel AVG task contributed to skill acquisition and retention in typically developing young adults, finding that participants who played under the enriching condition had greater generalized learning and complex skill retention. Self-reported engagement (User Engagement Scale; UES) was higher in the enriched group, but the only difference in self-reported motivation was in the Effort subscale of the Intrinsic Motivation Inventory (IMI), where the enriched group reported less effort as compared to the sterile group. The authors did not find a significant correlation between engagement, motivation and retention scores. A follow-up study using electroencephalography did not replicate the finding that the more enriched practice condition enhanced learning, it did show that more engaged learners had increased information processing, as measured by reduced attentional reserve [25].

Enjoyment, defined as ‘the state or process of taking pleasure in something’ [26], has less frequently been the subject of study in motor learning research, but has become popular as a way of describing patient interaction with VR/AVGs. Enjoyment may be hypothesized to be a precursor to both motivation and engagement. Given that the prevailing marketing of VR/AVGs is that they are ‘fun’ and ‘enjoyable’ [1, 3, 14, 27], it is important to evaluate its measurement in the context of other constructs.

Motivation, engagement and enjoyment in VR/AVGs may be influenced by the additional constructs of immersion and presence. Immersion is defined as “the extent to which the VR system succeeds in delivering an environment which refocuses a user’s sensations from the real world to a virtual world” [13, 28]. Immersion is considered as an objective construct referring to how the computational properties of the technology can deliver an illusion of reality through hardware, software, viewing displays and tracking capabilities [29, 30]. A recent systematic review [13] could not conclusively state effect of immersion on user performance. Immersion is distinct from presence, defined as the “psychological product of technological immersion” [31]. Presence is influenced by many factors, including the characteristics of the user, the VR/AVG task, and the VR/AVG system [28]. While presence is thought to be related to enhanced motivation and performance [32], relationships between this and other constructs of interest require exploration. Table 1 outlines definitions of constructs of interest to this scoping review.

The purpose of this scoping review is to explore the impact of these affective constructs on motor learning after stroke. This greater understanding will enhance the clinical rationale for VR/AVG use and inform directions for subsequent research. Specifically, our objectives were to:

Scoping reviews synthesize knowledge about an exploratory research question to map a field of literature [33]. They are useful methodologies to address questions beyond effectiveness and to describe how a particular subject has been conceptualized or studied [33]. The study is structured according the original methodological framework for conducting scoping reviews [34] and the updated recommendations proposed by Levac et al. [35]. The updated methodological framework consists of six stages: stage 1) Identifying the research question; Stage 2) Searching for relevant studies; 3) Selecting studies; 4) Charting the data; 5) Collating, summarizing, and reporting the results, and 6) Consulting with stakeholders to inform or validate study findings. The consultation stage is optional and was omitted here. The review follows the PRISMA-ScR (Preferred Reporting Items for Systematic reviews and Meta-Analyses extension for Scoping Reviews) reporting guidelines [36].

This scoping review explored how motivation, enjoyment, engagement, immersion and presence were described or measured in VR/AVG studies in stroke rehabilitation. We also sought to identify potential links between these constructs and motor learning outcomes. Although the majority of studies mentioned at least one of the five constructs within their text, construct measurement took place in only 1/3 of studies. Multiple relational links between two or more constructs or between any construct and motor learning were described, though statistical analyses were not used to examine these links.

The emphasis by authors on enjoyment was a surprising finding of this review. Enjoyment was described as important because it underlies both engagement and motivation, and because it is central to essential game design principles of VR/AVG games. However, although it was the most frequently measured construct, it is important to note that measurement of this construct was undertaken with the use of instruments designed for other purposes. This included using instruments measuring flow or intrinsic motivation [85, 86] or self-designed subjective questionnaires lacking psychometric properties [72]. Hung et al. [73] were the only ones to use an enjoyment-specific scale (PACES), although its psychometric properties have not yet been validated in the stroke population or for exercise modalities other than sports [87]. Given that authors appear to consider this construct foundational both to the affective impact of VR/AVGs and to the mechanics of game design, it will be important to achieve consensus on optimal measurement.

A second important finding of the review was the inconsistency with which constructs were mentioned, described, defined and measured in these studies, and the fact that despite lack of tests of statistical inference or even measurement, authors stated assumptions or conclusions about constructs as fact. For example, Shin et al. [57] conclude that their device “encouraged the patient’s skill development, improved immersion, and motivated further rehabilitation by providing meaningful play, optimal challenge, and a flow experience” while acknowledging that they did not measure motivation. In addition, definitions did not consistently align with our a-priori understanding of the terms, and were often vague and interchangeable. Indeed, these terms are differentially operationalized and defined in various fields (e.g. psychology, sports medicine, rehabilitation). This issue of ill-defined terminology was identified by some authors [20]. For example, immersion was often described as a synonym for presence, as follows: “This allows users to experience a high degree of immersion; they feel as if they are in the virtual world, rather than looking at it.” [78] Presence was also described as an indicator of subjective immersion, for example in [19]: “(...) presence is a subjective measure used in VR studies to quantify how immersed a user is in a VE.” Also problematic is the fact that authors use a single instrument to measure several different constructs. For example, immersion was measured using the Presence Questionnaire, the same instrument as that used to measure presence, which was also used to measure what authors’ labelled as engagement [19]. Overall, the inconsistent and varying use of terms, as well as the use of single instruments to quantify different constructs presents a challenge for readers and should be addressed through the development of consistent terminology and a consensus on optimal outcome measures [20].

Among the studies in which a construct was measured, 44% of studies (N = 22) used validated instruments (e.g. IMI, IM-TEQ, PQ, ITQ, TSFQ, PACES and SFQ), however, most measures were not verified yet for the targeted purpose (e.g., the PACES), or population (e.g. PQ). Most used either indirect tools (e.g. taking training time or practice duration as a measure of motivation and engagement, N = 3), study-specific subjective questionnaires with untested psychometric properties (N = 15), or exclusively qualitative assessments (e.g. interviews or comments, N = 12) with varying rigour in data analysis (Table 4). Tatla et al. [22, 23] also found a lack of valid instruments used to measure motivation in pediatric interventions for children with cerebral palsy and acquired brain injury. As such, consensus is clearly required on instruments in order to align the field and facilitate interpretation and the advancement of knowledge. Existing instruments could be adapted and validated for use in VR/AVG interventions and with specific target populations. For example, Gil-Gómez et al. [88] have proposed the SEQ (Suitability Evaluation Questionnaire) that is based on the SFQ (Short Feedback Questionnaire) but has been updated to cover specific VR-related items.The use of direct or indirect objective measures of motivation, enjoyment, engagement is an option to overcome challenges of subjective self-report. Indirect measures include recording time spent interacting with the VR/AVG game (as undertaken by [19‐21], counting the frequency of repetitions, or measuring the intensity of physical activity (for example, using EMG measurement, as in Zimmerli et al. who considered physical activity intensity as an indicator of engagement in VR/AVG interventions) [89]. Clearly, this indirect approach is not without limitations, as there will always be a multitude of influences besides affective state on adherence, dosage and intensity (for example, the expectation of external rewards, or the pressure to maintain a strict treatment schedule). As such, more direct objective measures are also warranted [20]. Examples include electroencephalography, including use of event related potentials to evaluate attentional demand [25], spectra analysis for indicators of engagement, or other measures such as galvanic skin response, heart rate variability or functional near-infrared spectroscopy [90]. The use of such objective measures may elucidate the neurophysiological processes by which affective state influences motor learning [25].

Perez-Marcos [91] suggests that authors should distinguish between VR hardware and software to evaluate user experiences. Specifically, authors should be more specific about describing the components of their VR/AVG interventions to differentiate between systems, the games themselves, and the resulting user experience [91]. Results of our review indicate that game mechanics such as rewards, feedback, challenge, choice/interactivity, clear goals, and socialization [14] were frequently lauded for their influence on motivation, engagement and enjoyment. These game design features are different from the features of the VR system that is delivering the intervention, and can likely be delivered across different platforms. Interestingly, authors did not link these game design features to immersion or presence, indicating that these constructs are more aligned with the game context than with the viewing medium or interaction modality. Further unpacking the ‘active ingredients’ of VR/AVG interventions, and how they may be attached to game characteristics as opposed to hardware components, is a key area for future research [92, 93].

Results of the review illustrate the discrepancy between the frequency of construct description or actual measurement. One potential explanation is that these constructs are universally accepted as inherent to VR/AVG interventions, and as such, researchers are not compelled to measure them. No conclusions can be made about the potential impact of motivation, enjoyment, engagement, degree of immersion and level of presence on the motor improvements achieved in a VE. We recommend including these analyses in future work, where power analyses permit. Such calculations should be facilitated as the field continues to grow and study designs move beyond the feasibility and pilot study stage in which authors’ focus on demonstrating an effect or differentiating the intervention from traditional care.

To accompany the increasing evidence of VR/AVG effectiveness in stroke rehabilitation, it is important to better understand factors that may differentiate certain systems or modulate effectiveness in clients with differing characteristics. The growing emphasis on the role of affective factors in motor learning combined with our findings that many researchers use these constructs as a rationale for VR/AVG use highlight the need to better understand and measure whether affective state differentiates VR/AVG use from traditional interventions and whether it contributes to intervention outcomes. This body of literature currently demonstrates a discrepancy between description and measurement, one that might be explained by the early stage of the literature and the current feasibility-oriented research methodologies. Results of the review provide suggestions for researchers interested in measuring these constructs and emphasize the need for consensus on terminology and outcome measures. Finally, the results point to the need to better understand, through improved measurement and inferential analyses, the potential impact of affective constructs and technical level of immersion on outcomes achieved through practice in VR environments.