Background
Post-stroke hemiparetic patients generally exhibit pathological synergies in their upper-limb movements, resulting in global and stereotyped unnatural patterns of coordination of their arm joints. These impaired synergies can induce in the stroke survivors harmful compensations at the shoulder and trunk level, having a negative impact on the quality of movement performance and potentially limiting the long term prognosis [
1,
2]. Therefore, a central question in neurorehabilitation is whether or not it is possible to modify the patients impaired upper-limb synergy in order to regain a more natural and healthier control of the arm. The mechanisms of recovery after stroke are multifactorial and the quantification of the effect of rehabilitation programs is complex [
3], but it is known that physical training can lead to permanent improvements in motor function on patients with motor deficits [
4].
Since the mid 90s rehabilitation robotics has arisen as one possible solution to provide intensive goal-directed assisted training, with potentiality to improve human-led therapy [
5]. For this reason many robotic devices for rehabilitation have been recently developed [
6]; however, despite the large number of robotic exoskeletons, the studies addressing motor control learning, exploiting artificial force fields, were mostly performed using planar robot or manipulanda [
7], i.e. robots with maximum two Degrees Of Freedom (DOF), thus addressing end-effector 2D movements with no control on and interaction with the other upper-limb joints.
Among these, a landmark result on motor learning was given by Shadmehr and Mussa-Ivaldi in 1994 [
8]. Several healthy subjects were asked to perform straight-line point-to-point motions, under the effect of velocity-dependent deviating force fields, produced by a planar robot. After an initial failure phase, due to the deviating fields, the users were able to learn how to complete the required task by adapting to these disturbances. The term
adaptation describes the progressive lessening of the effect of the perturbation performed on the human upper-limb movements by the presence of the force fields. Once the force fields were removed, participants temporary showed an over-shoot on the opposite direction of the fields, as an
after-effects or
post-effects. Authors explained this phenomenon as the generation, by the human Central Nervous System (CNS), of an
internal model of the disturbances produced by the fields, which was utilized to compensate the deviating forces in a feedforward manner.
Learning is quite a complex process, concerning the presence of several phenomena such as motor adaptation, post-effects of this adaptation, spatial generalization, and temporal retaining of these effects. In particular, what is crucial for modern neurorehabilitation is the aptitude of the impaired patients to transfer the post-effects to different activities of daily living (generalization), and to maintain these effects during the weeks after the training (retaining).
Based on the internal model hypothesis and by adopting similar experimental setup, others successive studies were carried out, providing evidence that motor adaptation is influenced simultaneously by dynamic and kinematic factors [
9], and by the presence of visual feedback on the error [
10]. Nonetheless, participants were even able to transfer post-effects in regions of the workspace where no exposure to the force fields took place [
8,
10].
These studies were all limited to movements in a plane by the architectural properties of the utilized devices. Moreover, due to the limitations of the planar devices, they rarely directly addressed intermediate joint movements and inter-joint coordination analysis. Dipietro et al. [
11], for example, showed how the shoulder-elbow impaired independence was improved in a circle drawing task for 117 chronic stroke survivors, after training with a planar robot on a therapy of assisted point-to-point movements. However, the shoulder and elbow joint angles were only deducted from the measured hand path, through a simplified non-redundant two-link model of the human arm.
Modern robotic exoskeletons, on the contrary, thanks to their 3D structures and a larger number of DOF with respect to planar robots, are able to impose forces at each different joint of the upper-limb separately and to collect reliable measures of the intermediate joints movements, providing a promising framework to test adaptation and learning on 3D activities. Clearly, an interesting aspect of working with exoskeletons, is the possibility to investigate the way the human CNS deals with the natural upper-limb redundancy for common activities like pointing or tracking tasks. Indeed the CNS is known to exploit synergies, i.e. fundamental building blocks of the motor control [
12], to couple joints or muscles movements together, in order to decrease and face the redundancy of the system [
13]. Despite the large redundancy of the upper-limbs, the final posture, reaching a given position in the space, is quite reproducible for each individual subject [
14]. However the factors that determine these final postures remain disputed, in particular the relative importance of static versus dynamic constraints ([
15‐
17], see [
18] for a review).
We are interested in better understanding the possibility of former results on manipulanda to be generalized to inter-joint coordination in such a redundant system, i.e. the 7-DOF of the human arm. In fact, beyond performing therapy for upper-limb movements and functional recovery, there is a growing need for rehabilitating synergies of patients towards more normal patterns of motion, but yet fundamental studies and experimental results are lacking on how humans, even healthy, adapt to constraints imposed on the inter-joint coordination.
To the authors’ knowledge, only Mistry et al. [
19] preliminary investigated human force field adaptation using an exoskeleton with multiple coupled parts and joint level interactions (a 7-DOF
Sarcos Master Arm robotic exoskeleton). However, the authors of this experiment limited the application of the force fields to a single joint; in particular, they used the exoskeleton to perturb the elbow flexion/extension, during point-to-point reaching tasks, by driving a disturbing force based on the shoulder velocity (respectively the sum of the shoulder flexion/extension and the shoulder abduction/adduction). The authors found that the human nervous system exploited the redundancy of the upper-limb to minimize the effects of the force field on the realization of the endpoint trajectory. In fact, and after a period of adaptation to the force field (about 100 movements) the hand trajectories returned to baseline but the trajectories at the joint level remained changed. After-effects were present after the removal of the perturbation; however this result was very preliminary since it consisted of only 3 catch-trial movements, with no analysis of the wash-out.
The goal of this research is therefore to try to answer the following questions by utilizing a robotic exoskeleton. Can we modify the upper-limb synergy and teach a specific inter-joint coordination? If we apply some viscous distributed inter-joint constraints through an exoskeleton, without constraining the end-effector movements, how would subjects react? Would participants, exploiting the upper-limb redundancy in the perturbing environment, adapt their motor coordination and show similar after-effects as the ones appearing on the end-effector in state-of-the-art studies, in which, however, the effects on the inter-joint coordination were not taken into consideration?
Such knowledge could be fundamental to exploit the main characteristics of modern robotic exoskeletons – the possibility to control multiple joints separately, by imposing distributed coordination control – for neurorehabilitation and could lead to the opportunity, in a near future, to rehabilitate pathological synergies with robots.
Data processing
Data gathering
Data were directly and only collected through the sensors of the ABLE exoskeleton, without the use of any external motion capture system. The ABLE joint kinematic data were measured through differential encoders placed at the joints level. For the end-point kinematics (the tip of the rod) a direct kinematic model of the robotic upper-limb was used together with a measure of the subject forearm length. Post-processing was done using Matlab environment (Ⓒ The MathWorks, Inc.). All the recorded data were passband filtered through a 4th order Butterworth filter (cutoff = 5Hz).
Joint kinematics
We analysed the final arm-robot configuration at the time when the subjects were pushing the WAM button. The final posture joint angles q
i
, with i=1,2,3,4, were expressed relatively to their values at the end of the first movement in the preliminary phase, thus before any force fields exposition.
End-effector kinematics
We computed the speed profile (norm of velocity) of the endpoint motion and considered the following temporal parameters
-
motion duration T, for which we defined the beginning and end of each movement as 10% of its peak velocity;
-
end-point motion smoothness
η, as the spectral arc-length metric defined by [
24], for which large negative values mean reduced smoothness;
-
maximum end-point velocity v
max
.
The spatial charateristics of the trajectory were studied thanks to the
curvature parameter
Φ, defined by [
22] as the maximum deflection of the hand path from a straight line joining the initial and final positions, showing if the hand is deviated from its natural path:
$$ \Phi = \frac{1}{l}\max(d_{p}(t)) $$
(3)
where
\(l = \left \| \overrightarrow {P(t_{end})} - \overrightarrow {P(t_{in})} \right \|\) is the value of the straight line from starting to final position, and
$$ d_{p}(t)=\frac{1}{l} \left\| \left(\overrightarrow{P(t)} - \overrightarrow{P(t_{in})} \right) \times \left(\overrightarrow{P(t_{end})} - \overrightarrow{P(t_{in})} \right)\right\| $$
(4)
is the instantaneous distance of the vector position of the pointer, \(\overrightarrow {P(t)}\), from the straight line joining \(\overrightarrow {P(t_{in})}\) and \(\overrightarrow {P(t_{end})}\).
Principal Component Analysis
A Principal Component Analysis (PCA) was performed on the simultaneous coordination of the four joints,
i.e. a test to spot potential resulting
joint synergy. Each PCA was performed on the joint velocities of a single repetition of reaching tasks, therefore on four reaching movements for the GT (one PCA for each black circle in Fig.
4) and on sixteen reaching movements for the ET (one PCA for each couple of white circles in Fig.
4).
In order to determine if two motions were similarly coordinated, we applied the metrics previously developed in [
25] that is a distance metrics between subspaces defined by the three Principal Components (PCs):
$$ \psi\left(\mathbf{U},\mathbf{V}\right) = \sqrt{1-S_{min}^{2}\left(\mathbf{U}^{T}\mathbf{V}\right)} $$
(5)
where S
min
is the minimal singular value decomposition of the matrix U
T
V, with U and V the two subspaces. This function represents the sine of the minimal angle of rotation between the two subspaces, where ψ=0 stands for no rotation and ψ=1 for orthogonality. Thus, a small distance between subspaces, representing different motions, would mean that these motions were performed by using similar joint synergies, while a large distance would indicate that the subject has changed his coordination while performing the same task.
Statistics
For the statistical analysis on the whole group of participants, the analyses were performed separately for the ET and GT trials. The values of the end-point and the joint variables were averaged during five phases of the procedure, for each target. In particular we considered 2 blocks of PRE trials, the two first and two last blocks of the EXP trials, respectively Early and Late exposition to KSC (E-EXP and L-EXP), the first two blocks of wash-out WAS, and the first two of follow up FOL.
Non-parametric statistics were performed on these five different phases. A Mann-Whitney U test was used to compare the effect of mode (GDM versus PCT) in every different phase-target combinations (5 phases, 5x8 cases for ET targets, and 5x4 cases for GT targets). A Friedman ANOVA test was used to detect the effect of phase (PRE, E-EXP, L-EXP, WAS, and FOL) and target (ET versus GT) separately for the two modes. If these effects were statistically significant, further two-by-two comparisons were performed with the Wilcoxon test.
To analyse individual results, we focused on the shoulder abduction/adduction movements. For each subject we compared data from PRE and WAS (the mean final posture of the two movements in PRE with respect to the first two movements in WAS, therefore only during the first repetition of WAS), to determine the presence of after-effects, and data from early EXP and late EXP (the mean final posture of the first two movements in EXP with respect to the last two movements in EXP), to determine the presence of adaptation. On these two intervals, we therefore computed a Student’s t-test.
For the PCA analysis, we computed the distance metric (Section “
Principal Component Analysis”) between the subspaces describing the PRE trial and all the subsequent blocks in the same mode. These distance were then compared to subspaces describing the spontaneous variability observed during the spontaneous variability experiment (4 blocks of 5 repetitions). To this purpose, for each mode, we calculated the mean of the permutation of the distance between PCs subspaces and used this value as hypothesized value for a non parametric one-sample sign test.
Finally, correlation analyses were also performed between morphological and kinematic data.
All the statistical computation were performed using Statistica (Ⓒ Dell Inc.).
Discussion
Adapting to new upper-limb synergies
Results overview Twenty participants of our experiment performed a total of 404 pointing/tracking tasks, 240 of which under the effect of inter-joint perturbing viscous force fields. Subjects were all healthy, naive, without any known pathology OF the upper-limb.
Results of this campaign showed the capability of most of the subjects to learn from this unknown and unnatural environment, meaning that their natural upper-limb coordination, within an ABLE exoskeleton, was exhibiting effects of the force fields exposition in terms of adaptation, post-effects (still visible after a 30 minute rest sessions outside of the robot), and generalization. These results extended the previous conlcusions found on experiments with planar robots [
8] or, preliminary, with an exoskeleton [
19].
Poor adaptation during the exposition We observed adaptation to the original coordination, during late-EXP phase, only on 21% of the testing population. In most of the subjects, we did not observe changes neither in their coordination, nor at the end-effector kinematics, between E-EXP and L-EXP, i.e. they did not adapt by compensating for the external perturbation, as usual in state of the art experiments with planar robot. The absence of joint space adaptation was observed also in the results by Mistry et al. [
19], but differently from their experiment, since in our case no adaptation of the end-point movements occurred.
Post-effects: two observable distinct behaviours At the same time, distinct post-effects were observed after the removal of the KSC: they consisted, in some cases (30%), in over-shoot on the opposite direction of the natural original coordination – thus similarly to Shadmehr and Mussa-Ivaldi original experiment, but at the joint level, rather then at the end-effector – but the main pattern (55%) consisted of a persistence of the perturbation during the wash-out and even the follow up period. This persistence of the perturbation could be the direct consequence of the CNS not globally optimizing the motor behaviour, but rather tending to repeat suboptimal task-satisfying solutions, because of influenced by motor memory, as described by Ganesh et al. [
26]. In the remaining 15% of the cases, we did not observe any significant post-effect.
Different dynamics, similar results: comparing the two testing modes When considering the constraint of following a path with the end-effector (thus in PCT), PCA results – meaning the existence of larger PCs deviation from the preliminary motion with respect to the spontaneous variability experiment – are weaker than in GDM, but we can still see, almost always, persistent post-effects in movements towards both ET and GT. At the same time, during wash-out and follow up phases, in both cases most of the participants produced an upper-limb coordination strategy modified with respect to their natural preliminary one (respectively 63% persistent perturbation and 29% over-shoot in GDM, versus 48% and 33% in PCT). These results are interesting considering the different dynamics of the two tasks (PCT generally required slower movements and longer duration). Indeed generally GDM task corresponded to ballistic movements, thus probably mostly driven by feedforward control and offline adaptation, while PCT requested stronger visual feedback to correct online the trajectory, as demonstrated by longer duration, slower velocity, and poorer smoothness.
Application to rehabilitation/Limits
Differences from existing studies A clear difference from Shadmehr and Mussa-Ivaldi original experiments and most of the results described in the introduction of this paper, is linked to the nature of the requested task to the participants. In fact, while in 2D experiments, participants were asked to perform point-to-point reaching task following a straight-line path, thus explicitly requiring to contrast the deviating force fields, in our experiment we never asked for any specific behaviour in terms of coordination. This condition clearly allowed the subjects to voluntary decide to resist the effects of the robot, to follow the constraints imposed by the KSC, or, of course, any possible strategy in between. Additionally, in 2D experiments, the perturbations were applied in the task space, as velocity dependant fields on the end-effector, whilst in this case, applying the velocity dependant force fields at the joint space, the subjects were implicitly constrained while moving and while being focused on the reaching tasks.
Ability to impose a specific synergy The main purpose of utilizing a KSC-like strategy with post-stroke patients would be the possibility for the exoskeleton to teach an healthy natural upper-limb coordination, in order to correct common negative and pathological compensations in impaired subjects. Our results did not perform as expected, since after the KSC practice, we did not observe the desired coordination imposed by the force fields. This result, observable in the distance from the constraints of graph
10 and
15, could be related to the only dissipative nature of the version of the KSC that we adopted for this experiment. The original KSC [
27] presents also a non-dissipative viscous torque, developed to avoid energy waste by the exoskeleton and to encourage motions which satisfy the desired constraints. In the future we will need to verify if, by adding this assistive term to the control law, we could obtain better performance on the post-practice resulting coordination. Furthermore, it is reasonable to consider that imposing and retaining a non-optimal synergy in healthy subjects is highly challenging. Probably the effects of the exposition to the natural correction by the exoskeleton would achieve better outcomes when modifying impaired pathological synergies in post-stroke survivors.
Subjects awareness We observed learning in participants who were not aware – at least at the beginning of the training – of the effects of the control law imposed by the exoskeleton, who were only told to focus on the achievement of the task (pushing the button) rather than on the performance of the motion itself (i.e. the inter-joint coordination). The idea of
implicit learning of motor control was already analysed by Patton et al. [
10] in an experiment with a planar robot, observing a detectable reduction in the washout of the after-effects. Indeed, this phenomenon of long lasting effects seems confirmed in our case. Unfortunately, we did not assess quantitatively the level of awareness of the participants, for example, by using a questionnaire. This “unawareness" of the participants could have also interesting application for neurorehabilitation: we can imagine to train patients on rehabilitative exercises for the upper-limb inter-joint coordination while asking them to play video games, involving end-effector movements, on ad-hoc virtual environments. Post-stroke participants could then focus their attention only on the rewards given by the score of the video games and thus only on the end-effector tasks, rather than being involved in performing particular upper-limb motions and therefore being forced to consider multiple goals for the movement of the joints of the arm. Meanwhile the exoskeleton could impose healthy rehabilitative constraints at the joints of the arm, decreasing the cognitive load on the impaired subjects. Moreover, computing performances for inter-joint coordination could be more complex to judge and qualify in order to produce the rewards-metric, mostly due to the human upper-limb redundancy. Thus considering the feedback to the user from the end-point only, while leaving the robot taking care of the correct joint synergy, could simplify the overall rehabilitative therapy.
Behind no-adaptation and distinct post-effects An important inter-individual difference was observed, both at the level of presence of adaptation during the exposition, and at the level of consequent post-effects. At the same time, a smaller but not negligible variability within subjects of these parameters was also found. Personal participants data as weight, height, and grasp force were collected and correlations analysis were performed between these data and the results of the experiment. Learning generally happened when subjects were stronger, taller, heavier, and consequently moving faster, but the difference between any two groups of participants (by either defining the group in which we observed post-effects, and the one in which these effects did not happen, or by taking the group which adapted versus the one which did not adapt, or again, the group which resulted in persistent perturbation versus the one which resulted in over-shoot) were not statistically significant for any of these parameters. The fact that our exoskeleton has fixed-length joint, thus not adjustable to the user physical characteristics, and that the distance between the exoskeleton and the WAM manipulator was not changed with respect to the different height of the participants, may have affected the final results.
Assessing learning effects with an exoskeleton
A central issue for this experiment was clearly the definition of parameters to assess adaptation, post-effects and generalization occurring during the robotic-aided training. Most of the usual adopted indexes to qualify the presence of these phenomena – in literature it is often shown a sufficient number of kinematic parameters as joint trajectory, end-point trajectory, speed profiles of one or more representative subjects – have been mostly developed for analysing 2D tasks, for which the requested behaviour of the participants was clear (point-to-point movements by following a straight-line). We believe that utilizing PCA analysis helped us to quantify phenomena happening during the motion and not only resulting in different configuration at the end of the movements. However the PCs distance metric was often not enough appropriate in describing the arising adaptation: for example, being an unsigned index, it could not discern the observable different post-effects of Section “
Inter-individual differences”. Cycloids seemed to be a fine tool when compared to PCs distance, but their computation produced complex reports to analyse (six cycloids per motion for a 4-DOF robot as ABLE).
Also, it is difficult to translate the PC signification into associated physical interjoint phenomenons (i.e. to determine the joint motions expressed by each synergy/PC), especially since the expression of PCs varied a lot between subjects.
Conclusions
Using a 4-DOF robotic exoskeleton, we were able to modify more than to teach new upper-limb synergies on twenty healthy subjects, performing pointing and tracking tasks. The peculiarity of our experimental protocol allowed us to observe and analyse different effects of learning – adaptation to force field, presence of post-effects, transfer or generalization of the adaptation – separately on two sets of pointing targets. Although adaptation, during the exposition to the force fields, did not occur in the majority of the cases, the presence of different after-effects was persistent and observable even after 30 minutes from the last constrained movements, which represents a new result with respect to classical motor adaptation experiments with robots.
One reason of this long lasting effects could be linked to the unexplicit nature of the constraints imposed by the exoskeleton and consequently a decreased involvement of the participants in the process of synergy modification. This result, if confirmed, could have interesting effects on the rehabilitation of post-stroke patients and the creation of intensive therapy. Unfortunately, the chosen number of repetitions for observing a complete wash-out was not always sufficient, thus, in the next future, an effort will be done to study the effective duration of these effects.
Many questions remain still unanswered, above all concerning the subset of participants who did not adapt and did show post-effects during the training with the exoskeleton, and the presence of two distinct adaptive behaviours, not necessary correlated with any measurement performed or with the requested tasks. Additionally, one of this two behaviours seems not to follow the well-established internal model hypothesis, due to the absence of over-shoot in the post-effects phase.
Furthermore an interesting investigation should be performed on the effects on the participants when the upper-limb is detached from the robotic exoskeleton. Could we still observe any effect on their arm synergy with respect to their natural unperturbed strategy before the experiment? This fundamental question affects directly the generalization to ADLs, outside the clinical environment and without the assistance of the robot, and thus deeply concerns KSC or other corrective strategies potentiality on the rehabilitation efficacy.
Finally, we could also verify the reaction of the participants when testing the same protocol on multiple sessions, during successive days, to understand if possibly the post-effects are reinforced and/or anticipated in order to provide a better insight on the property of the human CNS and the role of motor memory.