Review of control strategies for robotic movement training after neurologic injury

There is increasing interest in using robotic devices to help provide rehabilitation therapy following neurologic injuries such as stroke and spinal cord injury [1, 2] (Figure 1). The general paradigm being explored [see Additional file 1] is to use a robotic device to physically interact with the participant's limbs during movement training, although there is also work that uses robots that do not physically contact the participant to "coach" the participant [3‐5]. As can be seen in Figure 2, there was an exponential increase in papers in this field over the past ten years.

Much of this new work has focused on developing more sophisticated, many degrees-of-freedom robotic mechanisms, in order to support movement training of more complicated movements, such as walking [6‐15], and multi-joint arm and hand movements [16‐26]. Work has also focused on making devices portable so that they can be used during activities of daily living [11, 27‐31]. There has also been a progression in the development of control strategies that specify how these devices interact with participants. The purpose of this paper is to review this control strategy progression and to highlight some needed areas for future development.

The goal of robotic therapy control algorithms is to control robotic devices designed for rehabilitation exercise, so that the selected exercises to be performed by the participant provoke motor plasticity, and therefore improve motor recovery. Currently, however, there is not a solid scientific understanding of how this goal can best be achieved. Robotic therapy control algorithms have therefore been designed on an ad hoc basis, usually drawing on some concepts from the rehabilitation, neuroscience, and motor learning literature. In this review we briefly state these concepts, but do not review their neurophysiological evidence in any detail, focusing instead on how the control strategies seek to embody the general concepts.

One way to group current control algorithms is according to the strategy that they take to provoke plasticity: assisting, challenge-based, simulating normal tasks, and non-contact coaching [see Additional file 1]. Other strategies will likely be conceived in the future, but presently most algorithms seem to fall in these four categories, and we will use this categorization to organize this review.

The most developed paradigm is the assistive one. Assistive controllers help participants to move their weakened limbs in desired patterns during grasping, reaching, or walking, a strategy similar to "active assist" exercises performed by rehabilitation therapists. We will use the term "challenge-based" controllers to refer to controllers that are in some ways the opposite of assistive controllers because they make movement tasks more difficult or challenging. Examples include controllers that provide resistance to the participant's limb movements during exercise, require specific patterns of force generation, or increase the size of movement errors ("error amplification" strategies). The third paradigm, called haptic simulation, refers to the practice of activities of daily living (ADL) movements in a virtual environment. Haptic simulation has flexibility, convenience, and safety advantages compared to practice in a physical environment, as reviewed below. Finally, there is some work on robotic devices that do not physically contact the participant but instead serve as coaches, helping to direct the therapy program, motivate the participant, and promote motor learning. For such devices, it has been hypothesized that physically embodying the automated coaching mechanism has special merit for motivating participants [3]. Clearly, these strategies are not mutually independent, and in some cases multiple strategies could be combined and used in a complementary fashion. Further, assistance and challenge strategies can be viewed as different points on a continuum of either assistance or challenge; i.e. assistance is simply less challenge, and challenge is less assistance.

The goal of this paper is to review "high-level" rather than "low-level" robotic therapy control algorithms. By "high-level", we mean the aspects of the control algorithm that are explicitly designed to provoke motor plasticity. For many robots, such "high-level" algorithms are supported by low-level controllers that achieve the force, position, impedance, or admittance control necessary to implement the high-level algorithm. Research in robotic therapy devices has advanced the state-of-art in low-level force control also, for example, in control of pneumatic [21, 22, 27] and cable-based actuators [8, 14, 18, 19, 26, 32‐35], but these advances are not the focus of this article.

Active assist exercise is the primary control paradigm that has been explored so far in robotic therapy development, and therefore the largest portion of this review is devoted to this topic. Active assist exercise uses external, physical assistance to aid participants in accomplishing intended movements. Physical and occupational therapists manually implement this technique in clinical rehabilitation on a regular basis, for both lower and upper extremity training.

Many rationales can be given for active assist exercise [see Additional file 1], none extensively verified in scientific studies. Active assist exercise interleaves effort by the participant with stretching of the muscles and connective tissue. Effort is thought to be essential for provoking motor plasticity [36, 37], and stretching can help prevent stiffening of soft tissue and reduce spasticity, at least temporarily [38, 39]. Another motivation is that by moving the limb in a manner that self-generated effort can not achieve, active assist exercise provides novel somatosensory stimulation that helps induce brain plasticity [40, 41]. Another rationale is that physically demonstrating the desired pattern of a movement may help a participant learn to achieve the pattern [6, 42]. Another rational, offered often in the context of locomotor training is that creating a normative pattern of sensory input will facilitate the motor system in reestablishing a normative pattern of motor output. Repetition of this normal pattern will reinforce it, improving unassisted motor performance [43, 44]. Physically assisting movements can also help a participant to perform more movements in a shorter amount of time, potentially allowing more intense practice [45]. Another rationale, valid for tasks like walking or driving in which poor performance could lead to injury, is that assistance allows people to practice a task more intensively by making the task safe [28, 46]. A related rationale is that assistance allows participants to progress in task difficulty, much as a young child learns to drive a bicycle with training wheels, starting with a tricycle and progressively reducing the support of the training wheels [6, 46]. Finally, active assistance may have a psychological benefit. To quote a person post-stroke who participated in one of our studies "If I can't do it once, why do it a hundred times?" [47]. This quote emphasizes the fact that active assistance allows participants to achieve desired movements, and thus may serve to motivate repetitive, intensive practice by reconnecting "intention" to "action ".

On the other hand, there is also a history of motor control research that suggests that physically guiding a movement may actually decrease motor learning for some tasks (termed the "guidance hypothesis" [48], see review of guidance studies in motor learning in [46]). The reason is that physically assisting a movement changes the dynamics of the task so that the task learned is not the target task. Guiding the movement also reduces the burden on the learner's motor system to discover the principles necessary to perform the task successfully.

Guiding movement also appears in some cases to cause people to decrease physical effort during motor training. For example, persons with motor incomplete spinal cord injury who walked in a gait training robot that was controlled with a relatively stiff impedance-based assistive controller consumed 60% less energy than in traditional manually-assisted therapy [49]. Likewise, persons post-stroke who were assisted by an adaptively-controlled, compliant robot that had the potential to "take over" a reaching task for them decreased their own force output, letting the robot do more of the work of lifting their arm [50]. These findings suggest what might be termed the "Slacking Hypothesis": a robotic device could potentially decrease recovery if it encourages slacking; i.e. a decrease in motor output, effort, energy consumption, and/or attention during training.

Because providing too much assistance may have negative consequences for learning, a commonly stated goal in active assist exercise is to provide "assistance-as-needed", which means to assist the participant only as much as is needed to accomplish the task (sometimes termed "faded guidance" in motor learning research). Example strategies to encourage participant effort and self initiated movements include allowing some error variability around the desired movement using a deadband (an area around the trajectory in which no assistance is provided) triggering assistance only when the participant achieves a force or velocity threshold, making the robot compliant, or including a forgetting factor in the robotic assistance, as reviewed below.

After reviewing the literature, we decided to group active assistance control strategies into four conceptual categories [see Additional file 1]: impedance-based, counterbalance-based, EMG-based and performance-based adaptive assistance.

Implementing assistance strategies, and indeed also many of the challenge strategies discussed in the next section, often requires a desired trajectory to be specified. The most common strategy for determining the desired trajectory is to model the trajectory based on normative movements (mathematical models of normative trajectories such as a minimum-jerk trajectory [18, 20, 24, 34, 50, 55, 56, 60, 80, 103], pre-recorded trajectories from unimpaired volunteers [7, 9, 10, 12, 71, 72, 76, 111], or pre-recorded trajectories during therapist-guided assistance [17, 21, 62, 69, 105]). We note, however, that there is no rigorous evidence that desired trajectories should be "normative" in order to maximally stimulate plasticity during motor training.

Another strategy for determining the desired trajectory, possible for bilateral tasks, is to base the desired trajectory on the movement of the "good" limb [45, 53, 54, 57, 58, 75, 111, 112]. This approach was used with the MIME robotic rehabilitation device [45]. Motions of the unimpaired arm were detected and replicated by the robotic device that directed the movement of the affected limb based on position-control, thus facilitating bimanual movement practice. A similar approach was employed in the BiManuTrack device [58], for which the unimpaired extremity guided the affected limb in a mirror-like fashion, and for the LOPES gait training robot [111] where the state of the unimpaired leg was mapped, through a technique called Complementary Limb Motion Estimation, to determine the reference motion of the paretic limb. Bilateral strategies may even have a neurologic benefit: researchers have postulated benefits from training with bimanual movements related to the neurology of bilateral control in both, upper extremities [45, 57, 113], and lower extremities [75, 112].

Adaptive approaches have also been used to adjust the desired trajectory. As mentioned above, one strategy adapted the desired trajectory based on contact forces between the robot and the limb [10]. Other strategies include re-planning the (minimum jerk) desired trajectory at every time sample based on the actual performance of the participant [114], or adjusting the replay-timing of the desired trajectory from time sample to time sample based on the difference between the actual, measured state of the participant and the desired state, with the effect of better synchronizing a compliant gait training robot to the participant [7].

The problem of determining the desired trajectory for a robotic therapy controller is essentially the problem of predicting human behavior for a given task – i.e. identifying a model of human motor behavior. For relatively simple tasks, such as point-to-point reaching, normative behavior is fairly well described (i.e. the. minimum jerk trajectory). Providing assistance for more complex tasks will require developing models of normative motor behavior for these tasks. For example, a recently developed controller predicts human steering motions during a driving task, allowing assistance to be provided in a beneficial way for this task [46].

Some robotic therapy controllers do not require desired trajectories. For example resistive strategies can be implemented without desired trajectories. EMG-proportional controllers do not require desired trajectories since the participant's self-selected EMG specifies the desired movement. Likewise, participants with counterbalanced limbs can participate freely in a wide variety of self-directed exercises. However, the ability to complete those exercises may be limited since a counterbalance approach may not restore full range of motion or coordination. For example, when the restraining forces due to neuromuscular tone and gravity were actively cancelled with a robot [96, 115] or by chemically blocking antagonist muscle activity [116], persons post-stroke did not recover full range of motion of reaching or hand opening, respectively, suggesting that position-dependent agonist weakness substantially limits active range of motion. Thus, robotic devices that intend to aim movements across a large workspace need to account for position, velocity, tone, and gravity. Allowing a participant to use a brain-computer interface to specify a desired trajectory, or even robot forces, may allow greater participant control over the movement to be performed [117].

Although the term "challenge-based controllers" is somewhat vague, we will use it to refer to controllers that in some ways make a task more difficult or challenging, as opposed to the assistive controllers reviewed above that make tasks easier in some way. This work on challenge-based controllers is providing insight that might be missed by focusing solely on assistive-type algorithms. As noted above, however, in some ways, challenge and assistive controllers can be viewed as being different points on the same continuum, a continuum along which task difficulty is modulated to optimally challenge the participant [118].

Robotic therapy devices can be used as haptic interfaces for interacting with virtual reality simulations of activities of daily living, such as manipulating objects [17, 18, 21, 63, 64, 138‐144] or walking across a street [15, 78, 145, 146]. Potential advantages of this approach over training in physical reality include: a haptic simulator can create many different interactive environments simulating a wide range of real-life situations, quickly switch between these environments without a "set-up" time, automatically grade the difficulty of the training environment by adding or removing virtual features, make the environments more interesting than a typical rehabilitation clinic (e.g. walking through Paris versus down a hospital hallway), automatically "reset" itself if objects are dropped or misplaced, and provide novel forms of visual and haptic feedback regarding performance. A variable of virtual environments was suggested by Lum et al. in [5], where real objects for manipulation where presented robotically. This technique resembles a robotic therapy system developed in 1989 [4] in which a robot arm was programmed to place physical targets for reaching and manipulation.

A final area of development of robotic therapy control algorithms is for mobile robots that do not contact the participant but rather operate beside the participant, directing and encouraging therapy activities [3]. The question immediately arises as to whether a robot is necessary for this function, as a computer alone could give auditory and visual instructions and feedback. There is evidence however that people respond differently to "embodied" intelligence [147]. Therefore, physically embodying the coaching system in a robot may bring novel and relevant neuro-psychological mechanisms into play during movement training.

In this field, the development of the robot control algorithms focuses primarily on questions such as "How should the robot move and talk to encourage effort by the participant? " and "What type of exercises, and what practice order, should the robot specify to maximize learning? " The emergence of this field serves to highlight the key role that motivational factors and practice protocols play in rehabilitation therapy. A related field that is emerging in motor learning research and could be used to help design robot "coaches" for rehabilitation therapy is that of using computational models of learning to determine the best sequence of movements for maximizing adaptation to novel dynamic environments [148].

We close this survey of control strategies with a brief review of the experimental evidence of the effectiveness of the various control strategies for robotic therapy. For reviews of studies that have examined the effect of mechanical assistance on motor learning by unimpaired subjects, we refer the reader to [46, 149].

Many studies of assistance techniques in robotic therapy have examined the effect of assistance given to chronic neurologic participants, with the participants' baseline motor status used as their own control. These studies show that robot assistance modestly but significantly decrease motor impairment, including at long-term follow-ups, using standard clinical scales as the outcome measures [56, 58, 59, 77, 83, 88, 98, 113, 135, 150‐156]. Other studies have shown that an additional dose of robot assistance relative to a group that received normal therapy improved recovery [157‐160]. Impairment reduction with robotic therapy is often small enough to have marginal clinical or functional significance [161].

Perhaps more interesting for the purposes of this review are the few studies that have compared assistive control strategies to other controllers or rehabilitation strategies. For the upper extremity, one study compared impedance-based assistance to conventional therapy, and found a marginally significantly greater benefit [45]. A more recent study compared bilateral, unilateral, and combined bilateral and unilateral with conventional therapy and found that although combined unilateral and bilateral robotic training had advantages compared with conventional therapy, the differences did not hold in long term retention (6 month follow-up) [162]. A study that compared triggered assistance to no assistance and found no significant differences [79]. Training with impedance-based assistance compared to a smaller number of FES-triggered movements for wrist movements resulted in a significant and substantial advantage for the robot assistance strategy [57]. A comparison of an impedance-based assistance strategy to a resistance strategy for reaching after stroke found no significant difference [163]. Hogan et al. [86] compared performance-based progressive assistance to historical data from non-progressive assistance, and observed larger gains with the progressive assistance technique. A comparison of counterbalance assistance to traditional table top therapy found a small benefit with regards to impairment reduction, and revealed that participants strongly preferred the counterbalance assistance [164]. Similarly, a comparison of impedance-based robotic assistance to traditional sling suspension therapy found that the rate of recovery in the robotic group was greater than the sling suspension group for most subjects [165]. A recent study of a hand robot (HWARD) found that persons with chronic stroke who received a greater dose of time-triggered robotic assistance therapy applied using the robot experienced greater behavioral gains than a group of participants who received a smaller dose, plus active non-assist therapy (i.e. therapy in which the subjects did all the work and the robot does not assist) [157]. This may be the first direct evidence that robot assistance can be differentially beneficial as opposed to a matched amount of unassisted practice.

For gait training robots, there have been several recent studies that have compared impedance-based assistance to conventional rehabilitation techniques. One recent study found that intensive locomotor training from the electromechanical gait trainer GT I [166] plus physiotherapy resulted in a significantly better gait and basic activities of daily living ability in subacute stroke patients compared with conventional physiotherapy alone [158]. A study that compared impedance-based robotic assistance from the robot-driven gait orthosis Lokomat plus conventional therapy with conventional physiotherapy alone in hemiparetic patients after stroke found no significant difference between groups [159]. However, two recent studies [167, 168] compared robot assistance from Lokomat to therapist-assisted conventional therapy for chronic stroke participants, and found substantially greater improvements in speed and single limb stance time from the conventional therapist-assisted locomotion training. One study in spinal-injured mice compared two forms of impedance control [71]. One strategy used a position-dependent, velocity force field with a deadband that assisted spinal-transected mice to step following a nominal step trajectory with bilateral coordination, and was compared with a fixed training trajectory, and an assist-as-needed strategy without enforced interlimb coordination. It was found that although all the training strategies increased stepping ability, the number of steps and periodicity (consistency of step timing) increased significantly more when the mice were trained with assistance-as-needed with interlimb coordination. The differential training effects were small, however.

Evidence of differential clinical benefits of training with challenge-based controllers is sparse. In what appears to be the only randomized controlled study of resistive versus assistive forces, Stein et al. [163] compared the motor outcomes of chronic stroke persons who exercised while receiving viscous resistance from MIT-MANUS, with a group that exercised while receiving impedance-based assistance. They found that both groups improved in various outcome measures, but that there were no significant differences between groups.

For robot control algorithm studies using a constraint-induced philosophy, a comparison study of the Guided Force Training algorithm with training free reaching and conventional occupational therapy found that persons post-stroke trained with the robotic device significantly increased upper extremity Fugl-Meyer scores, significantly decreased the time to perform the task and demonstrated a transfer of motor learning to functional tasks [169]. However robot training did not show greater gains when compared to the non-robotic strategies. A study with the MIME robotic system [113] provided some evidence of the effectiveness of the active-constrained mode robotic therapy reporting that directional force generation errors were reduced in six of eight movement patterns. Furthermore, low-level subjects increased their extent of reach, and high-level subjects increased their speed. The Driver's Seat approach increased effort from the unimpaired side [130].

Studies using training with error-amplification control strategies have shown short-term improvements in curvature during reaching [133, 135] or interlimb coordination during walking, following chronic stroke. These improvements were not achieved with movement practice without error amplification. The long-term benefits of error-amplification (e.g. benefits of aftereffects) are unknown.

For haptic simulation techniques, there is some experimental evidence of effectiveness. One pilot study found that training with a haptic simulator/hand rehabilitator increased finger and thumb range of motion and/or speed in all 8 persons post-stroke [140]. Improvements showed during training in the virtual environment transferred to gains in functional real-world movements. Training in a virtual environment with a PHANToM™ haptic device increased participant's grip force generation, movement endurance and generated a more correct motor pattern [10]. Training in reaching and interacting with real objects did not show any detectable advantage over training with simulated objects with MIT-MANUS [170]. Training with a web-based virtual environment with a force feedback joystick improved movement ability in a person post-stroke, and was highly motivating [144]. Testing of a web-based haptic joystick rehabilitation suste, [171] and an ankle robot connected to a virtual reality simulator [172] resulted in high acceptance and satisfaction in a person post-stroke. Significantly, addition of virtual reality to robot-assisted lower extremity training was recently found to improve therapeutic outcomes, compared to robot-based training alone [173].

Finally, clinical testing with non-contact robotic coaches is still in an early stage. There are positive reports of participant compliance and satisfaction with the robot-specified exercises [3‐5].

To summarize, while many studies have demonstrated that training with different robotic control strategies can significantly reduce motor impairment as assessed with standard clinical outcome measures, few studies have found differential benefits of particular robotic control strategies with respect to other robotic control strategies. Two recent studies [167, 168] actually found that a particular form of robot assistance during gait training (relatively rigid, rote assistance) was substantially less effective than an equivalent dose of manual assistance from a physical therapist during the same motor task (walking on a treadmill). This negative finding highlights the important concept that the specific form of robot control selected for a rehabilitation application does indeed matter.

We reviewed the development of robotic therapy control algorithms intended to promote neuroplasticity and motor learning during rehabilitation after neurologic injury. Even though a substantial amount of work has now been done, the field is rapidly evolving. The question of the most effective control algorithms is still wide open, in part because the randomized controlled trials necessary to identify these algorithms are expensive and time-consuming. Fundamentally, it is still even unclear whether robotic control approaches have the potential to produce greater benefits than is possible with simpler techniques, such as rote, unassisted practice [174]. We conclude by suggesting three directions for future research.

The first direction is to focus randomized controlled trials in this field on rigorous comparison of control algorithms with each other, and with simpler, non-robotic therapy approaches. There are now many studies that examined the effect of robotic therapy on chronic patients with the patients' baseline as their own control [56, 58, 59, 77, 83, 88, 98, 113, 135, 150‐156], and studies in which an additional dose of therapy in the form of robot therapy was given [157‐160]. Giving the same dose of robotic therapy with two different control algorithms will help define which features of the control algorithms enhance motor recovery.

A second direction is to initiate use of more precision in defining what control algorithms are most appropriate for which rehabilitation tasks, what types of neurologic injuries, and at what stage of recovery. Motor learning and neuroplasticity mechanisms, and rehabilitation exercises themselves, are wonderfully diverse. Robotics has the opportunity to make a quantum leap by systematically implementing and controlling therapies, and by enabling systematic adjustment of treatment parameters. There is the beginning of a library of robotic therapy control algorithms from which to select, as reviewed here, to address these mechanisms and exercises. There are also experimental techniques that can precisely define features of neurologic injuries (e.g. medical imaging) and associated impairments (e.g. methods for quantifying weakness, tone, incoordination, and sensory deficits). Tailoring the control algorithm to the participant-specific pathophysiology, recovery stage, and the specific activity being rehabilitated may improve its therapeutic benefit. Mixtures and progressions of different robotic control strategies will likely end up being best; mixtures of robotics and FES or other training strategies are another possibility [76]. The form of feedback provided during robot-assisted training may be as important as the form of robot mechanical intervention itself [60, 136, 173].

The third direction is to develop better computational models of motor learning and recovery, in order to inform robot therapy control design. Developing such models may help in developing therapeutically better control algorithms using an optimization framework once the variables that drive adaptation are more clearly defined.