Computational neurorehabilitation: modeling plasticity and learning to predict recovery

Modeling activity-dependent plasticity requires a quantitative description of activity that stimulates plasticity. Historically, sensorimotor activity during neurorehabilitation has been characterized in research studies and clinical practice primarily by the amount of time spent in assigned therapy sessions [9, 27]. It is also possible to simulate training sessions, in order to derive theoretical inputs for models, as has been done for most initial models described below. However, one reason that computational neurorehabilitation models have the potential to soon become much more elaborate and powerful is that researchers are beginning to quantify more precisely the sensorimotor activity that a patient experiences. There has been increased interest in quantitative, observational studies, and in new sensing technologies, including robotics and wearable sensors.

Examples of the type of data available from robotic and sensor-based therapy devices include the number of movements made and the trajectories achieved while making these movements. Other key variables relate to kinetics, such as the forces applied by the robot to the patient [56, 57] or by the patient to the robot [58], or amount of positive and negative work done on the patient during therapy with the device [59, 60]. Isolated sensors can also quantify the physical interaction forces and motions that therapists apply during hands-on therapy [61]. Such biomechanical measurements can be combined with measures of Electromyography (EMG) to generate estimates of muscle activity during training, and, increasingly, brain imaging techniques, including Electroencephalography (EEG) [62], Near-Infrared Spectroscopy (NIRS) [63], and functional Magnetic Resonance Imaging (fMRI) [64], to provide insight into brain activity during training.

Wrist accelerometry is typically used to detect the amount of time spent moving the arm using a thresholding approach [73]. If sensors are worn on both arms, the amount of bimanual activity can be quantified, and the activity of the stroke-impaired arm can be compared to that of the less affected arm [74]. Indeed most human motor activity seems to be bimanual in nature [74, 75], which has implications for how computational neurorehabilitation models should be structured. New wearable sensing approaches are making it possible to non-obtrusively quantify finger and hand activity as well as gross arm movement during daily life [76, 77].

Given a quantitative description of sensorimotor activity during stroke recovery, a computational neurorehabilitation model uses this description to drive a mathematical model of activity-dependent plasticity mechanisms. Here, we briefly overview two types of activity-dependent plasticity that will play a key role in computational neurorehabilitation models – one related to spontaneous biological recovery, and one related to motor learning. For reviews see [78‐80]. Note that for ease of presentation we speak of plasticity and learning rules as if they were independent form the model structure, but for most models they will not be. The model will need to consider how the necessary anatomical and functional structures support learning and plasticity, regardless of the abstraction level of the model.

Soon after stroke, abnormal cortical patterns of excitation and inhibition occur both near [84‐87]) and far from the lesion [88]. Homeostatic plasticity, which is ubiquitous in the brain, acts to maintain desired firing rates and patterns [81]. After a lesion, because of the loss of interneuronal connections, the activities of neurons neighboring the lesions, or neurons previously connected to neurons within the lesion, are affected. Homeostatic plasticity may be crucial for network recovery, as measured by re-establishment of lesion-affected inputs [89]. In addition, sensorimotor activity might modulate this homeostatic plasticity, which is of importance for computational neurorehabilitation models, as it is one example of how sensorimotor activity appears to modulate spontaneous recovery [6, 31, 89, 90].

LTP, LTD and neural structural plasticity such as dendritic and axonal sprouting, are also modulated by sensorimotor activity, and also change as a function of time. Following stroke, some features of brain function revert to those seen at an early stage of development, with the subsequent process of “recovery recapitulating ontogeny” [91], but there is also a distinct, age-related pattern of gene expression, a “recovery transcriptome” [92]. Genetic changes in the perilesional area allow for a window of increased plasticity that makes it easier for the perilesional neurons to modify existing connections and form new ones in response to sensorimotor activity [81]. Increased LTP may also potentially lead to maladaptive plasticity and poor cortical reorganization if existing inputs are further strengthened at the expense of the reemergence of weak afferent synapses [89]. In summary, underlying mechanisms assumed to contribute to the non-linear time course of recovery of movement in the first 3 months after stroke presumably reflect the interaction between a period of heightened plasticity mechanisms, occurring in a limited time window, and sensorimotor activity [81, 93‐95]. A practical implication is that, when new patterns of movement that are a consequence of specific combinations of muscle weakness (e.g. increased trunk flexion and abduction of the shoulder during reaching) are attempted repeatedly during this period of heightened plasticity, they may become the new ‘norm’ – hence patients get stuck in local minima. Further, use of the less-impaired arm may subvert the heightened plasticity of the stroke-affected hemisphere, preventing it from improving the paretic arm function [14].

There are as yet few computational models of spontaneous biological recovery, much less of the interaction between spontaneous biological recovery and sensorimotor activity. Computational models of the effects of stroke to date have primarily focused on the network effects of deleting cells or of altered connectivity. For example, one early model used a difference-of-Gaussians connectivity pattern to explain rapid changes in the size of cellular receptive fields after stroke lesions [96, 97]. Other models have studied interhemispheric effects of lesions [23, 24], and used connectome data to model brain regions as graphical network nodes, evaluating the effects of node deletion on network dynamics [98‐100]. One of the first models to study the effects of network changes on movement kinematics evaluated the effect of lesion size on post-stroke reach variability using cortical cells that were tuned to preferred reach directions, but did not simulate plastic processes after lesion [101]. A recent model studied the interaction between homeostatic plasticity and Hebbian-plasticity after stroke in the somatosensory cortex, and suggests that after a lesion, a delay preceding rehabilitation would allow a return of homeostatically-determined desired firing in cells neighboring the lesions, and thus may allow a faster network recovery in the rehabilitation training compared to no delay [89]. It will be increasingly important to compare models that incorporate spontaneous biological recovery mechanisms to ones that do not, to determine how modeling these phenomena improves explanatory power. New analytical approaches to examine structural and functional connectivity within well-defined macroscopic brain networks, as briefly reviewed in Section II C below, will increasingly play a role, and will integrate plasticity rules with the necessary anatomical and functional structures.

Unsupervised learning is related to the concept of use-dependent learning, which refers to the phenomenon that the motor system can modify its performance through pure repetition of movements, without external feedback as to the success or failure of the movement [103, 111]. Several initial models of network dynamics after stroke incorporate unsupervised learning (see review [25]). Unsupervised learning, together with homeostatic plasticity, likely plays a role in map and neural reorganization post-stroke, and presumably in decreasing movement variability and thereby improving functional performance [89, 112].

Adaptation studies have shown that humans interact with novel environments by minimizing error (e) relative to the planned movement, and effort (u) [118, 119]. This can be modeled as the minimization of the cost function.

A key recent result is that a simple neural algorithm, which is a “sunken-v”, muscle-specific activation update law that relates the error experienced in muscle coordinates to the change in in muscle activation on the next movement trial, implements this minimization, while simultaneously shaping arm impedance to the task at hand [118‐120]. Another factor involved in movement generation is that subjects tend to minimize time to complete an action, which stands in tradeoff with the required effort [121].

Whereas behavioral observations suggest that at least two learning processes are involved in adaptation, it is unclear how many distinct memories the brain actually updates. In addition, it is unclear whether these putative multiple motor memories reside within a single neural system that contains a distribution of possible timescales, or in qualitatively distinguishable neural systems. A recent study addressed these issues using a model-based fMRI approach [127]. The behavioral data of subjects adapting to two opposing visuo-motor perturbations were first used to derive a large number of possible memory “states”, each with different dynamics, which were then correlated with neural activities. Regional specificity to timescales were identified. In particular, the activity in inferior parietal region and in the anterior-medial cerebellum was associated with memories for intermediate and long timescales, respectively. A sparse singular value decomposition analysis of variability in specificities to timescales over the brain identified four components, two fast, one middle, and one slow, each associated with different brain networks. Then, a multivariate decoding analysis showed that activity patterns in the anterior-medial cerebellum progressively represented the two rotations. These results thus support the existence of brain regions associated with multiple timescales in adaptation and a role of the cerebellum in storing multiple internal models.

Note that these multiple-time constant models assume error-based learning mechanisms. A recent summary of behavioral evidence concluded that while there are at least two components of motor adaptation in response to perturbations, they cannot be fully characterized by first order processes driven by error. For example, the slow process is implicit and learns form errors, while the fast process is explicit and is sensitive to success and failure, among other key differences [128]. The evidence for reward-based and use-dependent mechanisms in motor adaptation suggest they operate at multiple time constants as well, and are likely to be of more relevance to restitution rather than compensation [129‐131].

Such state-space models can account for short-term motor adaptation as well as multiple task learning and the contextual interference effect in post-stroke individuals [132, 133]. At least one initial computational neurorehabilitation model successfully used state-space models inspired by supervised learning data [56]. In addition, robotically amplifying errors can help stroke patients eliminate steady state directional reaching errors [116]. Further, training with amplified errors may increase arm movement recovery after chronic stroke [113]. The beneficial effects of sensory augmentation may be due to the larger error available to the brain for perception and for learning. A recent study however suggests that augmenting errors can decrease motivation in a way that persists beyond the experience of the augmented errors [134], and motivation plays a key role in neurorehabilitation [112].

Note that both supervised and reinforcement learning likely operate simultaneously as both error and reward feedback are often available [137]. For instance for fast reaching to targets by unimpaired subjects, it has been shown that different time constants of learning, and forgetting, may be associated with supervised and reinforcement learning [130]. In rehabilitation therapy, receiving error feedback from a therapist can be rewarding and reinforces behavior. In the absence of external feedback, the learner still has access to intrinsic feedback and this can strongly promote self-learning. Thus, what is presumed to be unsupervised learning can be instead reinforcement learning driven by self-generated feedback. Also, it is the self-generated feedback that the patient needs to rely on when returning to his or her home environment. Accordingly, whenever external feedback is provided, it is important not to become too dependent on this source of augmented information by gradually weaning the learner from external feedback during practice, i.e. to learn to rely on self-generated feedback [137, 138].

Humans do not always appear to minimize error or maximize future rewards, however. In some instances, humans tend to perform a motor task by using the same strategy as they had used in previous trials, even if they had previously experienced a strategy using much less effort [139‐142]. This suggests that rather than attempting global minimization of effort, the sensorimotor system might rather repeat a strategy that it knows will achieve the goal, a finding with implications for modeling use of compensatory movements by stroke patients.

Generalization refers to the concept that training on one task can improve performance on other tasks. Patterns of generalization are complex, in that generalization has been found to be limited in some conditions [143], but rather broad in others [54]. For example, after training to reach in one direction with a planar robotic perturbation, there is little transfer to other directions [144], but relatively broad generalization across certain arm postures [145]. The concept that motor generalization is rather limited has helped drive a strong focus on task-specific training after stroke [146]. However, a key qualifier of this concept is that the organization of practice may determine how much generalization occurs. If one trains one specific task or task variant, there may be little transfer. However if many task variants are practiced, transfer will likely be larger. This is called the variability of practice hypothesis [147], and has clear relevance for computational neurorehabilitation models.

Finally, motor learning not only involves building new action patterns but also suppressing or modulating pre-existing patterns or synergies. This is clear in bimanual skill learning in which the learner gradually overcomes the effect of pre-existing preferred coordination modes (such as in-phase and anti-phase patterns) that are part of the intrinsic dynamics of the system in order to acquire new coordination modes (such as less intrinsic, relative phase patterns) [75]. Like any motor learner, individuals with a stroke may be more constrained by preferred coordination modes and/or by basic synergies that need to be overcome to develop skill. Similarly, previously acquired coordination modes can hamper the acquisition of new coordination modes, a phenomenon called negative transfer [148]. Further, neural damage itself may fundamentally constrain the solutions possible for motor learning.

Currently, in rehabilitation, behavior is usually described with relatively coarse scales, which often sum scores of performance on many tasks, and are typically taken at widely spaced time points. Ideally, computational neurorehabilitation models will bridge the causal link between network plasticity and behavioral changes, which will require higher resolution measurements of behavior at many repeated time points.

Note that data sets that evaluate outcomes differ from the data sets discussed in Section I A in that they quantify how much and how well a patient can move, rather than the total amount and features of rehabilitation training activity. There can be overlap between the two data sets, however, in that measurements made during training can be used to assess movement outcomes, and measurements made of movement during daily life can be used to quantify both training inputs (inasmuch as daily movement serves a training function), as well as serve as a way to quantify outcomes. For instance, kinematic measurements from a robotic rehabilitation device obtained during the course of robotic therapy have been shown to predict standard functional outcomes, without the need for dedicated assessment procedures [149].

Higher resolution outcomes data are becoming available through detailed kinematic studies of upper extremity movement in stroke recovery. In one study that serves as an example, patients with active proximal and distal limb movement within the first 2 weeks after stroke participating in the VECTORS trial were studied with kinematics and electromyography, identifying deficits in movement accuracy, reduced muscle efficiency, delayed muscle onsets, and a reduced ability to modulate muscle activity [150, 151]. Within the first 3 months after stroke, muscle onset times and percentage of muscle capabilities were similar to a neurologically-intact control group, but deficits in the ability to modulate muscle activity remained [151], including an inability to efficiently open and close the fingers on a target object [152, 153]. No computational neurorehabilitation models have yet to our knowledge attempted to model these outcomes.

Other longitudinal movement data from multiple labs around the world are accumulating [7, 154, 155]. For example, a recent kinematic study with intensive repeated measurements in the first months post stroke used principal components analysis to show that individuals with stroke learn to dissociate shoulder and elbow movements mainly in the early phase post-stroke, but do not achieve fully dissociated movements even at 26 weeks [7]. Likewise, recovery in smoothness in reaching and hand aperture was mainly predicted by progress of time alone and almost plateaued within the first 8 weeks post stroke [156]. Again, no models that incorporate plasticity mechanisms have yet attempted to model these findings.

Ideally, motion capture data sets would include the effect of different interventions. For example, there is an ongoing debate on the issue of whether recovery of functional movement is best achieved through restitution (such as reaching with normal kinematics) or compensation (such as using the less affected extremity or leaning forward with the trunk) [4, 157, 158]. At the present time, there are only small amounts of movement data collected pre- and post-intervention to address this issue. In a pilot trial of intensive, progressive, task-specific upper extremity training for people with stroke [41], kinematic and kinetic movement data were examined pre- and post-intervention to examine how movement changed [159]. The results suggest that recovery of function via restitution versus compensation is not an all-or-none phenomenon, but varies within and across individuals. All patients demonstrated improvements in function on clinical scales. In contrast, some movement variables in some subjects indicated restitution of normal movement patterns, while other variables in the same or different subjects indicated the adoption of compensatory movement patterns [159].

Just as wearable sensors will drive computational neurorehabilitation models with data from self-training of the arm during home exercise or daily life, they will provide the descriptors of movement recovery that the models seeks to predict. Such sensors will provide data at a much finer temporal resolution than previous clinical data, which typically are obtained only at baseline, post-intervention, and at one or two follow-ups. This fact, along with the fact that the sensors provide kinematic data, will facilitate simulation of neural networks controlling movement recovery. Such technology-based measurements are also being found to map well to clinical outcome scales [149, 160‐162]. Thus, these measures have validity in terms of established clinical measures, while enhancing the interpretation of these measures, facilitating more fine-grain modeling, and developing new measures. Further, sensor-based measures may catch improvements or differences in behavior when clinical assessment suffers from lack of resolution or floor/ceiling effects, e.g. [163]. Again, we are at a propitious time for computational neurorehabilitation because of the rapid rise of new wearable sensing technologies, the data from which can be used to quantify functional outcomes important to patients in clinicians.

Prospective cohort studies suggest that 33 to 66 % of stroke patients with a paretic upper limb do not show any recovery in upper limb function 6 months after stroke [164, 165]. In contrast, depending on the outcome measures used, 5 to 20 % will achieve full recovery of activities at 6 months. Multivariate regression models that are aimed to predict these outcomes are based on identifying variables (i.e. “markers” or “predictors”), usually measured at baseline, that are linearly or logistically associated with patient outcome at a later time post stroke. Baseline behavior markers that have been found to be useful in such models are voluntary movement ability measured at key joints, initial gross severity of stroke, initial disability, initial severity of motor deficits (e.g. Fugl-Meyer motor scores), and initial kinematics of reaching movements [7].

Two independently conducted prospective cohort studies showed that 98 % of individuals who preserve some voluntary finger extension and some shoulder abduction when assessed within the first 72 h post stroke regain some function at 6 months [20, 166, 167]. However, only 25 % of patients without voluntary control at 72 h regained some function at 6 months. The small proportion of false positives (≈2 %) and relatively large proportion of false negatives (≈25 %) in the SAFE model (Shoulder Abduction Finger Extension) [167] suggests that this clinical model may be too pessimistic in identifying patients who are likely not to recover meaningful function. The preservation or early return of some finger extension most likely reflects the necessity of some fibers of the corticospinal tract system in the affected hemisphere to remain intact in order to activate muscles of the forearm and hand [166, 168].

Numerous studies have also shown that the initial overall severity of stroke measured within 72 h after stroke onset, for example by using the NIH Stroke Scale [169] and initial disability, for example measured with the Barthel Index [170], are highly associated with the final outcome at 6 months measured with the NIH Stroke Scale, Barthel Index, or Functional Independence Measure. Retesting the Barthel Index at regular intervals significantly improved the model accuracy. These findings indicate that the timing of clinical assessment post stroke is an important factor that defines the accuracy of predicting final outcome [20].

The upper extremity Fugl-Meyer score, an assessment of arm movement ability in which various test movements are scored on a three point ordinal scale and then the component scores are summed to form a single number, also predicts motor recovery. In 2008, it was shown that approximately 70–80 % of stroke patients follow a “proportional recovery rule”, recovering about 70 % of their maximal potential recovery at 3 months based on the initial Fugl-Meyer motor scale [171‐173] (Fig. 3). This can be expressed as:

where ΔFM is the predicted change in upper extremity Fugl-Meyer score at 6 months and FM_initial is the score measured within 72 h. This rules suggests either that 1) patients receive a dose of therapy proportional to their impairment, 2) some basal amount of rehabilitation is required for spontaneous recovery, or 3) current rehabilitation does not strongly modulate impairment recovery [10], hypotheses that could be explored with computational neurorehabilitation models. In addition, at lower (i.e., more severe) values of FM_initial, this relationship is not as accurate, with approximately 20–30 % of patients showing a much smaller ΔFM than that predicted by the model [171‐173]. Outliers not fitting the line of proportional fixed recovery suffered from more severe hemiparesis and multimodal impairments such as sensory deficits and neglect [133].

Note this equation is a first order equation in FM. Thus, proportional recovery appears to describe a rule of spontaneous biological recovery that follows first order dynamics [10]. In this framework, any early functional improvement that ensues is mainly happening via reductions in neurological impairment (i.e., restitution). Regression analyses of change scores in early started intensive repeated measurements post-stroke have shown that this time-window of spontaneous neurological recovery appears to be restricted to the first 3 months post stroke [174]. Beyond this time, improvements in function appear to be mainly driven by compensation strategies in which patients learn to use their intact end-effectors to optimize motor performance [7, 79, 156, 175]. See also reviews about definitions, terminology [119] and phenomenology [6] of stroke recovery.

A recent development in prognostic modeling is the use of multiple input variables in non-linear regression models such as neural networks. Specifically, a set of kinematic measures assessed with a robotic device, then mapped through a nonlinear mapping algorithm, predicted clinical outcomes after stroke with higher precision than baseline clinical measures alone [162]. This 30 min assessment asked patients to reach in the horizontal plane to visual targets, to draw circles, and to move against robotic resistance. Metrics included deviation from a straight line, aim, average speed, peak speed, movement duration, smoothness of movement, and features of the sub-movements of movement trajectories, including number, duration, overlap, peak, interval, and skewness of sub-movements. Model performance plateaued when using about eight of these features in the predictive model; that is, additional features did not improve prediction, presumably because of high correlations between some of these features. The nonlinearity of the model was essential to its improved predictive power, which suggests that successful computational neurorehabilitation models will incorporate nonlinearities as well. Note that this model, like “classical” regression models, is purely statistical, and does not explicitly model brain reorganization or learning following stroke.

Another way to look at proportional recovery is that anatomical damage sets a limit on the extent of possible recovery by restitution and therapy takes advantage of plasticity mechanisms (whether enhanced during the early period of spontaneous biological recovery or normal in the chronic phase) to help patients achieve the maximum recovery possible. Predictive models thus need to include neuro-anatomical variables [176]. However, given that level of initial severity is the best predictor of final outcome, we need to ask whether brain imaging data can improve model performance. In other words, we are interested in the factors that determine recovery over and above those that cause impairment, as these are different things [172].

Residual structural and functional architecture after stroke can also be used to estimate clinical outcome. Diffusion tensor imaging (DTI) is able to assess integrity of white matter tracts and when measured within 3 weeks of subcortical stroke, corticospinal tract (CST) integrity correlates with both initial and 6 month upper limb impairment [177]. In a separate study, damage to the CST at the posterior limb of the internal capsule 12 h post-stroke correlated well with motor impairment at 30 and 90 days [178]. Stinear and colleagues have proposed an algorithm for sequentially combining simple clinical, TMS, and DTI measures to predict upper limb function [168]. The PREP (Predicting REcovery Potential) algorithm was tested in a sample of 40 sub-acute stroke patients and performed well in predicting motor function based on Action Research Arm Test scores at 12 weeks post-stroke. The performance of DTI in this setting should be improved by making the tracts specific to particular functions e.g. upper limb [179] and developing ways for the assessment of tract integrity to be done in a standardised [180] and automatic [181] manner.

Corticospinal tract integrity correlates with initial upper limb severity, which explains why it often correlates with final outcome. However, it remains unclear whether it can explain outcomes over and above initial impairment. The role of intact cortical regions in supporting motor recovery is unknown, but has been explored in language recovery. Predicting language outcome and recovery after stroke (PLORAS) uses the whole structural brain scan from which voxel-wise estimates of the likelihood of damaged tissue are derived [182]. This ‘lesion-map’ for each patient is added to (i) time since stroke and (ii) a detailed assessment of various language capabilities. Using a machine learning approach, a new subject's lesion image is compared with those from all the other patients already in the database to find one with a similar lesion. The language scores for all the similar patients are plotted over time, enabling the time course of recovery for the new patient to be estimated [183]. The potential for such an approach extends to many domains including motor and cognitive outcomes. Using this type of neuroimaging complex biomarker discovery [184] it should be possible to provide accurate prognostic models allowing accurate goal setting in neurorehabilitation and stratification in clinical trials [185]. Note that for goal setting in neurorehabilitation, for any patients for whom a given model predicts little potential benefit of treatment, future research will hopefully reveal new, modifiable factors that can be targeted for such patients.

Multivariate machine learning approaches have also been applied to functional MRI data to predict outcome. For example, fMRI data acquired within 2 weeks of stroke in patients with aphasia was used to predict outcome in language [186]. Accuracy in predicting good and bad outcome at 6 months was 76 % and improved to 86 % when age and baseline language impairment were added to the classification model. In the motor domain, fMRI data acquired in the first few days after stroke has been used to try to predict a subsequent change in motor performance [172]. Using a multivariate analysis, a specific pattern of activated voxels was identified as highly predictive of clinical change over the subsequent 3 months, a finding that was independent of initial stroke severity and lesion volume. Anatomical hypotheses could not be tested using this multivariate approach – the study simply indicated that predictive signal was present in a pattern of activation.

Predicting outcome will be useful for clinical and research stratification, but what a clinician would like to know is what are the chances of a patient responding to a specific intervention. Stinear and colleagues [187] set out to determine whether characterising the state of the motor system would help in predicting an individual patient's capacity for further functional improvement at least 6 months post-stroke in a subsequent motor practice programme. A combination of TMS, structural and functional MRI was used to suggest a method for determining who would respond to training. This approach has also been used to predict likely response to robotic training, with both structural [188] and functional imaging [189] data making some contribution.

When it comes to assessing the effects of treatments thought to enhance the potential for experience dependent plasticity, less work has been done. Currently, there is a problematic explanatory gap between molecular (from animal studies) and behavioral (from human studies) accounts of the mechanisms of recovery after stroke. Lack of progression of knowledge from animal models to benefit for human stroke patients has led to the search for ways to study these mechanisms in human subjects. There are exciting advances in how human neuroimaging data can be analyzed that suggest a way forward. Specifically, it is now possible to examine changes in organization of the human brain after stroke at multiple levels of brain architecture, ranging from large-scale networks to alterations in synaptic physiology.

A number of analytical approaches are available to examine connectivity within well-defined macroscopic brain networks. For example, graph theory can be used to determine ‘efficiency’ of information transfer around small world networks such as the brain [190]. This allows inferences to be made about functional connectivity at the level of whole brain, hemisphere or specified network level and can be applied to fMRI, EEG, and MEG data. Dynamic Causal Modeling (DCM) is a method of analyzing data from a dynamic system such as the brain. Bayesian inversion of a specified anatomical model given the empirical data allows the determination of model parameters, which reflect effective connectivity between brain regions (i.e. the task-dependent influence of one brain region on another) [191]. DCM of induced responses acquired with MEG [192] is particularly appealing as it allows partitioning of the effective coupling between regions at the same spectral frequency (linear coupling) and across frequencies (non-linear coupling). Non-linear coupling in particular is important in functional integration [193].

Another recently developed approach to studying brain dynamics after stroke is to use patient-specific structural connectivity data obtained from MRI to set-up an individualized Virtual Brain model of the patient. By optimizing model parameters, such as long-range coupling or local inhibition, in order to match resting state BOLD signals, insight can be gained into how those parameters vary with different types of stroke [194].

At the mesoscopic scale, the spectral characteristics of brain oscillations measured with MEG (or EEG) in the gamma and beta frequency are dependent on the balance of activity between populations of excitatory (glutamatergic) pyramidal cells and inhibitory (GABAergic) interneurons [195] and are candidate biomarkers of the potential for both local- and network-plasticity [196]. More recently, it has become possible to define plausible biophysical DCMs to examine mesoscopic interactions between populations of excitatory and inhibitory cells in specific cortical regions using data from MEG [197]. This approach has been validated using local field potentials in animal preparations where independent pharmacological/microdialysis assays have served to corroborate modelling results [198]. A recent example of how DCM of canonical microcircuits can provide mechanistic inferences is the finding that psilocybin, a 5HT-2A agonist, reduced oscillatory power in posterior association cortex by increasing excitability of deep-layer pyramidal neurons [199].

This range of neuroimaging tools and computational approaches will provide the appropriate intermediate level of description with which to bridge the gap between what we know about recovery after stroke from animal models compared to what we know from studies of behavior in humans. A more detailed knowledge of how these processes are related to impairment and recovery following stroke will provide a mechanistic framework for understanding how to treat patients more effectively. It will open the way for functional brain imaging to become a clinically useful tool in rehabilitation, particularly for our ability to predict outcomes and response to novel plasticity enhancing therapies.

In summary, there is a rich history and promising future to using behavioural status and brain structure and function soon after stroke, to predict long-term motor recovery. Prognostic regression models can inform computational neurorehabilitation models and are a key benchmark against which the predictive power of computational neurorehabilitation will need to be tested.

A key aspect of stroke motor recovery is that individuals with a stroke can elect not to use their impaired arm, since they usually have a relatively unaffected arm to perform most needed tasks (i.e. they can compensate). This “learned non-use” may logically be expected to contribute to loss of motor control of the hemiparetic arm (just as an athlete or musician who stops practicing becomes rusty), although to our knowledge there is no research that has yet documented this loss. Han et al. developed one of the first computational neurorehabilitation models to study the interactions between adaptive decision making related to learned non-use and motor relearning after a simulated motor cortex lesion [112] (Fig. 1c). The inputs to the model are targets for bilateral reaching practice and the outputs are the choice of arm to use to reach to the given target, and the kinematic accuracy of the reaching movement. The model incorporates a reward-based learning mechanism for arm selection, and an error-based learning mechanism for refining the neural population code in primary motor cortex that specifies reach direction.

This model predicts a loss of motor cortex representation without rehabilitation, and a reversal of cortical representational loss with rehabilitation (cf. [93]). Furthermore, the model predicts that if spontaneous recovery, motor training, or both, bring function above a certain threshold, then training can be stopped, as the repeated spontaneous arm use provides a form of motor learning that further bootstraps function and spontaneous use (Fig. 4a), that is, a “virtual cycle” is entered. Below this threshold, motor training is in vain: the model exhibits learned non-use, and compensatory movements performed with the less impaired hand are reinforced; that is, a “viscious cycle” is entered. Evidence for this threshold prediction at the group level was subsequently found in data from the EXCITE clinical trial, a large study of constraint-induced movement therapy after stroke [65].

One clinical impact of the model is that it suggests a novel therapeutic paradigm, the “Train-Wait-Train” paradigm, that not only would test the threshold hypothesis, but also which could potentially be a more cost-effective way to deliver rehabilitation therapy. For example, in computer simulations, a virtual patient was first given a set of 200 trials of therapy, simulated by forced use of the affected arm. Then the patient experienced repeated cycles of 1000 trials in a free arm choice condition (wait), followed by 100 treatment trials (train). The initial training trials were not sufficient to reach threshold, and spontaneous use stayed low. After further training sessions, however, the simulated patient entered the “virtuous circle.” Experiments are currently testing this Train-Wait-Train paradigm.

The model also provides insights into the time constants of stroke motor recovery. The main recovery time constant was on the order of thousands of trials. However, the time constant controlling the change in the decision to use one hand or the other was much smaller, and as a result, the model developed learned non-use soon after stroke. Because how fast learned non-use actually develops after stroke is unknown, hand choice and kinematic data collected soon after stroke would lead to better parameter estimation and hence better predictions from these models.

This model, however, has a number of limitations that need to be addressed to better understand and predict individual recovery as a function of use and motor training. For example, the model is simply bi-stable: either the patient is “above threshold” and fully recovers, or below and does not. In addition, the model ignores interactions between the two cortices. An extension of this model, which includes inter-hemispheric inhibition, has been proposed to account for the beneficial effects of bi-manual training compared to uni-manual training [200]. Further, the model is only concerned with recovery of the control of movement direction, with no attention paid to arm muscle activity, other kinematic features, or functional upper extremity behaviors.

Motor recovery after stroke is characterized by a seemingly disparate set of behavioral and brain imaging observations, many reviewed above, but could these observations arise from a few fundamental features of human sensorimotor plasticity? Reinkensmeyer et al. approached this question by focusing on the modeling of the recovery of distal upper limb strength, the rationale being that strength strongly predicts upper extremity functional activity [136, 201]. The inputs to the model are attempts to flex the wrist, and the output is the flexion force achieved. For the motor system to recover wrist flexion strength, the model assumes that the motor system must learn appropriate activation of residual corticospinal cells (i.e. those preserved after the stroke – thus, within the terminology adopted in this paper this is again a study of neural compensation not restitution) via repeated movement attempts followed by reward feedback. A simplified corticospinal network model was developed based on several key observations from primate neurophysiological experiments – that corticospinal cell activity sums to create a net excitatory flexor or extensor drive to a joint, that different corticospinal cells have different “gains” for exciting motoneuronal pools, and that the relationship between a corticospinal cell’s activity and its individual contribution to muscle force is linear up to a peak firing rate, then saturates for higher activity levels. Finally, the model assumed that the key underlying mechanism of plasticity driving strength recovery after stroke was a reinforcement learning mechanism, in which the sensorimotor system modifies corticospinal cell activations based on repetitive movement experiences, using stochastic random search, such that limb force output is maximized.

This model predicts a broad range of the features of stroke recovery. It predicts the dose-response curve of rehabilitation, and putative modifiers of this curve, such as that the same dose of exercise has a smaller effect when given to a more severely impaired patient, and that the same dose of exercise given early has a larger differential effect than when given later. This latter finding suggests that increased sub-acute plasticity may arise in part due to normative, compensatory network learning dynamics rather than solely as a function of the stroke-altered tissue microenvironment.

Further, and unlike the Han et al. model reviewed above, the model predicts exponential-like strength recovery curves that never quite reach an asymptote, but instead exhibit a residual capacity for further recovery with further movement practice, as has been observed experimentally [202]. At the level of structural brain imaging observations, the model predicts that patients with a larger residual corticospinal network will recover more, a known stroke phenomenon [203]. It also predicts a key functional imaging observation, which is that movement-related activation in secondary motor areas increases following a stroke that damages output from the primary motor area [204].

Mathematically, two key mechanisms drive the observed network dynamics. First, the saturation in the individual relationships between corticospinal cell firing rates and their contribution to muscle force slows later learning, thus leaving room for residual capacity (Fig. 4d). This is because force results from the summed contributions of many cells. Once the activations of a subset of these cells have evolved in the correct direction to maximize force output, and become saturated, then those neurons can no longer contribute to further increases in force. As saturated neurons randomly vary their output from trial to trial, they exhibit an increased probability of decreasing the force produced on subsequent trials, since they can now only vary their output in the direction counterproductive to further force increase. Second, the activations of more strongly connected cells tend to be modified first, as changes in their activations are more likely to increase force output (Fig. 4c).

A key clinical implication of the model is that, if a single learning mechanism – reinforcement learning via stochastic search – can explain such a broad range of features of stroke recovery, then perhaps this mechanism should be targeted in rehabilitation training to facilitate the search process. Adding variability and chances for exploration to training regimens and providing augmented feedback on the teaching signal are two possible ways to achieve this; a large literature on reinforcement learning in computer science may also be brought to bear on rehabilitation protocol optimization (see above). The model predicts another unexpected way that an improved search might be achieved, an “Inhibit-Train-Release” paradigm, which is to temporarily inhibit cells that are already optimized (perhaps using a localized inhibition scheme, such as infused muscimol or focused transcranial magnetic stimulation, for example), train the remaining cells, and then release the inhibited cells. Stochastic search tends to optimize cells with stronger effects on muscles, and once optimized these cells block further optimization as explained above. The model predicts that temporarily inhibiting optimized cells will allow non-optimized cell activations to become optimized.

As discussed above, this model successfully accounts for a large number of existing neural and behavioral data. However, it has several limitations. It has cartoon-like complexity, modeling only a single flexor and extensor muscle, and simplifying corticospinal activation to static input-output relationships. In addition, it does not model spontaneous recovery or restitution, as we have defined them here. Further, while severe weakness precludes control, mild to moderate weakness is less well correlated with motor control. For example, the ipsilateral limb after stroke shows abnormal motor control but strength is normal [205], so motor control requires more than strength.

Models such as the Han et al. model or the Reinkensmeyer model above are “qualitative”, i.e., they make general qualitative predictions of recovery. Parameters in these models are “hand-tuned”. As such, they are only loosely based on actual data and cannot be used to predict recovery of individual patients. As reviewed above, during rehabilitative training mediated by a robot it is possible to observe and record the patient’s performance as well as the level of assistance that the patient received. Can the trial-to-trial evolution in movement success during rehabilitation be modeled, and, if so, what are the key learning parameters needed to describe this evolution? Casadio et al. [56] developed the first computational model driven by data from actual robotic training sessions, which describes the trial-by-trial evolution of the recovery process induced by robotic training. This model provides insights into the role of assistive force in the recovery process, and the extent to which learned changes in voluntary control decay over time and transfer to subsequent training sessions.

The model characterizes the recovery process related to robot-assisted training for improving arm extension in chronic stroke survivors as a linear, discrete-time, shift invariant dynamical system. The model posits that motor performance is a function of a voluntary control component, an assistive force component, provided either by a robot or a therapist, and a performance noise term. Thus, the level of physical assistance provided during rehabilitative training is taken as one of the system’s inputs. The voluntary control component is the internal state of the model and its temporal evolution is described as the combination of three additive terms: a ‘memory’ or ‘retention term’, accounting for how much voluntary control depends on previous experience; an ‘assistance’ component expressed as the magnitude of the assistive force; and a learning component that accounts for how much voluntary control on the actual trial is affected by the previous value of the driving signal. The input signals that drive recovery are movement error or a performance measure. A process noise term accounts for the portion of the recovery not due to these three terms.

The most striking result from this model is that the retention parameter predicts the percent change of the Fugl-Meyer score at the 3-month point following the end of the robotic treatment (Fig. 4b). This result, which needs to be confirmed with a larger cohort, is potentially important for the individualization of training. After one or few initial training sessions with the robot, the model can be fitted to the data. Then, by examining the retention parameter, one could potentially determine who will benefit from additional robotic motor training. Note however that this model is based on a set up in which the arm is constrained to move in two dimensions with shoulder and elbow movements. Thus, here again, for example, the issue of restitution versus compensation is difficult to address and the conclusions are limited to simplified movements at only two joints.

In the same vein of using computational models for predicting individual recovery, Schweighofer and colleagues developed a first-order dynamical model of stroke recovery with longitudinal data from participants receiving constraint-induced movement therapy in the EXCITE clinical trial [206]. The goal of the model was to better understand the interactions between arm function and use in human post-stroke following therapy. The model shows the existence of time-varying interactions between self-reported amount of daily arm use and recovery of function via a dynamical model of stroke recovery in the 2 years following therapy. The time constants in this model were on the order of several weeks. Of most importance, by comparing the model parameters before and after constraint-induced movement therapy intervention in participants receiving the intervention 1 year after randomization, therapy increased the parameter that controls the effect of arm function on use. An increase in this parameter, which can be thought of as the confidence to use the arm for a given level of function, led to an increase in self-reported spontaneous use after therapy compared to before therapy. Thus, here, the parameters of the model can be viewed as “states” that have physiological meaning, which may expedite the testing of new experimental hypotheses.

However, as in the previous model of Casadio et al., this is a highly simplified model of recovery. It contains only a few parameters and two time constants, one for functional recovery and the other for hand use. The reason for this simplicity is the very sparse data set on which the model was built. Increasing the complexity of the model, and thus the number of parameters was not possible because this would have led to overfitting, i.e., the model would fit the data very well but would not generalize.

Rehabilitation therapy often involves interactions between patient and a trainer, be it a rehabilitation therapist, a robotic device, or a computer game. Modeling the patient-trainer interaction could potentially provide insight into movement recovery.

Reinkensmeyer used an adaptive Markov model to examine the role of external mechanical assistance from a robotic device or therapist in promoting movement recovery [207]. Following research in gait training after spinal cord injury, the model assumes that motor control is characterized by repeated transitions between sensory and motor states; for example, sensing of full hip extension at the end of stance triggers leg flexion. It further grossly characterizes these states as abnormal and normal sensory states (e.g. appropriate or inappropriate hip extension), and abnormal and normal motor states (effective or ineffective leg flexion). The model uses a Hebbian-inspired model of plasticity in which the transitions between specific sensory and specific motor states become more reliable with repetitive activation of that transition. The action of a skilled external trainer (robot or human) who is assisting in movement is modeled as a mediated increase in the probability of transferring to a state of normal motor output (e.g. the trainer helps generate hip extension sensory inputs to enable effective leg flexion). Assistance-as-needed is simulated by mediating this transfer only when the patient is in an abnormal motor state, while assistance-always is simulated by mediating this increase on every movement repetition.

The model predicts that assistance-as-needed can enhance recovery beyond what is possible with unassisted practice, that assistance-always is not as effective as assistance-as-needed, that the trainer’s skill in assisting toward normal motor output matters in reinforcing normal state transitions, and that assistance is not useful when sensory input is less directly coupled to motor output. While these predictions may sound somewhat intuitive, this is perhaps because they mirror opinions that are often expressed in the current clinical milieu. The model in this case served to verify that these opinions could be mathematically supported relying on a simple but plausible plasticity rule.

Another approach toward modeling patient-trainer dynamics is based on what was first developed as a model of human movement adaptation. As briefly described above, the motor system uses a sunken-v muscle adaption rule to alter force, impedance, and trajectory, explaining a wide variety of experimental findings of reaching in dynamic environments [119, 208]. This update rule can be viewed as the motor system implementing a greedy minimization of a cost function of error and effort [118]. If the patient’s motor system is minimizing a cost function from movement to movement, then, what cost function might the trainer minimize to assist in learning? A gradient descent of a similar cost function of error and effort was shown to provide an efficient assistive controller for rehabilitation [209]. Jarrassé et al. expanded and formalized this approach, providing a framework for the description and implementation of a broad range of interactive behaviors of two agents (such as a patient and a robotic therapist) performing a joint motor task, including rehabilitation [210]. For example, one may assume that a robotic trainer just moves a patients’ limb while this patient is passive. To model this situation, the trainer would take all of the effort off of the patient by minimizing its cost function:

where e and u correspond to the patient’s error and effort state as observed by the trainer. However, individuals with stroke improve their motor function more when actively attempting to move, rather than relying on the robot to move their arm [211]. To avoid the patient becoming passive, the trainer can modify its cost function toi.e. minimize its own effort u_t ². If one assumes that the patient behaves according to the sunken-v adaptation rule, if the trainer is slightly more lazy (i.e. β_t > β), it will help the patient fulfill the task (as the trainer also has to minimize e²) only if the patient is not able to do so by itself, and disappear if the patient can do it. Thus, this method and the cost function of Eq. (4) can be used to design assist-as-needed control of rehabilitation robots. In terms of computational neurorehabilitation, this work suggests that sensorimotor rehabilitation may be able to be modeled in terms of the cost functions that the trainee and the trainer seek to implement, as well as the algorithms they use to implement those cost functions.