Introduction

Myoelectric control of prosthetic upper limbs is an established technology [1]. While cosmetic features, weight savings, battery life, and components have significantly improved over time, the fundamental control strategies in the clinical setting have not changed [2•] (Fig. 1). Amputees still control prostheses by using one muscle group to open the hand, and another to close the hand, with some advanced devices allowing movements at the wrist or specific grip patterns. The early strategies of on/off and proportional control have been shown over a number of decades to perform well, with consistent reliability [3, 4]. However, this is far from the capability of the natural human hand, which can frustrate prosthetic users, especially in an era of relentless technological innovations.

Fig. 1
figure 1

Examples of myoelectric hands commercially available. Although they differ in complexity and components, with some offering different grip patterns (adjustable by external means), all are based on the earliest control strategies to achieve grip. a Ottobock DMC Plus. b Touchbionics iLimb. c Ottobock Michaelangelo hand. d RSL Steeper Bebionic. b–d The most advanced clinically available hand prostheses at time of publication, while (a) is typical of a terminal device that is widely used in clinical practice

Full size image

In contrast to early approaches that work on one to two degrees of freedom, pattern recognition and other machine-learning algorithms have the potential capability to map more movements over multiple degrees of freedom, by deciphering the useful features present within the electromyographic (EMG) signal [5]. This capability is promising, but has not yet been applied in widely available prosthetic hands, despite being in development since the mid-1970s. Deciphering EMG signals in real-time is not a cost-free exercise and requires significant processing power. Moreover, the reliability of pattern recognition methods when used in daily living conditions is challenged by several factors of variability, such as changes in arm posture, electrode repositioning, and fatigue [6].

This review sets out to examine from a clinical perspective why simple proportional and threshold-based strategies are currently the only options for clinical devices, and what challenges still need to be addressed for surpassing the limitations that more advanced academic systems face before their more extensive use will be possible.

Early Developments

Historically, the first electronically-driven hand prostheses were pioneered by Reinhold Reiter at the end of World War Two [7], followed by subsequent groups [8–12]. Uncomplicated in nature, these early myoelectric hands used EMG signals from either one or two muscle groups to determine an ‘on’ or ‘off’ state to control the myoelectric limb.

In the case of one muscle group, engineers took advantage of the varying intensity of muscular contractions to determine suitable activation thresholds that would trigger a particular action or ‘state’ of the prosthesis. Variously known as on/off control, crisp control, finite state machine control, or onset analysis, these threshold-based methods allowed actions such as slight contraction to close the hand, strong contraction to open, and no muscle activity to halt the device [8] (Fig. 2a). The same technique could be applied to two opposing muscle groups, such as flexors and extensors, to control hand opening and closing [11]. Various thresholds can be used to determine which EMG activity is relevant and to discriminate from the background noise [13]. Although both fast and applicable to real-time applications [14], the number of movements generated is limited and the control is sequential in nature, unlike the smooth actions expected of the human hand.

Fig. 2
figure 2

A simple illustration of the early control strategies used when a one electrode is used to generate three states, and b two electrodes determine opposing actions

Full size image

An early alternative to the binary nature of on/off control was to vary the velocity and force of the prosthetic hand in a continuous and proportional manner to the recorded EMG signal from two opposing muscle groups to achieve hand open/close [15, 16] (Fig. 2b). This type of control has been previously defined as proportional control, where users control “at least one mechanical output quantity of the prosthesis (whether that be force, velocity, position, or any function thereof) within a finite, useful, and essentially continuous interval by varying his/her control input within a corresponding continuous interval”, and is found in many clinically available devices [17•]. While this type of control is acceptable for large gross movements, such as hand open/close, it is insufficient for finer movements such as individual finger flexion.

Proportional control could, in principle, be refined further to accommodate fine movements by mapping individual EMG signals to individual functions of a prosthetic device in a process known as direct control [18]. However, this requires overcoming the interference of several muscles contracting together, which generate crosstalk of the resulting EMG signals, reducing the selectivity of the electrodes for individual EMG signals [19]. One method to overcome crosstalk is to implant recording electrodes directly into [20, 21] or onto [22] muscles, which have been shown to be a feasible approach in animal studies.

Whilst fast and robust, the number of movements generated by proportional and direct control systems is limited by the number of independently controllable EMG channels. As a result, current prosthetic upper limbs are mostly limited to such simple functions as generating power grip or elbow flexion/extension, far from the multifunctional control of the human arm. In light of this, newer pattern recognition control strategies have been developed experimentally to overcome the lack of multifunctional control [23], by providing many more degrees of freedom while using the same number of channels.

Pattern Recognition Control

Introduced to the prosthetic research community around 1970, and tested in research conditions, pattern recognition algorithms provided a means for multifunctional control by classifying different muscle activation patterns [23–25]. These algorithms fundamentally included a sequence of events where the raw EMG data was first windowed, and useful features extracted and classified, before being used for control (Fig. 3). However, these systems did not generate great interest until 1993, when Hudgins et al. [26] showed that a more refined interpretation of the content of the EMG signal could be achieved, setting the basis for a rising wave of pattern recognition control algorithms [5, 27].

Fig. 3
figure 3

Comparison of the structure for pattern recognition (upper) and regression control (lower). For pattern recognition, a classification algorithm identifies the currently active function, resulting in a sequence of decisions at constant update rate. For the regression algorithm, a mapping function translates the information extracted from the EMG signal into continuous kinematics that can be used as control signals for a prosthetic device

Full size image

The development of these algorithms included increasing the number of input channels [25], which improved the accuracy of classifying features [28] and the resulting multifunctional control. This has allowed increased movement patterns to be classified beyond just grip, to include wrist flexion and extension, forearm pronation and supination [29, 30], humeral rotation [31] and ulnar and radial deviation at the wrist [32], with the current trend moving towards identifying muscle patterns to control individual fingers [33–36]. However, it is important to note that pattern recognition is only capable of classifying these movements in sequence and not simultaneously (Fig. 4a).

Fig. 4
figure 4

a Pattern recognition is able to classify different movement patterns, but only in sequence, which limits multifunctional control. b Regression control is able to identify different movements at the same time, leading to more intuitive prosthetic control

Full size image

While these advances in pattern recognition are encouraging, there are several factors that prevent transition from the laboratory to clinical application. Most laboratory testing involves either healthy volunteers or amputees in static conditions [37]. This is notable, as the training and testing datasets used for pattern recognition [38•] have been collected in these conditions, when muscle contractions would be ideal and exclude the interaction of the user with the control system in real-life situations. The muscle contraction patterns can change when the user has to act in stressful situations or react to wrong movements. In addition, during active movement, the effects of sweating and displacement of the electrodes, as well as changes of the muscle size and shape post-surgery, can influence the EMG signal characteristics. The effect of some of these factors can be mitigated, such as increasing the distance between surface electrodes [39] and alignment with respect to muscle fibres [40] to improve signal acquisition. However, the results from offline validation often cannot be easily translated into real-world situations [41].

An example of the difficulties in translating laboratory results to real-world applications is the way in which performance is reported in research studies, as classification accuracy. The association between classification accuracy and prosthesis usability is very weak. For example, in real-time testing of myoelectric pattern recognition control of a virtual arm by transradial amputees, the completion rate of predefined hand movements with the prosthesis was only ~55 % when the classification accuracy was ~85 %, suggesting that these algorithms are insufficient to reliably control real-time hand grasp even for relatively high classification performance [6]. Moreover, pattern recognition methods inherently require sufficiently long intervals to extract useful classification features without delaying response time. Optimal controller delays have been suggested to be 100 ms for fast prehensors and 125 ms for slow prehensors, without degrading the prosthesis performance [42], while delays longer than 200 ms have been shown to negatively affect a user’s performance [43]. Despite these considerations, laboratory studies on pattern recognition [44–47] usually report results for internals of 300 ms. Although in simple cases of one degree of freedom control, it has been shown that high accuracies are still obtained with shorter intervals [48], the performance usually degrades substantially as the processing window decreases.

In real-world situations, losing prosthetic control, such as while holding a pan of boiling water, would not only be undesirable but unsafe. Thus, a robust system is preferable to one that is technologically complex yet unreliable. Supportive of this view, a review of the English language literature over a 25-year period observed that, while technology had improved, prosthetic limb rejection rates have not been decreasing, with 35 % of paediatric and 23 % of adult amputee populations discontinuing use of their prosthetic limb [49]. In a separate study, 13.4 % of Norwegian major upper limb amputees discontinued use of their prosthesis mainly due to poor prosthetic comfort, function and control [50].

Regression Control

Improving how patients control prostheses, while overcoming the limitations of existing control strategies, is necessary to improve acceptance. Two major limitations of pattern recognition approaches are the lack of simultaneous multifunction control and of proportional control. In the case of a prosthetic hand with the four functions of hand open, hand close, wrist pronation and wrist supination, a classifier can only detect one function at a time. A simultaneous opening of the hand together with supination, for instance, to prepare the hand to grasp an object, is not possible [51]. In order to provide a solution for this problem, some research groups have proposed using additional classes for all combinations of functions [52] (for the above example, an additional class would be “hand open + wrist supination”). However, the combined classes increase the number of patterns that need to be trained, which is undesirable in clinical practice as it increases the time of the user necessary to train the system. Secondly, pattern recognition does not support proportional control of the prosthesis, crucial for optimising the speed of the response. Like a switch, a classifier can only decide an on/off basis, but, in most cases, the user also needs to be able to influence the speed or strength of the prosthetic device. This requires adding a proportional component to the control signal [17•, 26, 53], which can increase the overall complexity of the device and decrease the robustness of the control.

Newly developed regression algorithms overcome the need to increase the complexity of the device whilst providing simultaneous and proportional control. In contrast to methods only allowing classification of one motion at a time, regression algorithms are able to continuously estimate multiple control signals directly from the EMG signal (Fig. 4b). For the same prosthesis functions as above, this results in one control signal for the rotation angle of the wrist and a second estimate for the opening angle of the hand. Consequently, the regression approach provides greater flexibility for the user, leading towards a more intuitive prosthesis control paradigm. Examples for this control concept found in the literature are based on Non-Negative Matrix Factorization [54, 55, 56•], Artificial Neural Networks [57–62], or Linear Regression [63], allowing two or more degrees of freedom to be activated simultaneously and proportionally. Notably, some of the limitations of pattern recognition systems have been addressed using regression approaches, such as electrode shift [64]. The regression method has so far provided promising performance, allowing direct and intuitive control for the user, and developments in the near future are likely to further strengthen the robustness of this approach.

Surgical Methods to Improve Control

An alternative to developing algorithms to decipher EMG signals is to record from individual muscle groups. Aside from implanting electrodes, another method to overcome EMG crosstalk is to surgically isolate nerves and amplify their signals through targeting muscles of interest.

In 1995, Kuiken et al. [65, 66] demonstrated in rats that injured muscles could be hyper-reinnervated with additional nerves to encourage increased strength and muscle mass, while also proposing that this technique could provide additional EMG control signals for some amputees. In subsequent studies, it was also noted that subcutaneous fat dampened EMG quality [67], and that reduction of this fat could increase EMG amplitude and independence [68]. In 2004, the group went on to show, in a bilateral shoulder disarticulation amputee, that four independently controlled muscle units (three of which survived at 5 months) could be created from residual brachial nerves anastomosed to nerve segments originally supplying the pectoralis major and smaller muscles [69]. Termed targeted muscle reinnervation (TMR) (Fig. 5), this landmark procedure allowed the patient to perform simultaneous control of two degrees of freedom at the elbow and wrist of a prosthetic arm, using standard reconstructive techniques and without the need for implantable devices [70]. TMR has since developed, incorporating more patients [71, 72], and also describing the transfer of hand sensation to the area where nerves have been re-directed [73]. Notably, this technique has not only allowed multifunctional control of prosthetic limbs in real-time [74] but has also acted as a catalyst for further development into EMG processing for prosthetic control.

Fig. 5
figure 5

A simple schematic of the TMR procedure. Surviving nerves are co-apted to nerves supplying surviving muscle groups. In the case of this image, the pectoralis muscles are used to amplify the EMG signals generated by radial nerve activity

Full size image

Clinically, prosthetic limbs only use a few electrodes (between 2 and 6) to describe the most important movements, even in patients who have undergone TMR surgery. However, recent attempts to harness the full spectrum of EMG signals available have shown that high density (HD) EMG can provide the full discharge patterns of active motor units [74–77], which may be a future alternative source of control.

Future Directions

Closing the Loop

One particular area that needs to be addressed clinically to improve prosthetic control is to complete the control loop itself. Delicately holding a fragile object requires an intact sensory–motor integration system that finely balances executed motor commands in response to appropriate sensory stimuli. Current prosthetic technology does not possess sufficient tactile feedback, if any at all, to replace that of human touch, so current control strategies rely primarily on visual feedback [78].

To address the reliance on visual feedback, new research is attempting to improve the use of tactile feedback. TMR has demonstrated that not only were motor nerves transferred to new muscles but also some sensory nerves. In a shoulder disarticulation patient’s reinnervated regions, tactile stimuli gave the sensation of being touched on the missing limb, while still being able to differentiate hand from chest sensation [73]. In addition, using a haptic device, which transmitted vibration from the prosthetic limb to the amputee’s reinnervated skin, encouraged the patient to include the prosthesis as part of his self-image, as opposed to a foreign entity [79]. Separately, in a transradial amputee, in whom intraneural implant electrodes were stimulated to provide sensory feedback via the median and ulnar nerves to complete a motor control task, it was shown that success rates increased significantly compared to just visual feedback [80].

Virtual Rehabilitation Environments

Another concept that works around the limitations of only having visual feedback is the use of a virtual rehabilitation environment (VRE) to improve prosthetic control (Fig. 6). In the context of this review, a VRE refers to a computer-based system, presenting to the subject a visualization of a task to be completed. The subject is able to control elements of the VRE using muscle activation, perceiving visual feedback by graphical representation. The VRE is capable of both assessing the performance of the control algorithms as well as training prosthetic users before actual use of the device.

Fig. 6
figure 6

Future perspectives in enhancing prosthesis control and training. By providing feedback, the user can get additional information for instance about the grip force. Possible modalities are haptic feedback (a), electrocutaneous stimulation (b) or an augmented reality [92], i.e. with Google Glasses (c). Finally, virtual rehabilitation environments (d) can be used to provide early training after amputation

Full size image

In terms of evaluating control algorithm performance, a VRE can be used in online scenarios. First demonstrated in 2003, VREs allowed the user to control virtual hand movements using EMG patterns [81]. More advanced VREs have since been developed, providing more realistic graphics and increased function [82, 83], as well as interactions with virtual objects [84]. Formal performance assessment of control algorithms was validated using the target achievement control tests [38•], in which prosthetic users steered a virtual prosthesis into a predefined target position. This concept was streamlined using an abstract but more intuitive VRE paradigm of a simple arrow reaching circular targets [55].

VREs are also beneficial in training prosthetic users. After an amputation, the patient usually undergoes a long rehabilitation process (up to 6 months), including the healing phase of the affected upper extremity, before they can be fitted with their prosthesis. Currently available training systems are simple, such as the Otto Bock MyoBoy (http://www.ottobock.de/cps/rde/xchg/ob_com_en/hs.xsl/3795.html), and can be used to train the muscles necessary for direct control of commercially available hand prostheses. However, it would be beneficial to train users in more challenging scenarios, such as grasping virtual objects, which would facilitate the learning process. Consequently, VRE systems can provide an environment shortly after amputation to perfect prosthesis control even before fitting [38•, 74, 85, 86]. Despite these advantages, this application of VRE in myoelectric control is still not widely established.

Implantable Methods

Whilst surface EMG control methods are currently the most applicable, there has also been significant progress in implanted recording methods. Peripheral nerve function has been shown to remain intact many years after amputation [87], and successful direct nerve recording systems have been demonstrated in amputees [80, 88, 89], as well as cortical recording in patients with spinal cord injuries [90, 91]. In addition, it may be possible to permanently link implanted electrodes through a bone-anchored conduit [22] or wirelessly [20]. While these examples are either animal or single case studies and are far from general clinical use, they offer an insight into alternative avenues of how future prosthetic limbs might be controlled.

Conclusions

Considerable work has been done in feed-forward control to improve the manner in which amputees interact with their prostheses. However, there are still challenges that are not easily overcome for clinical use. Key areas to address are the speed and reliability of control in achieving multifunctional movement. Newer regression control algorithms may combine the best features of the preceding algorithms but are only in the infancy of their development. In addition, improving the sensory input beyond just visual feedback will be key to more fluent control. In the meantime, innovative systems such as VREs enable current amputees to engage with their prosthesis and hopefully reduce the rejection rate.