Multi-subject/daily-life activity EMG-based control of mechanical hands

Ten healthy subjects joined the experiment after having given their informed consent. The subjects were two women and eight men, nine right-handed and one left-handed, average age 30.9 ± 8.45 years, standard Caucasian weight and height. They were given no knowledge of what the experiment was about.

The experiment consisted of two phases. During phase 1, after a "rest" condition was sampled to define the baseline EMG activity, the subject would keep her/his arm still and relaxed on a table, and was asked to grasp a force sensor using, in turn, three different ways of grasping it (Figure 1): index precision grip, other fingers precision grip and power grasp. While gathering data, a label {1, 2, 3, 4} denoting the grasp (or rest) was attached to each sample, in order to build the ground truth values.

Each subject's experiment resulted in something more than 1200" of data. Data were sampled at 2 KHz, resulting in about 2.4 × 10⁶ samples for each subject, equally distributed in each phase.

We employed Aurion ZeroWire wireless surface EMG electrodes [17], in order to ease the FA phase, which required free movement in the laboratory. A FUTEK LMD500 Hand Gripper force sensor [18] was used to detect the force applied while grasping. (See Figure 2.) A standard digital acquisition board (National Instruments NI-USB6211) was used to record the signals, connected to the receiver of the EMG wireless device and to an amplifier, in turn connected to the force sensor. The sampling rate was set at 2 KHz in order to correctly sample both signals (the EMG signal relevant bandwidth lies between 15 and 500 Hz). The board was connected via a USB port to an entry-level laptop. We used a custom National Instruments' LabView VI block to acquire the signals.

Seven electrodes were glued on each subject's dominant forearm, according to this anatomic guideline:

These positions were chosen, according to the medical [19] and bioengineering [20] literature, to detect the activity of the flexor and extensor muscles of the forearm which are most relevant during grasping. Figure 2 (rightmost panel) shows the typical electrode positioning. Notice that there may be remarkable inter-arm differences depending on the subjects' age, gender and physical fitness. Moreover, some of the aforementioned muscles are deep into the forearm, so that muscle cross-talk cannot be avoided. This is a well-known problem in the EMG literature [1, 13].

The root-mean-square (RMS) of the EMG was evaluated using a time window T _RMS . The optimal value of T _RMS was evaluated independently for classification of the grasping posture and force detection, via grid search, in a preliminary phase of the experiment, and set to 500 ms for classification and 100 ms for regression. The choice of the RMS, as opposed to the simpler rectification and filtering, is motivated by its well-known relationship to the force exerted by the related muscle [1, 2, 13]. Rectification plus filtering would likely work as well, and it is indeed employed in some commercial myoelectrodes such as Otto Bock's MyoBock ones [21].

Notice that the right choice of T _RMS can be, in general, crucial: a small value will make the system more responsive (i.e., implies a smaller delay) but a higher value will be more informative and improve the performance (especially in the case of classification, as we verified). On the other hand, it is known that the EMG signal anticipates the muscle movements by a few hundreds milliseconds; therefore, in a practical application derived from this experiment, a wider lag would be more acceptable than one would expect. The electromechanical delay (EMD) of a muscle is defined as the interval between the onset of the electrical activity of the muscle (EMG) indicating its activation by the neural system and the onset of the resulting change in the mechanical variable observed. The delays reported range from 25 to 100 ms for different muscles and tasks [22].

Figure 3 (left) shows the typical EMG signal (red) and force (blue) recorded by the force sensor. Clearly the amplitude of the envelope of the EMG is related to the force, as is indicated in literature. The right panel of the Figure shows the bandwidth of the EMG. Figure 4 shows the effect of the RMS on the frequency components of the EMG, for three different values of T _RMS . In all cases, the RMS signal bandwidth is upper-bounded by about 25 Hz (left panel, for T _RMS = 20 ms) to 10 Hz (right panel, for T _RMS = 0.5s), as expected (larger values of T _RMS correspond to a better filtering but also to a larger delay). According to these figures, we subsampled the RMS of the EMG signals at 25 Hz by taking one sample in 80 of the original sequence, resulting in about 30.000 samples for each subject.

Lastly, samples for which the applied force was lower than a specific threshold were removed. After verifying several choices both numerically and visually, the threshold was uniformly set at 20% of the mean force value obtained for each subject and phase.

According to previous literature (e.g., [14, 16]), the statistical analysis was carried on using Support Vector Machine (SVM). For a comprehensive tutorial on SVMs refer to [23, 24]. SVMs are a statistical learning method able to build an approximated map between an input space and a label (classification) or a real value (regression). Classification is here used to classify the type of grasp according to the EMG signal, whereas regression is used to understand how much force the subject is exerting, independently from the grasp type. The input space is ℝ⁷, one coordinate for each EMG electrode. We used the ground truth values as labels and the force value given by the force sensor for the regression. Notice that SVMs work here in real-time, associating a grasp type and a force value to an EMG value at each instant of time. Grasp type and forces are then predicted almost at the onset of the grasping movement, differently from what happens in other approaches (e.g., [14, 25]) in which all values of the input signal over a further time-window are employed as the input space.

In order to ease the computational burden we employed uniformisation [16] to reduce the size of the training sets. The samples in a training set are considered one by one in chronological order, as it would happen in an on-line setting, and each new sample is added to the training set if and only if its Euclidean distance from all training samples retained so far is larger than a predefined value d. Values of d were set to 0.02 for the SA phase and 0.032 for the FA phase. These values were chosen in order to get not more than one thousand training samples for subject 1. The choice is arbitrary, but notice that (see [16] again) the performance of such systems changes linearly as d changes, whereas the training set size varies polynomially; thus, it is always possible to find a polynomially smaller training set, if needed, which will degrade the performance only linearly. This really means that the initial choice of d is not crucial. Also, notice that testing sets have not been uniformised, in order to give a more realistic result.

SVM analysis was performed for each subject and for each phase, to check how the performance depends upon subjects and conditions. For classification, the performance index is, as is customary, the percentage of overall correctly guessed labels. For regression, the performance index is the correlation coefficient evaluated between the predicted force signal and the real one. The choice of the correlation coefficient is suggested by this consideration: when driving a prosthesis we are not interested in the absolute force values desired by the user/subject, since mechanical hands usually cannot apply as much force as human hands do, for obvious safety reasons (or, e.g., in teleoperation scenarios, they could be able to apply much more force than a human hand can). Rather, we are concerned about getting a signal which is strongly correlated with the user/subject's will. Anyway, we also report about the normalised root mean-square error (NRMSE), in order to give a broader view of the results. Normalisation is done against the signals' ranges (notice, though, that correlation is the criterion used to find the optimal parameters during grid search). We employed a well-known freely available SVM package, libsvm v2.83 [26], in the Matlab wrapped flavour; the Gaussian kernel was chosen, since it is a standard choice in previous literature. EMG data were normalised along each dimension, as is customary, by subtracting the mean value and dividing by the standard deviation. 5-fold cross-validation was used to assess the generalisation error for each training set; this measure was then used for grid-searching the typical Gaussian kernel hyperparameters of a SVM, called γ and C. Once these parameters were found, the overall performance was evaluated as the mean and standard deviation of the performances obtained on each fold.

Figure 5 shows the main results. Classification accuracy (top panel) for the SA phase ranges from 99.58% ± 0.17% (subject 5) to 91.37% ± 0.89% (subject 8); for the FA phase, it ranges from 98.40% ± 0.08% (subject 2) to 82.43% ± 1.24% (subject 8 again). On average over all subjects, the classification accuracy is 97.14% ± 2.90% for SA and 95.24% ± 4.77% for FA. Notice that the performance is consistent by subject and by phase, meaning that (a) hard subjects in the SA phase are hard as well in the FA phase and viceversa, and (b) the FA phase is always harder than the SA phase.

Figure 6 shows the real and guessed force values for a typical subject, namely number 6, FA phase. Strong correlation between the guessed and true values is visually apparent, in agreement with the performance values outlined before. On the other hand, Figure 7 shows the (average) confusion matrices for the SA and FA phases. Clearly, most of the classification errors, for both phases, regard the "power grasp" being mistaken for the "other fingers precision grip". This is intuitively sensible, since gripping with middle, ring and pinkie finger involves co-contracting the index finger too, to some extent. This makes the former grip quite similar to the latter, from a muscular point of view.

As far as hyperparameters grid search is concerned, Table 1 shows the average values of (the logarithms of) γ and C for the optimal models obtained via cross-validation. The grid search ranges were [0, 3] for log₁₀(C) and [-1.85, 0.16] for log₁₀(γ) (these are standard values in literature, given the dimensionality of the input space). The average value of log₁₀(C) is around 1.5, but its standard deviation is rather wide with respect to the range, at least in the case of classification. The standard deviation is smaller for regression than for classification in both cases, which seems to indicate that regression is more stable a problem with respect to the hyperparameters.

In order to check whether the FA phase is really indicative of what a patient might do in her/his DLAs, we have trained a machine on the data gathered during the FA phase and then tested it on the data gathered during the SA phase. Figure 8 shows the results of testing FA-models on SA data, and viceversa.

FA-models tested on SA data obtain an average accuracy of 75.11% ± 12.34% for classification and 0.8056 ± 0.1151 for regression; whereas testing SA-models on FA data gives 70.17% ± 11.99% in classification and 0.7530 ± 0.1153 in regression. The advantage of FA models over SA models is apparent, uniform and consistent. Notice that here we show no error bars, since, for each subject and phase, there is just one training set and one testing set.

Lastly, let us consider the worst result of the per-subject analysis -- subject 8 in the FA phase, as far as classification is concerned. One of the possible causes of this comparatively low performance (82.43% ± 1.24) is that too many samples are missing from the original training set (d too high). In order to test this hypothesis, we let d linearly range around the pre-set value of 0.032 and check (a) the size of the resulting training set and (b) the performance obtained by the system. Figure 9 shows the result of this test.

The Figure confirms that the training set size has a decreasing polynomial trend, while the performance changes linearly [16]. In particular, for d = 0.032 the previously shown performance appears, whereas if a larger performance is required, one can increase the number of samples in the training set, or, which is equivalent, reduce the magnitude of d. For instance, to get an accuracy of about 90% d must be set at 0.2 ending up in a training set with some 1600 samples.

Recall that in this experiment, for all subjects, the EMG electrodes were carefully positioned on the forearm according to an anatomical guideline, meaning that noise due to inter-arm differences should be to some extent avoided. We can therefore check how well each model performs on each subject by building a cross-subject performance matrix A, for both classification and regression, and for both phases, in which A _ij is the performance index attained by a model trained on data gathered from subject i while predicting data gathered from subject j. Figure 10 shows the matrices.

In previous work it was shown that a significant (inverse) correlation appears between the cross-subject performance matrices and the cross-distance matrices D, obtained by evaluating a mean distance D _ij between two sample sets S _i and S _j like this: