Going beyond PA: Assessing sensorimotor capacity with wearables in multiple sclerosis—a cross-sectional study

A convenience sample of 90 patients was recruited at the Centre for Clinical Neuroplasticity, Medical Park Loipl, a specialist clinic for neurology and rehabilitation in Germany (Table 1). Inclusion criteria were a diagnosed MS (according to the patient file) and the willingness to participate in the study. Diagnoses were, dependent on the date of diagnosis, based on different criteria, which were McDonald (2017) in 23% (n = 21), McDonald (2010) in 30% (n = 27), McDonald (2005) in 13% (n = 12), McDonald (2001) in 12% (n = 11), Poser in 19% (n = 17), and Schumacher in 2% (n = 2) of the cases. None of the initial diagnoses have been revised. In that sense, we assumed a clean sample of persons with multiple sclerosis. Exclusion criteria were the presence of neurological comorbidities like, for instance, stroke. Further, due to the nature of the rehabilitation facility, a certain preselection of potential participants was unavoidable, excluding participants with EDSS scores higher or equal than 8.5, intense need for nursing, and psychiatric disorders. Ethical approval was given by the ethics committee of the Medical Faculty of the Technical University of Munich (approval identifier: 478/19 S-SR). All participants gave written informed consent to participate in the study.

Participants underwent a sensorimotor test battery at the clinic’s Neuro Assessment Lab (NAL), which included a set of tests of upper-limb sensorimotor capacity and gait performance. On the same or following day, participants were equipped on the dominant or better functioning (i.e., predominantly used in everyday life) upper-limb with a Huawei gt2 smartwatch (Huawei Investment & Holding Co., Ltd., Shenzhen, China) for one day. The smartwatch comprised a custom Android application (TUM:W—standing for Technical University of Munich Watch) that has been developed using C# Xamarin (Visual Studio 2019; Microsoft Corp., Redmond, WA). Participants wore the watch for one full (working/therapy) day between 08:00 and 18:00. Amount and content of therapies was not controlled for. Participants were told that the watch aims to assess the quality of everyday movements rather than PA (as part of the informed consent). As visible in Table 1, not all participants had full datasets, which was in eight cases due to an inability to carry out balance and gait tests in higher EDSS grades. One participant (1 out of 1) with an EDSS of 8.0 was not able to carry out neither the balance nor the gait tests. Two participants (2 out of 4) with an EDSS of 7.5 were not able to execute the balance tasks. One participant (1 out of 2) with an EDSS of 7.0 was not able to execute neither the balance nor the gait tests and a further one (1 out of 2) was not able to carry out the balance task (total 2 out of 2). Three participants with an EDSS of 6.5 (3 out of 6) were not able to execute all gait tests.

We recorded the following key characteristics: The age (in [a]; AGE), the time since first manifestation of MS (in [a]; FM), and the EDSS.

All tests of the Neuro Assessment Lab (NAL) were used to estimate the sensorimotor capacity of the participants.

Upper-limbs: We recorded the summed performance of dominant and non-dominant upper-limb for the nine hole peg test (due to a non-normal distribution we used the reciprocal 1/ sum of trial durations which lead to [1/s], PEG), a grip strength assessment using a isometric dynamometer (sex-specific z-scores of the summed force equivalent in [kg], GRIP), a upper-limb tapping task (sum of frequencies over the course of 10s in [Hz], TAP), and a pursuit task (sum of deviations from target in [mm], DOT).

Information processing capacity: Simple and go/no go reaction times (mean of 10 (go) trials per task in [ms], RT and GNG) were checked to coarsely estimate the information processing capacity of our subjects. Participants were asked to use their better functioning upper-limb to execute the tasks.

Postural control and gait: Further, body sway in parallel stance with open eyes and the Romberg index in a closed eyes condition (sway closed eyes/ sway open eyes) (10s trial durations, sway as the mean acceleration of the frontal part of the hip in [mm/s²]; SWAY and ROMBERG), and the time to execute the timed stand up and go (with 2 × 3m walking distance], in [s]; TSUG), the time to walk 10m at normal and maximum pace (in [s]; NORM10 and MAX10) were assessed. Lastly, in gait at self-selected (“normal”) pace, the step frequency (in [Hz]; FREQ), the smoothness of gait (in % with 100% being perfect; COMPLEXITY), the signal to noise ratio of gait (in % with 100% being perfect; NOISE), an gait asymmetry index (ratio of second highest and highest weighting of the frequency spectrum of the acceleration signal at the sternum in % with 0% being perfect; ASYM), and the limping (relative deviation of acceleration maxima between the two legs during gait in % with 0% being perfect, LIMP) were assessed. An upper- and lower-limb laterality index (LAT) was computed as the relative interlimb differences of the following tests: nine hole peg test, grip strength, tapping, smooth pursuit, and limping during gait. Lastly, the WSS was determined on the basis of sensorimotor performance. All tests, the underlying algorithms and methodologies, as well as the validation of the WSS have been previously described in Gulde [18].

All parameters of the smartwatch (TUM:W) were used to assess the level of physical activity and explore the potential of a set of mostly gyroscope-derived parameters to estimate the sensorimotor capacity that has been examined in the NAL (see above).

The measurement frequency of the accelerometer was 100Hz and of the gyroscope 25Hz. The built-in pedometer was accelerometer-based. The gyroscope data was used for kinematic analyses (spatiotemporal characteristics of behavior), so we smoothed it using a moving average of 0.42s [14]. All data processing to derive the used parameters was executed on the smartwatch to minimize needed storage. Figure 1 is an illustration of TUM:W and its derived parameters (additionally summed up and explained in Table 2).

Accelerometer (incl. pedometer): Every minute, the number of taken steps, based on the built-in pedometer, was read out. The 95th percentile of the cadence (of all minutewise data points which exceeded 5 steps/min to exclude non-gait intervals) was calculated (CADENCE95), as well as the mean of all minutewise pedometer data points exceeding 5steps/min in [steps/min] (CADENCE). The coverage of a whole minute allowed to emphasize outdoor walking. Indoors, especially in a room or flat, continuous gait with durations of more than a minute would most probable not take place. The 95th percentile was thought to catch the achieved maximum (excluding outliers, which were expected to occur less frequent than in, e.g., 5s measurement periods; in this sense, the 95th percentile was used instead of the 90th percentile), while the mean also represented fragmentation of distance and activity, i.e., walking, but not continuously, within every included minute.

The 90th percentile of the mean acceleration in [g] (MEANG), the mean amplitude deviation (MAD; the average deviation from the mean of the signal) in [milli-g], and the taken steps (i.e., movements recognized as steps) per second were read out and the frequency of steps (STEPFREQ) was derived, based on consecutive measurements of 5s. The shorter interval for gait assessment was thought to emphasize covering short distances (going to the bathroom or from the patient room to the cafeteria) and therefore potentially a different aspect of gait than the cadence parameters (which were covering 1 full minute).

Gyroscope: Every 5s, the mean angular velocity in [°/s] was calculated and the 90th percentile was computed (ACTI). This parameter was thought as the gyroscope-equivalent to MEANG.

Every 2.5s for the past 5s, the highest coefficient of autocorrelation (AUTOCOR) and its frequency in [Hz] in a band of 1.0–3.0Hz were calculated (AUTOCORFREQ). This was thought to catch cyclic movements, for instance gait. The relatively short time windows (i.e., 5s) of assessment was chosen to enable to also catch short movement durations (for instance, washing hands or walking indoors).

Every 30s, the last 60s of angular velocity data were read out and a set of kinematic parameters was derived. For each described parameter, the 90th percentile (i.e., reproducible best performance) was computed. The five highest velocity peaks in [°/s] (MAX5), the standard deviation of the velocity in [°/s] (STD), the standard deviation divided by the mean of the velocity (STDMEAN), the ratio of velocity peaks that exceed the last minimum by 0.17°/s (in order to exclude peaks by “human” noise; sensor noise was cancelled by signal smoothing) and all velocity peaks (PEAKRATIO; the ratio of intended peaks and the sum of intended and unintended peaks as an estimate of movement smoothness), the number of velocity peaks per 360° (PEAKSPER360, in contrast to the other parameters, high values represent the worst performance; as an estimate of movement smoothness [15]), the mean of the velocity peaks [°/s] (MEANPEAK), the standard deviation of the velocity peaks [°/s] (PEAKSTD), and the standard deviation of the velocity peaks divided by the mean (PEAKSTDMEAN).

Summary parameters: For the full recording, the ratio of activity bouts (MAD > 100 milli-g) of at least 30s and of at least 5s was computed (FATREV; accelerometer, higher values indicate lower fragmentation). Further, the relative amount of time of at least light physical activity (MAD > 100 milli-g) (RELPA; accelerometer), and the amount of gait with low autocorrelation (at least 5 recognized steps/min and a coefficient of correlation below 0.30) (LOW; gyroscope) were derived.

A summary of all used parameters is given in Table 2. Based on a-priori assumptions, we assigned dimensions, i.e., PA and beyond PA, to the parameters. PA means that we assumed a parameter to be closely related to energy expenditure. This included the gait parameters (CADENCE95, CADENCE, STEPFREQ), parameters describing average acceleration (MEANG) and the average of its fluctuation (MAD), and parameters that are based on volume (RELPA) and fragmentation (FATREV) of energy expenditure. We assumed all gyroscope parameters to rather describe the quality than the intensity or volume of behavior, since they mostly rely on wrist rotations that move only small masses (the hand and everyday objects like a pen or glass of water)—and even rotations of the whole upper-limb (by turning of the shoulder) are expected to rotate along the axis of shoulder and hand and in direction of gravity when executing tasks with higher torque demands. The variance in these measures is therefore rather connected to strength, dexterity, and potentially personality (speed-accuracy trade-off) than the cardiovascular capacity and physiological workload. Further, we expected the gyroscope derived parameters to be driven by (rotational) upper-limb, especially hand, movements that are not made to move larger masses like the own body mass during gait (leading to meaningful energy expenditure). Still, rotational movements can be more or less intense. This can be very well described by peak-based metrics. Such metrics extract local maxima, ignoring phases of inactivity and only taking actions into account. MEANPEAK would describe the average reached angular velocity of movements, for instance, when opening a lid or combing one’s hair. Higher values would indicate that these movements were carried out quicker (with higher peak velocities), meaning more intense. Deriving the standard deviation of the reached peaks (PEAKSTD) would further describe to what extent one can adapt the movement intensity—scooping a hot soup should not necessarily be too intense. However, higher values (i.e., variance) can only be achieved if higher intensities are reached (the lower limit of zero cannot be shifted), so PEAKSTD indicates the intensity of movements, but is also taking into account that the intensity should be adapted (to a task). In order to extent this thought, we also derived the quotient of standard deviation and the average (PEAKSTDMEAN), examining the adaptability within the range of one’s capacity to carry out intense movements.

Patients revealed different behavioral stereotypes (as previously reported for self-reported fatigue [19], that strongly differed in the amplitude of upper-limb PA, gait-related PA, as well as their clinical status. Cluster 1 revealed greater amounts of upper-limb PA, while showing little gait PA, with an overall good clinical status. Cluster 2 showed average upper-limb PA and clinical status, but a high level of gait PA. Cluster 3 was slightly inactive (upper-limb PA), showed little gait activity, and had the most severe clinical status. Cluster 4, although having a good clinical status, showed little gait activity and the least upper-limb PA. Cluster 3 showed an expectable behavior concerning the clinical status (i.e., more severe clinical status, physically slightly inactive, and very little gait activity). Cluster 1 and 2 suggest that there could be different emphases of PA, either trying to maximize ambulatory activity or upper-limb movements, although it remains unanswered how these upper-limb PA levels were achieved—by ADL, upper-limb trainings, gesticulating while conversing, or recreational activities (knitting, gaming, nose-picking, etc.). Although cluster 4 had the best clinical status, they revealed little to very little PA of upper-limbs and gait. These could be indicators of sedentary behavior, potentially caused by psychological factors like experienced fatigue, depression, or low motivation; it has been previously reported that higher levels of experienced fatigue are rather accompanied by lower levels of PA than activity fragmentation, which would be the case for cluster 4 (FATREV [fragmentation of PA] cluster 4 = 0.56 ± 0.08 vs. FATREV all participants = 0.54 ± 0.15, p = 0.642) [17, 30]. Taken together, the clusters show that (a) just because one can walk doesn’t mean that one will walk, (b) the amount and intensity of usage of upper-limbs is quite independent from the clinical status, and (c) that sedentarism might be limb-specific, i.e., persons with a lot of gait activity do not necessarily show a lot of upper-limb activity. It will be necessary to examine the impact of personal interests, therapy content, intensity, applied therapists, medication, as well as cognitive and emotional status on the translation of clinical status (e.g., the ability to walk fast or use the hands in a dexterous way) into everyday behavior.

We explored three TUM:W parameters of particular interest; parameters with strong unidimensional loading for each of the extracted components. Each of the extracted three components represented a certain aspect of capacity and/or everyday behavior. If being able to gather information of sensorimotor capacity (e.g., grip strength, finger dexterity, or gait quality) from wearable-derived parameters, it would allow us to shift from a lab-based scenario with discrete measurement points to a continuous assessment strategy. This could also cover fluctuations, the course of therapy or disease, and provide the opportunity to go beyond PA, for instance, distinguishing between a sedentary healthy person and a physically active person with a neurological disease, or early detecting persons at risk/without diagnosis. Beyond PA, in this context, is the estimation of sensorimotor capacity, independent of physical activity.

PEAKSTD [standard deviation of angular velocity peaks] showed connections with gait and upper-limb performance and the EDSS (Figs. 4, 5). When examining the sensorimotor capacity of the upper-limbs, only GRIP was not fully mediated by the EDSS. In this respect, the standard deviation of the angular velocity peaks was a moderate predictor of sex specific grip and—considering the gait speed associations—general physical strength (as grip strength has been shown to estimate general physical strength in clinical populations [34]. Increased physical strength would allow an individual to fully adapt its actions to the demands of different actions (e.g., slow manipulation vs. quick transport), while weaker individuals tend to rather show monotonous behavioral patterns due to their limited task-adaptability as it has been shown for elderly with higher levels of frailty [32]. It is important to note that the parameter was based on angular velocity, i.e., rotation and not translation of the wrist was the driving factor. The ability to predict the sex-adjusted grip-strength makes PEAKSTD a parameter with great potential for populations with a (pathological) loss of muscle mass and/or function (e.g., frail elderly), not only to assess the progression, but also to observe rehabilitation outcomes in an externally valid way (actual use in everyday life).

The derived mean cadence (CADENCE) of recognized minute wise ambulatory activity was associated with almost all lab assessments (Fig. 4), covering the full range of upper-limb capacity, gait performance, as well as clinical estimates. The strong association with the calendar age of patients, stronger than the correlation with FM [time since first manifestation of MS], indicates that age should always be concerned, independent of neurological health status [27]. CADENCE was a good predictor of the global condition of a subject, but very unspecific (i.e., CADENCE was a good predictor of a multitude of parameters,see Table 4, Figs. 4, 5). One can derive the status or the changes of status without knowing the underlying mechanisms. Gait is frequently called the “sixth vital sign” [11] and is, as in our case, often a strong, but unspecific symptom [29, 35]. Additionally, cluster 4 indicated that gait-related PA alone can be misleading, as it had a good clinical status (with an emphasis on gait capacity) but revealed only little gait. Walking little can stem from an impaired gait function, a low motivation to walk, or a combination of both. Further, one should keep in mind that a low fitness level has been shown to be a risk factor for developing MS [6]. Cortese et al. [6] (successfully) used the duration for a 3000m run of 19a men (Norwegian conscription measure) to estimate the impact of aerobic fitness on the prevalence of MS with onsets later than 10a after the fitness test, therefore controlling for potential confounding early disease signs. In this sense, i.e., low fitness levels being a risk factor for MS, CADENCE alone might not be the best measure to go beyond PA, although adding the percentile-based metric of the step frequency or cadence to the currently used daily step counts [1, 4, 5, 17, 20] might be a valuable addition (not only in neurological samples). The remaining association of CADENCE and TAP [upper-limb tapping] after controlling for the EDSS indicates that the average cadence is already deteriorating with decreasing conductivity (tapping tasks have been shown to be markers of reduced central nerve conductivity [20] before reduced gait distances become apparent in the EDSS scoring system. The connection of CADENCE with DOT [pursuing a moving target with the finger] (approx. 15% shared variance) rather underlines the missing representation of non-self-paced time pressure tasks in the disease rating (DOT have been shown to be moderately associated with the EDSS [18], although the multiple sclerosis functional composite used to include the paced auditory serial addition task [26]. Interestingly, but in accordance with [36], fast lab walkers were apparently not automatically fast everyday walkers, as there was no connection between gait velocity (in the lab) and CADENCE after controlling for the EDSS. One potential alternative explanation for the connection of CADENCE with TAP and DOT would be that strong upper-limb movements have been wrongly recognized as steps by the pedometer and persons with better upper-limb capacity would show more misrecognizable activity. However, a missing cross-loading on PA (UL) (Fig. 2) speaks against this.

FATREV [fragmentation of PA], the inverse ratio of short and long activity bouts, with higher values indicating less fragmentation, revealed, like CADENCE, a very global connection to the almost the complete test battery of the NAL (Figs. 4, 5). Same as CADENCE, it appeared to be comprehensive but unspecific. A deeper examination, however, showed that FATREV was well able to go beyond PA (after controlling for EDSS mediation), being a moderate to good predictor of upper-limb capacity and even weakly of information processing capacity. FATREV (EDSS adjusted) was able to predict the (inverse) nine hole peg test performance, grip strength, upper-limb tapping, pursuit under time pressure, and reaction times (simple and go/nogo). Especially the grip strength, tapping, and pursuit task predictions had quite strong |β|-weights of > 0.50. Studies from ageing and stroke have revealed for complex ADL that increased fragmentation (active times during task execution, usually referred to as relative activity) during execution is a good predictor of overall performance (i.e., trial duration) in elderly [13, 16], further that performance in neurological populations (i.e., stroke survivors) could be globally reduced (i.e., independent of sub-action demands) [12], and that higher individual demands of a tasks can also lower the execution speed and therefore sub-pass the threshold of ‘activity’ (i.e., activity being meaningful for increasing the energy expenditure) for wearables [16, 32]. Too what extend fatigability [19, 30], cognitive task demands [12, 13], or limited sensorimotor capacity is leading to a reduced FATREV, however, stands to be answered in future studies, as we did not include tasks on fatigability (motor/&cognitive in the NAL test battery).

The potential advantage of the three discussed parameters in comparison to conventionally assessed capacity measures is that they can be derived from wearables and therefore be used in a continuous way (see discussion above). Further, assessing persons in their home environment (including the nearby community), would allow us to have externally valid information of sensorimotor capacity (and its fluctuations and changes), as an additional dimension to physical activity measures.

The use of the 90th percentile was aiming at assessing the best possible performance during daily life without being too prone to outliers or measurement errors. In future it would be of interest to see if such derived values tend to be stable over longer measurement durations. Further, gyroscopic data appeared to be better suited to go beyond PA, since there appeared to be less “cross-talk” with gait, like it is often observed in pure accelerometric assessments [7, 23, 24], and due to the fact that rotation (and orientation) of the forearm is crucial for ADL performance [9, 23], however, a direct comparison of comparable gyroscopic and acceleration based parameters is still lacking. Different sensor placements might also allow to assess gait and postural stability in daily life, as there have been promising approaches [10]. The extent of transferability of our findings to world settings (not inpatient rehabilitation), longer measurement periods, or healthy samples will be covered in future studies. So far, battery life remains the strongest barrier in this field of research (without losing information by down-sampling) [7]. Another promising approach would be to examine the impact of therapy volume, exercise intensity, and the executing therapist on behavior and its changes. Such data could be used to optimize and personalize therapeutic measures.