This work aimed at identifying the barriers and facilitators for technology-driven assessments for sensorimotor function, as perceived by different stakeholder groups. The survey was carried out via an online questionnaire to obtain a broad overview of different viewpoints from across the globe. Our results reflect the opinion of 140 respondents from 23 countries from the Engineer, Clinician, Neuroscientist, Patient and Medical Industry stakeholder groups. There were disparate views on the current bottlenecks to technology-aided sensorimotor assessments, due to differences in stakeholders’ professional backgrounds, their goals, and the primary focus of their work. Nevertheless, we found that stakeholders generally agreed on the value of technology-aided sensorimotor assessments and how to move towards clinical practice.
Current barriers
An obvious barrier to the adoption of technology-aided assessments could be the lack of demand for such tools, i.e., if assessments are not routinely performed. Previous studies [
6,
8,
9] reported low implementation rates of assessments in routine clinical practice, which is consistent with the beliefs of Research Engineers, Basic Scientists and Medical Industry professionals in our questionnaire (section 3.2.1). However, all clinicians reported that they use standard clinical assessments, with approximately 75% using them on a regular basis (50% at least once a week and 24% at least once a month; Table
4). It is possible that the inconsistency between previous studies and our results on clinicians’ reported use of assessments reflects a biased view from our respondents, who may be more inclined to use assessments. But it is also possible that assessments are currently performed more regularly than in the past (based on literature references from 5 to 9 years before our survey). Furthermore, Medical Industry professionals reported that there is a strong interest among their clients in tools for sensorimotor assessments, foreseeing an increase in demand in the coming years. Medical Industry professionals also reported that they invest resources in the development of such tools, and that assessment capabilities are a selling point for their devices. Thus, the lack of need is likely not the reason for the poor adoption of these tools in clinical practice.
It is possible that the available technology does not meet the user needs for sensorimotor assessments due to differences in understanding of the role of assessments in clinical practice among the different stakeholder groups (especially between developers and users). However, we found general agreement across stakeholders. There was agreement among stakeholders that individual reasons were the most important reason for performing assessments (Table
6). Stakeholders also agreed, in general, about the level of detail required from assessments for different purposes (Table
7): communication with government bodies, insurance companies and patients were perceived to require the lowest amount of detail, followed by evaluation or selection of therapies, and finally determining medications or invasive interventions. These results indicate that there is an alignment across stakeholder groups about the general reasons for doing sensorimotor assessments, and the level of detail required for different goals.
Another barrier for adoption of technology-aided assessments could be the lack of access and exposure to available tools. We did not find this to be the case among the respondents in this study; the majority of stakeholders had used or at least observed technology-aided assessments being used (Fig.
5). In particular, around 50% of Clinicians worked with technological tools on a regular basis (at least once a month), and 60–80% of Clinicians used technological tools to measure a wide range of variables relevant to neurorehabilitation (Fig.
4). Although these percentages are not sufficient to suggest widespread adoption of technologies, they are encouraging as they indicate that the main user group of assessment technologies has at least been exposed to such tools. Thus, in a population with access and exposure to neurorehabilitation technology, there were other barriers that were preventing routine use of technology-aided assessments.
The survey identified that practical considerations (e.g., assessment duration, cost, etc.) were some of the main factors hindering clinical use of standard assessments (Table
10). This is in line with Jette, et al. [
8], where therapists identified: (a) duration for patients to complete assessments, (b) duration for clinicians to analyze data, and (c) difficulty for patients in completing assessments independently, as the three main barriers for routine assessments. We also found this for technology-aided assessments, where, across stakeholders, lack of time to perform assessments and high cost of technologies were the top two hindering factors (Table
11). Interestingly, Medical Industry professionals chose the lack of reimbursement for time spent in assessments as the second most hindering factor. This highlights the importance of financial aspects that directly impact delivery of care, adding to the cost associated with technology-aided assessments.
Unlike other stakeholders, Research Engineers scored the lack of understanding on how to interpret and use assessment results, and the lack of standardization/ device-dependence of assessment results as the most important factors hindering technology-aided assessments. These highlight the differences in the roles of stakeholder groups: Research Engineers, who are primarily tool developers (Fig.
5a), are aware of the limitations of these devices and the applicability of the different methods used in quantitative assessments. However, this aspect seems to be lost to other (user) stakeholders, which can easily lead to inappropriate use of the developed tools, e.g., application of measures in conditions where they are not valid, comparison of metrics with same name but that are not calculated in a comparable manner, etc. Thus, deeper interdisciplinary interactions need to be fostered to decrease the potential of overlooking stakeholder-specific factors.
In general, Clinicians showed a strong preference for standard tools for carrying out assessments in their work (Table
5). Despite this, Clinicians scored the lack of standards and interpretability of results below the financial burden of technology-aided assessments, indicating that the cost of existing technology was a more pressing problem than the lack of standards. Across stakeholders, the poor understanding of using technology-aided assessments for clinical decision-making, and the lack of standards were rated the third and fourth important factors hindering clinical acceptance. The lack of standards is a known problem with technology-aided assessments, as there is no general consensus on what features of sensorimotor behavior to measure and how, especially in the context of neurorehabilitation. Differences in technologies (e.g., robots vs. sensor-based systems) affect the nature of human-machine interaction between the different devices. This further exacerbates the problem of standardization and could partly explain the current lack of standards given the wide landscape of available technological tools. The tendency is that technology will continue to improve at a fast pace, and this is one reason why it is critical to select a solid base to best explore technological developments rather than re-evaluate every new device.
The issues of standardization, the associated time and cost to perform assessments, and reimbursement for clinical use of technology-aided assessment are intermingled. The circular nature (“chicken-and-egg” problem) of these issues was also raised in the open comments (Table
13): reimbursement requires standardization, standardization requires clinical use, and clinical use requires reimbursement/standardization. These issues are unlikely to be resolved immediately, and require high-quality evidence in favor of technology-aided assessments to demonstrate the value added by these procedures to clinical practice. In their current state, technology-aided assessments are likely not adding much value to clinical practice, and have poor benefit-to-cost (money and time) ratio. In contrast, technologies such as MRI are routinely used given the value they add to clinical practice, and there is little discussion around whether or not to use these procedures, even though they are many times more expensive than most technologies related to sensorimotor assessments. Thus, clearly showing the added value of technology-aided assessments to clinical practice, in terms of improved and efficient assessment, and better clinical decision-making and outcomes, is critical to support their future adoption.
Suggestions for the future of technology-aided assessments
There was general agreement between the different stakeholders about the activities to be pursued for the next 5 years (Table
12), with standardization of technology-aided assessments ranked highest. This was followed by activities to: (a) understand the link between the different levels of assessment, which is important for therapy planning and clinical decision making; and (b) make routine assessment reimbursable and economically viable. The least ranked activities were cross-disciplinary education, and the development of taxonomies for the field.
The suggestion to focus on standardization of technology-aided assessments is a logical one, but a difficult issue to address. Even without technology, developing standards in sensorimotor assessments requires a number of inter-related choices: the tasks performed and the assessment protocol, tools used for measurements, and the methods used for analysis and interpretation of the results. Technology in this context specifically refers to the tools used for assessment, and it directly influences the tasks that can be performed, the physical variables that can be measured, and the interpretation of results.
In the last decades, robotics has emerged as a viable and safe tool to complement traditional therapy (see [
10‐
12]) and it is a popular technology used by different stakeholders (Fig.
4). These robotic developments, primarily designed for delivering therapy, have also been programmed to measure a great variety of parameters related to sensorimotor function (e.g., [
3,
13,
14]) and they have been suggested as standard tools for sensorimotor assessments. Although an appealing idea, using therapeutic robots for assessment is limited because each outcome measure is influenced by robot-specific factors such as the topology of the robot’s kinematic chain, control modalities, etc., which confound the results obtained from these tools. Nevertheless, robotic devices have the potential to become standards in assessments, but they must be specifically designed for the purpose of assessment, e.g., robotic dynamometers or ergometers.
Another way forward is by using technology that has the potential for quick and widespread use with minimal interference to human movements, to promote focused and quicker convergence to a common set of measures/standards. One such example is inertial measurement technology, whose popularity is increasing and is likely to be ubiquitous in the near future. Inertial measurement technology has been used to quantify many existing assessments [
15] and movement behaviour in natural (non-laboratory) conditions [
16‐
18]. Thus, this technology could help establish standards in technology-based sensorimotor assessments. However, this technology also has its limitations. For instance, there is currently no standard algorithm for extracting movement kinematics from these devices; several different algorithms for activity counts or steps [
19], energy expenditure [
20,
21], smoothness [
22,
23], etc., exist but were validated in different populations (e.g., walking detection algorithms that work very well in the unimpaired population may not work well with impaired users [
24]). This is not ideal for establishing a standard, but it is likely that coming years will see increased work with this particular technology, which hopefully will result in accurate, robust and sensitive methods for quantifying movements. To accelerate and facilitate this, it is relevant to encourage the community to openly share their algorithms and data. Standardization is unlikely to happen if every research group develops their own version of different measures; as highlighted by Ince et al. [
25], non-availability of the original software code is a serious impediment to reproducibility – even if algorithms are perfectly described in the literature.
Moreover, there currently exist no standards for accurate and robust data collection with these devices in both laboratory/clinic and real-life settings. In the laboratory/clinic setting, there is a need to standardize tasks, so comparisons can be made between studies. In the real-life setting, there is still a big challenge in interpreting data from these devices operating in unstructured environments and without contextual information. Nevertheless, interesting and potentially useful methods have started surfacing in recent years, e.g., [
17,
26], and hopefully these will be extended further in the coming years.
Activities towards standardization can also help promote reimbursement of assessment procedures used in clinical practice, which can address issues with the cost associated with technology-based assessments. Apart from agreeing on a common set of tools to be used for sensorimotor assessments, another important issue that needs to be addressed for developing standards is the development of suitable language and taxonomies for the field [
27]. This is essential to ensure there is a common understanding of important constructs in the field among the different stakeholders. Such language is also crucial for formalizing procedures, e.g., [
28], and the development of appropriate measures for quantifying different sensorimotor constructs relevant for research and clinical purposes. There have been recent international efforts towards standardizing outcome measures for sensorimotor assessments with and without technology, e.g., [
29,
30]. An encouraging thought is that the difficulties currently faced in standardization of technology-aided assessments are not very unlike problems encountered in other fields before standards were established (e.g., [
31‐
33]), or at least openly discussed (e.g., [
34]). It would be essential to continue our technical and clinical work in the different fronts, while being aware of how our activities fit in the bigger picture of a standard for technology-aided assessments.
Cross-stakeholder communications would also be essential in this process, and a common language will also make it easy to implement cross-disciplinary education among the different stakeholders. We found surprising that, while different respondents reported having had many interdisciplinary interactions (Table
3), many technology developers had not used nor seen technology for assessments being used (Fig.
5). This could be reflected as a lack of proper understanding of user needs when developing technology, which could lead to devices not being used, and highlight the importance of fostering quality cross-disciplinary education and interactions among stakeholders.