Background
Methods
Implementation
-
adjusting the healthcare protocol to the context of the location,
-
organizing technical support,
-
adapting and translating information letters for patients,
-
training healthcare coaches to use the platform, developing new work processes, and
-
mapping healthcare costs.
Evaluation of the implementation
Statistical analyses
Results
Study sample and response rates
Patients participating in pilot phase (N = 86) | Patients participating in evaluation (N = 71) | ||
---|---|---|---|
Age (in years) | (median, IQR) | 62.0 (14.5) | 61.0 (15) |
ALSFRS_R-score at the start of the pilot phase | (median, IQR) | 37 (12) | 37 (10) |
Gender | |||
Male | (n, %) | 51 (59.3%) | 42 (59.2%) |
Female | (n, %) | 35 (40.7%) | 29 (40.8%) |
Diagnosis | |||
ALS | (n, %) | 65 (75.6%) | 53 (74.6%) |
PMA | (n, %) | 14 (16.3%) | 11 (15.5%) |
PLS | (n, %) | 7 (8.1%) | 7 (9.9%) |
Duration of the disease | (median, IQR) | 1 (2) | 1 (3) |
Implementation outcomes
Adoption
Acceptability
(Totally) Agree | Neutral | (Totally) Disagree | |
---|---|---|---|
% | % | % | |
Panel A – Patients | |||
Acceptability | |||
I am satisfied with the care concept ALS H&C | 71.7 | 25.4 | 3.0 |
I am satisfied with the ALS H&C app | 68.7 | 23.9 | 7.5 |
I find the chat function useful | 74.7 | 11.9 | 13.5 |
The healthcare coach usually responds to my messages in a timely manner | 88.1 | 4.5 | 7.5 |
I am satisfied with the number of contact moments with the healthcare coach | 76.1 | 20.9 | 3.0 |
I like that I can monitor my own measurements (well-being, weight, ALS symptoms) | 73.2 | 23.9 | 3.0 |
I find it confronting to monitor my own measurements (well-being, weight, ALS symptoms) | 7.5 | 43.3 | 49.2 |
I find the emails I receive to remind me of the measurements annoying | 4.5 | 20.9 | 74.6 |
I find the frequency of the emails to remind me of the measurements too high | 7.5 | 80.6 | 12.0 |
Appropriateness/Compatibility | |||
I find the care concept ALS H&C fits in well with my care needs | 50.7 | 34.3 | 14.9 |
I like that with ALS H&C a healthcare provider can always see how I am doing | 68.7 | 26.9 | 4.5 |
I find it annoying that I am being monitored with ALS H&C | 1.5 | 11.9 | 86.6 |
Feasibility | |||
Using ALS H&C is quickly becoming a routine | 62.6 | 23.9 | 13.4 |
I often forget to open and use the ALS H&C app | 29.9 | 23.9 | 46.2 |
I am not able to make it a habit to use the ALS H&C app | 17.9 | 26.9 | 55.2 |
Panel B – Healthcare providers | |||
Acceptability | |||
I am satisfied with the care concept ALS H&C | 64.9 | 29.7 | 5.4 |
I am satisfied with the ALS H&C web portal | 48.6 | 43.2 | 8.1 |
I feel comfortable using ALS H&C in my work | 35.6 | 42.5 | 21.9 |
Appropriateness/compatibility | |||
An e-health care concept like ALS H&C fits with the way I work | 70.9 | 26.4 | 2.8 |
An e-health care concept like ALS H&C fits with my organization | 80.6 | 18.1 | 1.4 |
I find it important to be able to offer ALS H&C to patients | 76.3 | 22.2 | 1.4 |
I like that with ALS H&C I can always see how a patient is doing | 62.5 | 30.6 | 7.0 |
With ALS H&C I can better tailor the care to the needs of the patient | 40.3 | 38.9 | 20.8 |
Feasibility | |||
Working with ALS H&C quickly became routine | 23.6 | 54.2 | 22.2 |
I often forget to open and use the ALS H&C web portal | 59.7 | 25.0 | 15.3 |
I am not able to make it a habit to work with the ALS H&C web portal | 37.5 | 37.5 | 25.0 |
ALS H&C can be easily combined with my other work activities | 47.2 | 34.7 | 18.1 |
Innovations such as ALS H&C reduce the workload of healthcare providers | 13.9 | 54.2 | 32.0 |
Appropriateness/compatibility
Feasibility
Fidelity
Main steps in the original implementation plan | Deviations from the original implementation plan |
---|---|
Implement ALS H&C in 10 teams in 3 cycles (3 teams in Cycle 1, 3 teams in Cycle 2, 4 teams in Cycle 3), so that learnings of the first Cycle could be incorporated in the next Cycle and so on. | Due to the COVID19-pandemic the original planning of the first implementation cycle had to be adjusted slightly. As a result, Cycle 1 was not finished yet before the start of Cycle 2 and therefore there were less opportunities to implement the learnings of the first cycle into the next. |
Each participating center will form a project team, consisting of the physiatrist(s); two or three allied health professionals (i.e., one of them will become the healthcare coach, and one allied health professional will fulfill the role of knowledge broker); the manager; one scheduler; someone who will become the administrator of the platform and can provide technical assistance if necessary; one or two ALS patients; and one or two informal caregivers. The members of each project team will be involved in the implementation of ALS H&C within their organization. | Based on the original implementation plan, each team needed to have one healthcare provider who was willing to take on the role of knowledge broker. This had to be someone with an affinity for implementation, who knew the organization well, and who would be the driving force behind the realization of the action plans. In Team 6 the knowledge broker stopped after the first meeting due to personal reasons and the project team was not able to assign this role to another team member for a while. Team 5 did not include an informal caregiver in the project team. |
Hold three preparatory meetings in which 1) ALS H&C will be introduced to the main stakeholders, 2) the target group and setting will be analyzed and the expected barriers/facilitators for implementation will be identified by the project team, and 3) the project teams develop action plans to address the expected barriers. These meetings will be held on site. | Due to the COVID19-pandemic physical visits were not possible. Instead, all meetings, except the first meeting with one team (Team 1), were held digitally via Zoom (videoconference). The third meeting of Team 5 (developing action plans) had to be rescheduled because there were too many no-shows at the official meeting. The healthcare professional that would take on the role of healthcare coach in Team 7 was not involved in the preparatory meetings (introduction, identifying barriers/facilitators, developing action plans) due to personal circumstances. |
Each team will have three months for a pilot study with 5–10 patients to test and execute the implementation plans and to provide care with ALS H&C. | One team (Team 5) had some technical issues in the first month of the pilot phase and therefore it was decided to extend their pilot study with one month. |
For each team there will be mid-term evaluations at 6 weeks (by phone/videoconference) and a final evaluation after three months (online surveys). | No deviations. |
Core elements of ALS H&C | Deviations from the original innovation |
ALS H&C consists of an application for patients that runs on smartphones and tablets, but can also be accessed through a computer. The application consists of a chat function for easy communication between patient and healthcare coach, a library where received information links can be saved, and three measurements: 1. A well-being question that can be answered with one of 10 smileys ranging from sad to happy and a written explanation/elaboration (optional) 2. Body weight 3. Functional status (ALSFRS questionnaire) The data will be passed on to a central server, where a healthcare professional can view it. The healthcare coach receives automated alerts whenever there is a significant change in well-being or body weight. The healthcare coach checks and follows up on the alerts and messages whenever necessary. They will monitor the data at least once a month with the monitor function on the platform. Data is shown in graphs and any significant changes are clearly indicated. The healthcare coach provides personalized feedback via a message in the app. | No deviations, but every team experienced some small temporary technical issues with the app/platform. These bugs were all resolved relatively quickly by the provider of the application. |
One healthcare professional is assigned the role of healthcare coach. This person will perform the monitoring and will be the first point of contact for the patients. There is a low-threshold for patients to contact the healthcare coach, preferably via the chat in the app. | No deviations. |
The patient is (as much as possible) in control. | No deviations. |
The default frequencies for the measurements are daily for well-being, weekly for weight, and monthly for functioning, but the exact frequency of the measurements can be adjusted based on the wishes of the patient. | No deviations. |
A healthcare protocol, which is based on the most recent treatment guidelines for physiotherapy, occupational therapy, speech therapy, etc., gives guidance to the healthcare coach for the monitoring, providing feedback and for sending information links. Participating centers are allowed and encouraged to slightly adjust the healthcare protocol to match their context, but without changing the core elements. | No deviations. |
The healthcare coach provides at least once a month feedback to the patient regarding their measurements, even if there were no changes since the last monitoring. | Team 7 did not comply with/adhere to the healthcare protocol for the monitoring with regard to the monthly feedback. The protocol states patients should always receive feedback on their measurements (once a month) even when there are no changes in their situation since the last monitoring, but Team 7 did not always do this. |
A fixed frequency of outpatient consultations at the clinic for all patients is not necessary anymore, because with ALS H&C the patient can be monitored continuously. Outpatient consultations can be planned based on the needs of the patient. | Most participating teams hold 3- to 4-monthly outpatient consultations with the physiatrist and other health care professionals of the ALS team to monitor disease progression of all patients. None of the teams felt comfortable letting go of this routine (completely) just yet. |
Providing information is based on the patients’ needs. | No deviations. |
Sustainability
Technology acceptance and usability
Attitude (individual context)
(Totally) Agree | Neutral | (Totally) Disagree | |
---|---|---|---|
% | % | % | |
Panel A – Patients | |||
Attitude | |||
I find the time it takes to answer the questions in the app is worth it | 67.2 | 20.9 | 11.9 |
I find it important that the option of home monitoring is offered to patients with ALS | 80.6 | 13.4 | 6.0 |
The usability of the app is an important determinant in my choice to use the app or not | 80.6 | 16.4 | 3.0 |
The attractiveness of the app is an important determinant in my choice to use the app or not | 32.9 | 40.3 | 26.9 |
I think it is good to use ALS H&C for my care | 79.1 | 16.4 | 4.5 |
I like using ALS H&C for my care | 64.2 | 35.8 | 0.0 |
I find the use of ALS H&C in my care helpful for my care | 59.7 | 31.3 | 9.0 |
I find the use of ALS H&C in my care burdensome | 6.0 | 17.9 | 76.1 |
I find the use of ALS H&C in my care time consuming | 7.5 | 16.4 | 76.1 |
I think the care I receive with ALS H&C is better than the care I received without this care concept | 35.8 | 41.8 | 22.4 |
Yes | No, just right | No | |
% | % | % | |
I find the frequency of the well-being measurements too high | 52.2 | 47.8 | 0.0 |
I find the frequency of the weight measurements too high | 16.4 | 83.6 | 0.0 |
I find the frequency of the ALSFRS-R measurements too high | 4.5 | 95.5 | 0.0 |
I find the frequency of the feedback of my healthcare coach on my measurements too low | 16.4 | 83.6 | 0.0 |
Perceived (Technology) Control | |||
I can decide for myself whether to use ALS H&C | 82.1 | 4.5 | 13.5 |
I experience more control over my care with ALS H&C | 55.5 | 29.9 | 14.9 |
Yes | Sometimes | No | |
% | % | % | |
I can use the ALS H&C app without assistance from others | 89.6 | 4.5 | 6.0 |
I let other healthcare providers (e.g., my GP) have a look in the app | 28.4 | 6.0 | 65.7 |
(Totally) Agree | Neutral | (Totally) Disagree | |
Perceived usefulness | % | % | % |
I find the app useful | 76.1 | 17.9 | 5.6 |
I find the app insightful | 83.6 | 11.9 | 4.5 |
I find the measurements informative | 70.2 | 22.4 | 7.5 |
I find the measurements insightful | 71.6 | 22.4 | 6.0 |
I find the messages/feedback I receive from the healthcare coach in the app easy to understand | 80.6 | 19.4 | 0.0 |
I find the messages/feedback I receive from the healthcare coach in the app informative | 70.2 | 29.9 | 0.0 |
I find the messages/feedback I receive from the healthcare coach in the app useful | 71.7 | 28.4 | 0.0 |
I find the e-mails I receive to remind me I need to complete the measurements useful | 79.1 | 14.9 | 6.0 |
Perceived ease of use | |||
I found it easy to install the app | 62.0 | 22.5 | 15.5 |
I find logging into the app difficult | 9.8 | 22.5 | 67.6 |
I find logging into the app is slow | 29.6 | 32.4 | 38.0 |
I find it easy to operate the app | 76.0 | 19.7 | 4.2 |
I find it easy to learn how to operate the app | 83.1 | 14.1 | 2.8 |
I find the app slow | 29.6 | 42.3 | 28.2 |
Yes | Sometimes | No | |
% | % | % | |
I was able to answer the well-being question with the smileys in the app | 91.4 | 2.9 | 5.7 |
I was able to add additional explanations to the well-being question in the app | 92.9 | 1.4 | 5.7 |
I was able to complete the weight measurements in the app | 93.0 | 0.0 | 7.0 |
I was able to complete the ALSFRS-R questionnaire in the app | 98.6 | 0.0 | 1.4 |
I was able to read the feedback and messages from the healthcare coach in the app | 87.3 | 5.6 | 7.0 |
I was able to send messages to the healthcare coach in the app | 88.6 | 1.4 | 10.0 |
I was able to open the information links the healthcare coach added to the app | 89.9 | 0.0 | 10.1 |
I have had problems with logging into the app | 23.9 | N.A. | 76.1 |
(Totally) Agree | Neutral | (Totally) Disagree | |
Peer influence | |||
People who are important to me think that I should use ALS H&C | 37.3 | 43.3 | 19.4 |
My physiatrist(s)/ALS care team find it important that I use ALS H&C in my care | 53.7 | 40.3 | 6.0 |
Intention | |||
I intent to keep using ALS H&C in the next six months | 88.0 | 9.0 | 3.0 |
I would recommend ALS H&C to other patients with ALS | 79.1 | 17.9 | 3.0 |
Panel B – Healthcare providers | |||
Attitude | |||
I find it a good idea to offer home monitoring as part of regular care to patients with ALS | 80.8 | 13.7 | 5.5 |
ALS H&C is beneficial to my patient care and management | 46.5 | 43.8 | 9.6 |
I think it is good to use ALS H&C in my work | 64.4 | 32.9 | 2.7 |
I like using ALS H&C in my work | 31.5 | 65.8 | 2.7 |
I find the use of ALS H&C in my work helpful for the care I provide | 58.9 | 28.8 | 12.3 |
I find the use of ALS H&C in my work burdensome | 23.3 | 39.7 | 37.0 |
I find the use of ALS H&C in my work time consuming | 27.4 | 49.3 | 23.3 |
I think the care I provide with ALS H&C is better than the care without ALS H&C | 32.9 | 49.3 | 17.8 |
Perceived (Technology) Control | |||
I would have the ability to use ALS H&C in my patient care and – management | 75.4 | 16.4 | 8.2 |
I can decide for myself whether to use ALS H&C in my work | 45.2 | 24.7 | 30.2 |
I do not have enough knowledge to use ALS H&C properly | 9.6 | 24.7 | 65.8 |
I have enough skills to use ALS H&C properly | 60.3 | 23.3 | 16.4 |
I can use the ALS H&C web portal without assistance from others | 76.4 (yes) | 16.7 (sometimes) | 6.9 (no) |
Perceived usefulness | |||
ALS H&C cannot improve my patient care and management | 12.3 | 31.5 | 56.1 |
ALS H&C cannot enhance my effectiveness in patient care and management | 17.8 | 41.1 | 41.1 |
ALS H&C cannot make my job any easier | 17.8 | 31.5 | 50.6 |
I find ALS H&C not useful for my patient care and management | 5.5 | 24.7 | 69.9 |
ALS H&C is an improvement of the regular ALS care | 58.9 | 34.2 | 6.8 |
E-health has no added value for ALS care | 2.7 | 15.1 | 82.2 |
I find being able to view the monitoring data of patients on the ALS H&C web portal insightful | 71.3 | 23.3 | 5.5 |
I find being able to view the monitoring data of patients on the ALS H&C web portal informative | 73.9 | 21.9 | 4.1 |
I find being able to view the monitoring data of patients on the ALS H&C web portal useful | 69.8 | 27.4 | 2.8 |
Perceived ease of use (only health care coaches) | |||
I find it easy to create accounts for patients | 85.7 | 14.3 | 0.0 |
I find it easy to set up measurement trajectories for patients | 85.7 | 14.3 | 0.0 |
I find it easy to communicate with patients via the chat function of ALS H&C | 92.8 | 0.0 | 7.1 |
I find it useful to communicate with patients via the chat function of ALS H&C | 78.6 | 0.0 | 21.4 |
I find it easy to send information links to patients via the ALS H&C web portal | 64.3 | 14.3 | 21.4 |
I find it useful to send information links to patients via the ALS H&C web portal | 50.0 | 21.4 | 28.6 |
I find it easy to switch between the different tabs in the ALS H&C web portal | 100.0 | 0.0 | 0.0 |
I find it confusing to switch between the different tabs in the ALS H&C web portal | 7.1 | 14.3 | 78.6 |
Finding a patient in the ALS H&C web portal is very easy | 28.5 | 35.7 | 35.7 |
Handling new messages in the ALS H&C web portal is easy | 78.6 | 7.1 | 14.3 |
Handling new alerts in the ALS H&C web portal is easy | 64.3 | 21.4 | 14.3 |
Setting up additional measurements for patients is easy | 85.7 | 14.3 | 0.0 |
Adjusting the frequency of measurements for patients is easy | 71.4 | 14.3 | 14.2 |
Perceived ease of use | |||
I found it difficult to learn how to work with ALS H&C | 8.0 | 41.3 | 50.7 |
I find logging into the ALS H&C web portal difficult | 13.5 | 45.9 | 40.6 |
I find logging into the ALS H&C web portal slow | 16.3 | 48.6 | 35.2 |
I have had problems with logging into the app | 37.8 (yes) | N.A. | 62.2 (no) |
I find navigating the ALS H&C web portal slow | 8.1 | 58.1 | 33.8 |
Peer influence | |||
People who influence my clinical behavior think that I should use ALS H&C in my work | 18.1 | 52.8 | 29.2 |
It is expected of me that I use ALS H&C in my work | 41.7 | 33.3 | 25.0 |
I experience pressure from others to use ALS H&C in my work | 9.7 | 38.9 | 51.4 |
Intention | |||
I intent to keep using ALS H&C in the next six months | 66.7 | 27.8 | 5.6 |
I would recommend ALS H&C to colleagues of other ALS care teams | 55.5 | 36.1 | 8.3 |
Perceived (technology) control (individual context)
Perceived usefulness (technological context)
Perceived ease of use and usability (technological context)
Peer influence (implementation context)
Advantages and disadvantages
Intention
Differences in implementation outcomes between teams with sustainable and unsustainable implementation
Discussion
Limitations
Conclusions
-
Sufficient training of healthcare providers on how to use the innovation.
-
Integration of the e-health web portal with the local electronic health record system.
-
Ensure high fidelity by not changing the core elements of the implementation nor of the innovation .
-
Pay special attention to potential communication/trust issues within a team and the teams’ attitude towards the innovation at the start of an implementation project;
-
Use positive experiences of patients to convince the healthcare providers of the value of the innovation;
-
Ensure at least one person is selected as project champion who is responsible for and who is the driving force behind the realization of action plans;
-
Ensure that the right people are selected for the crucial positions in the team and 5) ensure that they are involved right from the start.