Need-based care of multi-morbid patients – supporting general practitioners with algorithm-generated recommendations of healthcare services (telemedicine-project ATMoSPHÄRE)

The telemedicine project ATMoSPHÄRE started 2015 in Saxony and developed, employed, and evaluated a technology-based information and communication platform and a complementary home-based telemonitoring application. The aim of these tools was to improve the primary care of older, multi-morbid patients and therefore preserve and extend their ability to live independently. The telemedicine platform contained, in addition to the data provided by the telemonitoring application, information about socio-demographics, diagnoses, and therapy, as well as the results of various geriatric assessments. Detailed information about the study, including patient recruitment, study design, and additional information about the telemonitoring application developed for the patients, has been published elsewhere [13, 14].

The development and implementation of a digital recommendation system for external healthcare services by vital.services GmbH was a pivotal part in the second stage of ATMoSPHÄRE.

The DRS was intended to offer need-based, individual recommendations for medical and social care services to the study GPs, thereby assisting them in the planning of treatment.

Embedded into the telemedicine platform, the DRS used an algorithm to compare functional limitation of a patient with the treatment targets of a healthcare service (e.g. targets of occupational therapy), and determine how closely they matched. The DRS ranked all services according to how well they matched, and presented the closest matching services to the study GPs as recommendations on the telemedicine platform (Fig. 1).

The functional limitations were gathered using mostly commonly-employed geriatric assessments (Instrumental Activities of Daily Life (IADL) [15], Mini Mental State Examination (MMSE) [16], clock test [17], Timed Up-and-Go [18], ANGELINA [19], Geriatric Depression Scale (GDS) [20]). The treatment targets of services were taken or derived from different catalogues [21, 22]. Comparing functional limitations and treatment targets required to classify them within the framework of the International Classification of functioning, disability and health (ICF) [23]. Apart from providing a consistent and distinct vocabulary, the ICF possesses two additional properties utilized by the the DRS. Firstly, the mono-hierarchical structure allows definition of a proximity between categories. Secondly, the ICF-qualifier ratings signify the severity of disability within the corresponding category, which the DRS interprets as an indicator of need. The higher the severity of a functional limitation, the higher the need of a patient for a service which aims to alleviate that limitation.

Provided the limitations in functioning of a patient are classified within the ICF, each with the corresponding qualifier rating, and a set of healthcare services exists where the treatment targets of each service are converted into ICF-categories, a ranking of these healthcare services can be established. The treatment targets of a service or intervention are a set of ICF-categories representing the areas of functioning the service aims to support or improve. A service is considered more fitting the higher the qualifier rating of the function is, and the closer the function category and treatment target category are. An ICF category represents a specific function. The ICF qualifier indicates how severely the function is impaired. The qualifier ranges from 0 - meaning no impairment to 4 - indicating complete loss of function.

The functionality is shown schematically in Fig. 1 using the example of an impaired patient function indicated by the “Timed-Up-and-Go-Test” (relevant mobility impairment in need of clarification). The algorithm compares the ICF-coded assessment result of an individual patient to the ICF- categories representing the treatment targets of “walking frame”. For instance, the assessment result d4500.3 represents a severe impairment of the ability to walk short distances, this matches closely the treatment target d450 – Walking of the walking frame indicating this service could be appropriate. The closer the match and the higher the qualifier of the patient the higher the services will be ranked.

This publication bases on the results of the plausibility check of the DRS (Version 3.1.3) carried out by the Department of General Practice of the TU Dresden, which examined the adequacy of the DRS regarding its individually-compiled service recommendations for multi-morbid patients. This article is not intended to show the results of the entire plausibility check but rather presents the results of the first Delphi round and the interviews which entail the original answers and recommendations of the study GPs.

The operationalization of the adequacy of the DRS was achieved by a comparison with a determined reference value. The number and character of algorithm-generated recommendations were compared to the reference value regarding matching, missing, and additional fitting recommendations.

The reference value should represent the shared knowledge of a certain group, e.g. experts in a specific area [24]. In this context, the reference value is the set of the study GPs’ recommendations deemed appropriate for the treatment of a specific multimorbid patient.

The plausibility check followed a mixed-method approach and consisted of a) a two-staged Delphi procedure, evaluating the recommendations for two case vignettes (cv), and b) interviews with ATMoSPHÄRE study GPs, evaluating recommendations for selected study patients.

Based on a previous analysis of the most common supply needs of the study cohort, two profiles for multimorbidity were created. Patients corresponding to profile 1 all had indications for polypharmacy and were additionally diagnosed with type 2 diabetes mellitus and hypertension. Profile 2 patients showed irregularities in dementia-related assessments (MMSE or clock-test), or were already diagnosed with dementia. Using these two profiles, two corresponding study patients were selected and then utilized as anonymous case vignettes during the Delphi procedure. The data of these patients stand vicariously for the ailments and treatment issues patients of these profiles present during consultation hour. In the same way, two study patients were selected to be evaluated by study GPs for the evaluation via interviews.

The Delphi procedure was carried out in writing and consisted of two survey rounds. In both rounds the study GPs received a two-part questionnaire. The original GP- recommendations for the case vignettes (Part 1) as well as their approval or disapproval for the DRS-recommendations (Part 2) were collected in the first Delphi round. In the second round of the Delphi process the GPs were confronted with their own recommendations/answers and the recommendations of the other participating GPs and given the opportunity to reconsider decisions made in the 1st round (Fig. 2).

The purpose of the second round was to establish a consensus among the GPs which in turn was used to determine the reference value needed to evaluate the DRS-recommendations. This round is therefore the basis of the plausibility check but is not presented in this article.

In the first part of the questionnaire two case vignettes were introduced, including socio-demographic context, diagnoses, medication, and results of the geriatric assessments (see Fact sheet in Tables 1 and 2). The study GPs recommended healthcare interventions and services which they considered helpful for the case vignettes at hand. Part 1 was therefore used to determine the reference value. In Part 2 the recommendations of the DRS for the two case vignettes were presented to the study GPs. They were asked to judge which recommendations they considered as appropriate or inappropriate (see Additional file 1).

Additionally, two actual study patients (not case vignettes) representing the 2 profiles were discussed with their GP in the interviews in the same manner (Part 1 + Part 2). Furthermore, the study GPs were interviewed about the DRS in general. The interview questions for the plausibility check were developed based on the SPSS principle according to Helfferich [25]. The interviews were conducted by a trained researcher at GPs’ office.

The statistical analysis (with SPSS Version 25) of the quantitative data consisted, in addition to data description, of a comparison of means of the number of recommendations (non-parametric Wilcoxon Rank Sum Test for paired samples) between GPs and DRS. To determine the relevance of the results, the effect size according to Cohen was calculated. Effect sizes with values of d ≥ 0.8 are considered strong effects [26]. A significance level α equal to 5% was defined.

The vignette-specific listing of healthcare services and interventions by the study GPs showed a high variability regarding character and quantity of recommendations (Tables 1 and 2).

The study GPs indicated between three and ten (mean = 6, sd ± 2.9) recommendations for case vignette no. 1 (Table 1), and between one and nine (mean = 5.1; sd ± 2.9) for case vignette no. 2, respectively (Table 2). In comparison, the average number of appropriately rated algorithm-generated recommendations was significantly higher (asymptotic Wilcoxon-Test for both case vignettes: z = − 2.53, p = 0.011, n = 8), and roughly twice as many (case vignette no.1: mean = 11.9, sd ± 2.6; case vignette no.2: mean = 12, sd ± 2.6). The effect size according to Cohen was above 2.0 for both case vignettes (Fig. 3).

Examining the individual GP recommendations shows that only the first three recommendations for case vignette no.1 were made by a majority of physicians, meaning more than 4 of the 8 participating study GPs (Table 1). Likewise, only three recommendations were proposed by a majority for case vignette no. 2 (Table 2). After presentation of the algorithm-generated recommendations, the judgements converged (case vignette no.1: majority approval of 10 recommendations; case vignette no.2: majority approval of 12 recommendations) (Additional file 1).

A similar trend was found on conducting the interviews, where Part 1 consisted of the question: “Which prescriptions, healthcare interventions or services would you propose to your patient x?” Here, too, a significantly higher number of algorithm-generated recommendations were judged appropriate than previously listed by the study GPs for the patient (asymptotic Wilcoxon-Test: z = − 2.37, p = 0.018, n = 8). The effect size for the recommendations in the interviews was d = 1.76, which means a strong effect (Fig. 3).

Comparing the character of the recommendation categories shows that the DRS-recommendations additionally judged appropriate can be mainly assigned to social care (e.g. emergency-call service, pick-up and return services) and assistive products (e.g. pill dispensers). Occasional GP-made recommendations not included in the DRS pertain to special nursing services, or special assistive products, respectively.

All four interviewees confirmed in principle that the DRS is of added value to the GPs and the patient care. Requesting for specific assessment of the DRS prompted diverse responses. On one hand, the DRS was considered a helpful addition and reminder: “[...] you might ask yourself: “Useful or not?”, but as a check or suggestion, that I do like.[...] I don’t always consider foot care if patients don’t have diabetes. That surely is something.” (A1, f, 43) or: “Well, it is useful to get a reminder. That’s alright, I guess.” (A2, m, 44). On the other hand, the vagueness due to a lack of consideration of the patients’ priorities was criticized: “It’s tough. Because I believe a lot of hard and soft data is missing for the digital recommendation system. The patient wishes for instance, the prioritisation, what is most important to the patient at that time,[...].” (A3, f, 48).

As shown in other studies regarding the employment of assessments, geriatric assessments of multimorbid patients are only the basis for shared decision-making of therapeutic goals and healthcare decisions [28‐30].

The enduring participation of GPs during the plausibility check testifies to the interest of physicians in this subject. Due to the small sample size and restriction to patient profiles in the plausibility check, the results cannot be generalised. A heightened interest into telehealth applications by the participating GPs can be assumed, leading to a likely selection bias. This might have intensified the positive judgement of a digital tool.