With its digital health application (DiGA)-system, Germany is considered one of Europe's pioneers in the field of evidence-based digital health. Incorporating DiGA into standard medical care must be based on evidence-based success factors; however, a comprehensive overview of the evidence required of scientific studies for their approval is lacking.
Objective
The study aims to, (1) identify specific requirements defined by the Federal Institute for Drugs and Medical Devices (German: Bundesinstitut für Arzneimittel- und Medizinprodukte; BfArM) to design adequate studies, proving a positive healthcare effect, and (2) to assess the evidence given for applications permanently listed in the DiGA directory.
Methods
A multi-step approach was used: (1) identification of the evidence requirements for applications permanently listed in the DiGA directory, (2) identification of the evidence available supporting them.
Results
All DiGA permanently listed in the DiGA directory (13 applications) are included in the formal analysis. Most DiGA addressed mental health (n = 7), and can be prescribed for one or two indications (n = 10). All permanently listed DiGA have demonstrated their positive healthcare effect through a medical benefit, and most of them provide evidence for one defined primary endpoint. All DiGA manufacturers conducted a randomized controlled trial.
Discussion
It is striking that— although patient-relevant structural and procedural improvements show high potential for improving care, especially in terms of processes — all DiGA have provided a positive care effect via a medical benefit. Although BfArM accepts study designs with a lower level of evidence for the proof of a positive healthcare effect, all manufacturers conducted a study with a high level of evidence.
Conclusion
The results of this analysis indicate that permanently listed DiGA meet higher standards than required by the guideline.
Hinweise
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Abkürzungen
BfArM
Bundesinstitut für Arzneimittel- und Medizinprodukte
CG
Control group
COS
Core Outcome Set
DiGA
Digital health application
DRKS
Deutsches Register klinischer Studien
ICD-10
International Classification of Diseases
IG
Intervention group
MDR
Medical Device Regulation
RCT
Randomized controlled trial
Background
Digital health applications (known by their German abbreviation, DiGA) are increasingly present, with health systems around the world creating different legal frameworks for their integration into standard care. Together with Belgium, Germany was one of the first countries in Europe to develop an official framework to reimburse DiGA use. Other countries, such as England, Denmark, the Netherlands, Norway, Sweden, and the USA, are still in the development process [1]. France has already announced its intention to adopt the German system [2].
DiGA are certified medical devices with a primarily digital function, through which the medical purpose is achieved. Mainly used by patients, they detect, monitor, treat, or alleviate disease, injury, or disability. Shared use between patients and healthcare providers is also possible, however devices that merely read and transmit data are not considered DiGA [3]. They offer the potential of improved health outcomes [4], higher health standards, and improved and equal access to health services [5]. In addition, DiGA cost-effectively improve patient care [6].
Anzeige
The Federal Republic of Germany’s national parliament passed The Act to Improve Healthcare Provision through Digitalization and Innovation (Digital Healthcare Act – DVG) in 2019. With this law, healthcare providers were given the option of prescribing DiGA. To be part of standard care, the DiGA must be listed in the directory in accordance with paragraph 139e of the fifth book of social code (§ 139e SGB V), first requiring evidence-based proof of a benefit. The benefit is defined by the manufacturer guideline of the Federal Institute for Drugs and Medical Devices (German: Bundesinstitut für Arzneimittel- und Medizinprodukte; BfArM) [3]. First, certification as a medical device in a low-risk class (I or IIa) according to the Medical Device Regulation (MDR) is required. Basic requirements are set for data protection, interoperability, robustness, and user-friendliness, but the central criterion is proof that the DiGA has a positive healthcare effect. DiGA manufacturers must provide this evidence with a comparative scientific study in an International Classification of Diseases (ICD-10)-defined patient group. The positive healthcare effect can be a medical benefit and/or a patient-relevant structural and procedural improvement. The fast-track process offers two paths into the DiGA directory; if all criteria are fulfilled and BfArM reaches a positive decision, the DiGA is permanently included in the DiGA directory; if no positive healthcare effect has yet been demonstrated, the DiGA can be provisional included. In case of the latter, the expected positive healthcare effect must be proven within a period of 12 months with a previously approved evaluation study [3].
The DiGA manufacturer guideline specifies what type of evidence from positive healthcare effect studies is acceptable [3, 7]. After using the fast-track DiGA approval process, Brönneke et al. (2021) highlighted the importance of measures by legislation to help decide if innovations benefit patients in standard healthcare [7]. Similarly, Heimann et al. (2021) described factors required using the fast-track listing, including internal or self-commissioned external audits before submitting the application, consultation with BfArM on positive healthcare effects, and responses to queries from BfArM [8]. Löbker et al. (2021) reported on their experience from consultations on DiGA showing most (80%) of DiGA directory application manufacturers have taken advantage of a consultation during the process. The rate of withdrawn/rejected applications was higher, if manufacturers had not sought advice (63%), compared to manufacturers who had previously discussed key content with BfArM (35%) [9]. Düvel et al. (2021) qualitatively identified potential solutions to improve DiGA access to statutory standard care, recommending a central advisory office [10]. Lantzsch et al. (2022) concluded that there is room for improvement in the fast-track process, particularly in study reporting, as well as in outcomes for patient-relevant improvement of structure and processes. They demand new study designs to pave the way for the use of real-world data [11]. To generate high-quality evidence of DiGA positive healthcare effects, Stern et al. (2022) recommends further research on the impact of missing data, study endpoints, control group, multimodal interventions, study question, equity, generalizability, confounders and fit for purpose [12]. Geier et al. (2021) described different perspectives of the German Digital Health Association on DiGA, arguing the need for accompanying research to address the specific challenges of study design and methods in generating evidence for DiGA [13]. Hemkens et al. (2021) also emphasized the relevance of a robust evidence-based benefit assessment in DiGA approvals, stating that sustainable and efficient DiGA benefit assessment requires continuously adjusted evaluation in everyday care; central to this are randomized study designs that are integrated into standard care [14].
Incorporating DiGA into standard medical care must be based on evidence-based success factors; however, a comprehensive overview of the evidence required of scientific studies for their approval is lacking [15]. We want to investigate which methodologies used to generate evidence-based proof of benefit have been successfully implemented, based on the already permanently listed DiGA.
Our study aims to, (1) identify specific requirements defined by BfArM to design adequate studies, proving a positive healthcare effect, and (2) to assess the evidence given for applications permanently listed in the DiGA directory.
Anzeige
Methods
All permanently listed applications in the German DiGA directory (as of November 15th, 2022) were included in this study.
The study used a multi-step approach: First, we identified the evidence requirements for applications permanently listed in the DiGA directory, based on an analysis of the BfArM manufacturer guideline (as of March 18th, 2022) using the PICOS scheme: Population, Intervention, Control, Outcome, Study Design [16]. Then, a single researcher extracted these requirements, and then they were double-checked by a second independent researcher. All extracted data were transferred and categorized to a data extraction sheet (MS Excel). Finally, the evidence available supporting the permanently listed DiGA was identified, using the following sources: DiGA directory; study registries (German registry of clinical studies (DRKS), clinical trials.gov, ISRCTN registry); published study protocols; published study reports; submitted publications of permanently listed DiGA; and finally, manufacturer websites.
The sources (including studies, study protocols, reports) were used to extract data using the pre-defined data extraction sheet. Data extraction was performed by one researcher, followed by quality assurance of extracted information carried out by a second researcher. The methodological approach is represented in Fig. 1.
Fig. 1
Methodological approach
BfArM = Federal Institute for Drugs and Medical Devices (German: Bundesinstitut für Arzneimittel- und Medizinprodukte, DiGA = Digital health application
×
Results
All DiGA permanently listed in the DiGA directory (13 applications) are included in the formal analysis (Table 1). Most of these were initially accepted into the DiGA directory with an application directly for permanent listing. Four DiGA manufacturers (Kalmeda, Selfapy Depression, Vivira and Zanadio) initially applied for a provisional listing during testing, before being permanently listed. All four applications with initial provisional inclusion in the DiGA directory extended the trial period; Kalmeda by three months, Selfapy Depression and Vivira by four months, and Zanadio by ten months.
Table 1
Overview of indication area and quantity of ICD-10-Codes (three, four and five-digit)
DiGA Digital health application, ICD International Classification of Diseases
aOriginally targeted number of indications
We identified seven relevant categories from our BfArM manufacturer guideline document analysis: (1) patient population, (2) positive healthcare effect and study endpoints, (3) study design, (4) study location, (5) observation period and observation times, (6) sample size and drop-out, and (7) study results. These categories are used to guide the step-by-step reporting of findings.
Patient population and patient characteristics
A positive healthcare effect must be provided for at least one defined patient population. The delineation is based on one or more indications listed in the ICD-10 catalog; this international classification provides a specific disease definition. Three- or four-digit ICD-10 codes are permissible. This can be used to distinguish whether the positive healthcare effect should be demonstrated for all patients with a specific disorder (e.g., F32.-Depressive Episode) or for a specific patient population (e.g., F32.0-Mild Depressive Episode). If several indications are given, the evidence must be provided for each defined patient group. Thus, the ICD-10 code chosen is a measure of the specificity and concreteness of the patient population addressed. If the positive healthcare effect among the indications is comparable, the proof can be pro-vided for several indications combined [3].
Of 13 DiGA applications, seven address indications in mental health. One DiGA works at the interface of mental health and metabolism (Hello Better Diabetes and Depression), and the others are relevant to the nervous system (Elevida, Vorvida), the musculoskeletal system (Vivira), ears (Kalmeda), and hormones and metabolism (Zanadio). The majority (n = 10) of the listed DiGA can be prescribed for one or two indications, except for Deprexis with six, Velibra with four and Vivira with 20 indications. The manufacturers of Selfapy Depression originally sought listing for nine indications but were only able to prove a positive healthcare effect for two of these. Vivira manufacturers were able to provide evidence of a positive healthcare effect for 20 of the 45 indications they originally sought listing for. Table 1 shows the DiGA indication areas addressed, and the number of ICD-10 codes (separated into three-, four-, and five-digit codes) for which listing in the DiGA directory was achieved.
Except for Hello Better Vaginismus Plus, all permanently listed DiGA were developed for both women and men. In all studies, the proportion of female participants was higher than that of men. This trend was evident in both intervention groups (IG) and control groups (CG). Zanadio was approved for women only, due to an insufficient number of males in the trial. All DiGA manufacturers defined a minimum age of 18 years as inclusion criterion. On average, the study participants’ mean age was between 28.0 (Hello Better Vaginismus Plus) and 51.3 (Hello Better Diabetes and Depression) years. In all studies already published (except for Hello Better Vaginismus Plus) the average age of the participants was between 35 and 55 years. There were no significant differences in age between IG and CG. To date, participants with a higher education and a permanent job are more frequently represented in DiGA studies (Table 2).
Table 2
Overview of patient population and patient characteristics
DiGA Digital health application, IG Intervention group, CG Control group, m Male, f Female, M (SD) Mean (standard deviation)
Anzeige
Positive healthcare effect and study endpoints
For permanent or provisional inclusion in the DiGA directory, manufacturers must demonstrate one or more positive healthcare effects by means of a scientific study. This proof can be provided as a medical benefit (improvement of the state of health, reduction of the duration of the disease, prolongation of survival, or improvement in the quality of life), and/or a patient-relevant improvement of structure and processes (e.g., coordination of treatment processes, alignment of treatment with guidelines and recognized standards, or adherence). The positive healthcare effect to be demonstrated must relate directly to the insured person and be proven within the framework of the study to be conducted by means of defined endpoints [3].
All permanently listed DiGA have demonstrated their positive healthcare effect through a medical benefit. The manufacturers of Velibra and Vorvida demonstrated a patient-relevant structural and procedural improvement, in addition to the medical benefit. The most frequently proven medical benefit of all permanently listed DiGA can be categorized as improvement in the state of health. The Vivira application manufacturers additionally aimed for a reduced disease duration and an improved quality of life, but were unable to provide evidence for either. The Velibra manufacturers demonstrated a patient-relevant improvement of structure and process, reducing therapy-related efforts of patients and their relatives, and the Vorvida manufacturers, an improvement in patient autonomy (Table 3). Most DiGA provide evidence for one defined primary endpoint, except the Velibra manufacturers, who defined a total of five primary endpoints and proved four of them (the subscale, mental health, of the Short Form Health Survey-12, could not be proven). Depending on the defined patient group, various primary endpoints are attempted. The operationalization of the primary endpoints was carried out using validated questionnaires (Table 5).
Table 3
Overview of positive healthcare effects, indications, study designs, observation durations, sample sizes, IG, CG
DiGA
Positive care effect
Medical benefit
Patient-relevant improvement of structure and processes
DiGA Digital health application, IG Intervention group, CG Control group, RCT Randomized controlled trial, UC Usual care, n.a. Not applicable
Study design
A quantitative comparative (retrospective or prospective) study is required to provide evidence of a positive healthcare effect. Studies intended to prove a positive healthcare effect should reflect the actual healthcare reality and be conducted with data close to the healthcare system. A comparison can be made intra-individually or inter-individually. In an intra-individual, single-arm comparison, the data can, for example, be collected before and after the application of the DiGA. In an inter-individual, two-arm comparison, the intervention group data (DiGA use) are compared with control group data. The following options exist for the design of the control group: (1) treatment without the use of a DiGA, (2) non-treatment, and (3) treatment with another, comparable DiGA already permanently listed in the DiGA directory at the time of application. It is important that the selection of the control group corresponds to the standard of care [3].
All permanently listed DiGA manufacturers conducted a randomized controlled trial (RCT) using an inter-individual comparison of an intervention group with a control group. Almost all DiGA were tested against standard of care. The manufacturers of Selfapy Depression originally designed two intervention groups: one with, and one without psychological support. Evidence could only be proved for DiGA use without psychological support. Most control groups (standard of care) were offered access to the DiGA after the end of the study. Three DiGA manufacturers provided additional materials to the participants: Hello Better Diabetes and Depression enabled the control group to use an online knowledge transfer program, Kalmeda provided general information on the topic of tinnitus, and the manufacturers of Selfapy Depression sent weekly emails with standardized mindfulness exercises (Table 3).
Anzeige
Study location
Studies to prove a positive healthcare effect must be conducted in Germany, as the actual healthcare setting is closely linked to the positive healthcare effects. The comparison of the intervention against a control group without DiGA use is only meaningful, if a treatment in the German healthcare system is addressed. If the comparability of the healthcare situation can be proven, a study in other countries is also permissible [3].
Evidence of positive healthcare effect was obtained for 11 DiGA in Germany, and for two applications in Switzerland (Table 3).
Observation period and observation times
There are no concrete specifications of observation periods and times in the BfArM guideline. Baseline data collection is considered reasonable; the data collection periods (incl. possible follow-ups after the intervention phase) should be described [3].
The observation periods ranged from six weeks (Somnio) to nine months (Kalmeda, Zanadio). Most DiGA studies had an observation period of eight to twelve weeks. In addition to collecting data at baseline (t0) and at the fixed primary endpoint collection time, ten DiGA studies also collected follow-up data. In most cases, follow-up data were collected after six and/or twelve months. Almost all studies date back several years, so that existing data was used to prove a positive healthcare effect (Table 3).
Anzeige
Sample size and drop-out
BfArM requires studies to have an adequate sample size calculation/planning. In confirmatory studies, the sample sizes should be estimated based on the primary outcome measure and the relevant effect size. BfArM also requires studies to record the number of drop-outs and the respective reasons for these, with a transparent presentation in a study flow chart. A high drop-out rate should also be considered in connection with the respective clinical picture. Studies on addictive diseases, for example, show higher drop-out rates in actual care reality compared to diseases with recognized therapies and high responder rates. A high drop-out rate does not indicate the success of the application and can only be accepted to a limited extent [3].
The manufacturers of Somnio were able to provide evidence of a positive healthcare effect with the lowest sample size (n = 56), and the manufacturers of Deprexis used the largest sample (n = 1,013). Proof was provided for Hello Better Panik based on 92 cases. Five studies had sample sizes between 100 and 199, and six studies between 200 and 299. All other studies worked with sample sizes larger than 300. The distribution of the individuals to the intervention and control groups was done by means of 1:1 randomization in most studies; one exception was the Selfapy Depression study, with two intervention groups (IG1 = 151, IG2 = 150, CG = 100). The drop-out rates at the time of the primary endpoint survey ranged from 6.7% (Zanadio) to 30.0% (Vorvida). In the pre-post study of Vivira (intended to transfer the results to other indications), drop-out rates of more than 90% were observed, which is why the evidence was assessed as not provided. Drop-out rates were higher in intervention groups than in control groups, except for two studies (for Hello Better Panik and Selfapy Depression applications). Drop-out rates at the last follow-up time point ranged from 5.9% (Somnio, a respondent who missed t1 provided data at t2) to 61.3% (Selfapy Depression) (Table 4).
DiGA = Digital health application, IG = Intervention group, CG = Control group, n.r. = Not reported, n.a. = Not applicable
Study results
A positive healthcare effect is considered proven if the outcome is (clinically) relevant, patient-relevant, and statistically significant [3].
All studies were able to deliver significant differences between the intervention and control groups at the primary endpoint and thus, significant results. The effect sizes, according to Cohen et al. were small effect sizes (< 0.5) for three DiGA; medium effect sizes (0.5–0.8) for six applications, and in the range of large effect sizes (> 0.8) for four DiGA. No effect sizes were reported for four DiGA. For the Deprexis study, the effect sizes of the two published studies were used, and for the Velibra study, the effect sizes were those related to the four primary endpoints. The effect sizes increased for five DiGA at the follow-up time point, and decreased for two DiGA. No effect sizes at the follow-up time point were reported for six DiGA. DiGA that reported the minimum clinically relevant difference also exceeded this effect size (Table 5).
Table 5
Overview of primary endpoints and study results (based on the intention-to-treat analyses)
Unless otherwise stated, all values are given as: mean/marginal mean/mean difference (standard deviation), effect size [95% confidence interval]
CI Confidence interval, DiGA Digital health application, MCID Minimal clinically important difference, IG Intervention group, CG Control group, M Mean, mM Marginal mean, MD Mean difference, ITT Intention-to-treat, PP Per protocol, n.r. Not reported
aSurvey time of the primary endpoint
After analyzing the requirements and implementation of the evidence of permanently listed DiGA, our research question, which methodological success factors regarding evidence-based proof of benefit can be derived from the already permanently listed DiGA, can be summarized with the following Fig. 2.
Fig. 2
Overview of methodological success factors regarding evidence-based proof of benefit of permanently listed DiGA
DiGA = Digital health application, ICD = International Classification of Diseases
×
The requirements stated in the BfArM guideline only provide a frame of reference. The factors identified in the present analysis can help successfully implement these requirements in the standard of care.
Further results
Although this was not a study objective, we have found the DiGA directory to lack reporting quality and transparency, which is relevant to further development of the DiGA approval process.
First, of the four DiGA initially demonstrating positive healthcare effects through systematic data evaluation, hardly any pilot study results were available. Only the Selfapy applications provisional results could be found in the DiGA directory.
Second, the DiGA directory was not sufficient for a complete analysis of the seven identified categories and the evidence produced; further sources, such as study registries, study reports and publications, had to be obtained. Finally, we found one case in which the results reported in the DiGA directory did not match to those from the study report.
Discussion
The aim of this study was to identify BfArMs requirements of studies that prove a postulated positive healthcare effect, and, to assess the evidence of applications permanently listed in the DiGA directory.
Permanently listed DiGA meet higher requirements for demonstrating a positive healthcare effect, than those specified in the guideline. Most DiGA focused on one or two indications (ICD-10 codes), and proved a medical benefit through a RCT. All DiGA studies were able to prove patient relevance and achieved statistically significant results, however, observation periods, sample sizes and drop-out rates differed substantially among studies.
Patient population and patient characteristics
Permanently listed mental health DiGA are most frequently represented in the DiGA directory. A focus on one or two indications (ICD-10 codes) seems to predict success for a permanent listing. This could be explained by specific and homogeneous patient populations being needed to provide adequate evidence. Surprising is however, that the BfArM guideline clearly specifies that inclusion in the DiGA directory is only possible for three or four-digit ICD-10 codes. Nevertheless, three manufacturers (Hello Better Panik, Velibra, Zanadio) achieved permanent listing using only a five-digit ICD-10 code.
In all studies aiming to demonstrate a positive healthcare effect, the proportion of women represented in the populations (in both IG and CG) was higher than that of men. A report from a German health insurance company also showed that more women use DiGA than men. Since gender-specific disease prevalence among insured patients in the (as of yet) main DiGA indications can sometimes differ greatly, this may be a result [53].
In almost all studies, the mean age of the participants was between 35 and 55 years. An analysis by a German health insurer of the age structure of all insured adults with at least one DiGA claim as of December 31st, 2021 showed that 27% were aged 50–59 years, followed by 22% aged 30–39 years, and 20% aged 40–49 [53].
DiGA studies published to date show that individuals with better education and employment status participated more frequently. International studies also show that the use of mobile apps is unevenly distributed in society; a US-America study showed that people with an academic education had a higher (2.8-fold) chance of using a health app, compared to those without a high school diploma. The study also showed that higher education was associated with more frequent use of digital health services [54]. A study from the Netherlands also points to social differences in the use of health apps, and showed that people who used a health app were younger and more educated, compared to those who did not [55]. An important factor that should be considered in relation to these discussed aspects is the willingness to participate in a clinical trial. Gouveia et al. (2022) showed that the willingness of younger patients to participate was significantly higher compared to older patients. In contrast, gender, lifestyle, employment status, monthly income, or education showed no influence on willingness to participate [56]. Another point influencing participation in a clinical trial is personal treatment preference [57]. In addition to willingness, attitude, and patients’ motivation to use a DiGA and/or participate in a trial, their previous experience with digital tools and their habits to use such tools for their disease self-management needs to be considered [58, 59]. Due to the influencing factors mentioned above, and the scarce data on DiGA implementation, it is not fully understood whether DiGA address the entire defined study population, or only certain subgroups. This fact underlines the high relevance of subgroup analyses for the manufacturers.
There is a need for further research on the monitoring of the actual use of DiGA by all statutory health insured in Germany, to assess whether the study populations correspond to the actual target groups. At the same time, during the process of the DiGA approval, responsible authorities should monitor whether pilot/feasibility studies, and those needed for permanent listing, may have the potential to recruit more diverse populations, and thereby increase external validity of the trials. The inherent concern is that digitally competent and literate patient populations are recruited, and other specific populations overlooked. This may further amplify the digital divide in Germany [60]. Also Stern et al. (2022) point out that equity aspects must be taken into account to ensure that health inequalities are not reinforced, or even created in the first place [12]. As such, DiGA manufacturers should be encouraged to take additional efforts to effectively recruit all genders, as well as a variety of cultural backgrounds and different levels of digital health literacy. This would enable subgroup analyses, identify those requiring additional support for adequate DiGA usage, and ultimately support individualized DiGA recommendations, according to patient characteristics, competencies and preferences [61].
Positive healthcare effect and study endpoints
It is striking that— although patient-relevant structural and procedural improvements show high potential for improving care, especially in terms of processes — all DiGA have provided a positive care effect via a medical benefit. Lantzsch et al. (2022) pointed out needed improvement in using outcomes for patient-relevant improvement of structure and processes regarding an improved evaluation [11]. All applications were able to demonstrate an improvement in health status. Only one DiGA additionally aimed for an improved quality of life, and a reduced disease duration, which, however, could not be achieved. DiGA that provide a medical benefit in the form of a health status improvement seem to have the greatest likelihood to be permanently listed in the DiGA directory. Focusing on one primary endpoint seems to predict success for a permanent listing; the majority of the DiGA provide evidence for one defined primary endpoint. With regards to the study endpoints and measurements used, the analysis found that multiple endpoints were used, which is not surprising, given the range of indications covered. However, the heterogeneous measurements used to assess these endpoints offer a standardization potential, which will be increasingly relevant as more DiGA for the same indications are listed. To close this gap, it is possible to use Core Outcome Sets (COS). The idea of COS, is to provide a minimum set of outcome domains and a consensual set of measurement tools to be used in every clinical study with a comparable intervention and target population. Development of new, or expansion of existing COS for the evaluation of DiGA, can improve comparability of study results [62, 63].
Study design
Although BfArM accepts study designs with a lower level of evidence for the proof of a positive healthcare effect, all manufacturers of permanently listed DiGA conducted a study with a high level of evidence. Implementing a RCT was therefore a factor increasing the likelihood of permanent inclusion in the DiGA directory. RCTs are not only conducted in the medical sector, but increasingly also in the technology sector [64], and are therefore considered a promising study design for DiGA. However, the comparatively short innovation cycles for new technologies can be an obstacle to conducting RCTs in a DiGA context. The DiGA and its individual components are usually continuously adapted and further developed by the manufacturers, so that new versions are often already available before the evaluation of the original version is completed. One success factor here can be a continuous, learning evaluation of the continuously changing DiGA process [14]. Stern et al. (2022) cite actual or perceived risks of regulatory uncertainty, as possible reasons for deciding against non-RCT studies. It may then be more risky or costly for manufacturers to subsequently switch to a more traditional study design after an unsuccessful trial [12].
Although both inter- and intra-individual comparisons are allowed, all manufacturers chose to compare an intervention group with a control group. Different methods are proposed for this comparison. DiGA manufacturers have compared to treatment without DiGA, or to no treatment. The third option — comparison with another comparable DiGA that is already permanently listed in the DiGA directory at the time of application — will become more relevant as the DiGA directory grows. Increasing in popularity is a fourth conceivable pathway; the design of a placebo DiGA, allowing blinding to be maintained. Stern et al. (2022) emphasize the relevance of establishing best practices and methods to accurately define the comparison group [12].
Observation period and observation times
BfArM requires a description of baseline data, data collection periods, and follow-up times, but leave manufacturers flexibility in observation periods. Despite this, choosing a plausible observation period fitting to the indication, is important. If the observation period is shortened, justifying transferability of the achieved effect(s) to the prescribing period, is advisable. Stern et al. (2022) consider planning of washout phases, in which patients do not receive any therapies prior to the start of the actual intervention [12]. In any case, the observation dates must be clearly defined. In fact, almost all studies date back several years, meaning already existing data were used to prove the positive health effect. Since May 27th, 2020, manufacturers can submit applications to be included in the DiGA directory. The directory was started on October 6th, 2020. Only three trials (Kalmeda, Vivira, Zanadio) started in 2020/2021. The study for the application Vivira led to permanent listing in the DiGA directory in only 12 weeks, with a total study duration of five months.
Study results
According to BfArM, a positive healthcare effect is considered proven if the outcome is (clinically) relevant, patient-relevant, and statistically significant. All DiGA study results were able to prove patient relevance and achieve statistical significance, and are central to successful entry in the registry.
Although the questionnaire scores improved after the application of the DiGA, the symptoms partly remained similar. One example is Hello Better Panik; here, though participants’ scores on the PAS scale (German version: Panik- und Agoraphobieskala) improved by 6.45 (IG) and 1.83 (CG) from baseline (compared to the primary endpoint), symptoms remained moderate. Similarly, Hello Better Stress und Burnout showed improved Perceived Stress Scale scores (by 8.01 (IG) and 2.19 (CG) from baseline compared to the primary endpoint), however symptoms also remained moderate. These examples show the limitations of DiGA. Somnio, however, showed differences in the Insomnia Severity Index of 7.58 (IG) and 1.22 (CG), thus improving from moderate insomnia to subthreshold insomnia; DiGA also have the potential to greatly improve users’ health.
Further results and recommendations
In comparing the evidence required with the evidence provided by permanently listed DiGA, it was clear that transparency is not yet fully apparent. Particularly, gaining a fully comprehensive picture on some aspects (patient population, study endpoints, study location, drop-out rates, and study results) was not possible, using the DiGA directory alone. Other sources had to be consulted, such as study registers, study reports and publications. In one case, the results reported in the DiGA directory did not match those in the study report. The claimed user-friendliness of the directory must also be questioned. The user only gets an insight into the mean values/mean value comparisons. An interpretation aid for the meaning of these figures is missing.
In the future, the DiGA directory should be an essential tool for medical professionals. An important step for the spread of health apps and their integration into clinical practice, is the education of clinicians regarding available technologies [4]. A study with 51 physicians showed that half of the respondents expect to be able to identify high-quality DiGA using the DiGA directory. This group is more likely to prescribe DiGA [65]. To increase DiGA acceptance and willingness to prescribe among physicians, a transparent, complete, and correct presentation of the information is indispensable.
Future research needs are identified in the continuous evaluation of the fast-track process, with special attention to requirements of DiGA manufacturers. As an innovative and learning system in healthcare which is constantly being developed and improved (with increasingly specific requirements for DiGA manufacturers, and the growing DiGA directory), an accompanying monitoring could be a central success factor.
Limitations
The current study combined multiple methods to analyze the available evidence of listed DiGA. However, only available studies, data, and information on the applications permanently listed in the DiGA directory were included in the analysis, due to the lack of evidence for provisionally listed applications. Therefore, recommendations may not be transferable to DiGA manufacturers aiming for provisional listing. Furthermore, evidence available for permanently listed DiGA was not identified through a systematic literature search. To limit the risk of overlooking relevant studies, numerous independent sources were used to identify DiGA-specific information. In addition to the DiGA directory, entries in study registries, study protocols, study reports, submitted publications, and manufacturers' websites, were used as information sources.
Conclusion
The results of this analysis indicate that permanently listed DiGA meet higher standards than required by the guideline. Before prescribing a DiGA, its evidence should be carefully examined. The identified success factors provide healthcare practitioners with a transparent overview of the status quo of applications already tested. They can also support future manufacturers in the development and evaluation of their DiGA. With regards to the various endpoints used in the presented studies, future DiGA trials should focus on the most relevant outcomes, and strive towards comparability of results (especially among DiGA for the same indication). In addition, an analysis of DiGA-use in everyday care should become the subject of further implementation research. Overall, there is a need for accompanying monitoring; from the application development, through its testing during the evaluation study, to its use in everyday care. A growing DiGA system could be a beneficial improvement for the whole healthcare system.
Acknowledgements
We thank Joanna Diesing for the linguistic optimization of our manuscript.
Declarations
Ethics approval and consent to participate
Not applicable.
Consent for publication
Not applicable.
Competing interests
The authors declare no competing interests.
Open AccessThis article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
