Effectiveness of online and mobile telephone applications (‘apps’) for the self-management of suicidal ideation and self-harm: a systematic review and meta-analysis

We searched for literature indexed in seven electronic databases covering a wide range of disciplines, including: computing and information technology (Applied Science & Technology) clinical trials (Cochrane Central Register of Controlled Trials [CENTRAL]), medicine (Embase; Medline), psychology (PsycARTICLES; PsycINFO), and public health (Global Health) as well as a database that specifically indexes literature on interventions for the prevention of suicide (Centre for Research Excellence in Suicide Prevention [CRESP]). Clinical trial registries were also searched using these same keywords to identify ongoing studies. All databases were searched from their respective start dates until 31 March, 2017.

A two stage process was used to locate relevant studies. At the first stage, keywords inclusive of digital interventions and platforms were combined. Next, using standard Boolean operators, these were combined with keywords related to suicidal ideation and self-harm. There were no restrictions either on publication language or status. For further information on this electronic search strategy, please see the Appendix.

Ancestry searches were also conducted by manually screening the reference lists of included studies and previous reviews [22‐26]. Where information on either study design, methodology, or results was either unclear or missing from the published study, we sought clarification from corresponding authors.

Methodological details were extracted from included studies using a standardized extraction form by two authors (KW and AM) working independently of one another. Disagreements were resolved through consensus discussions. Methodological details included: research design, treatment setting, and outcome ascertainment. We also assessed whether those evaluating the intervention were independent from those who developed the intervention.

Data on the primary outcome, suicidal ideation, were also extracted by KW and AM working independently of one another. To make maximal use of the available data, information was extracted irrespective of whether these outcomes were measured continuously, for example as scores on a psychometrically validated measure of suicidal ideation or as numbers of repeat episodes of self-harm, or categorically, as the proportion of participants reporting thoughts of suicide or number of self-harm events. Care was taken, however, to ensure the items used to determine these outcomes were comparable between studies.

Secondary outcomes included: episodes of self-harm, attempted suicide, and completed suicide measured according to self-report and/or hospital or medical records. Where a study reported outcomes at multiple time points, for example at six and 12 months’ follow-up, only data for the longest follow-up period were extracted, in line with recommendations [27].

Risk of bias was assessed using the Cochrane Collaboration tool for randomised and pseudo-randomised controlled trials [35] and, for controlled before/after designed studies, the Risk of Bias In Non-Randomized Studies of Interventions (ROBINS–I; [36]). The Cochrane Collaboration tool assesses bias in seven domains including: adequacy of the random sequence generation, allocation concealment, blinding of participants, clinical personnel, and outcome assessors, as well as incomplete data, selective outcome reporting, and other bias. The ROBINS–I assesses bias in eight domains including: confounding, participant selection, classification of the intervention, departures from the intended intervention, missing data, measurement of outcomes, selection of the reported results, and overall bias.

The funder had no role in study design, data collection, data analysis, data interpretation, or writing of the report. KW and AM had full access to all the data in this study, and all authors had final responsibility for the decision to submit for publication.

The majority of these studies had been conducted either in Australia (four studies: [37‐40]) or the USA (four studies: [41‐44]). Two studies were conducted in Germany [45, 46], one in the Netherlands [47], one in Sweden [48], and one in Switzerland [49]. One further study recruited participants through online forums and therefore included participants from a number of different countries, including countries in North America, Europe, and Australasia [50].

Most studies evaluated the effectiveness of online programs [37‐39, 41‐49]. Only two evaluated the effectiveness of mobile telephone apps [40, 50]. Most programs were developed for the self-management of depression [37, 38, 42, 43, 45, 46, 48, 49]. However, as these studies assessed the effectiveness of these programs on to at least one suicide-related primary or secondary outcome (i.e., suicidal ideation, self-harm, attempted and/or completed suicide) they were nonetheless eligible for inclusion in this review. Only five programs were developed specifically for the self-management of suicidal ideation [39‐41, 44, 47], and only one was developed for the self-management of self-harm [50].

In terms of study design, eight were randomised controlled trials [37, 40, 41, 44, 46, 47, 49, 50], four were observational pre-test/post-test studies [38, 39, 43, 48], and one was a pseudo-randomised controlled trial in which sequential participants were alternately allocated to the intervention and control conditions [45]. For one RCT, although participants were randomised to the intervention and control conditions, all participants received access to the digital intervention [42]. For the purposes of this review, data from this RCT was therefore treated as pre-test/post-test comparisons instead. For two additional RCTs, outcomes at pre-test/post-test were also reported for the intervention group, enabling their inclusion in pre-test/post-test comparisons as well as in RCT comparisons [45, 49]. However, to ensure results from these studies were not double-counted, analyses were pooled separately by study design for all outcomes reported in this review. In one further RCT, participants assigned to the wait-list control condition crossed over to receive the intervention after six weeks [40]. To avoid contamination from any ‘carry-over’ effects, we only extracted data for the first six week period prior to cross-over as recommended [51].

Most programs were developed by clinical psychologists and/or psychiatrists with experience treating suicidal ideation and/or self-harm [37, 39‐45, 47‐50], and were evaluated by those who developed the intervention [37‐40, 42, 44, 46‐50]. In terms of therapeutic approach, most programs were based on the principles of cognitive behavioural therapy (CBT). Some also included elements of ‘third wave’ CBT, such as mindfulness [45], dialectical behaviour therapy [47], or mentalisation-based cognitive therapy [47]; all of which hold the rationale that challenging thoughts, a principle feature of CBT, is less important than understanding and accepting thoughts in a non-judgemental manner. Other programs included a variety of treatment approaches, including: acceptance-based therapy [40], problem-solving therapy [47], interpersonal therapy [42], mood monitoring [46], and crisis planning [46]. Only one program utilised gamification in which participants were presented with a series of visual stimuli pairs designed to condition aversive reactions to self-harming thoughts or behaviours [50].

For the eight RCTs and one pseudo-RCT designed trials, the interventions were compared against a number of types of control conditions, including: wait-list control [37, 40, 45], attentional control [41, 47, 50], psychoeducation [44], treatment as usual [46], or face-to-face psychotherapy [49].

An additional 10 ongoing studies were identified at the time of the systematic search [52‐59]. Further details of these ongoing trials are reported in Table 2.

Four studies reported data on the number of participants self-reporting suicidal ideation. At post-intervention, there was some suggestion of a reduction in the proportion of participants self-reporting suicidal ideation in three observational pre-test/post-test studies (OR 0.36, 95% CI 0.27 to 0.49, 3 studies, I ²  = 0%, p < 0.0001; Fig. 2). However, by the final follow-up assessment, data from one RCT suggested no evidence of a treatment effect for these interventions (Fig. 2). There was no evidence of a significant difference in the magnitude of the treatment effect by study design for this outcome (χ² = 0.61, df = 1, p = 0.44). As all four studies included in this analysis investigated the effectiveness of digital interventions specifically developed for the self-management of depression symptoms (rather than suicidal ideation or self-harm specifically), sensitivity analyses could not be undertaken.

One study covering three related RCTs reported information on the frequency of self-reported episodes of suicidal ideation at post-intervention (for all three studies) and at the conclusion of the final follow-up assessment (for one of these three studies) [50]. However, no evidence of a treatment effect was found for these interventions at either time point (Fig. 3). As only one RCT was included in this analysis, neither tests for subgroup differences nor sensitivity analyses could be undertaken.

Eight studies reported data on suicidal ideation scores using a number of different psychometric instruments, including: the BSSI [44, 47, 49], the Depressive Symptom Inventory–Suicidality Subscale (DSI-SS; [60]) [40], the Suicide Behaviors Questionnaire–Revised (SBQ–R; [61]) [45], the Suicidal Ideation Questionnaire–Junior (SIQ–J; [62]) [39], the four item suicidal ideation sub-scale of the General Health Questionnaire (GHQ–28; [63]) [37], and the suicidal ideation item of the Hopkins Symptom Checklist (HSCL–20; [64]) [43]. As raw means and standard deviations were not reported for suicidal ideation scores in one RCT, data were instead estimated from graphics in the original report for this study [37].

At post-intervention, there was evidence these interventions were associated with a significant reduction in suicidal ideation scores in five RCTs (SMD -0.26, 95% CI -0.44 to −0.08, 5 studies, I ²  = 0.0%, p = 0.005; Fig. 4). There was no evidence of a significant treatment effect for these interventions in either one pseudo-randomised controlled trial or four observational studies, however (Fig. 4). The test for subgroup differences was non-significant suggesting there was no difference in magnitude of the effect size by study design for this outcome (χ² = 0.25, df = 2, p = 0.88). Sensitivity analyses including only those studies in which the intervention was specifically developed for the self-management of suicidal ideation or self-harm also did not materially affect these results (results not shown).

Two RCTs reported data on suicidal ideation at the final follow-up assessment [37, 44]. For one of these trials, however, data on suicidal ideation scores at final follow-up had to be estimated by review authors from graphics presented in the original trial report [37]. Data from these trials suggested these digital interventions were not associated with a significant treatment effect for suicidal ideation by this time point (SMD -0.34, 95% CI –0.70 to 0.01, 2 studies, I ²  = 0.0%, p = 0.06; Fig. 4). Given that both studies included in this analysis were RCTs, neither tests for subgroup differences nor sensitivity analyses could be undertaken. One further RCT presented data on outcomes at final follow-up for the intervention group only [65]. Analyzing these data as pre-test/post-test comparisons suggested a significant reduction in suicidal ideation scores at the final follow-up assessment in this trial (MD -4.90, 95% CI -7.07 to −2.73, 1 study, I ² = not applicable, p < 0.001). As only one RCT reported data at the final follow-up assessment, neither tests for subgroup differences nor sensitivity analyses could be undertaken.

One study, covering three related RCTs, evaluated the effectiveness of digital interventions on frequency of self-harm episodes [50]. At post-intervention, there was no indication of a treatment effect for these interventions on either self-reported frequency of self-cutting or non-suicidal self-injury in these three RCTs (Fig. 5). There was also no indication of a treatment effect for this intervention on frequency of self-reported self-cutting or non-suicidal self-injury at the final follow-up assessment (at one month) in one of these RCTs (Fig. 5). Sensitivity analyses could not be undertaken for this outcome as only one study investigated outcomes relating to repetition of self-harm. As all three studies were RCTs, subgroup analyses to investigate the impact of study design on the magnitude of the effect size could also not be undertaken.

One RCT evaluated the effectiveness of a digital intervention on attempted suicide and self-harm [46]. As results for self-harm could not be disaggregated from that for attempted suicide, they are instead analysed here as a combined outcome. There was no evidence of a reduction in the proportion of participants who attempted suicide and/or engaged in self-harm over a 24 month follow-up period in this study (OR 2.11, 95% CI 0.19 to 23.81, 1 study, I ²  = not applicable, p = 0.55). Once again, as only one study investigated outcomes relating to combined attempted suicide and/or self-harm, sub-group and sensitivity analyses could not be undertaken.

One RCT evaluated the effectiveness of a digital intervention on attempted suicide [47]; however, no evidence of a reduction in the proportion of participants self-reporting a suicide attempt was noted by the post-intervention assessment in this study (OR 0.58, 95% CI 0.16 to 2.02, 1 study, I ²  = not applicable, p = 0.39). As only one study investigated outcomes relating to combined attempted suicide, sub-group and sensitivity analyses could not be undertaken.

Most studies were of low to moderate quality, as assessed using the Cochrane Collaboration’s tool for randomised and pseudorandomised controlled trials [35], or the ROBINS–I tool for controlled pre-test/post-test studies [36], with biases most apparent for the domains of participant, clinical personnel, and outcome assessor blinding. Performance and detection bias therefore cannot be ruled out.

Of those studies utilising an RCT design, most compared the intervention to either waitlist control [37, 45] or attentional control [41, 47, 50] conditions. Variability in the control condition has been found to be associated with the magnitude of the treatment effect in office-based psychosocial interventions for depression and anxiety [66, 67]. Yet, despite this, variability in the control condition is a rarely investigated source of heterogeneity in meta-analyses of digital interventions.

The included studies report data on a variety of different outcomes. Although all studies included at least one suicidal ideation and/or behaviour relevant outcome, these were measured in a variety of different ways, including from psychometric scales, self-report, or according to hospital or medical records. Given that these outcomes were measured using the same methodology within individual studies, the effect of outcome measure definition on case identification at the post-intervention and follow-up assessments could be expected to affect the intervention and control groups equally. Between studies, however, outcome measure definition may have affected our results in light of research findings suggesting that self-reported self-harm underestimates hospital-treated self-harm [68]. We were unable to assess whether variability in the magnitude of the treatment effect size was related to the way in which the outcome measure was assessed in this review, however, owing to the small number of studies included in any one meta-analysis. Further work will be necessary to pick apart the influence of outcome definition on the apparent effectiveness of interventions, including digital interventions, for the prevention of self-harm.

Most studies (71.4%) also reported information on depression [38‐40, 42‐45, 47‐49]. Fewer studies reported data on other clinically relevant outcomes, such as hopelessness [40], and none reported information on problem-solving following the completion of the intervention. The National Institute for Health and Clinical Excellence (NICE) guidelines recommend that all interventions for self-harm should, at a minimum, investigate the effect of psychological interventions for self-harm on potential mechanisms of action, including depression, hopelessness, and problem-solving [69], echoing more recent calls to this effect in the international scholarship [70, 71]. Online resources may normalize self-harm and may provide vulnerable individuals with access to self-harm content and imagery, including information on methods of self-harm [72]. Additionally, digital interventions that provide some form of active psychosocial therapy, and particularly those that include a focus on mood monitoring, may lead to increased negative affectivity and rumination [73]. Outcomes relating to negative affect and rumination should therefore also be reported for all evaluations of digital interventions for this patient group in future.

Most programs (57.1%) were developed for the self-management of depression [37, 38, 42, 43, 45, 46, 48, 49]. Recent findings for psychosocial interventions, however, suggest that those developed specifically for the management of suicidal thinking are associated with greater impacts for the reduction of attempted and completed suicide as compared to those targeting indirect symptoms associated with suicidal behaviour, such as anxiety, depression, or hopelessness [31]. Future studies in this area should evaluate the degree to which these digital interventions lead to meaningful change in the proposed mechanism(s) of action in order to identify the treatment module(s) associated with the greatest impacts in reducing suicidal ideation and self-harm in this population.

The included studies also examined interventions of varying intensity. Whilst it could be expected that treatments of greater intensity will have greater impacts on reducing suicidal ideation and self-harm, a recent meta-regression review found no evidence to suggest that treatment intensity, measured as the total number of available treatment sessions, was associated with greater effectiveness for office-based psychosocial therapy for self-harm repetition [74].

Finally, it is also likely that these populations will have a very low risk for suicidal behaviour as all studies recruited participants from the community. Despite this, almost all (78.6%) studies included indicated samples, such as callers to telephone or online counselling services [37, 45, 50], or those already in contact with primary care [38, 42, 43], counselling [39, 40, 49] or psychiatric services [46, 48].

The majority of these interventions were based on the principles of standard cognitive behavioural therapy (CBT), which has been found to have efficacy in reducing repetition of self-harm in clinical populations in a recent systematic review of psychosocial interventions for the treatment of self-harm [10]. The findings of the present review significantly extend this by suggesting that digital interventions which incorporate the principles of standard CBT may have promising effects in reducing suicidal ideation in non-clinical populations, at least in the short-term.

Whilst we utilized a comprehensive search to locate all relevant trials of digital interventions for the self-management of self-harm, we identified only two mobile telephone apps [40, 50]. Given that a recent Australian study identified a total of 24 apps for the prevention of suicidal behaviour are currently available for download from the Australian Google Play and iOS store [75], this would suggest that a large number of these apps have no evidence to support their effectiveness. It is likely that a similar proportion of online interventions would have little evidence to support their effectiveness.