The BEACON study: protocol for a cohort study as part of an evaluation of the effectiveness of smartphone-assisted problem-solving therapy in men who present with intentional self-harm to emergency departments in Ontario

In a linked separate paper, we have described the study protocol for a cluster randomised controlled trial (cRCT) to test the effectiveness of a new service that delivers blended problem-solving therapy (PST) compared with usual care (UC) in men aged 18 years or older who present to emergency departments (EDs) in Ontario with intentional self-harm. In the cRCT, 25 EDs were randomised to either UC or the implementation of a smartphone-assisted PST service for men who present to the ED for an episode of self-harm. All outcomes in the cRCT will be obtained from provincial administrative health databases, with the primary outcome being a composite measure of the proportion of men re-presenting to the ED with self-harm and who die by suicide within 1 year of their index presentation. This protocol outlines a cohort study designed to assess the impact of a blended therapy intervention on men enrolled at the intervention sites of the cRCT.

Self-harm has a strong association with suicide: 1.6% of people presenting to EDs with self-harm will die by suicide within 1 year (95% confidence interval, 1.2–2.1%), with the incidence rate in men being almost double the rate in women (2.7% vs. 1.2%) [1]. After 5 years, approximately 3.9% of individuals who have presented to the ED for the treatment of self-harm die by suicide [1]. This risk is more than 50 times greater than the general population rate and is associated with a 40-year reduction in average life expectancy [1]. A recent retrospective study of individuals who died by suicide in southwestern Ontario identified a history of self-harm in more than one-third of decedents [2]. Because presentation to the ED with self-harm is a major identifiable risk factor for suicide, with at least one-fourth of deaths by suicide being preceded by a hospital visit due to non-fatal self-harm in the previous year [3, 4], it is likely that any reduction in the repetition of self-harm will be mirrored by a decline in subsequent deaths by suicide. The Canadian Association for Suicide Prevention Blueprint for a Canadian National Suicide Prevention Strategy has also identified people who have attended hospital because of non-fatal self-harm as a high-risk group to target in order to prevent suicide [5].

Mortality of non-suicidal causes is also high for those who self-harm, with significantly more than the expected numbers of deaths of natural causes and due to accidents [6]. A population-based cohort study investigating administrative datasets in the province of Ontario found all-cause mortality following a first episode of self-poisoning was 1107 per 100,000 person-years, a rate 5.5 times that of the control population. Nearly half of all deaths in this study were attributed to suicide (23.4%), accidents (16.4%), or undetermined intent (5.4%) [7]. Approximately 10% of those who present in an ED following self-harm will engage in repeat self-harm in the following month, and up to 27% will do so after 6 months [8]. Recurrent self-harm is associated with significant distress and many unresolved interpersonal problems [9]. Individuals who self-harm are also frequent users of health and social services [10, 11]. For instance, Morrison & Laing [12] reviewed death records of Alberta patients aged 25 to 64 who died by suicide to develop healthcare use profiles of those who died by suicide compared with those who died of other causes. They found that those who died by suicide averaged more than twice the number of health service visits per person when compared with those who died of other causes [12]. Similarly, those who died by suicide were more likely to have had an ED visit, inpatient hospitalisation separation, or community mental health service than those who did not die by suicide [12].

Numerous studies have also shown that mainstream media reports on suicide influence some people in the general population to engage in self-injury and die by suicide through a copycat phenomenon known as the “Werther effect” [3‐16]. This has not specifically been described in males who present to the ED after an episode of self-harm, and the potential impact of the Werther effect was observed with roughly a 10% increase in suicide deaths across the entire United States in the months following the death of the actor and comedian Robin Williams [17]. We will assess exposure to media reports on suicide and access to the internet to inform methods. This could inform future prevention efforts at a population level.

The intervention we will study aims to address this gap and build on previous work, extending the range and intensity of PST by supplementing it with a sophisticated smartphone application and electronic case management that has already demonstrated effectiveness in men with substance abuse disorders [18]. The addition of electronic support to psychotherapy has been called “blended care,” referring to the combination of online and face-to-face therapy in one treatment [19]. Only a small number of studies have investigated blended care, and none has examined blended care in patients who are at high risk for suicide. Large trials in routine clinical settings are needed to assess the effectiveness of blended care interventions in those with mental disorders. In recognition of this, the European Commission has funded a large study, the European Comparative Effectiveness Research on Internet-Based Depression Treatment project (E-COMPARED), in which the effectiveness of blended care for treating depression will be assessed in a randomised controlled trial (RCT) in eight European countries [20]. However, the E-COMPARED study specifically excludes suicidal participants and those with co-morbid mental disorders, such as bipolar disorder or substance abuse.

The present cohort study will assess the impact of treatment with smartphone-assisted PST on suicidal thoughts and behaviour in men over the age of 18 years who present to an ED with intentional self-harm.

The primary objective of this study is to examine the relationship between the number of blended PST sessions attended by men who present to the ED with self-harm and suicidal ideas in the year following their enrolment in the study. Secondary outcomes include health service use (including re-presentation to hospital with self-harm), change in measures of depressive symptoms, anxiety symptoms, symptoms of post-traumatic stress disorder, health-related quality of life, meaning in life, social supports, and substance misuse.

We will also examine the effect on outcomes of the following confounding variables: conformity to “masculine” gender norms, self-reported exposure to reports of suicide in the media, and whether participants searched the internet for methods of suicide for their index presentation.

We will also investigate a theoretical process of change by measuring problem-solving skills assessed by the means–ends problem-solving procedure and the Social Problem-Solving Inventory–Revised Short Form (SPSI-R:S).

This study’s research objectives, primary and secondary research questions, and associated hypotheses are outlined in Tables 1 and 2.

This is a cohort study embedded in a cRCT in which the unit of randomisation is hospital EDs in Ontario. This protocol has been developed according to the Standard Protocol Items: Recommendations for Interventional Trials (SPIRIT) statement.

This study protocol describes a cohort study embedded in a multicentre cRCT that will be conducted at ten sites in Ontario, Canada. Once site allocation is completed, the principal investigator (PI) (SH) will contact members of the department of psychiatry and department of emergency medicine at each intervention site to be site co-PIs to assume responsibility for all study-related activities at their respective site. The PI will then travel to each study site in order to present information about the study in a rounds-style presentation and meet with key stakeholders who will be responsible for the implementation of the study.

Eligible participants will be men aged 18 years or older who present to EDs in the intervention arm of the cRCT with intentional self-harm. They will be eligible regardless of whether they are hospitalised or discharged during the index ED visit (Table 3).

There are no exclusion criteria beyond the opposite of the inclusion criteria. Participants are not required to have a smartphone with a data plan in order to participate. Participants who do not have a smartphone with a data plan will be provided with one for a period of 1 year from the date of their study enrolment.

All informed consent discussion will be completed by a site-delegated study staff member, such as a research coordinator or research assistant.

On the consent form, participants will be asked if they agree to use of their data should they choose to withdraw from the trial. Participants will also be asked for permission for the research team to share relevant data with people from the universities taking part in the research or from regulatory authorities, where relevant. This trial does not involve collecting biological specimens for storage.

Neither a systematic review nor a large multicentre study found a clear relationship between the nature and intensity of standard hospital care and subsequent fatal or non-fatal repetition of self-harm [1]. Given this, rather than randomising participants to a control group, we opted to use a within-subject design in which participants act as their own control subjects. To assess whether completion of PST results in decreased suicidal thoughts and behaviours, we will evaluate changes in suicidality from baseline throughout the course of the study using repeated-measures analysis of variance (ANOVA).

The cohort study will be stopped if there are more than 12 deaths per year.

The use of a blended therapy approach with the addition of the BEACON prescription management system to face-to-face PST may increase adherence to the face-to-face sessions. The patient-facing smartphone application was previously identified as increasing feelings of connectedness in participants between face-to-face sessions [25]. To improve adherence to the use of the BEACON prescription smartphone application, the application has included daily activities to encourage daily use, as well as the option to set reminders by either the clinician or the participant, which will push notifications to the participant’s phone, serving as an additional prompt to continue using the application.

Not applicable; participants will continue to receive UC while enrolled in this study.

In the event of a study-related injury or illness, participants will be provided with appropriate medical treatment and care. Financial compensation for lost wages, disability, or discomfort due to an injury or illness is not generally available.

The number of sites for the BEACON (Smartphone-Assisted Problem-Solving Therapy in Men Presenting to the ED with Self-Harm) study was determined by the cRCT. Using IC/ES data, we determined that identifying a cohort of men who present with intentional self-harm over the course of 1 year and then following up with them for 12 months at 10 intervention sites and 15 control sites in Ontario would give us 80% power to detect a relative reduction in repetition rate of 41% (i.e., from a control arm rate of 13% to an intervention arm rate of 7.7%). For this substudy embedded within the intervention arm of the trial, we anticipate that we will be able to recruit 20 to 50 men at each of the intervention sites with an average of 35 men per site, for an anticipated total of 350 men. To test our primary hypothesis that the severity of suicidal thinking at 1 year will decrease with the number of PST sessions completed by participants, we will use both linear and dichotomous modelling approaches. For linear correlations, we will have 80% power to detect a correlation of r = 0.15 between number of PST sessions and severity of suicidal thinking measured by the BSS. For the dichotomous approach, we will divide participants into those who have received fewer than three sessions (zero to two sessions) and those who have received three or more sessions. The rationale for this is that three sessions of PST is the minimum number to complete one “cycle” of problem solving. Previous studies have found a mean score of 18 on the BSS for individuals who present with self-harm [40]. To achieve 80% power to detect an effect size of 0.3, or an absolute decrease of 2.4 points from a mean of 18 points on the BSS, we will require 350 participants. Assuming the ratio of those with three or fewer sessions to those with more than three sessions is between 0.75 and 1.25, we anticipate 175 in each group (e.g., 0–2 sessions/3+ sessions). This assumes a common standard deviation on the BSS of 7.9 points based on a comparison of suicidal and non-suicidal samples of adults receiving psychiatric care [40].

Men who present with intentional self-harm to an ED that has been randomised to receive the study intervention will be approached by ED or study staff with information about the study. A delegated study staff member at each site will also routinely review the electronic medical records at their respective site to ensure that no potentially eligible patients have been missed. These patients will be provided with information about the study by telephone. Patients interested in participating in the study will be scheduled for a baseline intake appointment with a delegated study staff member. We will manage the transition from the ED to outpatient care by providing staff training, written information for patients, and an electronic referral service at each site.

At their baseline intake appointment with a research assistant, eligible participants will download the BEACON prescription smartphone application, and they will be introduced to the on-boarding process and set-up of their profile. Once this is complete, participants will be asked to complete the baseline intake assessments and will be referred for their first PST session. In order to limit participant burden, they will have the choice either to complete their baseline visit and first PST session in one study appointment or to split these visits into two study appointments.

This is a cohort study embedded in a cRCT, so no randomisation will take place.

Not applicable.

Not applicable.

There is no blinding in this study, because all patients who consent to participate in this study will receive the study intervention.

Not applicable.

All outcome measures will be administered via paper-based questionnaire at the baseline visit. Questionnaires will then be administered via the BEACON suicide prevention smartphone application as per Table 5.

A major difficulty in all clinical trials is ensuring that patients attend their baseline appointment. As such, a key component to our patient recruitment strategy is to ensure minimal loss to follow-up between the point of referral and study enrolment. One way that we are addressing this is through the use of an electronic referral system in which referring clinicians can schedule a baseline intake appointment with interested patients while still in the ED. The goal is to schedule these appointments within 1–2 business days of the presentation.

Once enrolled in the study, standard procedures for loss to follow-up will be followed for patients who do not complete their follow-up appointments. This is an escalated response that may include any combination of the following: continued attempts to contact the participant by postal mail, telephone, and/or email; contacting a participant’s emergency contact; and contacting a participant’s family physician.

The BEACON suicide prevention smartphone application will also be designed to increase participant engagement with the study. For instance, participants will have instant and time-delayed communication with their research therapist through the messaging feature of the mobile application. Research therapists will also have access to a clinician dashboard that will allow them to identify patterns of use which might indicate that a participant may be at risk of being lost to follow-up (i.e., through decreased use of the smartphone application). This will provide research therapists with an opportunity to reach out to participants and attempt to re-engage them in the intervention.

All data, with the exception of the BSS and SPSI-R:S, will be completed directly by participants using a centralised electronic data capture (EDC) system, developed and maintained by the Ottawa Methods Centre. Participants will create a unique login to the system and will complete the questionnaires remotely or on-site at the time of their visit. The BSS and SPSI-R:S will still be completed in paper format due to unavailability of an electronic license. On a monthly basis, each site will upload their de-identified study data via a secure network OHRI SharePoint folder (password protected and encrypted) to the research coordinator via email as per N2 Standard Operating Procedure 106 File Transfer and its associated OHRI addendum. Data exported from the EDC will be combined with the uploaded site data and combined into a single study master database in IBM SPSS Statistics software (IBM Corp., Armonk, NY, USA). This database will be stored on the OHRI hospital server at the coordinating site, and only the trial management committee (TMC) will have access to it.

All hard copies of original study documentation will be stored in the participant research charts, which will be categorised in numerical order, according to sequential numbering (i.e., 001 to 350). Data collection via the EDC will be by direct entry, and no paper copy will exist. Once all data monitoring, validation, and cleaning activities are complete, an export of the final EDC database as well as any paper records will be archived at a secure storage facility for a period of 10 years, as required by International Conference on Harmonisation of technical requirements for registration of pharmaceuticals for human use good clinical practice (ICH GCP).

All study-related documentation will be double-locked in areas with limited access at the appropriate study site. All participants will be assigned a unique participant identification number, which will appear on all documentation included in a participant’s research chart, including study forms, questionnaires, participant progress notes, and correspondence, in order to maintain participant confidentiality. All correspondence with participants will be de-identified in order to remove names and other potentially identifying information prior to being included in the study chart. All documentation, including eligibility screening forms, signed informed consent forms, and messaging logs, will be stored in double-locked filing cabinets in areas with limited access and stored separately for participant research charts to avoid linking a participant’s name and unique identification number.

Each study site will have a separate master tracking log that will link participants’ names and identification numbers. The combined master tracking log for all study sites will be stored at the coordinating study site. These will be password-protected, and only delegated research staff at that site will have access to it. Participant information will be stored on the secured hospital servers at each study site. The password-protected documents containing participant information will be transferred to the coordinating study site by email, as per N2 and OHRI guidelines. All data are encrypted at rest as well as in transit. Participants’ study information will not be released unless a delegated study staff member obtains written permission from the participant or when required by law. Participants will not be identified in study presentations or publications. All participant records will be kept for a period of 10 years, as indicated in the ICH GCP guidelines.

If this study protocol is amended to include any ancillary studies, upon approval of the REB, all participants involved in these ancillary studies will be asked to sign a consent update form. If a separate informed consent form is required, a copy of the consent form will be stored with the BEACON study consent documentation. Copies of all REB approvals for the ancillary studies will be stored at the coordinating study site. A data file tracking all signed ancillary consent forms must be maintained by the ancillary study and provided to the clinical research coordinator of the BEACON study.

Categorical participant characteristics, such as gender identity, marital status, and education level, will be reported using descriptive statistics with frequencies and percentages. Continuous characteristics, such as age, will be reported using mean ± SD for continuous variables that are normally distributed and as median and 25th and 75th percentiles for non-normally distributed variables. Non-normally distributed variables will also be dichotomised and analysed as categorical data, as described above. Changes in participants’ scores from their baseline visit to follow-up at 1 year will be by repeated measures ANOVA with generalised linear mixed modelling to account for missing variables. Multivariate linear regression analyses will be performed to determine which participant characteristics moderate primary and secondary treatment outcomes.

An interim analysis will be performed after 9 months by an external party who is blinded to the site allocation. The results of this analysis will be reviewed with the data and safety monitoring committee (DSMC), which will make the appropriate recommendations regarding continuation, modification, or termination of the study.

Additional subgroup analyses will be carried out to determine the impact of smartphone-assisted PST for the following subgroups: first-time presentations of self-harm compared with repeaters, Francophone versus Anglophone, men with substance abuse disorders versus no substance abuse disorders, and rural versus urban residence.

The analysis of the primary outcome will be based on self-report data collected through the BEACON suicide prevention smartphone application. As such, the completeness of the data will be impacted by participant withdrawals. In order to minimise the impact of participant drop-out, withdrawal, and loss to follow-up, the research team will follow up with participants regarding the completion of the study questionnaires at their PST appointments. The smartphone application will also be used to prompt and remind participants to complete the study questionnaires. Should participants be lost to follow-up, a delegated study staff member will follow up with them directly to complete the questionnaires either by telephone or by mail. When possible, a delegated study staff member will attempt to ascertain the reasons for drop-out or withdrawal from participants in order to address any issues within the research team’s control in order to prevent future study withdrawals. As such, it is possible that there may be missing data. Characteristics of participants with missing data will be compared with those of participants with complete data to examine the assumption of missing at random. In the case of substantial missingness (e.g., > 5%), missing outcomes will be imputed using multiple imputation prior to analysis.

Anonymised research data will be deposited in an online repository hosted by the Open Science Framework (OSF; https://​osf.​io/​). OSF is a tool that promotes open, centralised workflows by enabling capture of different aspects and products of the research life cycle, including developing a research idea, designing a study, storing and analysing collected data, and writing and publishing reports or papers. It is developed and maintained by the Center for Open Science, a non-profit organisation founded in 2013 that conducts research into scientific practice, builds and supports scientific research communities, and develops research tools and infrastructure to enable managing and archiving research.

The DSMC is comprised of four members from the following fields of expertise: statistics/biostatistics, epidemiology, methodology, psychiatry, and the ethics of clinical trials.

Its responsibilities are as follows:

During the active treatment period (baseline visit to session 6), the following occurrences will be routinely collected and assessed by delegated study staff members: death by suicide, subsequent self-harm, visits to the ED or other unscheduled hospitalisations, and re-presentations to the ED for self-harm. All AEs will be reviewed and classified by the site co-PI at his/her discretion. Investigators will determine relatedness of an event to the study intervention based on a temporal relationship to the study intervention, as well as whether the event is unexpected or unexplained, given the participant’s clinical course, previous medical conditions/history, and concomitant medications or interventions. It is estimated that approximately 1% of participants enrolled in the study will die by suicide and 1% of participants will die of causes other than suicide in each year of the study [1]. This results in an anticipated rate of approximately six deaths per year. To address this, the research team has developed standard operating procedures to manage participant suicidality and mitigate potential staff burnout.

Throughout the study, participants will be able to contact the appropriate local mental health crisis teams at any time. This emergency information will be provided to participants during their baseline visit. Suicidal thoughts and self-harm will be monitored closely at each participant follow-up appointment using both the BSS and the Columbia-Suicide Severity Rating Scale [42]. If the participant’s suicidality worsens, this will be recorded as an AE or SAE, as appropriate, and monitored. Participants will also have access to a licensed professional between study visits up until the 6-month study follow-up and will be connected with mental health services as needed. Between the 6- and 12-month follow-up visits, participant activity will be monitored via the clinician dashboard and follow-up by a delegated study staff member as needed.

Following site initiation, an internal monitor will be selected. This monitor will not be involved in data collection activities and will be one step removed from the clinical trial. The internal monitor will perform the first monitoring visit at each site shortly after the site has recruited their first participant to ensure that research personnel have implemented the appropriate recruitment processes and procedures, such as eligibility sign-off and consent. This visit will be completed prior to the site recruiting more participants. Any corrective actions implemented regarding inconsistencies identified during the previous monitoring visits will be assessed for completeness. On the basis of research category and participant/institute risk exposure, remote monitoring visits will occur every month after the first monitoring visit. The internal monitor may schedule more visits or on-site visits as needed.

Any subsequent modifications to the study protocol, including changes to study objectives, study design, patient population, sample sizes, study procedures, or significant administrative changes will be agreed on by the steering committee and submitted to the REB for review and approval prior to implementation.