Background
The use of technology for the psychological treatment of mental disorders is on a rapid ascent. While the potential ways of using technology to enhance treatment have been discussed for decades [
1,
2], the recent explosion in information technology and telecommunications, and the widespread use of mobile devices have resulted in a dramatic increase in the use of both mobile devices and the internet to enhance and facilitate access to treatment. Several review articles have been published summarizing the bourgeoning body of data being generated [
3‐
7]. Results have generally been supportive of both efficacy and feasibility, though several issues have been identified, such as confidentiality, privacy, crisis management, technological competence, and ethical issues [
3,
8]. As with all innovations, new practice guidelines have been developed to address the unique challenges presented [
9‐
11].
Cognitive behavior therapy (CBT) is an empirically based treatment that is uniquely suited to enhancement by new technologies [
5,
12]. It is highly structured, typically manualized, follows a sequential progression, emphasizes self-responsibility, self-monitoring and homework, and includes ongoing outcome measurements. A variety of technology-enhanced CBT applications across a range of mental disorders have been reported. These include computer-administered CBT self-treatment (stand alone, no therapist contact), computer-assisted CBT treatment (computer-administered with some clinician guidance or contact), mobile monitoring and communication, psychoeducation, remote live treatment via videoconference, and online therapist training [
12‐
18].
The use of technology is particularly well suited for psychological interventions with youth and teens. Nine in ten teens in the USA (93 %) have access to a computer, 78 % have cell phones, and 74 % have mobile access to the internet via a cell phone, tablet or other device [
19]. Text messaging has become the preferred mode of communication among teens, with two-thirds reporting they are more likely to use their cell phones to text their friends than to talk with them. Half of teens in the USA send 50 or more texts per day [
20]. Mobile phone use by teens cuts across socio-demographic backgrounds, as more US families replace traditional land lines with mobile phones (e.g., 41 % of households have only wireless according to a 2013 survey by the National Center for Health Statistics; among poor households, the figure is 56 %) [
21]. Teens in both the USA and abroad have both the technical expertise with these technologies, and a favorable attitude toward their use in mental health care [
4,
22]. Three quarters of lifetime mental disorders begin in adolescence and young adulthood, making it a critical target age for prevention and intervention efforts [
23‐
25].
Given the compatibility between CBT and new technologies, and the affinity for new technologies by youth, the integration of new technologies into CBT treatment of youth has been rapidly increasing [
5,
7]. Applications have been developed for the treatment of a variety of disorders, including simple phobias, social anxiety disorder, generalized anxiety disorder, obsessive–compulsive disorder, encopresis, autism, eating disorders, depression, and substance abuse [
26‐
39].
Mobile applications such as text messaging [i.e., short messaging services (SMS)] are particularly well suited for youth and can help clinicians implement CBT treatment more effectively through the use of homework reminders, real-time self-monitoring and between session communication and feedback [
17]. Among mental health patients, text messaging is the most popular feature, and a higher percentage of mental health patients text compared to the general population [
40,
41]. Self-monitoring in particular has been found to improve treatment outcomes, both by itself and when added to therapy [
42,
43] and accounts for a significant portion of the variance in treatment outcomes [
44]. Text messaging may help overcome non-compliance (a primary reason for lack of treatment efficacy) by enabling encouragement and support between sessions. Interacting with each adolescent on a daily basis to encourage compliance with homework assignments, evaluate progress, monitor side effects, etc., would be prohibitively expensive if clinicians were required to personally send and receive the messages themselves. Fortunately it is not necessary, given the demonstrated feasibility of automating those functions. There is a large body of literature on the efficacy of text messaging for improving heath behavior and treatment outcomes in other areas of health care (e.g., diabetes, asthma, hypertension, obesity), with positive outcomes in 93 % of the published studies [
45]. Text messaging is also used in the treatment of psychiatric and substance use disorders in adults [
46,
47]. Data on the use of SMS in the psychological treatment of youth and young adults are beginning to emerge [
3,
48‐
52]. Teens have generally reacted favorably to use of SMS technology in treatment and prevention programs, with good compliance rates [
22,
53].
In response to the National Institute of Mental Health’s call for research on the use of technology to facilitate the dissemination of evidence-based treatments [
54], we developed a technology-enhanced intervention protocol to facilitate CBT treatment of adolescent depression. The program consists of three components, each using technology for a particular purpose: (1) online therapist training, (2) in-session use of tablets for teaching clients CBT concepts and skills, and (3) text messaging for between session homework reminders and self-monitoring. These three components help disseminate training to therapists, help therapists implement CBT with patients more effectively, and improve CBT treatment outcomes, respectively. The goal of this study was to evaluate the feasibility, user satisfaction, and effectiveness of this technology-enhanced approach for treating adolescent depression.
Results
Clinicians and patients
There were no significant differences between clinicians randomized to CBT and TAU in terms of age [t(16) = 0.42, p = 0.678), gender (X2(1) = 1.90, p = 0.168], or years’ experience [t(16) = 0.10, p = 916]. There were also no significant differences between patients in the CBT and TAU arms on age [t(63) = 0.076, p = 0.940], gender (X2(1) = 0.94, p = 0.432), or baseline depression severity (QIDS-A-Pat) [t(63) = 0.27, p = 0.787).
Online tutorial
Increase in didactic knowledge
We examined changes in scores on the 49-item pre-and post-tests of knowledge of CBT concepts covered in the tutorial. A significant increase was found in the number of correct items from the pre-test (24.4, SD = 4.42) to the post-test (33.9, SD = 5.11), t(8) = 7.02, p < 0.001.
Learning objectives
Twenty-three learning objectives were identified a priori as learning goals for the online tutorial (Table
3). After completing the tutorial, 97 % of the learning objectives were rated as met. The mean rating of how much they learned as a result of taking the tutorial was 4.4 (rated on a 1–5 scale (1 = very little and 5 = a great deal).
Table 3
Learning objectives: CBT tutorial
Module 1. Theoretical principles of CBT |
Describe the core concepts behind Becks Cognitive Theory of Depression, Learned Helplessness, & Social learning |
Describe the main idea behind Social Learning Theory |
Describe the nature of therapeutic relationship in CBT |
Module 2. Explaining the nature of depression to clients |
Provide key information to clients on the nature of depression |
Module 3. Explaining treatment rationale to clients |
Explain the rationale underlying CBT treatment to clients |
Help clients identify initial treatment goals |
Explain CBT session structure and format to clients |
Module 4. Mood monitoring |
Explain the rationale for mood monitoring to clients |
Teach clients how to monitor their mood |
Develop a plan for mood monitoring for the teen to use before the next session |
Module 5. Goal setting |
Teach clients the basic principles of goal setting |
Explain the rationale for breaking down goals into sub-goals, making them specific, and attainable |
Help clients set long- and short-term treatment goals |
Module 6. Behavioral activation |
Explain the rationale for behavioral activation |
Teach clients the skill of activity scheduling |
Teach clients the skill of increasing pleasant activities |
Module 7. Problem solving |
Explain the rationale for problem solving to clients |
Describe the steps in problem solving |
Describe emotional barriers to problem solving and how to deal with them |
Module 8. Cognitive restructuring |
Explain What Automatic Thoughts are to clients (called “unhelpful thoughts”) |
Teach clients to identify their unhelpful thoughts |
Teach clients how to replace automatic thoughts with more helpful and realistic thoughts through |
Teach clients how to use the three and 5 column mood monitoring forms for identifying and challenging unhelpful thoughts |
User satisfaction: technical aspects
The means score on the SUS for the online tutorial was 78.4 (SD = 20.44) (Table
4). This corresponds to a score of good user satisfaction on the SUS. Mean global rating of user-friendliness (rated scale range from 1 (worst imaginable) to 7 (best imaginable)) was 5.6, which is halfway between “good” and “excellent”.
Table 4
System usability scale scores for the online tutorial and teaching/text system
Worst imaginable | 12.5 | |
Awful | 20.3 | |
Poor | 35.7 | |
Ok | 50.9 | |
Good | 71.4 | 78.4 (20.4)-Tutorial |
84.4 (13.8)-Teaching/text |
Excellent | 85.5 | |
Best imaginable | 90.0 | |
User satisfaction: clinical content
Descriptive statistics were obtained on user satisfaction with the online tutorial (Table
5). All that subjects agreed or strongly agreed that the material was presented in an interesting manner, was clearly presented and easy to understand, and was useful and relevant to treating adolescent depression. All would recommend the online tutorial to others.
Table 5
Mean satisfaction ratings on tutorial scale clinical content
1. The material was presented in an interesting manner | 3.6 (0.55) |
2. The concepts were clearly presented and easy to understand | 3.7 (0.49) |
3. The content was useful and relevant to treating adolescent depression | 3.7 (0.48) |
4. I would recommend this course to others | 3.7 (0.48) |
5. Overall, how satisfied were you with this module? | 3.7 (0.48) |
Online teaching materials and text messaging
User satisfaction: clinicians
The means score on the SUS for the online CBT teaching materials and text-messaging system was 84.4 (SD = 13.80). This corresponds to a score between good and excellent. Ratings on individual SUS items are presented in Table
6. The mean rating for all items was between “agree” and “strongly agree”. Clinicians found the system ‘user friendly’ in terms of understanding how to utilize the system for teaching CBT skills, setting up text messages, and receiving text reports.
Table 6
Mean ratings on system usability scale items: online teaching materials and text messaging
1. I would like to use this system frequently | 3.4 (0.90) |
2. I found the system unnecessarily complex | 3.3 (0.85) |
3. I thought the system was easy to use | 3.3 (0.85) |
4. I think I would need the support of a technical person to be able to use this system | 3.2 (0.87) |
5. I found the various functions in this system were well integrated | 3.6 (0.66) |
6. I thought there was too much inconsistency in this system | 3.5 (0.58) |
7. I would imagine that most people would learn to use this system very quickly | 3.2 (0.75) |
8. I found the system very cumbersome to use | 3.5 (0.65) |
9. I felt confident using the system | 3.5 (0.53) |
10. I needed to learn a lot of things before I could get going with this system | 3.3 (0.69) |
User satisfaction: patients
Feedback was also solicited from adolescents on how helpful the teaching and text message system was. Eighty-five percent of patients felt the teaching materials presented on the iPad during sessions were helpful in learning new skills, 90 % felt the text messages between sessions were helpful, and 95 % said reviewing their text message responses on their homework and mood at the next session with their therapist were helpful. All patients said they would be willing to use text messaging again to communicate their feelings to their clinician between sessions.
Clinical outcomes
Both treatment groups significantly improved with treatment, with mean improvements on the QIDS-A of 6.09 (SD = 4.26) and 5.73 (SD = 4.71) for the CBT and TAU groups [
t(34) = 8.453,
p < 0.001 and
t(29) = 6.67,
p < 0.001 respectively]. Clinical outcome measures comparing the CBT and TAU groups are presented in Table
7. Therapist ratings of the therapeutic alliance (TASA) were significantly higher in the CBT intervention arm than in the TAU arm,
t(131) = 4.03,
p = 0.001. Measures of symptomatic improvement were greater on all other outcome measures for the CBT arm; however, none reached statistical significance. Effect sizes (Cohen’s d) [
69] ranged from small (QIDS-A) to large (TASA).
Table 7
Clinical outcome measures: CBT vs. TAU
CBT | 6.09 | 51.4 % | 71.4 % | 62.8 | 65.3 |
TAU | 5.73 | 46.7 % | 60 % | 60.02 | 58.4 |
Diff | 0.35 | | | 2.8 | 6.9 |
P | 0.753 | 0.825 | 0.332 | 0.03 | 0.001 |
Effect size | 0.08 | 0.11 | 0.28 | 0.31 | 0.70 |
95 % Confidence interval, effect size | | −0.433, 0.6436 | −0.289, 0.852 | −0.0273, 0.0306 | 0.7002, 0.3496 |
Text messaging
A total of 9,613 text messages requiring a response were sent. Of these, 3658 (38.1 %) were responded to. The correlation between improvement on the QIDS-A and percent of texts responded to was not significant (r = 0.165, p = 0.343).
Dropout rate
Seven subjects dropped out prior to week 12 in the TAU arm, compared to 4 subjects in the CBT arm.
Discussion
Results of this study provide support for the feasibility of this technology-enhanced CBT Intervention as a means of improving CBT treatment of adolescent depression. User satisfaction, a critical component of feasibility, was high for both adolescents and therapists on all components. The program was successful in increasing therapists’ knowledge of CBT concepts and principles. Teens found the online teaching tools useful for learning CBT concepts and skills. They also found the text messaging between sessions helpful, particularly for reviewing work done between sessions with their therapist. All teens indicated they would be willing to use the system again. Rather than put a barrier between the teen and the therapist, the technology improved the therapeutic bond, a critical factor in treatment outcomes. Improving the therapeutic relationship may help to keep teens in treatment, a critical factor for successful outcomes.
From a system delivery perspective, the use of this technology-enhanced intervention is designed to augment rather than replace existing one to one clinician care. As such it is not a low intensity intervention (i.e., an intervention designed to limit therapist time) [
70]—and keeps the same number and lengths of session as usual. This approach contrasts recent “stepped-care” models of treatment, which start with the least restrictive treatment with minimal therapist support. Future research can examine the use of this (and similar) technologies within a stepped-care model. This could include examining factors such as length of treatment, use of online self-help combined with therapist and non-therapist support, both with and without text-messaging augmentation.
Effect sizes on the clinical outcomes in the current study were small to medium. According to Cohen, a small effect size is one in which there is a real effect, but can only be seen through careful study. The current study used community clinicians (vs. academic research centers) to see how well the intervention works in a sample of community therapists that not had formal training in CBT. Taken in this light, small effects are encouraging. As the training continues to be evaluated and refined, the impact of additional follow-up training, or live applied training may further improve results. Prior studies with remote CBT training found the addition of live remote observation through a videoconference of trainees conducting CBT, with immediate feedback in real time significantly improved clinical skills [
16]. The addition of this applied training component may have improved clinical outcomes. A follow-up study is underway to examine the impact of the addition of live training on post-training treatment outcomes with community patients.
The current program utilized technology to integrate three components as part of a single intervention: therapist training, client education, and treatment implementation and outcomes. As the use of technology continues to be adopted and integrated into clinical treatment, more empirical evidence will help shed light on which components are useful and under what circumstances. At a minimum, the current intervention helps address the critical shortage of training on empirically based treatments. The potential ways in which technology such as text messaging and use of interactive educational tools can enhance treatment are at the start of a new era of clinical research. New possibilities are rapidly emerging, and to some extent, are outpacing our ability to empirically evaluate these new innovations [
71]. Some recent data suggest that the explosion of mental health apps has resulted apps of poor quality, or apps that do not reflect clinical practice guidelines or evidence-based practices [
72,
73]. However, while presenting many challenges, they also present exciting opportunities. Continued research should continue to generate empirical data to help guide both clinical practice as well as future research in this area.
Authors’ contributions
JCM and KAK were responsible for developing study design, instrument development, data analysis, and manuscript development. BK provided oversight of clinical content of tutorial, provided input into study design, and manuscript development. All authors read and approved the final manuscript.