A pilot and feasibility study of a cognitive behavioural therapy-based anxiety prevention programme for junior high school students in Japan: a quasi-experimental study

This study was conducted in collaboration with Chiba University and Kodomo Minna Project (‘Project for all the children’). This is a project in which ten universities collaborated and conducted a research, commissioned by the Ministry of Education, Culture, Sports, Science and Technology, for the purpose of improving school refusal and bullying, which are major issues in Japanese schools.

This is part of a research project on students from elementary to high school. In this study, data on junior high school students were collected and analysed. The Ministry of Education, Culture, Sports, Science and Technology recruited schools to participate in this programme. The Board of Education of a prefecture located in the western part of Japan applied to participate, and students in their first year of junior high school were selected to participate in the programme. Although it would have been desirable methodologically to recruit a control group from students in the same year, the Board of Education made a firm request for all first-year students in the participating schools to receive the programme at the same time; therefore, students in their second year of junior high school were recruited for the control group.

This was a universal quasi-experimental study with an intervention and a control group. The participants in the study were 472 students in their first or second year of junior high school (aged 12–14 years), attending three public junior high schools in a single prefecture in Japan. Intervention group participants received the anxiety prevention programme, and control group participants received no prevention programme.

In addition, the ‘Journey of the Brave’ programme was conducted as part of regular classes in schools. This study was a prospective observational study that collected and analysed students’ anxiety scores before and after the programme. It was approved by the Chiba University Review Board. In this study, consent was obtained in the form of an opt-out. Parents were given an informational letter about the study, and they could provide opt-out consent to exclude their child from participation. In addition, at the time of the survey, teachers distributed a written assent form for the students, for students to provide their assent to participate.

Table 1 provides a summary of the ‘Journey of the Brave’ programme. This is a programme developed with consideration for the psychological characteristics of children and adolescents and for the social and cultural background of Japan, with the following three representative features [31]. First, this programme specialises in the prevention of anxiety-related problems, to help children and adolescents understand the purpose of the programme and engage in effective learning. Second, in order to enable children and adolescents to enjoy the programme, likeable characters are presented in a story format. Third, group work is intentionally avoided in favour of emphasising an individual work format because of the psychological characteristics of Japanese adolescents. It has been pointed out that compared to individuals in Western countries, Japanese individuals tend to be more influenced by the way they are perceived by others [34]. Adolescents tend to feel more anxious about the relationships within the same age group [35], and it is necessary to consider that there may be some students with high anxiety in the class.

This programme consists of ten 45-min sessions; the content is taught according to a workbook and a teacher’s manual. The first half of the programme is dedicated to the development of ‘anxiety hierarchy’ and the experience of gradual exposure, while the second half mainly concerns cognitive restructuring. More precisely, after psychological education on anxious feelings (i.e., the notion that anxiety is a natural feeling that everybody has and plays an important role in protecting you from danger, but if excessive anxiety persists, it might lead to disturbances in life, etc.), each student is encouraged to establish his or her own goal for the programme, such as giving a presentation in front of all the students, an important test, and so on. In stage 3, relaxation skills such as breathing methods and muscle relaxation are taught. In stage 4, students develop a table of their ‘anxiety hierarchy’, consisting of 7 steps that will allow them to reach the goal set in stage 2. Stages 5, 6, and 7 encompass the process of gradually learning about the cognitive model (the relationship between cognition, behaviour, emotion, and bodily responses) as well as cognitive restructuring. At the same time, gradual exposure homework is given to address higher levels of anxiety in accordance with the anxiety stairs table developed in stage 4. Assertion skills to reduce interpersonal anxiety are taught in stage 8; stage 9 consists of an overall review session; and stage 10 involves a summary and graduation ceremony. In the workbook used by the students, realistic examples of many anxiety-provoking moments in their daily lives are provided so that they can deepen their understanding of anxious feelings and CBT.

The original ‘Journey of the Brave’ programme consisted of 10 sessions (administered once per week, each lasting 45 min). As this study conducted the programme in junior high schools, the research group elected to reduce the number of sessions in view of the fact that the length of class time was 5 min longer than in elementary school, and that junior high school students should be able learn more quickly. In addition, since the curriculum of the regular classes for the year has already been determined, the Board of Education requested that the number of classes be reduced to seven that were administered about once per week and lasting 50 min.

In this programme, the content of each session was based on CBT theory (Table 1), but the relaxation method (Stage 3) could be shortened as it was addressed in health class, and Stages 2 and 3 were consolidated into one session. The remaining content was implemented within the 7 class hours. As Stages 5 and 6 as well as Stages 9 and 10 had little individual work for students, we decided to summarize these in one session.

Additionally, the following three things were addressed as we utilized a group of practitioners who did not have specialized knowledge about CBT to allow them to lead this programme smoothly and effectively. First, we conducted a 6-h workshop, which was a training course. This training course was a free workshop, and participants received a certificate of completion. This workshop consists of lectures on the theory of CBT, role-plays for each session (lasting about 20 min per session), feedback from instructors, and time for questions and answers. Second, we devised a workbook with detailed contents that allowed the students to read and understand it themselves. Third, we had them utilize a teacher’s manual, which was distributed to the teachers. The teacher’s manual was attached with the Q & A and information on how to proceed with the class, which was created based on questions by teachers in past programmes. In addition, after the completion of stage 3, a template for reporting the progress of the class was attached to the teacher’s manual. In the report template, there is a field for comments and consultations for supervision. In addition, if the teachers wanted to have a consultation, they could do so at any time by phone or email during the intervention period. This was described in the manual and shared with the teachers at the workshop.

The preventive interventions were conducted from September to November 2017 in one participating school and from October to December 2017 in the other. In each case, the intervention was delivered by the class teacher, who had taken the ‘Journey of the Brave’ programme instructor training course. In total, the programme was implemented by the class teacher in six classes of two junior high schools.

All sessions were held in the classroom during regular class time. Every session was conducted according to the workbook and the teacher’s manual, and a piece of homework was to be assigned at the end of each session, to be worked on at home and returned by the next session, in order to help students consolidate the content. Students in the control group followed the regular school curriculum. The main assessments were a pre-test (Time 1; baseline), a post-test (Time 2; 2–3 months after baseline), and a follow-up test (Time 3; 2–3 months after the post-test). At each of these time points, self-report questionnaires were distributed to the students by the teacher in charge of each class, and all students (149 in the intervention group and 89 in the control group) completed the questionnaires. The teachers assisted students in this process by reading the questions aloud.

Students were asked to evaluate the programme after completing all seven sessions. An evaluation form was used to measure their acceptance of and satisfaction with the programme. The form comprised the following two sections: (1) the student’s evaluations of the content of the programme (5 items; for example, ‘Do you think that this programme helped you to cope well with your feelings of anxiety?’ with each item scored from 0 = disagree to 3 = agree; see Additional file 1: Table S1) and (2) the student’s accomplishment of their ‘anxiety hierarchy’ task (scored from 0 = none to 3 = complete).

For baseline variables, summary statistics are presented in the form of frequencies and proportions for categorical data, and means and SDs for continuous variables.

Analysis of the primary outcome measure consisted of a mixed-effects model for repeated measures (MMRM), with intervention group, time, and the interaction between intervention group and time as fixed effects; an unstructured covariate was used to model the covariance of within-participant variability. MMRM analysis assumes that any missing data occur randomly. Analysis of the secondary outcome measure was performed in the same manner. We also conducted subgroup analysis by comparing the intervention and control groups on their SCAS scores in a high-anxiety subgroup (SCAS score of 35 points or above in the pre-test) and a low-anxiety subgroup (SCAS score below 35 in the pre-test). Subgroup analysis was also performed in the same manner.

Additionally, the responses to the students’ evaluation questionnaires were aggregated. A repeated-measures analysis of variance (ANOVA) was conducted to examine the changes in SCAS scores at each time point according to the students’ responses regarding the extent to which they had accomplished their ‘anxiety hierarchy’ task (0 = none to 3 = complete).

All comparisons were planned and all p values reported are two-tailed. A p value < .05 was considered to represent statistical significance. All statistical analyses were performed using the SAS software program, version 9.4 (SAS Institute, Cary, NC, U.S.A.), and SPSS Version 24.0 (IBM, Armonk, New York, USA).

Additional file 1: Table S1 presents the number and percentage of respondents giving each response to each item on the programme efficacy section of the evaluation questionnaire.

According to the repeated-measures ANOVA to examine SCAS scores at each time point based on students’ responses regarding the extent to which they had accomplished their ‘anxiety hierarchy’ task (Table 5), there was a no significant interaction effect between group and time (p = .85).

The results for anxiety are in contrast to those of the original study of this preventive intervention on elementary school students [32], in which a significant reduction in the anxiety scores of the intervention group was observed. We consider two possible reasons for the absence of a significant reduction in anxiety scores in the present study. The first reason is that this programme was facilitated by classroom teachers with limited expertise for CBT, whereas in the original study, the programme was conducted by trained health facilitators. The second is that the environmental surroundings of junior high school students differ greatly from those of elementary school students, and the former group are at a sensitive and difficult developmental stage compared to elementary school students.

In the previous study with elementary school students, the programme was delivered mainly by trained health facilitators, but in the present study, the intervention was delivered by teachers. In a UK-based study of the effect of universal anxiety prevention programmes in schools, it has been reported that intervention by trained health facilitators is effective, but that teacher-led intervention may not be effective [25]. In this study, when classroom teachers acting as facilitators were asked about the amount of homework assigned, they reported that homework assignment and review was not practiced regularly at the two schools. Homework is considered one of the most important therapeutic components of CBT [41]. In CBT, the ultimate goal is for clients to be able to exercise control over their own emotions and behaviours, and the practice provided by homework is useful in establishing knowledge and skills, making use of them in daily life (generalisation), and improving self-efficacy. Previous studies in which this programme has been implemented have also shown that ongoing provision and review of homework helps students to consolidate their knowledge and change their behaviour [32]. Since it can be presumed that the facilitator’s level of expertise in CBT is particularly influential with regard to homework assignment and students’ accomplishments with gradual exposure (reported in the present study as part of the students’ programme evaluation questionnaires), it is possible that differences in the expertise of facilitators may have caused the disparity in effects between the original and the present study. In mental health interventions delivered by lay counsellors, supervision has been shown to be important in managing programme fidelity [42, 43]; therefore, it will help the classes progress more effectively if the supervision of the teachers who are leading the sessions can be enriched. In this study, there were no telephone or email consultation requests from the teachers. In addition, in the report template, the teacher reported the completion of stage 3 and the future schedule of the class at the midpoint of this programme. There was a section where comments and consultations from teachers were entered into this report template, but there were only comments on the programme and impressions about the class overall, and no records of consultations. Therefore, for supervision, it will be necessary to improve the report format so that teachers can easily complete assignments and consultations. Furthermore, in future implementation, in addition to using the report template, it will be important to set a time for conducting supervision sessions in advance.

In addition, the workbook used in this programme seems to be appropriate, because it deals with themes that are likely to present issues during adolescence, such as anxiety in interpersonal relationships, but it is possible that the content might not have been suitable for the developmental stage of junior high school students. Feedback from teachers who had been involved in delivering this programme was collected at the end of the intervention, and some teachers mentioned that ‘the illustrations may be too childish for the students’ and ‘some examples of anxiety scenarios don’t match the students’ level of development’. We propose that a future task should be to improve the content of the workbook so that it matches the developmental stage of junior high school students.

Furthermore, in the present study, the number of sessions was reduced from 10 to seven in view of the fact that junior high school students have a higher level of understanding than elementary school students. However, a meta-analysis of research on universal school-based preventive interventions [44] shows that the greater the number of sessions, the larger the effect; thus, it is probable that the negative outcome in the present study may be partially attributable to this reduction of the number of sessions.

We believe that the factors discussed above greatly influenced the students’ motivation for learning during this programme. Therefore, it will be necessary to revise the contents of the programme further, based on the developmental stage of junior high school students and taking into account the evaluations provided by participants in the programme, and to deliver the full 10 sessions in future administrations of the preventive intervention.

A final point to consider is that, in general, it is desirable for participants in both the intervention and control groups to have comparable scores on the outcome measure at baseline; however, in this study, the anxiety scale (SCAS) scores significantly differed between the groups. The participants in this study were recruited from the first year (intervention group) and second year (control group) of junior high school. The first year of junior high school in Japan is a year in which students experience major changes in their educational environment. Research has reported that school refusal and the number of students whose study motivation declines is increasing rapidly [45]. It is estimated that the first year of junior high school is a time when anxiety greatly increases compared to other grades, and the difference between the groups in this study is possibly attributable to the fact that the groups were drawn from different academic years. Additionally, the small number of participants in this study might have influenced this result. The results of the original study (2018) revealed that the smaller the number of participants, the greater the difference in baseline scores between the intervention and control groups. In the future, we plan to verify the efficacy of the programme by recruiting an appropriate number of participants from the same academic year and who have comparable mean total scores on the anxiety scale (SCAS).

Based on the results of the questionnaire items in which students were asked to evaluate the efficacy of the programme (item 3: ‘Do you think that this programme helped you to cope well with your feelings of anxiety?’ and item 5: ‘Do you think that what you learned in this programme will be useful in the future?’), more than 70–80% of the students answered in the affirmative. One of the advantages of implementing a universal prevention programme in schools is the prevention of potential future deterioration of the mental health of students who do not present any symptoms or signs at the time of the programme, and the reduction in the risk of other comorbidities. Although no significant reduction in participants’ SCAS scores was observed on this occasion, we conclude that the delivery of this programme is useful in allowing participants to acquire knowledge and skills regarding how to manage their anxiety, and these techniques can be used to exercise control of their own emotions and behaviours in their future lives. By implementing this universal intervention programme for anxiety prevention in schools, students might acquire the knowledge and skills based on CBT and apply them to prevent mental health deterioration in the future. Therefore, longitudinal studies must be conducted to verify the long-term efficacy of universal preventive interventions [46]; doing so for the programme implemented here, through a follow-up assessment, is a future task.

Furthermore, the results indicated that there was no significant difference in SCAS scores at each time based on students’ responses regarding the extent to which they had accomplished their ‘anxiety hierarchy’ task (0 = none to 3 = complete). However, looking at the change in the score at each stage, we found that the students who reported positive progress in their responses to the item on the extent to which they were able to accomplish their anxiety hierarchy gradual exposure task also exhibited a decrease in SCAS scores at the post-test and follow-up test. In contrast, the scores of students who reported that they had not been able to complete any of the steps toward their task were reduced in the post-test, but subsequently increased again in the follow-up test.

As a second point, when examining total scores in the pre-test, we noticed that the higher the participant’s anxiety score, the lesser the extent to which they were able to accomplish their anxiety hierarchy task. The results of many tests of CBT treatments for anxiety problems in children and adolescents have shown that success with exposure therapy is important to alleviate anxiety [47], but the present study indicated that participants’ degree of exposure achievement was lower among students with higher anxiety scores. Therefore, it is conceivable that students with higher anxiety scores may not have been able to set feasible targets that matched their anxiety level (meaning that it was difficult for them to accomplish the exposure task in their daily lives). In the future, it may be necessary to improve the programme workbook, especially in relation to how to set a reasonable goal so that students can select achievable targets that match their individual capacities in class. Assistance for students with high anxiety who experience difficulty with gradual exposure will also lead to the provision of early intervention and support at school, which will be very helpful to such students.

There were several methodological problems and limitations with the present study, as follows. First, because this was a pilot study, the number of participants may have been insufficient. The study enabled the calculation of sample size to detect clinically significant differences in outcome measures. Using the PS Power and Sample Size Calculator Software version 3.1.2 with α equivalent to .05 and power (1−β) of .80, the required sample size for this type of research was found to be 200 participants each for the intervention and control groups [48]. Additionally, in this study, the anxiety scale (SCAS) scores differed significantly between the intervention and control groups, possibly due to differences in grade between the students in these groups. In the future, we aim to verify the efficacy of the programme by recruiting intervention and control groups with an appropriate number of participants from the same academic year.

Next, according to systematic reviews and meta-analyses of school-based anxiety and depression prevention programmes, the effect size of such preventive programmes is small, but it has been indicated that, even with a small effect size, there is a possibility that it can be useful for preventing the onset of these disorders in youth [19]. Additionally, research reports that young people (aged 7–14 years) with anxiety commonly worry about how others perceive them, and thus tend to give socially desirable responses instead of providing valid self-report [49]. In the future, in order to evaluate the effects of preventive programmes, it will be necessary not only to evaluate efficacy using questionnaires (i.e., self-report), but also to design a long-term study in which a follow-up study of participants’ changes in anxiety score and the number of school refusals is conducted.