School learning modes during the COVID-19 response and pre- to during pandemic mental health changes in a prospective cohort of Canadian adolescents

The Cannabis, Obesity, Mental health, Physical activity, Alcohol, Smoking, and Sedentary behavior (COMPASS) Study is an ongoing prospective study designed to collect hierarchical survey data annually from a rolling cohort of students in grades 9 through 12 (Secondary I–V in Quebec) and their schools [29]. School boards and schools were purposefully selected based on whether they permitted active-information passive-consent parental permission protocols, which are shown to better reach students at risk of depression [30]. All students attending participating schools and not withdrawn by their parents were eligible to participate. Students could withdraw anytime. Participant codes were generated to enable individual-level data to be linked across waves [29]. Pre-pandemic student data were collected using paper-based surveys completed during one classroom period, with an average response rate of 83.4%. Since March 2020, when schools first closed due to the COVID-19 pandemic, surveys were conducted online using Qualtrics XM survey software, with an average response rate of 58.0%. A survey link was emailed to all students by their schools, followed by a reminder email one week after [31]. Additional details regarding COMPASS study methods can be found online (www.​compass.​uwaterloo.​ca) or in print [29], including online requests for the full student questionnaires. All procedures received ethics approval from the University of Waterloo (ORE#30118), Brock University (REB#18–099), CIUSSS de la Capitale-Nationale–Université Laval (#MP-13-2017-1264), and participating school boards.

We used 2-year linked COMPASS data from 8274 students attending 41 secondary schools that participated before the onset of the COVID-19 pandemic response (October 2019–February 2020; time 1), when all schools were open to in-person learning, and during the COVID-19 pandemic in the 2020–2021 school year (December 2020–May 2021; time 2). At baseline, 23,269 students participated in the 41 schools that participated in both study waves. Students with missing values on learning mode or covariates (n = 562) or on any of the three outcomes (n = 442) were removed, leaving a final analytic sample of 7270 students. An attrition analysis compared baseline characteristics of the full baseline sample with the final analytic sample. See Table 1 for pooled t-test and Cohen’s D results for mean differences in continuous outcomes and Chi-square tests and Cramer’s V results for proportion differences in categorical variables. The full baseline sample had poorer mental health scores, and the analytic sample were more likely to identify as female and white, be in earlier grades, and perceive a happy home life. However, the Cohen’s D and Cramer’s V results indicate a small magnitude of difference, with the exception of grade, as expected in a rolling cohort design; many non-linked students were in the final year of secondary school and would have graduated out of the cohort.

Depressive symptoms were measured using the 10-item Center for Epidemiologic Studies Depression scale Revised (CESD-10) [32]. Students were asked how often they experienced symptoms of major depressive disorder (e.g., “I could not get ‘going’”; “I felt everything I did was an effort”; “I felt hopeful about the future” [reverse scored]) within the last 7 days using a 4-point Likert scale (none or less than 1 day; 1–2 days; 3–2 days; 5–7 days), with items scored 0 to 3. Two positively framed items were reverse scored. Sum scores had a plausible range from 0 to 30, with higher scores indicative of more frequent symptoms. Anxiety symptoms were measured using the 7-item Generalized Anxiety Disorder scale (GAD-7) [33]. Students were asked how often they experienced each symptom (e.g., “becoming easily annoyed or irritable”; “not being able to stop or control worrying”) in the last 2 weeks on a 4-point Likert scale including “Not at all”, “Several days”, “Over half the days”, and “Nearly every day” (scored 0, 1, 2, and 3, respectively). Sum scores ranged from 0 to 21, with higher scores indicating more frequent symptoms. Psychosocial well-being was assessed using Diener’s 8-item Flourishing Scale (FS) [34]. Students were asked to indicate their agreement with statements regarding their relationships, life purpose and satisfaction, engagement and interest with daily activities, optimism, and self-esteem and competence. The original 7-point Likert scale was reduced to 5-points (“Strongly agree” to “Strongly disagree”) by removing the “Slightly agree” and “Slightly disagree” options to be suitable for a large school-based survey [35]. Sum scores ranged from 0 to 32, with higher scores indicating higher perceived well-being [36]. All three scales have demonstrated validity in adolescents [37‐42] including strict measurement invariance by gender and grade in the COMPASS study [40, 41]. Continuous scores on the CESD and GAD-7 correlate strongly with comparable measures [33, 34, 38‐40, 43‐46], and scores of 10 or higher on the CESD and GAD-7 have demonstrated specificity and sensitivity for detecting clinical depression and generalized anxiety or other anxiety disorders, respectively [33, 34, 47]. Scores on the Flourishing Scale correlate strongly with other psychological and social well-being measures [35, 48]. Internal consistency was α = 0.81, α = 0.89, and α = 0.90 for the CESD-10, GAD-7, and FS, respectively. For each scale score, person-mean imputation was used for students missing 1 or 2 items to minimize missing data [48, 49].

Schools across Canada first closed to in-person learning mid-March 2020. They remained closed for the remainder of the 2019–20 school year and implemented emergency remote learning, which varied greatly across and within schools and boards, but was minimal for many students [50]. Schools resumed in-person learning in the fall of 2020, with staggered re-openings over September 2022 in Ontario schools [51]. Localized partial school closures were again mandated in the 2020–21 academic year in areas and at times of high COVID-19 case counts, particularly in January–February 2021, during the second COVID-19 wave and the emergence of the Omicron variant, and again in May–April 2021 [50]. During closures, schools shifted to virtual remote learning, which involved a mix of synchronous and asynchronous instruction. Outside of closures, schools across provinces typically instituted cohort or “bubble” models to reduce student contact. The blended model consisted of alternating cohorts between in-person attendance and virtual remote learning, either by alternating weeks, days (i.e., every other day in many Quebec schools), blocks of days (e.g., 3 days at school, 2 days at home, etc.), or morning/afternoons. Quadmester scheduling was common, in which the school year was divided into four terms and students took two courses per term, either alternating courses weekly or two courses/day. Alternatively, some schools implemented Octomesters, in which students took one course at a time for eight terms. Additional safety protocols were initiated in schools, such as mandatory face masks and shortened lunch periods and school hours. Parents/students could also choose to remain remote when schools were open to in-person learning [51]. Students learning remotely when optional either attended virtual schools or engaged in hybrid learning, meaning educators taught students learning in-person and virtually at the same time and in the same classroom.

In 2020/21, student surveys asked what form of curriculum delivery they were currently participating in: “100% in-person (I physically go to school every day)”, “100% online (I attend school remotely over the internet every day)”, and “Alternating in-person and online (Some days I go to school, some days I attend online)”, hereby referred to as Blended. Given potential self-selection into learning mode, students in schools that were mandated to be closed to in-person learning at the time of data collection were categorized as “Virtual mandated”. Remaining students who selected 100% online were categorized as “Virtual optional”, meaning that they or their guardian(s) had opted for virtual learning when schools were open [52].

Models controlled for student-reported gender (male; female; “I describe my gender differently”/”I prefer not to say” [collapsed due to small sample size]), grade-level at follow-up (Grade 9 to 12 in Ontario, Alberta, and BC; Secondary 2 to 5 in Quebec, classified as equivalent to Grade 8 to 11), race/ancestry (Asian, Black, Latin American/Hispanic, Multiple [selected more than one response], Other [selected ‘other’ or Métis, First Nations, or Inuit; ethics restrictions precluded the identification of students with Indigenous heritage for separate study], White), household size including themselves, province, home life happiness (agreement with statement: “I have a happy home life”; responses dichotomized as Strongly Agree/Agree vs. Neutral/Disagree/Strongly Disagree), and socioeconomic status (SES). The home life happiness single item was derived from the Canadian Health Behaviour of School-aged Children (HBSC) study and added to the COMPASS student survey as a pragmatic measure to assess the home context, based on the need for brevity in a large school-based youth survey with multiple objectives, and the constraints of passive consent protocols. While passive consent protocols are advantageous in improving response rates, limiting selection bias, and reducing burden on schools, they prohibit the inclusion of items that may require identification of students (e.g., questions regarding experiences of abuse) [53]. An exploratory analysis of home life happiness in the HBSC study found associations with higher perceived health, fewer subjective health complaints, and lower depression and engagement in risky behaviors [54]. Adolescents that reported a happy home life were more likely to not live in a foster home and to eat meals as a family and have higher family support [54]. SES scores ranged from 0 to 8, with higher scores indicating lower SES, derived from six questions: “About how much money do you usually get each week to spend on yourself or to save?” (Zero-$5 = 2; $6-$100 = 1; > $100 = 0); “In your house, do you have your own bedroom?” (Yes = 0; No = 1); “Do you sometimes go to bed hungry because there is not enough money to buy food?” (Yes = 1; No = 0); “Would you say that you and your family are more or less financially comfortable than the average student in your class?” (More comfortable = 0; As comfortable = 1; Less comfortable = 2); “How true are the following statements about COVID-19 for you right now? I am worried about my family being able to pay bills and expenses” (True/Mostly True = 1; Neutral/I don’t know/Mostly False/False = 0); and “If you do not eat breakfast every day, why do you skip breakfast? There is nothing to eat at home” (Yes = 1; No = 0). SES responses were scaled for students missing 1 or 2 items; students missing 3 or more items were set to missing. While school data collection dates are kept similar each year, at an individual level, the number of days between data collection dates was subtracted from 365 days and entered into models to control for any differences in time between data collection dates among participants.

Average baseline, follow-up, and within-person change in scores were calculated for CESD-10, GAD-7, and FS by each covariate. Conditional change models (i.e., lagged dependent variable models) were used with time 2 CESD-10, GAD-7, and FS scores as the outcomes to examine learning mode as a predictor of change in mental health score. Linear mixed effects regression models with random intercept for school were used to account for the clustering survey design of students within schools. All models controlled for gender, grade, race/ancestry, household size, home life happiness, SES score, province, and difference from 365 days between data collection dates. Additionally, baseline mental health scores were included as a confounding predictor of change scores [54, 55]. Learning mode interactions with gender and perceived home life happiness were tested based on hypothesized moderation effects and statistical significance of main effects of these two covariates. Note that no main effects were found for household size or race/ethnicity. Least squares means estimates for student learning mode were calculated to allow comparison of adjusted changes in mental health outcomes across learning mode interaction groups. SAS 9.4 was used for all analyses and PROC GLIMMIX was used for the linear mixed effects models.

Table 2 presents descriptive statistics of the sample. Over half (56.5%) of the sample identified as female, while 1.8% reported describing their gender differently or preferred not to say. Most participants identified as White (78.0%) and attended schools in Quebec (52.8%) or Ontario (31.1%). About 80% perceived having a happy home life. Students were distributed across the four learning mode categories; virtual mandated was the most common mode (28.5%), blended was the least common (22.3%), and in-person (25.2%) and virtual optional (24.0%) modes each accounted for about one-quarter of the sample.

Table 2 also presents mean scores for the three mental health outcomes at time 1 and 2 by learning mode and all covariates. Mental health mean scores were lower in time 2 than time 1. There was less of a difference in all three mean unadjusted mental health outcome scores pre- to post-COVID-19 onset among students attending school fully in-person in comparison to the score differences among students in the other learning mode categories, while students attending virtually when mandated had relatively greater differences in depression scores across waves. Males had less of a difference across time 1 and 2 in unadjusted mental health outcomes relative to females and students that describe their gender differently or preferred not to say. Students in earlier grades had greater differences in scores relative to students in later grades. The largest differences in mental health scores across waves were among students that did not perceive their home life to be happy, in comparison to students with home life happiness and in all other subgroups examined.

Table 3 shows results of the main effects conditional change models; mental health outcomes from before to during the COVID-19 response were regressed on learning mode, controlling for the corresponding pre-pandemic mental health outcome, covariates, and school clustering. A main effect for learning mode indicated that students learning in a blended mode had a marginally greater increase in anxiety scores (Est. = 0.45, 95% CI [0.10, 0.81]) relative to students learning in-person. No main effect for learning mode resulted for depression or psychosocial wellbeing. Students that identified as female, described their gender differently or preferred not to say, and did not perceive their home life to be happy had greater declines in all three mental health outcomes from before to during the pandemic response, relative to their peers identifying as male and with happy home lives, respectively.

Table 4 shows results of the conditional change models with the interaction terms added. Virtual learning when optional was associated with greater increases in depression (Est. = 1.28, 95% CI [0.60, 1.95]) and anxiety (Est. 0.79, 95% CI [0.18, 1.39]) in females, and greater declines in psychosocial wellbeing in students without home life happiness (Est. =  – 1.35, 95% CI [ – 2.07,  – 0.62]), relative to males and students with perceived happy home lives that were learning fully in-person, respectively. Virtual learning when mandated was also associated with greater increases in depression in females relative to males learning in-person (Est. = 0.75, 95% CI [0.10, 1.40]); whereas, students without home life happiness that were learning virtually when mandated had less of an expected increase in depression symptoms relative to their peers with home life happiness and learning in person (Est. =  – 1.03, 95% CI [ – 1.80,  – 0.25]).

Least square means estimates are presented in the Supplementary Table S1. Figure 1 displays the least squares means estimates for changes in mental health outcomes by learning mode and gender (Fig. 1a) and learning mode and perceived home life happiness (Fig. 1b) before and during the COVID-19 response. Females learning fully in-person during the pandemic year had marginally less of an increase in depression scores relative to females learning in other modes, whereas males learning virtually when optional had less of an increase in depression than males in other learning modes. As above, students learning in a blended mode had greater increases in anxiety that their peers in other learning modes, regardless of gender and home life happiness. In students without happy home lives, learning virtually by choice was associated with greater declines in psychosocial wellbeing relative to other learning modes. Caution is advised in interpreting differences among students identifying their gender differently given the large confidence intervals and smaller cell sizes.