Psychometric properties of the Shirom-Melamed Burnout Measure (SMBM) among adolescents: results from three cross-sectional studies

Burnout is a complex phenomenon with more than 40 years of empirical research [1]. Burnout was initially defined as a work-related syndrome that gradually develops when people are exposed to chronic emotional and interpersonal stress at work [2]. While researchers generally agree that burnout is a multidimensional construct [3‐5], different conceptualizations of burnout have been proposed by different groups of researchers [6]. Drawing on Hobfoll’s Conservation of Resources (COR) theory, Melamed et al. [7] argued that the central characteristics of the burnout construct are emotional exhaustion, physical fatigue, and cognitive weariness.

Based on Shirom, Melamed et al.’s [8, 9] definition of burnout, researchers have shown that burnout has an impact on both physical and mental health outcomes: For instance, burnout was found to be a risk factor for increased total cholesterol, low-density lipoprotein cholesterol and triglyceride levels [7, 10‐12], increased fasting glucose and risk of developing type 2 diabetes [13, 14], increased inflammatory markers [12, 15], increased leukocyte adhesiveness [16], increased diurnal cortisol levels [17], elevated cortisol response after awakening [18], increased risk of musculoskeletal pain [19], and a higher likelihood of fertility problems [20]. As regards psychological dimensions, data revealed that burnout symptoms constitute a risk for poor life satisfaction and quality of sleep [17, 21]. Similarly, significant relationships were observed between burnout and depression, with varying degrees of overlap [15, 22‐24].

While the above findings underscore that burnout symptoms are a cause of concern from a public health perspective [25], most of the existing evidence is based on working adults and therefore cannot be generalized to young people. Nevertheless, several researchers have claimed that the concept of burnout is pertinent beyond the occupational context and also concerns student populations [26, 27]. More specifically, Salmela et al. [27] have argued that school-aged adolescents and university students – by attending classes, completing assignments, taking examinations, and acquiring a degree – also execute work. As a consequence, researchers have developed instruments specifically designed for young people. For instance, based on the Maslach Burnout Inventory (MBI) [4], Schaufeli et al. [26] have created a 15-item instrument for university students, which consists of three dimensions: exhaustion, cynicism, and professional efficacy. This instrument was well received by the scientific community, and has stimulated a large number of studies to examine the associations with academic performance and various health-related outcomes [28‐34]. Some years later, Salmela-Aro and Näätänen [35] developed an instrument to assess burnout among school-aged children, the School Burnout Inventory (SBI). On the basis of the Bergen Burnout Indicator 15 (BBI-15) [36, 37], burnout was conceptualized as a three-dimensional construct consisting of the following subdomains: exhaustion at school (four items), cynicism toward the meaning of school (three items), and sense of inadequacy at school (three items). Evidence supported the validity and reliability of the SBI [27], and based on this tool, researchers were able to gain further insights into risk factors [38‐44], time courses [42, 45], and health consequences associated with school burnout [46‐48] among adolescents.

In summary, it can be concluded that research on burnout among school-aged adolescents has increased significantly in the last ten years. However, it is also obvious that the existing research has been dominated by the SBI. While this instrument has sound psychometric properties [27], the school-specific conceptualization of the construct makes it difficult to compare the SBI scores with those of adult populations, and complicates research focusing on the transition from school to working life. As highlighted by Walburg [49] in her review on adolescent school burnout, only a few alternative instruments have been applied in this age group. Nevertheless, some researchers have used the Shirom Melamed Burnout Measures (SMBM) [7, 16] as an alternative to assess burnout symptoms in adolescent students. In these studies, higher scores on the SMBM were associated with more depressive symptoms, more sleep problems, lower life satisfaction, and poorer quality of sleep [50]. Moreover, adolescents who accomplished recommended levels of physical activity reported lower SMBM scores [51], and the relationship between adolescents’ stress and burnout scores was moderated by their levels of mental toughness [52].

Although the SMBM seems to have satisfactory internal consistency among adolescents [50‐52], the validity and reliability of the SMBM have not yet been examined systematically in young people. Compared to the SBI, the advantage of the SMBM is that the measure is rooted in Hobfoll’s Conservation of Resources theory [53], and thus has a clear theoretical background. Moreover, the items of the SMBM are context unspecific, and thus allow a comparison between varied samples of adolescents. Finally, the SMBM allows a comparison between burnout symptoms of adolescents and adults, is well suited to examining transitions from adolescence to adulthood, and provides a cut-off pointing towards clinically relevant levels of burnout [54]. Therefore, the purpose of the present study was to validate the SMBM in three samples of adolescents attending different types of public schools in Switzerland. We claim that these analyses are warranted because the SMBM has been widely used in (adult) burnout research during the last 25 years [24], and because it is time to find out whether this instrument is also suitable for younger people.

In the present article, six hypotheses will be tested: First, we expect that the SMBM will produce adequate internal consistency across all student samples. Thus, we expect inter-item correlations of ≥ .20, Cronbach’s alpha coefficients of ≥ .70, and item-total correlations of ≥ .30 [16, 55, 56]. Second, regarding factorial validity, we expect that adequate model fit will be found for a three-factor model [55, 56]. We also expect that adequate model fit will occur for a first- and second-order model [55]. More specifically, with reference to the standards defined by Comrey et al. [57] and based on previous findings [55, 56], we expect very good factor loadings (≥ .63) across all items on the corresponding factors. Third, we expected to find adequate convergent validity. That is, we hypothesize that the SMBM subscales and the SMBM overall index will be moderately to strongly (and positively) correlated with perceived stress [58‐60] and the School Burnout Inventory (SBI) [51],¹ while we expect a moderate (negative) relationship between the SMBM and adolescents’ life satisfaction [61]. Fourth, as an indication of discriminant validity, we assume that only moderate (positive) correlations will exist between the SMBM and self-reported depressive symptoms [24, 61]. Fifth, we expect that girls will score higher on the SMBM than boys [59, 61, 62]. Sixth, we expect the measurement model to be invariant across samples.

The key finding of the present article is that among adolescents the SMBM has excellent psychometric properties and acceptable convergent/discriminant validity and can therefore be used as an alternative screening instrument in adolescent samples. Finally, our data confirm that clinically relevant burnout symptoms may already occur at young age.

In the introduction section, we have proposed six hypotheses, which we will now discuss in detail. First, we expected that the SMBM would have acceptable internal consistency across all samples [16, 55, 56]. This assumption was supported, with all Cronbach’s alpha coefficients exceeding the critical value of ≥ .70. Moreover, without exception, all inter-item correlations within the respective factor were ≥ .20, and all item-total correlations were ≥ .40. Based on the standards suggested by West et al. [76], we further found that the skewness (< 2) and kurtosis (< 7) of all SMBM indices were in the acceptable range.

Second, using CFA, evidence was found for the factorial validity of the SMBM across all our adolescent populations. Thus, our assumption that a three-factorial model would produce adequate fit was confirmed. In accord with prior research [55], almost all factor loadings were very good or excellent in our sample. The first-order model revealed moderate-to-strong correlations between the three latent factors, which is in line with previous research (cp. [6, 56, 62]). Shirom and Melamed [6] argued that energetic resources are individually possessed and expected to be closely interrelated, with deficits in one resource often leading to a deficit in other resource. Therefore, moderate-to-strong correlations between the latent factors were expected. Finally, in support of findings from data of Canadian workers [55], a second-order model produced equally good model fit, which corroborates the idea that the SMBM overall index can be used as a global/general measure of burnout.

Third, our analyses support the convergent validity of the SMBM across all samples. As reported in prior investigations with adult populations [58‐60], our data suggest that the SMBM indices correlated at least moderately and positively with measures of self-perceived stress. Finally, across all samples moderate negative correlations were found for most of the SMBM indices and adolescents’ satisfaction with life, which is consistent with a previous study with Swiss vocational students [50].

Fourth, our findings support the discriminant validity of the SMBM. While our findings corroborate previous studies showing that the SMBM measures are at least moderately correlated with depressive symptoms [22, 24, 61], the strength of the correlations varied across our three study populations (r = .39 to .51), with the highest correlations found in high school students. While we acknowledge that there is a certain overlap between these two constructs, the fact that the SMBM overall index only shared between 15 and 26% of variance indicates that symptoms of burnout and depression are far from being identical constructs among adolescents. As suggested by Melamed et al. [9], some degree of overlap between burnout and depression is expected because both the definition of burnout and depression include fatigue and loss of energy as characterizing criteria. Schonfeld and Bianchi [22], however, recently argued that past research might have underestimated the overlap between burnout and depression. In their study with 1386 teachers, for instance, they reported a correlation of r = .77 (p < .001) between the SMBM and the PHQ-9. Moreover, 86% of the teachers identified as burned out met criteria for provisional diagnoses of depression. Nevertheless, this finding needs to be interpreted with caution because Schonfeld and Bianchi [22] used an arbitrary cut-off to classify participants into groups with versus without burnout (≥ 5.50), which was considerably higher than the empirically validated threshold suggested by Lundgren-Nilsson and colleagues [54]. We are aware that Lundgren-Nilsson et al.’s cut-off is based on the SMBQ (Shirom-Melamed Burnout Questionnaire = a slightly extended version of the SMBM) and therefore might not be perfectly applicable for the SMBM (despite a strong overlap between the items of the two instruments). Nevertheless, we still prefer this empirically derived threshold in comparison to any arbitrary cut-off value. In summary, we believe that our findings corroborate the discriminant validity of the SMBM because the percentage of adolescents who were simultaneously classified into the group with high burnout levels and high levels of depressive symptoms was not too high, with 20% among vocational students, 26% among young elite athletes, and 53% among high school students.

Fifth, our assumption that female participants would score higher on burnout symptoms than male participants was partly confirmed. While significant gender differences were found in high school students and young elite athletes, boys and girls did not differ among vocational students. The inconsistent pattern of finding is in line with previous research. Thus, while studies with adolescents mostly supported gender differences [27, 45], among adult workers, male and female participants did not always differ from each other [59, 62]. In the present study, it is possible that the male and female vocational students did not differ from each other because they generally reported lower burnout symptoms than high school students or young elite athletes. Accordingly, floor effects might have decreased the odds for higher variances and for detecting a higher variability in the measurements. The fact that vocational students had the lowest SMBM scores was a surprise because it has been suggested that the vocational students are exposed to workloads similar to adult workers and are therefore at risk for elevated stress levels [51]. However, the vocational students participating in the present study were in their first year. Thus, it is likely that burnout symptoms will increase in the third or fourth year of vocational education and training. Finally, in line with previous studies [55, 62], physical fatigue turned out to be the most affected burnout dimension across all three samples, followed by cognitive weariness and emotional exhaustion.

Sixth, our data supported the notion that the measurement model is invariant across samples. A multiple group comparison showed that the overall model fit is excellent. Setting factor loadings and inter-factor correlations/regression weights equal across samples did not have a negative impact on the goodness-of-fit indices. Thus, we claim that the SMBM seems to perform equally well in various adolescent samples.

Despite the novelty of our findings, several limitations should be taken into consideration: For instance, due to the cross-sectional nature of our data, it was not possible to test predictive validity and test-retest reliability. Moreover, all three samples consisted of non-clinical populations and we only used measures of self-reported depressive symptoms to examine discriminant validity. Thus, without formal clinical diagnoses, we were not able to validate the cut-off for clinically relevant burnout (≥ 4.4), which was previously established by Lundgren-Nilsson et al. [54].

Implications for clinical practice are that 7–12% of adolescents reported symptoms pointing towards clinically relevant levels of burnout. This percentage is comparable to adult populations [23, 59] and shows that more systematic efforts are needed to prevent burnout symptoms. As known from adult studies, burnout symptoms have a relatively high temporal stability, and may track across extended periods of time [77, 78]. In future research, researchers could test strategies to empower students to maintain a better stress-recovery balance [79].

The SMBM is a frequently used tool in international burnout research. Our study reveals that this instrument has adequate psychometric properties and satisfactory convergent and factorial validity among adolescents. Hence, the SMBM can be used among young people to gather information for screening and treatment planning, which may be of particular interest during the transition from school to working life. There is a clear need to validate the cut-off score for clinically relevant burnout, in order to more accurately judge the clinical significance of the symptoms in early screening.