Sample 2
Sample 2 dataset includes 544 subjects with age ranging from 18 to 72 years, mean age = 32.9, SD = 10.4. There are slightly more females in the sample (n = 308, 56.6%) compared to males (n = 236, 43.4%). In terms of education level 386 (71%) have a bachelor’s degree or above. Furthermore, 189 (34.7%) specialization or working in health related fields, 62 (11,4%) previously diagnosed with mental health condition, and 105 (19.5%) living with a diagnosed relative with mental health condition.
All participants answered all 35 items in the questionnaire, therefore there is no missing data.
The mean score was 115.5 (Standard deviation (SD): 15.2; Range: 66–157; median: 117) out of 160 total possible score (Skewness: -0.29, Kurtosis: -0.05).
Internal reliability of the tool was first checked assuming unidimensionality. Cronbach’s α (0.850) and McDonald’s ω (0.863) values are above 0.80, suggesting very good reliability. This indicates the MHLS tool is possibly unidimensional.
Confirmatory factor analysis (CFA) was further used to test the hypothesized unidimenstionality (Table
1). One-factor CFA model shows the standardized loading estimates to be statistically significant for all except for three items (#12, 15, 23). Standardized loading values range from − 0.17 (item #20) to 0.65 (item #5 and 6), but the average is fairly low (0.37). Only 10 out of 35 items have standardized loading exceeding the suggested threshold of 0.5. These results suggest that it is unlikely that all items in the scale measure the same latent construct.
Table 1
Confirmatory factor analysis (CFA)
Factor 1 | Item_1 | 0.4296 | 0.0322 | 13.321 | < 0.001 | 0.5541 |
| Item_2 | 0.4833 | 0.0322 | 15.022 | < 0.001 | 0.6119 |
| Item_3 | 0.4445 | 0.0339 | 13.111 | < 0.001 | 0.5476 |
| Item_4 | 0.4376 | 0.0278 | 15.738 | < 0.001 | 0.6356 |
| Item_5 | 0.5014 | 0.0311 | 16.139 | < 0.001 | 0.6484 |
| Item_6 | 0.5034 | 0.0309 | 16.277 | < 0.001 | 0.6509 |
| Item_7 | 0.4715 | 0.0293 | 16.082 | < 0.001 | 0.6497 |
| Item_8 | 0.3701 | 0.0272 | 13.594 | < 0.001 | 0.5671 |
| Item_9 | 0.3147 | 0.0335 | 9.401 | < 0.001 | 0.4077 |
| Item_10 | 0.2088 | 0.0358 | 5.834 | < 0.001 | 0.2602 |
| Item_11 | 0.2825 | 0.0313 | 9.016 | < 0.001 | 0.3948 |
| Item_12 | -0.0375 | 0.0424 | -0.886 | < 0.001 | -0.0404 |
| Item_13 | 0.4135 | 0.0322 | 12.853 | 0.376 | 0.5390 |
| Item_14 | 0.3536 | 0.0377 | 9.388 | < 0.001 | 0.4078 |
| Item_15 | -0.0194 | 0.0439 | -0.441 | 0.659 | -0.0201 |
| Item_16 | 0.6478 | 0.0506 | 12.808 | < 0.001 | 0.5401 |
| Item_17 | 0.5455 | 0.0531 | 10.280 | < 0.001 | 0.4473 |
| Item_18 | 0.4542 | 0.0560 | 8.113 | < 0.001 | 0.3583 |
| Item_19 | 0.5227 | 0.0523 | 9.987 | < 0.001 | 0.4353 |
| Item_20 | -0.2039 | 0.0545 | -3.743 | < 0.001 | -0.1687 |
| Item_21 | 0.3625 | 0.0554 | 6.548 | < 0.001 | 0.2921 |
| Item_22 | 0.5194 | 0.0495 | 10.502 | < 0.001 | 0.4509 |
| Item_23 | 0.0765 | 0.0551 | 1.390 | 0.165 | 0.0637 |
| Item_24 | 0.4171 | 0.0579 | 7.204 | < 0.001 | 0.3218 |
| Item_25 | 0.2873 | 0.0613 | 4.685 | < 0.001 | 0.2111 |
| Item_26 | 0.4164 | 0.0516 | 8.062 | < 0.001 | 0.3566 |
| Item_27 | 0.4111 | 0.0574 | 7.167 | < 0.001 | 0.3186 |
| Item_28 | 0.4454 | 0.0527 | 8.449 | < 0.001 | 0.3712 |
| Item_29 | 0.2614 | 0.0556 | 4.704 | < 0.001 | 0.2168 |
| Item_30 | 0.4158 | 0.0570 | 7.293 | < 0.001 | 0.3322 |
| Item_31 | 0.3505 | 0.0565 | 6.198 | < 0.001 | 0.2854 |
| Item_32 | 0.3319 | 0.0566 | 5.860 | < 0.001 | 0.2706 |
| Item_33 | 0.1768 | 0.0522 | 3.385 | < 0.001 | 0.1564 |
| Item_34 | 0.4882 | 0.0616 | 7.927 | < 0.001 | 0.3558 |
| Item_35 | 0.5404 | 0.0598 | 9.042 | < 0.001 | 0.4018 |
CFA model was further assessed using multiple fit measures. A chi-square test shows a difference between estimated and actual variance-covariance matrix, χ²(560) = 4249, p < 0.001. This suggests the model does not have a good fit. CFI value is fairly low (0.416), well below the threshold of 0.95. RMSEA value is above the required 0.07 value (model RMSEA = 0.110). This further suggests poor model fit. Combined with loa loading estimates, we can say that MHLS does not have a unidimensional factorial structure.
Therefore, Exploratory Factor Analysis (EFA) was be performed to better understand the dimensional structure of the instrument. All 35 items were included into EFA model. Sample size is sufficient for EFA based on Kaiser-Meyer-Olkin test (value 0.865). Barlett’s test of sphericity (χ²(595) = 6745, p < 0.001) is statistically significant, which further confirms that items correlate with each other to the sufficient degree for EFA to be performed.
The initial EFA model (oblimin rotation, principal axis extraction, parallel analysis) has 6 factors. However, the last two factors have very little loading values (mean 0.47 and 0.48), low common variance explained (3.31% and 2.68%) and small eigenvalues (0.55 and 0.35). These factors also have only two items each. Only the first four factors have eigenvalue > 1.
Therefore, the last two factors were excluded and EFA model with 4 factors was constructed (Table
2). It explains cumulatively 37.8% of variability, each factor has eigenvalues > 1. Model overall has good fit with RMSEA = 0.0567 < 0.07. Most items have loading to one (and only one) factor, thus no cross-loading. Only 4 items do not load to any factors (item # 12, 15, 20, 22). Each factor has between 4 and 13 items.
Factor 1 includes items 1–11, 13, 14 which can be labeled as MH recognition. Factor 2 contains items 29–35 that describe attitudes towards people with MH. Factor 3 has items 21, 23–28 which can be interpreted as general attitudes towards MH. Factor 4 has items 16–19 all relating to information seeking about mental illness. High reliability (Cronbach’s α) was obtained for items within each factor: Factor 1 α = 0.857, Factor 2 α = 0.867, Factor 3 α = 0.764, Factor 4 α = 0.809. In addition, high test re-test reliability ICC was obtained for each factor: Factor 1 α = 0.926, Factor 2 α = 0.939, Factor 3 α = 0.819, and Factor 4 α = 0.829. The four-factor model is very similar to model determined in Slovenian validation of MHLS (Krohne et al.
2022).
In terms of known groups assessment, healthcare practitioners (Mean: 121.8, SD:14.1) scored significantly higher than the general population (Mean: 112.2, SD:14.7); t(542)=-7.4 p < 0.001. Moreover, those who were previously diagnosed with mental health condition scored significantly higher (Mean: 122.6, SD:13.9) than those who never been diagnosed with mental health condition (Mean: 114.6, SD:15.1); t(542)=-3.9 p < 0.001. In addition, those who are living with diagnosed relative (Mean: 120.7, SD:14.7) scored significantly higher than those who are not (Mean: 114.3, SD:15.1); t(542)=-3.9 p < 0.001. Moreover, those with bachelor’s degree or above scored higher (Mean: 116.8, SD:14.6) than those with less than with bachelor’s degree (Mean: 112.5, SD:16.3); t(542)=-3.0 p = 0.003. Finally, female (Mean: 118.8, SD:14.4) were significantly higher in mental health literacy score than male (Mean: 111.3, SD:15.1); t(542)=-5.8 p < 0.001.
Table 2
Exploratory factor analysis (EFA) with 4 factors
| Items | 1 MH recognition | 2 Attitudes towards people with MH | 3 General attitudes towards MH | 4 Information seeking about mental illness | Uniqueness |
| Item_1 | 0.556 | | | | 0.665 |
| Item_2 | 0.624 | | | | 0.586 |
| Item_3 | 0.582 | | | | 0.658 |
| Item_4 | 0.647 | | | | 0.552 |
| Item_5 | 0.622 | | | | 0.553 |
| Item_6 | 0.518 | | | | 0.578 |
| Item_7 | 0.743 | | | | 0.465 |
| Item_8 | 0.684 | | | | 0.570 |
| Item_9 | 0.405 | | | | 0.802 |
| Item_10 | 0.302 | | | | 0.894 |
| Item_11 | 0.483 | | | | 0.741 |
| Item_12 | | | | | 0.886 |
| Item_13 | 0.577 | | | | 0.643 |
| Item_14 | 0.455 | | | | 0.790 |
| Item_15 | | | | | 0.947 |
| Item_16 | | | | 0.613 | 0.464 |
| Item_17 | | | | 0.727 | 0.422 |
| Item_18 | | | | 0.591 | 0.629 |
| Item_19 | | | | 0.800 | 0.365 |
| Item_20 | | | | | 0.899 |
| Item_21 | | | 0.473 | | 0.703 |
| Item_22 | | | | | 0.780 |
| Item_23 | | | 0.385 | | 0.757 |
| Item_24 | | | 0.501 | | 0.633 |
| Item_25 | | | 0.570 | | 0.687 |
| Item_26 | | | 0.703 | | 0.468 |
| Item_27 | | | 0.621 | | 0.600 |
| Item_28 | | | 0.611 | | 0.569 |
| Item_29 | | 0.634 | | | 0.563 |
| Item_30 | | 0.757 | | | 0.393 |
| Item_31 | | 0.779 | | | 0.385 |
| Item_32 | | 0.787 | | | 0.380 |
| Item_33 | | 0.609 | | | 0.640 |
| Item_34 | | 0.635 | | | 0.549 |
| Item_35 | | 0.614 | | | 0.539 |