Measuring positive mental health and flourishing in Denmark: validation of the mental health continuum-short form (MHC-SF) and cross-cultural comparison across three countries

The 14-item MHC-SF measures positive mental health during the past month with one hedonic dimension corresponding to emotional well-being and two eudaimonic dimensions corresponding to social and psychological well-being (see Table 1). The items are all positively worded. All items are asked using the following format: “During the past month, how often did you feel …? ”. Response categories are coded as ‘never’ (0), ‘once or twice’ (1), ‘about once a week’ (2), ‘about two or three times a week’ (3), ‘almost every day’ (4), ‘every day’ (5). Item f was originally worded “society is becoming a better place for people like me”, but following Keyes’ [22] recommendation to change the wording of the item, the Danish version instead includes the revised item “society is becoming a better place for all people”.

The continuous total on the MHC-SF is calculated by summing the scores of each item, which results in a total score ranging from 0 to 70. Thus, a higher score indicates a higher level of positive mental health. In this study’s validation analyses, we primarily used the continuous total MHC-SF score when relevant (convergent validity, discriminant validity, content validity). In the cross-cultural analysis, we used both the continuous score as well as its categorical operationalization.

Categories for positive mental health were generated according to Keyes’ criteria [22]. Individuals who scored 4 (‘almost every day’) or 5 (‘every day’) on at least one item that measured emotional well-being, and also scored 4 or 5 on at least six of the eleven items of the combined scale of social and psychological well-being were categorized as ‘flourishing’. Individuals who scored 0 (‘never’) or 1 (‘once or twice’) on at least one item of emotional well-being and also scored 0 or 1 on at least six of the eleven items of the combined scale of social and psychological well-being were categorized as ‘languishing’. Individuals who were neither ‘flourishing’ nor ‘languishing’ were categorized as having ‘moderate’ mental health.

Before the initiation of the survey, the MHC-SF was translated into Danish through forward-translation and back-translation. The details of the translation methodology have been described by Sousa and Rojjanasrirat [23], and it has been applied successfully in translations of mental health and well-being measures in the Scandinavian setting [24, 25].

Covers overall well-being, five items which are scored from 0 to 5, then summed and multiplied by 4, and scored into a continuous scale from 0 to 100. High scores indicate high levels of well-being [26].

A single item for self-rated health which asks respondents to rate their overall health (physical as well as mental), five response options which range from poor to excellent (1–5). Higher scores indicate better self-rated health [27].

Six items which measure symptoms of discomfort and pain within the past 2 weeks; Shoulder or neck; Back or lower back; Arms, hands, legs, knees, hips or joints; Headache; Stomach-ache; Difficulties sleeping. Each item is coded 0 = symptom not present, 1 = symptom present. Items are summed to score on a scale ranging from 0 to 6, with higher scores indicating a higher number of symptoms [27].

Ten items that are each scored from 0 to 4. Positive items are reversed and summed into a scale ranging from 0 to 40. Higher scores indicate higher levels of perceived stress.

Data on poor mental health was collected using the PHQ-4 which asks respondents about their experience of core depressive and anxiety symptoms over the past 2 weeks as specified by DSM-IV [29]. There are four items for depression/anxiety; each item is given a score from 0 to 3 and then scored into a continuous scale ranging from 0 to 12 Higher scores indicate a high level of depression/anxiety.

Other variables included in the present study were: sex (male, female), age, education (primary or unknown, youth education, short-cycle higher education (2–2½ years of full-time study), medium-cycle higher education (3½-4 years of full-time study), long-cycle higher education (5–6 years of full-time study), employment status (employed, not employed or unknown), and living arrangements (single, married or with a partner).

We examined total scores for floor and ceiling effects. Instruments exhibit floor or ceiling effects if more than 15% of respondents record the lowest or highest score [32].

We conducted confirmatory factor analysis (CFA) over two randomly created subsets to analyze global goodness-of-fit (n = 694) and measurement invariance (n = 2814). We used an unweighted least squares estimator with means and variance adjusted (ULSMV). We modelled three different structures: two first-order models with one and three correlated factors respectively, and a hierarchical or bifactor model with three domain-specific factors. In the three-factor model, items a–c loaded on the latent variable of emotional well-being, items d–h on social well-being, and items i–n on psychological well-being [33]. In the bifactor model (see Fig. 1), every item loaded onto one of the three domain-specific factors — as specified in the three-factor model — and also on a general well-being factor [17].

As recommended by Hoyle and Panter [34], we used several fit indices including the Root Mean Square Error of Approximation (RMSEA), the standardized root mean square residual (SRMR), the Comparative Fit Index (CFI), and the Tucker-Lewis Index (TLI). Values greater than 0.95 for the CFI and TLI were considered to reflect good model fit. RMSEA and SRMR values of 0.06 or less were considered to indicate good fit, although values up to .08 were considered acceptable [35]. We evaluated measurement invariance across sex (women vs men), age groups (16–54 years of age vs 55+), and education (primary or unknown vs youth education vs short-long cycle educations), examining differences in Alternative Fit Indexes. We considered a model invariant when the respective constraint produced at most −.01 change in CFI, paired with changes of up to .015 in RMSEA, and .030 (for metric invariance) or .015 (for scalar or residual invariance) in SRMR [36].

Besides the general fit of the models to the data, we analyzed further the bifactor model to study the potential multidimensional and unidimensional uses of the MHC-SF scores implied in the three-factor and the one-factor solutions. The two alternatives have been considered plausible for the conceptualization of well-being, and several recent validations of MHC-SF versions have consistently included at least two of the three models [16, 17]. Following Rodriguez, Reise and Haviland [37], several aspects of the (Danish) MHC-SF were evaluated through a bifactor lens: (1) the reliability of unit-weighted composite scores; (2) the use of a set of items to compute factor scores or to identify a latent variable in an SEM context; and (3) whether multidimensional (bifactor) data are “unidimensional enough” to specify a unidimensional measurement model in an SEM context. For the first aspect, we calculated omega (ω), omega hierarchical (ω_hs), and their ratio (ω_hs/ω). For the second, we estimated indices of factor determinacy (FD, benchmark >.90) and construct reliability or replicability (H, benchmark: >.70). For the third aspect of the evaluation, we analysed the explained common variance by the general factor on all items (ECV) and on each individual item (I-ECV), the percentage of uncontaminated correlations (PUC), and the relative parameter bias as the difference between an item’s loading in the unidimensional solution and its general factor loading in the bifactor, divided by the general factor loading in the bifactor.

We assessed convergent validity by calculating correlations between the MHC-SF continuous score and WHO-5, and discriminant validity by calculating correlations between the MHC-SF and SRH, education, symptoms of discomfort and pain, PSS, and PHQ-4. We hypothesized that the MHC-SF scores would show a strong positive correlation with well-being (WHO-5) [38], moderate positive association with SRH, and moderate negative associations with scales measuring the negative aspects of physical or mental health status (symptoms of discomfort and pain, PSS and PHQ-4) [12, 14] (based on Cohen’s rule of thumb, i.e. small: r = 0.1; moderate = 0.3; large = 0.5 [39]).

Based on the findings of recent Danish health and morbidity studies, we hypothesized that the scale would show a positive association with education [27, 40]. The association was hypothesized to be weak to moderate based on recent studies suggesting that well-being is less sensitive to socioeconomic patterns compared to poor mental health [41]. Differences in scores across sex and education were assessed using linear regression analysis.

Cognitive interviewing techniques were used to examine the face validity of the scale (i.e., do people understand the questions in the way they were intended) and usability (i.e., how participants process and respond to the scale). Eleven face-to-face interviews, all in Danish, were conducted with six men and five women age range 20–77 years. Participants were selected with the aim to have variation in age, sex and education, which are attributes known to be associated with mental health and health literacy [42]. The interviews followed an interview protocol that was developed in accordance with the recommendations by Gray [43]. All interview questions were non-leading, non-directing and neutrally framed (e.g. “how did you experience responding to the questionnaire?”). During the interview, prior scripted and spontaneous open-ended probes were used. All interviews were recorded and summarized in writing using a literary style [44]. The software program QSR NVivo 11 was used to assist managing and analyzing the qualitative data; allowing to work with the written summaries and the audio files simultaneously. The interview data were analyzed using the Framework approach [45]. The applied framework consisted of six a priori themes based on the Tourangeau et al. model for survey response [46]: comprehension (overall), comprehension (item-specific), retrieval, judgement, response, and other (relevant passages, not fitting with other themes). Following a thorough familiarization with the data, the data were categorized according to the six themes. The content of each theme was compared across participants with the aim of identifying potential challenges in the response process and different ways of interpreting and answering the questions. Next, patterns and links between themes were identified to explain why problems occurred. Preliminary findings were discussed within the research team. A summary of the findings is presented in the results section structured according to the themes used in the analysis (the themes retrieval and other are not reported as there were no notable results pertaining to them). The findings are supported by illustrative quotes.

Total MHC-SF scores and categories for well-being were computed with weights applied to generate nationally representative estimates using the Stata svy command. The Danish scores were reported along with scores based on data representative of two other countries, specifically Canada and the Netherlands. The Dutch scores were, however, based on a version with revised response categories (see Appendix) to make it easier for the respondents to recall. These revised response categories were: never (1), rarely (2), sometimes (3), regularly (4), often (5), or (almost) always (6). The total MHC-SF scores and categories for well-being were reported for each country as well as stratified by age and sex. Information regarding survey and sampling in Canada and the Netherlands is provided in the Appendix.

Several participants experienced difficulties completing the scale, with reactions to the questionnaire being often negative. The main problems were comprehending the questions and their relevance. The layout of the questionnaire was considered to disrupt the flow of reading because the first part of the questions (Within the past month, how often did you feel …) was only written once at the top, i.e. respondents felt they had to read the first part again for every new item in order to comprehend the entire item. Overall, the items in the MHC-SF scale were considered as being unusual or characterized as something people “usually do not think about” or do not “consciously reflect upon”, making them hard to answer and prolonging the decision-making process.

Items f (That our society is becoming a better place for people) and h (That people are basically good) evoked reactions or comments from most participants. They found the questions problematic to answer and irrelevant regarding their personal state of well-being or mental health since they considered the questions to be about personal political values and views. For example, a respondent stated,

" This [item f] is really a sick question. Because it is very political and has nothing to do with me. Or, in some way it has." (male, 47 years)

A participant explained that some questions are very sensitive to the context and external conditions, e.g. political events, media and social interactions. This was especially evident for items f (That our society is becoming a better place for people) and g (That people are basically good) as participants based their responses, for example, on global political events (e.g. one participant mentioned the election of a new president in the USA in 2016 and how this might impact the societal situation both globally and nationally). Some considered the decision-making process for item c (Satisfied) to be highly influenced by the way the question is contextualized by the respondent (e.g. comparing oneself with a homeless or a child in Africa is different from comparison with the neighbour next door). Another participant pondered over the use of the word ‘society’ in item f, and remarked that the question is more about decision-makers (e.g. politicians) than about the participant:

"This is not so much about what I am in society, but how others are running our society or deciding how it has to be" (female, 57 years)

The wording of item f puzzled several participants, and two informants pointed out that the question was ambiguous because it focuses on two matters within the same item: the current state of society, and the current developmental direction of that same society:

"I can say, it is a shitty place, but it is getting better. Or, it is a good place, but it is getting worse. … Therefore, it is very ambiguous when you go into the phrasing here." (male, 47 years)

Two respondents found item g (That people are basically good) not to be in accordance with their worldview and their understanding of human nature, as they did not find it possible to categorize people as being ‘good’. Several participants considered the wording in item g too broad and vague as it was not clear who and what the word ‘people’ covers (close relations, the Danish population, or all people in the world). Item l (That you have experiences that challenge you to grow and become a better person) was considered complicated, which caused doubt about how it should be interpreted.

The broad and vague wording, as described above, complicated the decision-making process for some participants. Likewise, items b (Interested in life) and d (That you had something important to contribute to society) were considered hard to answer, because participants found them to be too broad. Also, respondents found it difficult to assess how many times they had experienced a given feeling. According to the participants, this issue occurred because they construed some items as asking about values (e.g. political views) rather than feelings per se. This issue was especially evident for items f (That our society is becoming a better place for people), g (That people are basically good) and h (That the way our society works makes sense to you).

Participants had varying opinions about whether the number of response categories was appropriate or not. Some participants found the number of response categories to be appropriate, while others suggested that there should be more response categories. One participant remarked that five response categories would be appropriate, as this would give a middle option. The wording of the response categories, specifically in connection with item i (That you liked most parts of your personality), f (That our society is becoming a better place for people), and h (That people are basically good), were considered to be “clumsy” and “random” as these items were interpreted as asking more about fundamental personal values than about the frequency of experiencing a given feeling:

" [ … ] it is hard to tell, if it is almost every day, or if it is one time a week, or if it is two to three times a week. Because it is a core value that I have, and therefore, I think, to me it must be almost every day, right?" (female, 65 years)

This issue is related to the problems caused by the difficulty in assessing how many times a specific/given feeling had been experienced, as well as whether one can experience a personal value or attitude less than all the time. Altogether, there were problems with several aspects of the scale, i.e. the overall layout, the wording and thematic content in several items, as well as possible response categories.