Rasch analysis of the Meaning in Life Questionnaire among adults from South Africa, Australia, and New Zealand

In the present study, the Meaning in Life Questionnaire [15] was examined against the assumptions of the Rasch model. This is the first known study where the scale is analysed using an item response theory (in particular, Rasch modelling) approach. By applying the Rasch model, we explored the unidimensionality of each subscale, the functionality of the response categories, and how well the sample was targeted by the scale. We also examined differential item functioning (DIF) of the scale for a range of demographic variables.

The sample (N = 601) consisted of about equal sized groups of adults from South Africa, New Zealand, and Australia, who all completed the original English version of the MLQ as part of a battery of scales used in the international Eudaimonic and Hedonic Happiness Investigation (EHHI) project [33]. Participants were selected to be fluent in English, have at least secondary education, and be between 30 and 60 years of age. The aim was to factorially cross gender, age (three age groups of 30-39 years, 40-49 years, and 50-60 years), and education level (secondary and tertiary education). The socio-demographic profile of the sample is summarised in Table 1.

A mixed-methods cross-sectional survey design was used, where participants responded to open-ended questions related to happiness, meaning in life, and goals, and completed a battery of quantitative measurement scales. For the current investigation, only responses to socio-demographic questions and the MLQ were used. In order to avoid the potential complications of missing values and imputation techniques in Rasch analyses, respondents who generated missing values on the MLQ were removed from the sample. This involved 15 participants from South Africa, whose removal was justified by the fact that the original South African sample was larger than the samples from Australia and New Zealand. The sample from New Zealand contained no missing responses, and for the Australian sample six respondents were removed. Ethical approval was obtained from the respective regulatory ethics committees in each country. Participants were recruited by research leaders within each country using poster and newspaper advertisements and the snowball-method. Participants were provided with information on the study prior to voluntary participation.

Data were analysed using the Rasch rating scale model [25], which assumes that the distances between the thresholds of polytomous items (i.e., the probabilistic midpoints between adjacent response categories) are equal across all items. The Winsteps® 3.81 software [36] was used for all analyses, except for the graphical presentation of the person-item threshold distributions (Fig. 2), which was obtained from RUMM2030™ [37]. The MLQ-P and MLQ-S were evaluated separately, since the scale was designed to yield two relatively independent subscales [15]. Since no single aspect of Rasch analysis is definitive in identifying the optimal data-model relationship, multiple tests and graphical representations should be used to examine the characteristics of the items and persons [30]. The following interrelated facets of Rasch analysis should be considered simultaneously to inform decisions.

The first significant finding that warrants discussion is the poor performance of item 9 (“My life has no clear purpose”), the only reversed item in the MLQ-P scale. In a review on misresponse to reversed and negated items, Weijters and Baumgartner [43] advocated for the inclusion of reversed items in measurement scales as it can provide many benefits (e.g., control acquiescence, disrupt careless responding, and promote a broader coverage of the content domain), but stressed that it should be done with caution. A reversed item that is merely the negation of an item in the main direction (in point of fact, item 9 is basically the negation of item 4, “My life has a clear sense of purpose”), does not hold the benefit of broadening the content domain tapped by the instrument, and has the disadvantages inherent in negated items (e.g., accurately assessing level of agreement with statements that contain negation requires considerable cognitive strain) and reversed items (e.g., cross-cultural differences in response styles such as acquiescence). We therefore follow the guidance provided by Weijters and Baumgartner [43], who advised against the use of negated reversals, and consequently we recommend the removal of item 9, which will result in a 4-item Presence of Meaning subscale.

Steger et al. [15] stated that the reversed item was retained in the hope of discouraging automatic response sets. It is our view that this concern is to a large extent already handled by the mixed administration of the Presence and Search subscales. If item 9 is removed, however, the remaining items 4 to 6 will tap presence of meaning and the last three items will tap search for meaning. To guard against careless responding and response sets, we recommend shuffling the last six items (item 9 excluded) so that the respondent does not respond to three items from the same subscale in sequence.

For both subscales, the response categories indicative of low and medium levels of the latent construct appeared to be redundant and for the search subscale, the category thresholds were disordered. These findings suggest that the respondents were unable to distinguish reliably among the categories, and consequently fewer categories should yield more consistent, reliable scores. Weijters, Cabooter, and Schillewaert [44] suggested that seven response categories may be acceptable for populations who are expected to have high cognitive abilities, verbal skills, or questionnaire experience, such as college students, but that a 5-point scale may be more appropriate for the general population. For future use, we recommend either a 6-point rating scale where the midpoint category 4 = Can’t say true or false is dropped, or a 5-point scale with categories 1 = Absolutely untrue, 2 = Untrue, 3 = Unsure, 4 = True, 5 = Absolutely true (the issue of whether to include a midpoint category is much debated in the literature [44, 45]).

In the present study, the average level of meaning in life captured by the items was substantially lower than the average level of presence of meaning manifested by persons who completed the scale, suggesting poor targeting. In fact, the scale provided little information for respondents with high levels of presence of meaning while at the same time most of the respondents fell within that range. This could have significant practical implications. Correlations in correlational studies will be largely influenced by the minority of people exhibiting lower levels of presence of meaning as reflected by lower scores on the MLQ-P, while nuances of presence of meaning at the higher end of the continuum will not be captured well. This can, for example, influence outcomes of studies where the associations between meaning in life and indicators of health and quality of life are studied significantly. In addition, in experimental studies or studies where intervention programs are evaluated, the MLQ-P would probably not detect changes in meaning in life of people on the higher end of the continuum, which involves the majority of people, as the scale is not sensitive to changes at the higher end of the continuum.

Another possible explanation pertains to the nature of presence of meaning as a construct and its distribution in the general population. The fact that the majority of the respondents endorsed high levels of presence of meaning according to their scores on the MLQ-P could simply tell us that most people indeed experience their lives as basically meaningful: Most respondents’ level of presence of meaning were higher than the levels where the scale had optimal information, merely because there is not much variability at the upper end of the underlying construct continuum. Such an explanation speaks to the findings of Heintzelman and King [46], who conducted a review of research on meaning in life from epidemiological data and studies using the MLQ-P [15] and the Purpose in Life Test [47]. They found that diverse samples rated themselves significantly above the midpoint on self-report measures of meaning in life and concluded that most people experience their lives as “pretty meaningful”. This line of thought can be linked to psychopathology literature where “quasi-traits” are distinguished. Reise and Waller [48] defines a quasi-trait as “a unipolar construct in which one end of the scale represents severity and the other pole represents its absence (depression versus not depressed)” which “is in contrast to a bipolar construct, where both ends of the scale represent meaningful variation (depression versus happiness)”. In psychopathology research, the existence of quasi-traits with their associated peaked information curves (with the peaks in the range representing severe levels of the trait) has been found in many item response theory applications and often led researchers to conclude that items needed to be added or adapted to provide information at low (less severe) levels of the trait continuum [48]. According to Reise and Waller [48] this reasoning is problematic when working with quasi-traits: If the underlying latent construct is a quasi-trait, such attempts may be futile – it will be difficult (if not impossible) to formulate items that yield information across the continuum of the trait. Similarly we can ask whether it would be possible to develop items designed to capture even higher levels of presence of meaning, or whether we should conclude that the variation of presence of meaning is limited at the higher end of the continuum, although the majority of people attain such high levels.

If we settle with the conclusion that the majority of the population attained maximum levels of presence of meaning, we will inevitably have to re-evaluate the usefulness of, for example, interventions that aim to enhance meaning in life in the general population (most of whom have attained high levels of meaning in life). The question would be what the (large) portion of people with high levels of meaning would gain from interventions that intend to enhance meaning. Accepting that the majority of the population have already attained levels of presence of meaning that do not allow for much improvement may pose further questions. For example, could it be possible that icons of eudaimonic living, such as Mahatma Ghandi, Mother Theresa, or Nelson Mandela, who sacrificed their lives for a greater cause, have experienced levels of meaning in life similar to the majority of people? Or should we rather conclude that the nuances of presence of meaning at higher levels are just not captured by the current conceptualisation and operationalization of the construct?

Another way to explain the poor targeting of the MLQ-P may be that the subscale applies a rather narrow understanding of meaning in life, with all items paraphrasing the notion of having found a sense of meaning or purpose in life. By repeating the same content using slightly different syntax, the scale actually operates in a similar way to a one or two-item measure, which could contribute to the inability of the scale to differentiate well at the higher end of the continuum. Alternative measures that capture a broader sense of meaning in life, such as the Sources of Meaning and Meaning in Life Questionnaire (SoMe) that operationalises meaningfulness through coherence, significance, direction, and belonging [11], may display better sensitivity.

In addition, one can argue that participants’ presence of meaning in life was not really as high as they indicated it to be – social desirability may have augmented their scores artificially. However, presence of meaning in life has been found to be unrelated to scores on measures of social desirability in several studies [15, 49] and, as argued by Heintzelman and King [46], high presence of meaning scores have been found consistently among diverse samples, including anonymous samples where social desirability may not have been a big concern. The high scores could have also been due to a generalisation effect – when asked to respond to items that concern global meaning in life, people may not be sure what meaning actually refers to. They may have a broad understanding of meaning and therefore think that they generally experience meaning. However, if the constituents of meaning are spelled out, they might realise that they don’t have as much meaning as they initially thought.

One may also reason that the lack of sensitivity to varying nuances of meaning in life at the higher end of the continuum relates to the fact that the scale relies on self-report and alternative avenues to capture meaning in life should be explored. This approach may be problematic because meaning in life is, at its heart, a subjective experience. Several studies have argued that self-report is the best way to capture meaning in life [46, 50, 51]. However, obtaining self-report using less structured approaches may add value, for example by using experience sampling methods [52] or qualitative methods.

The data in this study were gathered in three different countries and two gender groups, three age groups, and two levels of education were distinguished. Of all these demographic variables, significant DIF was only detected for items from the Presence subscale based on the country variable. The absence of DIF is the desirable outcome should data from the different demographic groups be combined or compared [53].

The significant country DIF for items from the Presence of Meaning subscale warrants further attention. Before removal of item 9 (“My life has no clear purpose”), the item exhibited DIF for country: Given equal levels of the latent trait, respondents from South Africa tended to respond more strongly towards the extreme True response categories than respondents from New Zealand and Australia, and, similarly participants from New Zealand were more inclined to extreme responses in the True direction than participants from Australia. After removal of item 9 and before collapsing the response categories, item 1 (“I understand my life’s meaning”) manifested DIF, where Australians found it harder to endorse the item than South Africans given equal levels of the latent construct. After collapsing categories, this finding was extended – respondents from both New Zealand and Australia found it significantly harder to endorse item 1 than respondents from South Africa given equal levels of the construct. Also, before collapsing categories, participants from New Zealand found it harder to endorse item 4 (“My life has a clear sense of purpose”) than participants from South Africa given equal levels of the latent trait. Last, given equal levels of the latent construct, participants from South Africa found it harder to endorse item 5 (“I have a good sense of what makes my life meaningful”) than respondents from Australia and New Zealand, both before and after collapsing categories. Country-specific parameter estimates may be needed for these items of the Presence subscale, that is, the dataset can be split by country and these items should be calibrated separately for each country [54].

The two items that respondents from Australia and New Zealand found harder to endorse than South Africans given equal levels of the latent construct (i.e., items 1 and 4) refer to comprehending one’s life meaning and having a clear sense of purpose – both can be seen as a global state of grasping one’s life meaning, without referring to the elements that brings meaning to one’s life. South Africa is a developing country and together with the many challenges the country faces come multiple opportunities for individuals to contribute and to have a sense of purpose. This may especially be the case for educated individuals who may feel that they have skills and knowledge that can really make a difference in a country with many challenges (based on the selection criteria of this study all participants had at least secondary education). Australia and New Zealand, on the other hand, are first world countries with a lot more stability and certainty. People from such countries may feel that things “go right” regardless of their contribution which may possibly lead to having a less clear sense of purpose and meaning comprehension. Another possible explanation may be connected with the fact that the specific South African group in this study exhibited a higher frequency of religious practice (mostly Christianity) than the participants from Australia and New Zealand. Religiosity may be associated with a clear sense of purpose and meaning comprehension.

The item that South Africans found harder to endorse than respondents from Australia and New Zealand given equal levels of the latent trait (item 5) refers to an awareness of the constituents of a meaningful life – the elements that make one’s life meaningful. One possibility is to infer that people (in this case, South Africans) who find it easier to agree with items referring to a global comprehension of one’s life’s meaning (items 1 and 4), may not have such a pressing need to know what the elements are that make their lives feel meaningful – one may argue that they take it for granted or that they spend less time attending to the specific details of why they find their lives meaningful. In contrast, people who find it more challenging to agree with items related to comprehending one’s life meaning and having a clear sense of purpose (in this case respondents from Australia and New Zealand), may be more attentive to the things that add life meaning.

Since all aspects of Rasch analysis are interconnected [30], the existence of cross-country DIF on the Presence subscale could have influenced the rest of the findings. Future research should explore whether the findings of this study replicate in more culturally homogeneous samples where DIF is not present.

While the study makes important contributions to the body of knowledge about meaning in life and the measurement thereof across three countries, it also possessed several limitations. This study made use of the Rasch model, which is considered to be a one-parameter IRT model that includes only item difficulty as a parameter. Although the Rasch model has very attractive mathematical properties, analysing MLQ data using more complex IRT models will also be of value.

In this study, recommendations regarding the removal of item 9 (“My life has no clear purpose”) and category collapses were made a posteriori based on removing the item from and collapsing categories of data attained using the original full scale. These recommendations should be tested in new datasets gathered with a revised scale.

The fact that the sample in this study comes from three different countries can be seen as a strength in the sense that diversity is reflected in the study of an already well-established scale. In addition, it allowed us to investigate DIF across the three countries. The fact that evidence was found for DIF across the countries, however, points towards the possibility that the scale may function differentially across the different country groups which could have had an influence on the rest of the results. This suggests the need for repetition of the study in more culturally homogeneous groups to investigate whether the findings replicate when cross-country influences are not present.

Another important avenue for future research is the revisiting of presence of meaning in life as a construct, in particular with regards to the higher end of the construct continuum. The content domain of presence of meaning should be explored qualitatively in order to deepen our understanding of the construct, especially at high levels. For example, by investigating lay people’s conceptualisations of meaning in life, we may identify sub-facets of meaning in life which may provide greater variance at the upper end of the continuum.

¹In the ability testing environment, where Rasch modelling originated, the term item difficulty is often used to refer to the level of the latent construct captured by the item, while person ability describes the level of the latent trait held by the respondent. For the purpose of the current study where meaning in life is studied, we deemed the terms item challenge and person intensity more suitable and used them in the manuscript.