Psychometric evaluation and predictive validity of Ryff's psychological well-being items in a UK birth cohort sample of women

Recent years have seen a widening interest in research on aspects of well-being [1‐4]. Extensive research on subjective well-being (SWB) which focuses mainly on how people feel, e.g. positive affect, negative affect and life satisfaction (see review by Diener et al.) [5], has begun to be complemented by a heightened interest in how well people perceive aspects of their functioning, e.g. the extent to which they feel they are in control of their lives, feel that what they do is meaningful and worthwhile, and have good relationships with others e.g. [6, 7]. This perspective is often referred to as psychological well-being (PWB) and is based on a eudaimonic perspective, rather than the hedonic perspective of subjective well-being research.

This new focus has necessitated the theoretical development of new constructs as well as questionnaire items to measure psychological well-being in clinical and population samples. The work of Ryff and colleagues has been at the forefront of this endeavour.

Ryff's scales of Psychological Well-being [8, 9] were designed to measure six theoretically motivated constructs of psychological well-being: autonomy – independence and self-determination; environmental mastery – the ability to manage one’s life; personal growth – being open to new experiences; positive relations with others– having satisfying high quality relationships; purpose in life – believing that one’s life is meaningful; and self-acceptance – a positive attitude towards oneself and one’s past life.

Despite the widespread interest in Ryff's theoretical framework, and application of the Ryff PWB items, the psychometric properties of the proposed sub-scales remain contentious. In particular there has been concern over issues of factorial validity and distinctiveness. Do the items intended to measure each theoretical domain, really do so? Do the items capture information from more than one domain? Are fewer dimensions actually revealed by empirical data collected to test the multidimensional theory?

Previous psychometric studies of the Ryff PWB are summarised in Table 1. To date no independent investigation of the factorial validity of Ryff's well-being items has unequivocally supported the a priori six-factor structure. Authors of existing studies either challenge the value of so many theoretical constructs, whose scores correlate >0.8 or 0.9, or have not confirmed the fit of the proposed model [10‐14]

Many of these studies have reached similar conclusions despite the analysis of different short and long forms of the Ryff scales. As shown in Table 1, versions with different numbers of items have been applied in a variety of settings and samples. The original instrument included 120 items (20 per dimension) but shorter versions comprising 84 items (14 per dimension), 54 items (9 per dimension), 42 items (7 per dimension) and 18 items (3 per dimension) are now widely used. It is important to note that the overlap among items in the alternative versions of the Ryff scales is limited; for example, the 18-item version has only six items in common with the 42-item version, one item for each dimension.

Ryff's own studies [7, 9] have reported high correlations among scores for the constructs that were proposed as independent. It is possible that the measures may not, in practice, adequately operationalise the constructs proposed by her theory. For example, in Ryff's first study, which employed 120 items, the inter-correlations among factor scores for the six dimensions ranged from 0.32 to 0.76. Associations were particularly strong between personal growth and purpose in life; self-acceptance and purpose in life; and environmental mastery and self-acceptance [9]. Indeed, the magnitude of these inter-factor associations prompted Ryff and Keyes [7] to estimate a second-order factor model which invoked a general PWB factor to explain associations among the first-order constructs, so clearly they acknowledged the high inter-dependencies among the six factors.

In a psychometric investigation of multi-samples, Springer & Hauser [14] factor analysed Ryff PWB items from three large North American studies; the Wisconsin Longitudinal Survey (42-items and 12-items); MIDUS – Midlife in the United States (18-items) and the National Survey of Families and Households (NSFH II) (18-items). Their results, based on internal construct validity arguments alone, seem to provide yet further evidence that the Ryff PWB items may either measure less than six distinct constructs, or that the theoretical constructs exist at two levels of definition.

Psychometric studies of multi-item questionnaires often see a need to isolate components of response tendency that are due to methodological features e.g. design or wording of items [15, 16]. Springer & Hauser [14] introduced a single latent variable (a method factor) to isolate the covariance among responses common to all negatively worded Ryff items. In their study, this component of their model was found to considerably improve model fit. In their response 17 to a commentary on their conclusions by Ryff and Singer 18 they reported a test of a 4-factor model based on the four most highly correlated dimensions (environmental mastery, personal growth, purpose in life and self-acceptance) and compared this to a 4-factor model using the same items but where item allocation was based on positive and negative wording and position (i.e. earlier or later) in the instrument. They demonstrated similar indices of model fit between the two models.

The penultimate column of Table 1 reports the factor analysis method used by existing psychometric studies. Most existing work has examined the dimensionality of Ryff PWB items using the traditional linear factor model, which assumes that responses are continuous scores on an interval scale metric [7, 11, 12]. Hauser and Springer's analysis [14, 17] was performed using a factor model that provide an ordinal/graded treatment of the Likert style response scales. Model estimation was based on polychoric correlation among items and weighted least squares methodologies (WLS). They argued that application of the standard linear model was inappropriate. Application of linear statistical models to ordinal data can result in biased estimates of factor loadings [19‐22]. Categorical data factor analyses models are considered to be more theoretically appropriate in their statistical underpinnings for Likert scaled (ordinal) data [23‐26]

In addition to these considerations that have focused entirely on issues of internal construct and factorial validity of the Ryff PWB items, it is important to consider evidence for the construct validity of the PWB in relation to other dimensions of mental health and well-being.

Ryff [7] reported correlations from three cross-sectional studies that included measures of happiness, life satisfaction and depression in addition to PWB items. Positive associations were found between measures of happiness and life satisfaction and all PWB dimensions but with the strongest correlations for self-acceptance and environmental mastery. Conversely, the severity of depressive symptoms were negatively associated with all PWB dimensions, but with the strongest negative correlations again evident for environmental mastery and self-acceptance. In a small European sample of Swedish white collar workers (N = 91) Lindfors [27] reported a correlation of -0.61 between the score on a short screening measure for minor psychiatric morbidity (the 12-item General Health Questionnaire) [28] using a total (sum) score from the 18-item Ryff. These results suggest 1) some overlap between reported psychological well-being and the absence of depressive symptoms, and 2) positive associations with other measures of subjective well-being. More external construct validity evidence is desirable since the convergent and divergent validity of PWB measures is still not well-understood. Longitudinal studies of PWB and related constructs are of value since it is of intrinsic interest to examine the consequences of PWB for other outcomes, and to contribute new data on predictive validity, which is currently absent. The existing studies are limited by being based almost solely on concurrent self-report data.

Motivated by the controversy over the dimensionality of Ryff PWB items and methodological developments described in existing studies (Table 1) we aimed to provide the first independent examination of the a priori structure of the Ryff PWB items in a UK population-based sample. In doing so we use methods that are theoretically appropriate for factor analysis of ordinal data and compare the fit of models with the following components:

Because few studies have reported any prospective consequence or correlates of population variations in levels of PWB we also examine the predictive validity i.e. the longitudinal association between the PWB constructs and a summary measure of psychological distress comprising the 28-item General Health Questionnaire [29].

Our analysis sample includes 1,179 respondents who completed at least 85% of PWB items (36 out of 42 questions); 957 had complete data on all items. Descriptive statistics revealed a general positive skew towards the well-being end of the response scales (Table 2). Responses to the most positive category were common (ranging from 12%–60%) and for just over half of the items this formed the modal category. These results indicated a ceiling effect on measurement of the individual items comprising the well-being scale. For questions including positive item content, responses to the lowest levels of well-being were few, often as little as 1–2% of responses to that question.

Each model is described in a single line in Table 3. This table includes a model reference number, the modelling components included, and fit statistics/information criteria.

In this study we provide the first confirmatory test of the factorial validity and structure of Ryff's Psychological Well-being (PWB) scales (42-item version) in the UK. In contrast to previous studies, our sample come from the UK and comprise only women who are surviving members of a national birth cohort study which began in 1946. This sample completed the Ryff items as part of an annual woman's health survey in midlife and also completed a mental health measure one year later.

In our psychometric modelling we evaluated the fit of categorical (ordinal response) factor models with single and six construct factors, first and second-order factors, and method factors, as well as providing a reference comparison to a model with random item-factor associations. Like all previous research we were unable to identify a model that fitted the data well, although a number of modelling components appeared to be useful in improving model fit to the data, and therefore determine our conclusions regarding the factorial validity of Ryff's measures, with reference to her theory, and in regard to these 42 items. Our results indicate the following:

We tested the fit of random-item factor models in our data. Our random item-factor models differed by 1,356 BIC points from the ssaBIC for the theoretical model. This indicates to us that there are still some fragile item factor associations in the six-factor model, otherwise this comparison would yield a much larger reduction in BIC when comparing theoretical to random models.

Previous authors have concluded that the empirical data are not consistent with a six-factor model [11]. We do not reject the six construct factors, but see the value of a more parsimonious model, based on a hierarchical representation of the proposed dimensions. This approach is common in mental health epidemiology and personality research but does not seem to be as frequently adopted in well-being literature.

Our second-order factor model requires the item to factor mapping established for the first-order factors, for its definition, since it is the second-order (more general) factor that is proposed as the explanation for the association among environmental mastery (E), personal growth (G), purpose in life (P), and self-acceptance (S). Examination of item content suggests that this second-order factor may encapsulate a motivational aspect of well-being which incorporates notions of goal orientation and self-direction. Our finding that there are three (rather than six) distinct factors – autonomy, positive relations and motivation/self-direction – is reminiscent of the work of Deci and Ryan [39, 40] which postulates that well-being results from the fulfillment of three basic psychological needs – autonomy, relatedness and competence. It could be argued that our second-order factor bears a relationship to Deci and Ryan's concept of competence. However it should be noted that while there is overlap between the autonomy concepts of Ryff and of Deci & Ryan, the latter focus on the core concept of personal control while Ryff's items include an element of not caring what others think. The three factor structure of well-being has also suggested by Kafka & Kozma [11]. Their factor analyses of Ryff PWB (120-items) (See table 1 for details) which also included the Satisfaction with Life Scales (SWLS) and Memorial University of Newfoundland Scale of Happiness (MUNSH) extracted main three factors, with the first comprising environmental mastery (E), personal growth (G), purpose in life (P), and self-acceptance (S); the second MUNSH & SWLS and the third factor autonomy (A) and positive relations (R). Given these results we advocate a more parsimonious approach to examine antecedents and correlates of general well-being, as defined by the second-order factor, and/or to examine the specific antecedents or correlates of the first-order factors. These are scientific questions at different levels of generality, and should be recognized as such.

Many aspects of our modelling results do suggest that some Ryff items may measure more than one of the six constructs in the theory. This possibility requires further theoretical work that was not undertaken here, but should form an agenda for future research, and for future factorial complexity studies.

Our psychometric analyses of the Ryff 42-item PWB suggests that the addition of two method factors to reflect positive and negative item content improves model fit. A revised model with a single second-order factor, loaded by four of the six first-order factors (environmental mastery, personal growth, purpose in life and self-acceptance), two method factors, and two more distinct first-order factors (autonomy and positive relations) provided the most parsimonious solution in this birth cohort sample. Psychological well-being was negatively associated with mental health, but further investigation of precision of measurement across the health continuum is required.