Abstract
Multiple sclerosis (MS) is a progressive disease characterized by neurological symptoms and sometimes heightened levels of distress. The Perceived Stress Scale (PSS) is often used in MS samples to measure stress but has not been validated in this population. Participants () completed the PSS as well as measure of depression, anxiety, and mental and physical health. Factor analyses indicated that the general factor of a bifactor model accounted for a large amount of the variance in the 14-item and 10-item versions of the PSS. The 4-item PSS had two factors, the Stress subscale and the Coping subscale, but a one-factor model also fits the data well. Total scores and both subscales had sufficient reliability and validity for all versions of the PSS, although a few items of the 14-item PSS had low item-total correlations. This study supports the use of the total score of the PSS in MS but also suggests that the 10-item PSS had better psychometric properties than the 14-item PSS.
1. Introduction
Multiple sclerosis (MS) is a neurological disease characterized by numbness, weakness, fatigue, vision problems, and spasticity and is often progressive [1]. People with MS often experience depression and anxiety, with as many as 50% of people with MS experiencing depression in their lifetime [2]. A common trigger of depression is stress [3], which can be defined objectively as stressful life events such as divorce or subjectively as perceptions of stress including perceptions that events are uncontrollable and unpredictable. One of the most common measures of subjective experiences of stress is the Perceived Stress Scale [4]. However, despite its use to measure perceived stress in MS [5, 6], this scale has not been psychometrically validated in this population.
The Perceived Stress Scale (PSS) measures the level to which a respondent considers his or her life stressful and was designed to assess how unpredictable and controllable a person appraises his or her life [4]. The original version had 14 items, seven negatively stated and seven positively stated. Two short forms are available, one with 10 items (six negatively stated and four positively stated items) and the other with four items (two negatively stated and two positively stated) [7]. The PSS is related to health behaviors such as smoking [8] as well as depression and anxiety [9]. Other measures of stress also are related to the PSS, supporting validity [10]. The 14-item and 10-item versions showed adequate internal consistency although the 4-item version had marginal reliability, such as a Cronbach’s alpha of .67 [8, 10]. Cohen et al. specifically posited that perceived stress will be sensitive to daily hassles and other frequent changes, so test-retest reliability is not expected to be high [4]. The 14-item version showed test-retest reliability of .85 after two days but .55 after six weeks [4] and a .73 test-retest reliability after two weeks [9], consistent with this prediction. The 10-item PSS had a test-retest reliability of .86 using a 7-day interval [11], .77 using a two-week interval [9], and .53 to .61 using a 12-month interval [12].
In Cohen’s original validation study, factor analyses of the 14-item and 10-item forms showed that two-factor model was the best model for the data while a one-factor model was a better fit for the 4-item version [7]. Cohen advised that for purposes of measuring stress, the distinction between the two factors was irrelevant. However, subsequent studies have consistently shown a two-factor structure for all three versions of the PSS [8, 10]. This two factor structure has been shown for the 10-item PSS in cancer patients [12], Turkish university students [13], and university teachers in Brazil [11]. One way of reconciling the use of the total PSS score and the two factor structure was the suggestion of a second order factor that influences the two factors [11, 12]. The first factor contains negatively stated items (unable to control important things and so forth) and has been termed “Stress” [12], “Negative stress” [11], and “Perceived Helplessness” [13]. The second factor contains positively stated items (things were going your way and so forth) and has been termed “Counter-stress” [12], “Positive stress” [11], and “Perceived Self-efficacy” [13]. Examination of the item content shows the first factor measures perceptions of uncontrollability, unpredictability, and inability to cope. The second factor items measure perceived success in coping. For the purposes of this study, the first factor will be referred to as “Stress” and the second factor as “Coping.”
One way to examine the contribution of the general factor and the two subdomains to the summary score is the bifactor model. The bifactor model consists of a general factor and all items load on this factor [14]. Additional “group” factors are included and only certain items load on each group factor. See Figure 1. The bifactor model can be used to determine whether a measure is sufficiently unidimensional to support using a total score while still accounting for the inevitable multidimensionality of psychological measures [15]. The bifactor model also directly addresses and tests unidimensionality, unlike the second order model that has previously been used with the PSS. As no study has used the bifactor model to quantify the contribution of the potential factors to the PSS overall score, we examined the bifactor model for the PSS. In addition to the general factor, the bifactor model included group factors for the Stress and Coping factors shown in previous studies.
(a)
(b)
(c)
The current study examined the psychometric properties of the PSS in a sample of people with multiple sclerosis (MS) as this scale has not been validated in this rehabilitation population. Unidimensionality is an assumption of any scale that uses a total score, like the PSS, and we investigated whether the total score was supported and examined the contribution of the two factors previously described. To do so, we used a confirmatory factor analytic approach to investigate different factor structures for the three forms of the PSS. For all PSS versions, we compared one factor and two factor models. We also fit bifactor models and examined the amount of variance accounted for by the general and group factors for the 14-item and 10-item PSS. A bifactor model could not be fitted to the 4-item version due to the limited degrees of freedom. In addition to assessing reliability and floor and ceiling effects, we assessed validity by correlating the total PSS and two subscales with measures of depression, anxiety, and health. As stress is substantially related to depression and anxiety, we hypothesized that the PSS total score and both subscale scores would be more strongly associated with the mental health measures than with the physical health measures.
2. Methods
2.1. Participants and Procedures
This study was approved by the University of Washington institutional review board before participants were recruited and informed consent was provided by participants before participation. Participants with self-reported MS were recruited for a longitudinal survey on symptoms and quality of life in multiple sclerosis through the greater northwest chapter of the National Multiple Sclerosis Society. The first survey was completed by 1,271 people with MS and 562 participants were randomly selected to participate in the longitudinal survey. Data for this study were collected between November 2008 and July 2009 at the 20-month followup (). A paper survey was mailed to participants and mailed back to study offices. In the event of missing data, study personnel called participants to obtain the missing data. Data were double entered and cleaned.
2.2. Measures
2.2.1. Perceived Stress Scale
The Perceived Stress Scale (PSS) was developed to measure how controllable and unpredictable people viewed their lives, an important component of the experience of stress. The original version had 14 items asking participants to rate their stress over the past month [4], with seven items negatively stated and seven items positively stated. For this study we refer to the subscale representing the negatively stated items as “Stress” and to the subscale representing the positively stated items as “Coping.” Two short forms were also developed [7]. The first short form has 10 items, six negatively stated and four positively stated. The second short form has four items, two negatively stated and two positively stated. The response set ranges from 0 (never) to 4 (very often) and positively stated items are reverse coded before items are summed with higher scores indicating more perceived stress. Scores for the 14-item form range from 0 to 56, for the 10-item form range from 0 to 40, and from 0 to 16 for the 4-item form. Norms from a United States probability sample were 19.62, 13.02, and 4.49 for the 14-item, 10-item, and 4-item versions [7].
2.2.2. Patient Health Questionnaire-9
The Patient Health Questionnaire-9 (PHQ9) contains nine items corresponding to criteria symptoms for Major Depressive Disorder [16, 17]. Respondents are asked to rate their symptoms over the previous two weeks. The response options range from 0 (not at all) to 3 (nearly every day) and items are summed to create total scores. Higher scores indicate more depressive symptoms. The PHQ9 has been found to be reliable and valid in numerous medical populations [18].
2.2.3. Patient Health Questionnaire Anxiety
The Patient Health Questionnaire Anxiety [16] contains seven items measuring worry and other symptoms of anxiety. Response options range from 0 (not at all) to 2 (more than half the days) and respondents rate their symptoms over the past four weeks. Item responses are summed and then divided by the number of items to derive total scores in which higher scores indicate more anxiety.
2.2.4. Short Form 8
The short form 8 (SF8) is an abbreviated version of the SF36 questionnaire [19]. It contains eight items corresponding to the SF36 domains (general health, physical functioning, pain, social functioning, role limitations due to physical health, role limitations due to emotional health, vitality, and mental health) and scores are transformed onto a t-score metric with a mean of 50. Higher scores indicated better functioning. For this study, the overall physical health and overall mental health scales were used. The SF8 has shown good reliability and validity [19].
2.3. Analyses
2.3.1. Factor Structure
Factor structure was analyzed using categorical confirmatory factor analysis in Lisrel 8.8. Polychoric correlations were used as the data for these analyses and a diagonally weighted least squares estimator was used. Confirmatory factor analyses were run on each version of the PSS (PSS14, PSS10, and PSS4). For the PSS14 and PSS10, bifactor, one and two factor models were fit. For the PSS4, one and two factor models were fit to the data. The one factor analyses had all items load on a single factor while the two factor analyses had all negatively stated items load on the first factor, labeled Stress, and all positively stated items load on the second factor, labeled Coping. The bifactor model had a general factor on which all items were loaded. Both bifactor models also had two group factors, Stress (negatively worded items) and Coping (positively worded items).
Fit was evaluated using the chi-square tests for perfect and close fit, root mean square error of approximation (RMSEA), expected cross validation index (ECVI), comparative fit index (CFI), the standardized root mean residual (SRMR), and Akaike Information Criterion (AIC). The following criteria were used to assess goodness of fit: nonsignificance on tests of perfect and close fit, RMSEA < .08 but preferably <.05, CFI > .96. ECVI and SRMR are used to compare models with lower levels indicating better fitting models so absolute values were not used for these measures. Lower levels of the AIC indicate a better fitting model and a difference of 10 was considered important [15].
2.3.2. Floor and Ceiling Effects
To examine floor and ceiling effects, we calculated the percentage of participants reporting the lowest scores (0 for all versions) and the highest possible scores (56, 40, and 16 for the PSS14, PSS10, and PSS4, resp.). We used the recommendations of Terwee et al. [20] to determine whether a floor or ceiling effect was present. Essentially, in any sample of 50 or more participants, having 15% or more of participants scoring at the lowest or highest score indicated a floor or ceiling effect.
2.3.3. Reliability and Validity
Reliability and validity analyses were conducted using SPSS. Cronbach’s alpha was calculated for the total scores as well as for each of the two subscales. Item-total correlations (the correlation of an item with total of the remaining items) were also calculated to examine reliability in addition to the coefficient alpha. Ideally, item-total correlations are over .40 [21]. Convergent validity was assessed by correlating the PSS total and subscale scores with the PHQ9, PHQ-anxiety, and SF8 scores. The PSS scores were predicted to be strongly positively associated with depression and anxiety (PHQ9, PHQ-Anxiety) and strongly negatively associated with the mental health score (SF8-Mental Health score) and, while still moderately correlated with physical health (SF8-Physical Health score), the PSS scores would have lower correlations with physical health compared to mental health.
3. Results
Participant demographics and disease-related variables are displayed in Table 1. The average participant was middle aged, female, Caucasian, and in a long-term relationship and was not currently employed. The majority had relapsing remitting MS and was more than 10 years post-MS diagnosis. The PHQ9 mean score of 6.66 indicated mild depression.
3.1. Factor Analyses
Results of the confirmatory factor analyses supported a bifactor structure for the 14-item and 10-item PSS forms. Factor loadings are reported in Table 2 and fit indices are reported in Table 3. For the PSS14, all items had acceptable factor loadings in the one factor model except for item 12. However, fit indices indicated that the one factor model did not have an optimal fit to the PSS14 data (RMSEA greater than .08, CFI < .96, both perfect and close fit tests were significant). The two factor model of the PSS14 had acceptable factor loadings and most fit indices indicated sufficient fit (RMSEA < .08, CFI > .96) although both tests of perfect and close fit were significant. When examining the bifactor model for the PSS14, the factor loadings on the general factor were all over .40 except for item 12 and some loadings on the group factors (Stress and Coping) which were low. The general factor in the bifactor model accounted for 45% of the total variance while the Stress factor accounted for 9% and the Coping factor accounted for 6%. The general factor accounted for 74% of the variance explained by the model and the group factors together accounted for 26%. The ratio of the total variance explained by the general and the group factors is consistent with what Reise and Haviland [22] suggested as supportive of sufficient unidimensionality.
For the one factor CFA model of the PSS10, the factor loadings were acceptable although, similar to the PSS14, several fit indices were not ideal (tests of perfect and close fit were significant, RMSEA > .08). The two factor structure of the PSS10 had adequate factor loadings and sufficient fit indices and the test of close fit was not significant (). The bifactor model of the PSS10 had some low factor loadings on the Stress and Coping factors but this was not as low as the PSS14. The general factor in the bifactor model accounted for 52% of the variance while the Stress factor accounted for 9% and the Coping factor accounted for 4% of the total variance. The general factor accounted for 80% of the explained variance and the group factors accounted for the remaining 20%. Similar to the PSS14, this is supportive of a general factor.
Factor loadings were acceptable for one and two CFA models of the PSS4. While fit indices for the PSS4 indicated that the two factor model fit the data better than the one factor model (lower RMSEA, RMSR, ECVI, and AIC), the one factor model still fits the data acceptably (RMSEA < .08, CFI > .96, test of close fit was nonsignificant). Results indicated that while the two factor model would be preferred, the one factor model was still tenable for the PSS4.
3.2. Floor and Ceiling Effects
The PSS was found to have negligible floor or ceiling effects. Only 2 (0.4%) responded “never” to all 14 questions on the PSS14, 5 (1.1%) were at the floor for the PSS10, and 32 (7.2%) endorsed the lowest category for PSS4. The ceiling effects were also low and ranged from 0 for PSS14 to 0.2 for PSS4. The breakdown of the responses at the floor and ceiling for the total score and subscales is listed in Table 4. More participants reported the lowest score than the highest score. For all the scales the proportion of responses at the floor and ceiling was far below the 15% threshold. Two subscales of the PSS4 had a higher number of participants reporting the lowest scores (12.3% and 12.6%) but they did not quite reach the threshold for a floor effect.
3.3. Reliability and Validity
Cronbach’s alpha ranged from .73 for the PSS4-Stress to .91 for the PSS14-Total (see Table 4) and indicated adequate internal consistency for all versions and subscales. The Cronbach’s alpha over .90 suggested likely redundancy in the full version (PSS14). Ranges for item-total correlations are also reported in Table 4. Some item-total correlations were below .40 for the PSS14 total score (2 of 14 correlations) and the PSS14 Stress subscale (1 of 7 correlations) but all correlations for the PSS14 Coping subscale were over .40. Items on the PSS14 total scale with the lowest correlations were items 4 and 12. For the PSS10 and PSS4 total score and Stress and Coping subscales, all item-total correlations were above .40.
As indicated in Table 5, all PSS total scores (PSS14, PSS10, and PSS4) and PSS subscale scores were statistically significantly related to the PHQ9, PHQ-Anxiety, SF8-Physical Health scale, and SF8-Mental Health scale, indicating convergent validity. All associations were of the expected magnitude and in the hypothesized direction. The PSS scores showed a stronger relationship to the SF8-Mental Health scale (’s ranged from −.620 to −.714) compared to the SF8-Physical Health scale (’s ranged from −.246 to −.349), consistent with predictions.
4. Discussion
The present study evaluated the psychometric properties of the Perceived Stress Scale (PSS) in a sample of people living with MS. The analyses provided evidence for sufficient unidimensionality of all forms of PSS and supported the use of the summary score. Reliability and convergent validity was acceptable for the 10-item and 4-item PSS forms as well as for the subscales on those forms. Cronbach’s alpha suggested redundancy of items for the 14-item PSS and some of the item-total correlations were low indicating that some items with low item-to-total correlations could be eliminated. These items, numbers 4 and 12, are also not included in the 10-item and 4-item PSS. The total scores and both subscales were related to measures of mental health and physical functioning.
The results of this study support previous evaluations of the PSS in healthy populations [7] and medical populations [12]. From one of the original validation studies, means for the 14-item, 10-item, and 4-item PSS total scores were 19.62, 13.02, and 4.49, respectively, in a healthy population [7]. These means for a healthy population were slightly lower than the means in this study suggesting that people living with MS report on average higher levels of stress compared to the healthy people. For women with cancer, means for the 10-item PSS were reported as 17.55 at initial cancer diagnosis, 14.13 one year later, and 14.04 two years later [12]. The mean of the 10-item PSS in this study was comparable to levels reported one and two years after diagnosis. In a sample of stroke patients, a mean on the 10-item PSS was reported at 12.27 right after hospital discharge and 10.47 at 12 months later, both lower than the present study [23]. Overall, people with MS reported levels of stress higher than those with other medical illnesses.
Previous studies have reported a two factor structure as opposed to a one factor structure [8, 10–13]. This study suggested that while there are separate Stress and Coping factors, a bifactor model suggested that the total scores of the PSS14 and PSS10 are driven by one underlying dimension: perceived stress. While the two factor model fits the 4-item PSS best, the one factor model still had acceptable fit indicating that a total score would still be tenable for the 4-item version. Unlike previous studies, sufficient reliability was shown for all forms of the PSS including the 4-item version suggesting that, at least for MS populations, the 4-item PSS is acceptable. This indicates that the total PSS scores are still acceptable to use. The shorter versions of the PSS or the subscales may be an appropriate choice for certain purposes, like epidemiological studies or patient screening and monitoring. The 4-item PSS may be particularly useful for research studies not primarily focused on stress and coping but for studies that may use stress and coping for secondary analyses. Ultimately these results suggest that the total score of the PSS is usable as the measure is sufficiently unidimensional. In addition, the PSS-10 appears to have better internal consistency than the PSS-14 and less participant burden due to fewer items and may be a better choice for MS researchers or clinicians.
The contributions of this study should be considered within the limitations. The sample in the study was a convenience sample and was not representative of people living with MS in the US. In addition, the diagnosis and the type of MS were self-reported. Also, as the full 14-item PSS was only administered at one time point in this study, we were unable to assess test-retest reliability. Regardless of the limitations, this study supports the use of the PSS in people with MS.
Conflict of Interests
The authors declare that there is no conflict of interests regarding the publication of this paper.
Acknowledgments
The authors would like to thank the study participants and the research staff who helped with this project. Dr. Wu is supported by a fellowship from the National Institute of Aging (T32 AG027677).