Psychometric properties of the IDS-SR₃₀ for the assessment of depressive symptoms in spanish population

In the present study we utilized the RESIST data set. The RESIST was a prospective, naturalistic, multicenter, nationwide epidemiological study. The main objective of the RESIST study was to compare residual symptoms between early and late remission in a large sample of depressive psychiatric outpatients in daily routine practice [31]. A regionally stratified sample of 400 psychiatrists was selected to participate in the study. They were proportionally distributed by regions within Spain's 17 regional communities. Each participant was asked to recruit 4 or 5 eligible outpatients. Patients were eligible for the study if they were 18 years or older, they met MDD diagnoses according to DSM-IV criteria and they had 6-8 weeks of antidepressant treatment.

A total of 1870 patients were initially recruited. 275 patients were excluded in the final analysis due to different reasons: change of treatment (9.1%), patients without second assessment (3.6%) incomplete or missing data (1.9%). Finally 1595 patients were included in the analysis.

Data collection took place from February to June 2009 after receiving the approval of the Clinical Research Ethics Committee of the Teknon Foundation. Data was collected during 2 routine visits after obtaining written consent from patients. As the RESIST study was originally designed to evaluate the residual symptoms of depression, it was required that patients had been treated with antidepressant at least 6 weeks to have depressive patients in remission at first assessment. Assessments were carried out after 6-8 weeks of antidepressant treatment and at 10 ± 2 weeks after first assessment. Treatment with any antidepressant was allowed by the study protocol. The antidepressant prescribed and the dose was entirely at the discretion of the psychiatrist. Change of treatment resulted in exclusion from the study. Interviews were done by clinical psychiatrists. Self-reports were completed on the same day during the clinical interview.

Case report form at first assessment was fulfilled by the psychiatrist and included the criteria of DSM-IV for major depressive episode that patients should meet. It included also collection of data about sociodemographic characteristics (age, gender, marital status, employment status, educational level, residence environment), and clinical features of the depressive disorder (age of first depressive episode, duration of episode, number of previous episodes, DSM-IV-TR comorbid psychiatric diagnoses and comorbid medical diseases).

The Spanish translation of the Self-rated Inventory of Depressive Symptomatology (IDS- SR₃₀) [14] was obtained from the authors and available on the website http://​www.​ids-qids.​org. There was a South American version that we adapted to the current use of the language in Spain.

The 21 item HDRS (Hamilton Depression Rating Scale) [8]: is a 21 question multiple choice questionnaire which rates the severity of symptoms observed in depression such as low mood, agitation, anxiety and weight loss. The clinician must choose the possible responses to each question by interviewing the patient and by observing the patient's symptoms. Each item has between 3-5 possible responses which increase in severity. The first 17 questions contribute to the total score and question 18-21 are recorded to give further information about depression such as if paranoid symptoms are present. The HDRS was used as a gold standard.

The Statistical Package for Social Sciences (SPSS) version 19.0 and the Mplus v5.1 were used to carry-out the statistical analyses.

Factor analyses. We made use of first assessment IDS-SR₃₀ scores for a principal component analysis (PCA). Then, a set of confirmatory factor analyses (CFAs) based on the exploratory result and on the scientific literature were performed on a new data set (second assessment scores) to find the best fitting factor structure for the instrument. Following the common assumption that the ratio of subjects per variable (item) is central for factor analysis, in both study periods we were able to satisfy the minimum of five participants-per-item ratio recommended by Kass and Tinsley [32].

Firstly, in order to make our results in the PCA comparable with those recently reported by Wardenaar [33], we performed a Horn's parallel analysis [34] to determine the number of factors to be retained. This is a Monte-Carlo based simulation method, considered more replicable than the traditional extraction techniques (Kaiser's criterion or Scree plot), that compares the unrotated eigenvalues to eigenvalues from a random sample with the same number of cases and variables. PROMAX was used for oblique rotation. To assess the suitability of the data for factor analysis, the Kaiser-Mayer-Olkin's (KMO) Measure of Sampling Adequacy [35] was computed. KMO scores above 0.90 are considered excellent. The Bartlett's test of Sphericity [36] was also applied to examine the extent to which the correlation matrices departed from orthogonality.

Secondly, we tested the fit of the following factor structures: The one-factor model of Trivedi [24] with all items loading on one depression factor, the two-factor model of Bernstein [37] with a "depression" factor (all items) and a "somatic" factor (items 25, 26, and 28), the three-factor model of Rush [14] with a "mood/cognition" factor (items 5, 8, 10-14, and 17-22), an "anxiety/arousal" factor (items 6, 7, 23-28, and 30) and a "sleep" factor (items 1-4). Items 9, 15, 20, and 29 loaded on factors 1 and 2. Item 24 loaded on factors 2 and 3. The recently three-factor model of Wardenaar [33] with a "mood/cognition" factor (items 5-8, 10, 15-23, and 29), an "anxiety/somatic" factor (11-14, 25-28) and a "sleep" factor (items 1-4). Items 9, 24, and 30 loaded on factors 1 and 2. Finally, we also tested the fit of our exploratory factor solution.

In ordinal items with a non-normal distribution, as the ones in the inventory, it may be expected that the covariances underestimate the true amount of relations among variables. Therefore, we proceeded to estimate the models from the matrix of polychoric correlations [38]. The Mean and Variance corrected Weighted Least Squares (WLSMV) was applied to test the fit of the factor models.

Although a model with a non significant chi-square estimate is generally considered a good fitting model, Hu and Bentler recommended combinational rules to evaluate model fit [39]. Therefore, the following indices were analysed (values in parentheses denote goodness-of-fit standards): the Tucker-Lewis, non-normed fit index (TLI ≥ 0.95), the comparative fit index (CFI ≥ 0.95), and the root means square error of approximation (RMSEA ≤ 0.08). We also report the Akaike's Information Criterion (AIC), a relative fit index especially designed for comparing alternative factor models. The model with the smallest AIC value has the best fit. Used together, these indices provide a more conservative and reliable evaluation of the solution. The goodness of fit index (GFI), the adjusted goodness of fit index (AGFI), the relative fit index (RFI), and the normed fit index (NFI) were not used because these indices would likely be affected by the large sample size. Model comparisons were performed based on a practical improvement in model-fit approach (TLI difference ≥ 0.01) [40, 41].

Internal consistency. Stratified Cronbach's coefficient [42] was used to assess the reliability in the presence of subscales. Subscale internal consistency estimations were conducted on the best-fit model observed following CFA.

Construct validity. Pearson product-moment correlations computed between the IDS-SR₃₀ factors and the HDRS. We took into account the Cohen's criteria [43] to evaluate the substantive significance of correlations (large correlations are those > 0.50, medium correlations are from 0.30 to 0.49, and small correlations are from 0.10 to 0.29).

The KMO measure produced a coefficient of 0.96, indicative of excellent sampling adequacy. Bartlett's test of Sphericity produced a figure of 21669.61 (P < 0.0001), indicating that the correlation matrix was unlikely to be an identity matrix and was therefore suitable for factor analysis. According to the Horn's parallel analysis three components were extracted. Factor loadings (after rotation) are displayed in Table 3. Tabachnick and Fidell [46] suggest that, in exploratory factor analysis, an item forms a part of a factor if its factor loading on that factor is at least 0.32 and at least 0.10 greater than its other factor loadings. Three items (items 3, 9, and 29) did not meet these criteria. Given that it is unlikely that the exclusion of the three items would yield a significant improvement in model fit in the subsequent CFAs, all further analyses were carried out on the IDS-SR₃₀. The present three-factor solution (labelled Mood/Cognition, Anxiety/Somatic, and Sleep), which accounted for 49.87% of the variance, was more similar to the three-factor structure reported by Rush [14] than to the three-factor structure recently reported by Wardenaar [33]: there were divergences (different primary loading) in 5 items (items 3, 15, 23, 29, and 30) and 8 items (items 3, 6, 7, 11/12, 13/14, 24, 29, and 30), respectively.

Fit statistics for the factor models are shown in Table 4. Although all models showed RMSEA values that are considered acceptable (< 0.08), the one-factor and two-factor solutions failed to provide good fit taking the other indices into account. Considering the findings collectively, all three-factor models seem adequate for the current sample. However, as the Hu and Bentler's guidelines for retaining a hypothesized model recommended, the factor structure posited by Wardenaar [33] received more support than the other three-factor structures because obtained the lowest AIC value and its CFI and TLI values reached the rule of thumb minimum value of 0.95. Furthermore, the TLI value was 0.01 greater (0.95 vs. 0.94). The correlations between the Wardenaar's factors were all positive and statistically significant (p < 0.01): Mood/Cognition * Anxiety/Somatic = 0.76; Mood/Cognition * Sleep = 0.78; and Anxiety/Somatic * Sleep = 0.71. We decided to retain Wardenaar's model for further analyses.

Prior to examining the internal consistency and convergent validity of the IDS-SR₃₀, the three items that had cross-loadings (9, 24, and 30) were assigned to the subscale with which they had the highest factor loading on the CFA to compute the subscale scores. Thus, items 9 and 30 were assigned to the Mood/Cognition sub-scale, whereas item 24 was assigned to the Anxiety/Somatic sub-scale.

The results of the reliability analysis are shown in table 5. The stratified overall Cronbach's coefficient was 0.94 at first and second assessment. Cronbach's α coefficient ranged from 0.54 to 0.93 in the three subscales derived from the Wardenaar's factor model. If item 4 was deleted, the Cronbach's α of the Sleep sub-scale would be 0.73 and 0.64 at first assessment and second assessment, respectively.

The IDS-SR₃₀ would show adequate convergent validity if the total and subscale scores correlated significantly with the HDRS. As can be seen in Table 6, the correlations between Mood/Cognition, Anxiety/Somatic, Sleep, the IDS-SR₃₀ total score and the HDRS were all significant and large at both assessments.