Comparing the reliability and validity of the SF-36 and SF-12 in measuring quality of life among adolescents in China: a large sample cross-sectional study

Stratified cluster random sampling was adopted [31], first dividing regions by geographical location: Dongguan, Shanghai, Shenyang, Wuhan, Xi’an, and Kunming represented the south, east, north, central, northwest, and southwest regions, respectively. These areas were chosen in order to ensure proper representation by including participants from geographically diverse areas. Second, middle schools were randomly selected and followed by grade (first grade of junior school to third grade of high school). The basic sampling frameworks were all middle schools, as reported by each city. In each city, middle schools were selected by simple random sampling according to a random number table. Finally, 17 middle schools were included (4 in Dongguan, 1 in Shanghai, 3 in Shenyang, 1 in Wuhan, 4 in Xi’an, and 4 in Kunming). The number of schools in each city was limited by the research group's local investigative capacity.

All students enrolled from the first grade of junior high school to the third grade of high school were included in the survey. The exclusion criteria were those with any physical or mental condition that made them unable to complete questionnaires or students and their parents who had not signed an informed consent form. The study was approved by the Institutional Review Board (IRB) at the Affiliated Hospital of Guangdong Medical University. Verbal informed consent for publication was obtained from the participants and/or their relatives, as approved by the IRB. The response rate was almost 80%. This present study included 19,428 adolescents with complete information on quality of life measures. The sample sizes for each region were Dongguan (4490, 23.1%), Shanghai (1039, 5.3%), Shenyang (3539, 18.2%), Wuhan (1371, 7.1%), Xi’an (4197, 21.6%), and Kunming (4792, 24.7%).

SF-36 (version 1) was used to assess QOL. Compared with version 2, the differences lie in two points. First, the answers-rank of RP, RE, MH, and VT are distinct, and second, the scoring rules are different [32]. Since the use of SF-36 (version 2) requires authorization, version 1 was used in this study. Based on the response to individual items comprising the 8 subscales and using a z-score transformation, the scores of each subscale were calculated [33]. First, the domain items were coded; second, the items were scored; and finally, the scores were converted as shown in Formula 1.

Scoring norms for the Chinese version of the SF-36 (version 1) and SF-12 are not given at present by studies, so scores of these instruments were mainly based on American norms in China that have been proven to be valid [23, 32, 34]. Using Z-transform scores and factor score coefficients, we calculated PCS and MCS scores of the SF-36 according to Formulas 2 and 3:

SF-12 component summary scores (eight subscales, PCS-12, and MCS-12) were calculated using the SF-12 items that are embedded in the SF-36 [35]. This method has been presented as being equivalent to calculating the SF-12 as a stand-alone questionnaire [17]. All summary scores range from 0 to 100, where higher scores indicate better QOL. We calculated PCS and MCS scores of the SF-12 according to the SF-12 scoring algorithm proposed by John E Ware in 1995 that has been widely used in China [36].

For descriptive analyses, we aimed to show overall demographics and QOL. We calculated average and standard deviations in QOL scores by SF-36 and SF-12. For testing their relevance, the Pearson correlation coefficient was used to show correlation between the domains of SF-36 and SF-12.

Cronbach’s alpha for domains composed of multiple items and construct reliability (CR) were used to evaluate the reliability of the SF-36 and the SF-12, and validity indicators were represented by criterion validity and convergence validity (average variance extracted, AVE). Criterion validity was expressed by the correlation between the response of each domain and self-reported health status. The calculation of the formulas for CR and AVE are shown in Formulas 4 and 5. λ = factor loading, θ = measurement error.

The sample was randomly split into a training set (50%) and a validation set (50%) to examine the construct validity of the SF-36 and SF-12. Using the training set, an exploratory factor analysis (EFA) was performed to explore the latent structure based on correlations matrix, and factor loadings (λ) were estimated by maximum likelihood estimation and rotated by promax. Using the validation set, confirmatory factor analysis (CFA) was used to validate the identified two-factor structure in some Chinese populations. The weighted least-square method was used for the estimation of CFA parameters. Factor loadings (λ) were taken for standardized regression coefficients. The classic goodness-of-fit χ² statistic and its degrees of freedom were reported. However, as the χ² statistic is highly sensitive in large samples, assessment of goodness-of-fit was based on the fit indices as recommended: the root mean square error of approximation (RMSEA, close to 0.06 or lower) and the comparative fit index (CFI, close to 0.95) [37]. The basic two-factor CFA model (Model I) without correlated errors was first assessed (PCS associated with PF, RP, BP, and GH, whereas MCS associated with VT, SF, RE, and MH). Subsequently, the factor structures PCS and MCS, associated with most of the 8 domains (Model II) as described above, were also incorporated [23]. Then, the EFA or CFA was repeated on another data set, and mean estimates were reported.

According to the evaluation results of the samples, and taking into account the characteristics of the ordered and multi-category forms of the instrument items, the Semejima grade response model (logistic two-parameter module) in item response theory was used to estimate the discrimination, difficulty, and average information of each item [38]. RStudio, Amos 20.0, and Multilog 7.03 were used to process data.

Of the 20,226 questionnaires received, 798 had no responses on some of the SF-36 items. In the end, 19,428 samples were included in the study. The mean age of the sample of respondents was 14.78 years (standard deviation [SD] = 1.77), and 49.4% (9,595) were boys. Among the SF-36 and SF-12 domains, the PF mean score was the highest, and the RE mean score was the lowest. PCS was better than MCS. The biggest mean difference in scores between the two instruments was in the domain of SF. Of the corresponding domains, the RE domains were the most relevant (r = 0.923), while the smallest correlation coefficient was in the VT domains (r = 0.670), which means domains of the SF-12 could reflect the information from 67.0% to 92.3% of the corresponding domains of the SF-36 (Table 1).

The parameter values and information content of the items according to the Samezima grade response model are shown in Table 6. The discriminations of items were between 0.45–2.73, with a large gap. The difficulty of the items ascended from the lowest level to the highest level unidirectionally, which met the difficulty assumptions estimated by the model. The average amount of information of each item was between 0.07 and 1.02.

In the SF-36, the domains of PF, RP, GH, and RE had acceptable discrimination of items (> 1), but the remaining dimensions were less differentiated, especially BP and SF, probably because for teenagers there was strong homogeneity between individuals in terms of physical pain and social function. On the other hand, in the SF-12, BP, SF, RP, and VT had higher discrimination of items than in the SF-36.

With reference to the relevant literature, the amount of information measured on the scales > 25 indicated that the quality of the evaluation items was good; the amount of information < 16 indicates that the evaluation items were poor [31]. Given the number of items on the instrument for the SF-36, we divided 25 and 16 by 36 to get the average information amount for each item, so as to obtain the determination criterion: the average information amount of excellent items was > 0.69 (25/36), while items < 0.44 (16/36) were judged to be poor. Similarly, for the SF-12, the average information amount of the excellent items was > 2.08, while items < 1.33 were judged to be poor. Except for PF05 and PF09, the items of the PF domain in the SF-36 were excellent, and the items of the GH domain in the SF-36 were excellent too, though the items of BP, VT, SF, RE, and MH were poor. On the other hand, the average amounts of information in the SF-12 items were poor (Table 6).