Evaluating the validity and reliability of the Chinese entrapment scale and the relationship to depression among men who have sex with men in Shanghai, China

A cross-sectional study was conducted from March to November 2018 in four districts of Shanghai. After removing missing data, a total of 304 participants were included in the analyses. The inclusion criteria were: biological males aged over 18 years who had engaged in sexual behavior with men over the past 6 months. The exclusion criterion was participants with mental or cognitive impairment, unconsciousness so that they cannot verbalize their real feelings or fill out questionnaires.

The hidden nature of MSM prevented this study from conducting a large-scale investigation with random sampling; therefore, a snowball sampling method was used [30, 31]. First, 5–10 eligible MSM were selected as initial “seeds” in each district with the help of the local Center for Disease Control and Prevention and non-governmental organizations. Then, these participants were tasked with recruiting eligible subjects from the same sociocultural background. These second groups of participants were also asked to provide information on other potential subgroup participants, and this process continued until no sample could be found through snowball sampling method.

The investigators were gathered together and trained to understand the survey content, methods, and relevant precautions. The investigators reached consensus on the health behavior questionnaire. Anonymous face-to-face interviews with participants were conducted as follows. First, the investigators explained the goal and procedure of the survey to participants in detail, answered any questions, and obtained their written informed consent. Next, each participant was asked to independently complete a self-administered questionnaire in a private room. The questionnaire took around 30 min to complete. After completion of the questionnaires, the investigators performed integrity checks and logic checks for each questionnaire, and resolved any problems in a timely manner to ensure accuracy of the collected data.

The 16 items of the Chinese version of the ES are divided into external entrapment (items 1–10) and internal entrapment (items 11–16). The response options for each item are “not at all,” “a little bit,” “moderately,” “quite a bit,” and “extremely,” which correspond to scores of 0–4. The total score ranges from 0 to 64. A higher score indicates a stronger sense of entrapment. The final Chinese version of the ES is detailed in Additional files 1.

The PHQ-9 was developed for criteria-based screening and diagnosis of depression [32]. The scale has been widely applied in primary care settings and demonstrated acceptable psychometric properties [33‐35]. Compared with other commonly used clinical depression assessment tools, the PHQ-9 has the advantages of having fewer items, being easier to understand, and less time-consuming. The scale comprises nine items that evaluate the frequency of depressive symptoms in the previous 2 weeks. Each item is scored from 0 to 3 (representing “not at all”, “a few days”, “more than half a day” and “nearly every day”). Total scores range from 0 to 27. The optimal cutoff point is ≥10, which was described as diagnostic depression in a systematic review [36]. Many studies have confirmed that the generation of entrapment and defeat may trigger depression and lead to poor psychological states such as lack of self-esteem and self-confidence [5, 8, 9, 37]. Therefore, this study used the PHQ-9 to assess depressive symptoms among MSM, consistent with previous studies [38, 39]. The internal consistency reliability (Cronbach’s α coefficient) of the PHQ-9 in this study was 0.874.

Participants’ sociodemographic characteristics were calculated by numbers and proportions. The ES scores were described as mean ± standard deviation (SD) and median (inter-quartile range, IQR). A histogram of the ES scores was provided in Additional file 2. Differences between sample subgroups were tested with non-parametric tests. Participants were randomly divided into two groups using a random number generator to perform exploratory factor analysis (n = 143) and confirmatory factor analysis (n = 161) to evaluate construct validity of the Chinese ES. In exploratory factor analysis, we performed Kaiser-Meyer-Olkin (KMO) test and Bartlett’s test of sphericity to determine the feasibility of factor analysis. Then principal component analysis was used to explore the dimensionality of the ES. Confirmatory factor analysis was conducted to compute the model fit indices of the scale. The ratio of chi-square and degrees of freedom (CMIN/DF) between 1 and 3, root of the mean square residual (RMR) under 0.05, normed fit index (NFI), incremental fit index (IFI), Tucker-Lewis index (TFI), and comparative fit index (CFI) greater than 0.9 indicate that the model is goodness of fit [40]. The root mean square error of approximation (RMSEA) between 0.08 and 0.10 indicates that the model is acceptable and has a mediocre fit; the value between 0.05 and 0.08 means reasonable fit and if the value is less than 0.05, the model fit very well [41]. In order to further confirm the dimensionality and model fit indices of the scale, we used package in R (‘mirt’) to run Item Response Theory (IRT) model and M2 test and the results were shown in Additional file 2. The criterion validity, the extent to which the ES scores relate to a gold standard, was evaluated between the ES and PHQ-9 scores using Spearman’s correlation coefficient. The correlation of at least 0.70 with the gold standard is considered a positive rating for criterion validity [42]. Cronbach’s α and Spearman-Brown coefficients were used to evaluate the internal consistency reliability and split-half reliability, respectively. Usually 0.70 is recommended as a minimum standard for reliability [42].

We performed two hierarchical regression analysis to test unique associations of sociodemographic factors, entrapment with depression. In step 1, unique association of significant sociodemographic characteristics to depression was tested. The entrapment factor was added in the step 2, testing the increment in the explained variance of entrapment in the prediction of depression beyond sociodemographic factors [5, 7, 43]. The coefficient of determination, denoted R², was used to indicate the proportion of the variance in the depression that is predictable from the sociodemographic and entrapment factors. Finally, receiver operator characteristic (ROC) curve analysis was performed to calculate optimal cut-off value of the ES for predicting depression [34]. The area under ROC curve (AUC) can measure the ability of an instrument to distinguish whether a subject has changed or not according to an external criterion [44]. An AUC of at least 0.70 is adequate [42]. P < 0.05 was considered statistically significant. All analyses were performed using SPSS 25.0, AMOS 24.0 and R software (version 3.6.1).

Table 1 presents an overview of participants’ sociodemographic characteristics and comparisons of the ES scores between subgroups. The age of participants ranged from 18 to 69 years, with a mean age of 29.91 years. Most (85.2%) participants were unmarried and 69.7% were self-identified homosexuals. The results suggested that only the difference of the ES scores between married and unmarried subgroups was statistically significant (P = 0.011).

There were some limitations in this study. First, participants might have had some concerns when completing the questionnaire because of privacy issues, which could have resulted in information bias. However, all investigators have participated in trainings and an anonymous, self-administered questionnaire was used to maximize the data quality. Second, the MSM population in the study setting is small and relatively hard to reach, meaning strict random sampling was impossible for this study. The snowball sampling method used in this study inevitably produces selection bias and sample representation problems. However, snowball sampling can identify more subjects that meet study requirements relatively easily at low cost and high efficiency. Third, only one scale of depression was evaluated in this study. Maybe other scales of depression can be added in the further study to get a more accurate understanding of depression in the MSM population. Another limitation is that ES was not given a second time in the same individuals due to the specificity and anonymity of the MSM population, hence the test-retest reliability cannot be measured. Finally, participants were limited to the MSM population in Shanghai, which is an economically developed and culturally open city; the social acceptance of MSM may be higher than in other areas. Therefore, this sample cannot represent all MSM populations in China. Use of the ES should be further explored in other areas of China.