Factor structure and diagnostic efficiency of the Myanmar version BDI-II among substance users

Current substance users were recruited for this study, through their contact with a local non-government organization (NGO) providing harm reduction services in Hseni Township, Northern Shan State of Myanmar. The inclusion criteria included being aged 18 years or older, reporting using drugs (opioids, heroin, methamphetamine, etc.) within the past month, and providing informed consent to participate. Participants were recruited from people who came to a services center for accessing harm reduction services. A respondent-driven sampling (RDS) method was applied for the main evaluation study, because RDS is known as an effective data collection method for a hidden population like substance users [22]. First, 11 seeds (initial key persons) with different age groups and residential places were purposely identified from these people in Hseni Township. Each seed was given three coupons to recruit up to three other participants who he/she knew, into the study; each participants recruited by the seed was also given the same number of coupons to recruit further participants. This recruitment chain stopped at the third wave. In total, 750 coupons were distributed, and 230 coupons were returned. On the other hand, 50 participants who visited the drop-in center (DIC) were assessed by a consultant psychiatrist if they had a diagnosis of depressive disorders using ICD-10 criteria. All participants were interviewed face-to-face using a semi-structured questionnaire including the mBDI-II. Participants were provided with some snacks as an incentive, and informed consent was obtained from all.

Data were analyzed using the Statistical Package for Social Science version 21 program (SPSS 21) (IBM Corp., USA). Descriptive analysis was performed to describe the socio-demographic characteristics. Mann–Whitney U test was used to compare the score distribution between the groups. Sensitivity and specificity of the mBDI-II score were plotted against the ICD-10 diagnosis by the psychiatrist as a standard. The performance of the mBDI-II was analyzed using receiver operating characteristics (ROC) curves, and area under the ROC curves (AUC) with 95% confidence intervals was calculated using a nonparametric method. Sensitivity and specificity were calculated for an original cutoff score (10+) [7], a cutoff score proposed by a previous study of substance users [14‐25], and the optimal screening cutoff score based on the closest point to the left hand corner, and the Youden index [25].

To examine the factor structure of the mBDI-II, confirmatory factor analysis (CFA) was done in this study. Using CFA, we compared four distinct models. Model 1 constrained all 21 items to load on a single factor (depression). Model 2 was based on the oblique two-factor solution (somatic–affective and cognitive) reported by Beck et al. [7] for BDI-II normative data from psychiatric outpatients. Model 3 was based on the oblique two-factor solution (somatic–affective and cognitive) reported by Beck et al. [7] for the BDI-II normative data from colleague students. For both Models 2 and 3, items were limited to loading only on the models as reported by Beck et al. [7], variance of each factor was fixed at 1, and the two factors were allowed to correlate. Model 4 was based on a three-factor solution (cognitive, affective, and somatic) reported by Buckley et al. [21] for substance users currently in chemical dependency treatment. Variance for each factor was fixed at 1, and the three factors were allowed to correlate.

For each of the four models, four indices were used to assess model fit. First, a Chi-square fit index was calculated to compare each given model’s covariance structure with the observed covariance matrix. A relative likelihood ratio (X²/df) (RLP) was calculated in this study. Second, root mean square error of approximation (RMSEA) adjusts for model complexity and provides a measure of predicted and observed model. Third, the Comparative Fit Index (CFI) compares the observed model fit with an independent model that assumes the latent variables are uncorrelated. The last index was the Expected Cross-validation Index (ECVI). Lower ECVI values mean better fitting of the model. The reliability of the mBDI-II was analyzed using the internal consistency coefficient and corrected inter-item correlations.

The mBDI-II median score for the participants who had been diagnosed based on ICD-10 criteria was 20.00 (0–38), and for the others, the median score was 14.00 (0–40) (p = 0.003) (Table 1). The participants who had been diagnosed based on ICD-10 criteria had significantly higher mBDI-II mean scores than the comparison group (p = 0.003).

The demographic characteristics of clinically depressed substance users and non-clinically depressed substance users are reported in Table 2. The majority of the participants in both groups were male, and 53.8% of the clinically depressed patients were ethnic people; however, 62.9% of non-clinically depressed substance users were Burmese. Non-clinically depressed substance users were slightly older than the clinically depressed group, and 50% of them had never attended school. Regarding drug use history of the participants, 81.0% of the participants reported having used heroin within 30 days, and 45.6% within 24 h and 29.2% of participants reported having used amphetamine-type stimulants (ATS) within 30 days and 10.6% within 24 h). These two drugs were the most commonly used drugs.

The mBDI-II showed moderate accuracy on the AUC, and it had an AUC of 0.69 (95% CI 0.45–0.92) (Fig. 1). The optimal threshold score for screening was obtained from the ROC curves comparing the mBDI-II and ICD-10. Table 3 shows the sensitivity and specificity of cutoff scores for the detection of clinical depression with different cutoff scores from this study and other studies among substance users. According to the highest Youden index of 0.48, the optimal cutoff score was 10 with a sensitivity of 0.78 and a specificity of 0.70. The previous optimal cutoff point of 13 had a lower Youden index, sensitivity, specificity, and likelihood ratio compared to the cutoff point 10 in this study. The left-most point in the ROC curve was also 10 in this study. The likelihood ratio for positive test results (LR+) was 2.6, and likelihood ratio for negative test results (LR−) was 0.31 in this study.

Table 4 describes summary statistics for the CFA of the mBDI-II. The “goodness-of-fit” test indicated that the poorest “fit” was obtained for the one-factor solution. Of the remaining four models, the next poorest “fit” was the two-factor model. Among the remaining two models, the two-factor solution was the poorest “fit.” Therefore, the three-factor solution “fits” the best for these data.

The Cronbach’s alpha for both groups was high, and Cronbach’s alpha for the clinically depressed group was 0.91 and for the non-clinically depressed substance users was 0.93. The item-total correlation ranged from 0.15 to 0.79 for the clinically depressed sample and 0.33 to 0.76 for the non-clinically depressed substance users. Lower item-total correlations were observed for item #9, “suicidal thoughts,” #10, “crying,” and #21, “loss of interest in sex” (0.28, 0.33, and 0.15, respectively).

In this study, we examined the factor structure and diagnostic accuracy of the mBDI-II among substance users. CFA indicated that a three-factor solution (cognitive, affective, and somatic) was the best fit for these data. To detect depressive symptoms, which we defined the clinically depressed and the non-clinically depressed substance users, the total mBDI-II score had an area under the curve of 0.68. The optimal cutoff score was 10 for the highest Youden index (0.48) and with high sensitivity and specificity (0.78 and 0.70). An acceptable level of internal consistency reliability was obtained: The Cronbach’s alpha coefficients among clinically depressed and non-clinically depressed substance users were 0.91 and 0.93, respectively.

The mean scores for the mBDI-II among the clinically diagnosed and the non-clinically depressed substance users were 19.9 and 14.4, respectively. This finding is similar to the mean scores in other studies among substance users ranging between 14.9 and 20.6 in the USA [26]. The mean score for the mBDI-II is also similar to the mean score for the BDI-II (13.4) in a study conducted in India, in which the study population was school adolescents [14]. The score distribution of the mBDI-II may be similar to that in other countries.

The AUC of the mBDI-II was moderate (0.68). The AUC is smaller than that reported in another study of substance users in the USA (0.82) [20]. The BDI-II may have less effective performance in screening depression among substance users in Myanmar than in high-income countries, such as the USA. In low- and middle-income countries, a smaller AUC was occasionally reported. For instance, the AUC of this study is similar to those in studies among school adolescent in India (0.66–0.67) [14, 15]. However, it should be noted that we did not diagnose depressive disorders among the non-clinically depressed sample; and thus, some of that sample may have depression. This may have resulted in underestimation of the screening performance of mBDI-II in this study.

The proposed optimal cutoff score in this study was 10 (sensitivity of 0.78 and specificity 0.70) for the diagnosis of depression among substance users. This cutoff score was less than the developer recommended cutoff score [7]. In contrast, the results for the clinically depressed substance users study [20] were between 14 and 25. Other studies of adolescents stated the cutoff score of 18 with sensitivity and specificity of 0.63 and 0.73 [15]. However, the previous studies [15, 20] did not describe the LR+ and LR−. The present LR+ 2.6 is reasonable to use, and it means that more positive test results will occur in the depressed substance users than in non-depressed users. The present LR-0.31 was close to “0,” and thus, it is less likely that the negative test result will occur in depressed substance users than in non-depressed users.

A three-factor solution (cognitive, affective, and somatic) was the best fit model for these data (Table 4). Not all indices, however, indicated an acceptable fit. The RMSEA (0.09) was higher than the values of 0.05, 0.48 and 0.67 obtained in previous studies of substance users [19‐21]. In addition, the CFI in this study was 0.86, while it was better (0.94) in the other study [19]. The lower fit of the solution in this study may be attributable to some of the items being less correlated with other items, as described later. The discrepancy might also be due to the study population. Our study population may be different from that in other studies in terms of the degree of substance abuse and/or treatment seeking.

The internal consistency of the mBDI-II in this study was acceptably high. This is in agreement with other studies conducted among substance users [17, 18]. A previous literature review reported that the internal consistency for the BDI-II ranged from 0.83 to 0.96 [26], which is similar to the present findings. We found a low item-total correlation of item #9 (suicidal thought), #10 (crying), and #21 (loss of interest in sex) in this study. The mBDI-II items #10 and #21 were reported in our pretest to be difficult to respond to as well. Suicide is often stigmatized more in some cultures [27]. This may be the case for substance users in Myanmar. The item on crying was reported not to converge with other depression items in some populations [28, 29]. Loss of interest in sex showed a different item functioning in a validation study of BDI-II some samples [30]. These items may be more affected by the culture of the study population and may not reflect depression appropriately.

Several limitations should be acknowledged for this study. Participants in this study were only substance users who identified by themselves; therefore, the result may not generalize to other populations. RDS depends on the social network of sampling population; therefore, it can limit the sample representatives. In addition, the sample size of our study is limited, and especially, the sample of participants who underwent clinical diagnosis was small. Depressive symptoms caused by substance use and withdrawal were not assessed in this study, and it can affect the validity of the instrument. An additional limitation is that we were not able to evaluate the validity of the mBDI-II by comparing responses to other depressive symptomatology instruments. Moreover, there may be clinical depressive cases in the community residents of this study and it can affect the validity of the instrument. Further studies will need to establish convergent validity and factor structure based on substance dependence severity.