Reliability, Validity, and Classification Accuracy of a Measure of DSM-IV Diagnostic Criteria for Pathological Gambling

Pathological gambling was formally recognized as a mental disorder by APA in DSM-III. These diagnostic criteria have since been revised twice, and the current standard is DSM-IV. In spite of the fact that the DSM diagnostic criteria are the primary tool for diagnosing pathological gambling, there is little information about the reliability, validity, and classification accuracy of measures of these diagnostic criteria. A few studies (1–3) have reported on the reliability and validity of DSM-IV, but most of these were pilot studies of small clinical groups. No studies have reported on the classification accuracy of the DSM-IV diagnostic criteria, to our knowledge. In a recent meta-analysis of prevalence studies of pathological gambling by Shaffer et al. (4), it was reported that the DSM diagnostic criteria have been used to measure the prevalence of pathological gambling in different types of groups; however, few of these studies have reported the reliability or validity of the diagnostic criteria in these groups. Given the lack of information about the classification accuracy of the DSM-IV diagnostic criteria for pathological gambling, it seems appropriate at this time to examine these properties of a measure of the DSM-IV diagnostic criteria for pathological gambling.

Method

Two study groups were recruited: 1) a general population sample obtained from the 1995 Minnesota State Survey conducted by the University of Minnesota Center for Survey Research (N=803) and 2) clients from six state-supported Minnesota gambling treatment programs who were recruited between January 1995 and June 1996 (N=259) (5). The general population sample was primarily women (55%), 96% were white, most had a high school education or higher (92%), and ages ranged from 18 to 90, with an average of 45 years. The gambling treatment group was primarily men (56%), 94% were white, most had a high school education or higher (95%), and ages ranged from 18 to 70, with an average of 39 years.

The general population survey instrument included the 19-item measure of DSM-IV diagnostic criteria as well as the 20-item South Oaks Gambling Screen (6, 7). In the gambling treatment group, the DSM-IV questions and South Oaks Gambling Screen were part of the 91-item paper-and-pencil client intake questionnaire that was part of the Minnesota gambling treatment outcome study (5).

The 10 DSM-IV diagnostic criteria were paraphrased into 19 questions. There are two items for each criterion, except criterion 4 (“is restless or irritable when attempting to cut down or stop gambling”), which has one question. Response options were “yes” or “no”; a “yes” answer was scored 1, and a “no” answer was scored 0. Nine of the 10 DSM-IV criteria have two questions, and if either question is endorsed in the pathological direction, the criterion is considered endorsed. The DSM-IV criteria are scored by summing across the 10 criteria; a cutoff score of 5 or more indicates a diagnosis of pathological gambling.

The general population sample was recruited through the 1995 Minnesota State Survey, which is an anonymous telephone survey of Minnesota adults (18 years of age and older) conducted annually by the University of Minnesota Center for Survey Research. Individuals were randomly selected from the population of Minnesota. A total of 803 telephone interviews were completed, with a response rate of 70%.

The gambling treatment group was recruited from clients seeking treatment at six state-supported gambling treatment programs in Minnesota (5). After complete description of the study to the clients, written informed consent was obtained. Clients who gave consent to participate in the study were administered the client intake questionnaire at admission assessment by treatment staff. Clients were informed that their answers were confidential and that their names would not be written on questionnaires.

Results

Reliability was estimated by using a measure of internal consistency, Cronbach’s alpha (8). Internal consistency for the 10 criteria were alpha=0.81, 0.77, and 0.98, for the general population, gambling treatment, and combined groups, respectively.

Construct and convergent validity were examined. Construct validity of the DSM-IV diagnostic criteria would be evident if the DSM-IV scores in the general population sample were significantly lower than the DSM-IV scores in the gambling treatment group (9). DSM-IV diagnostic criteria means and standard deviations were computed for the general population sample (mean=0.1, SD=0.6) and the gambling treatment group (mean=8.5, SD=1.9). An independent-groups t test yielded a statistically significant difference between the two groups (t=108, df=1055, p<0.01). Convergent validity was examined by computing correlations between the DSM-IV diagnostic criteria score and the South Oaks Gambling Screen, a convergent measure of the same construct. Evidence for the convergent validity of the DSM-IV would be demonstrated if the DSM-IV diagnostic criteria score were significantly correlated with the South Oaks Gambling Screen. The correlation between DSM-IV and the South Oaks Gambling Screen was r=0.77 (p<0.01) and r=0.75 (p<0.01) for the general population and gambling treatment groups, respectively.

With a standard DSM-IV cutoff score of 5 or greater and a cross-tabulation of DSM-IV diagnosis and group membership, the DSM-IV had the following classification accuracy indices (10, 11): hit rate=0.98, sensitivity=0.95, specificity=0.996, false positive rate=0.004, and false negative rate=0.05. Lowering the cutoff score to 4 yielded the following classification accuracy indices: hit rate=0.99, sensitivity=0.97, specificity=0.99, false positive rate=0.01, and false negative rate=0.03.

Table 1 shows the probability of group membership for all 10 DSM-IV scores. There was an increasing probability of being a member of the treatment group with each increase in DSM-IV score. A score of 0, 1, or 2 resulted in almost no chance of being a gambling treatment client. A score of 5 indicated an 82% likelihood of gambling treatment membership, and scores of 6 or greater indicated an almost certainty of being in the gambling treatment group. A score of 3 indicated a 29% chance, and a score of 4 indicted a 63% chance of being a gambling treatment client.

Discussion

The DSM-IV diagnostic criteria for pathological gambling, when operationalized into questions, demonstrated satisfactory reliability, validity, and classification accuracy. However, modest improvements in classification accuracy were obtained by lowering the cutoff score to 4. Most important, the false negative rate improved from 0.05 to 0.03 with a cutoff score of 4. False negative errors may be considered the most serious errors in the diagnosis of pathological gambling because they are likely to have greater and more serious consequences than false positive errors.

Clinicians can be fairly confident that the respondent is not a pathological gambler if his or her DSM-IV score is between 0 and 2, and the clinician can be fairly confident that the respondent is a pathological gambler when his or her DSM-IV score is 5 or higher. However, clinicians should be careful when diagnosing individuals who have DSM-IV scores of 3 or 4 because individuals in this score range had about a 50/50 chance of being a gambling treatment client.

It should be noted that although applying a cutoff score is standard diagnostic practice, respondents were distributed across the continuum of DSM-IV scores, and there were some members of the gambling treatment group whose scores fell below the cutoff score of 5. However, most had scores not far below the threshold. Probabilities of group membership associated with specific DSM-IV scores may prove helpful in the diagnosis of pathological gambling.

There were classification errors, and these errors could have occurred for a variety of reasons. It will be important for future research to explore the causes of these false positive and false negative classification errors. There are likely other signs and symptoms of pathological gambling that should be considered potential diagnostic criteria, and that may improve diagnostic classification. Future research will need to identify other signs and symptoms of pathological gambling and to test them out to see if they enhance current diagnostic methods.

Overall, the DSM-IV diagnostic criteria have demonstrated satisfactory reliability, validity, and classification accuracy. However, there continues to be room for improvement in diagnostic assessment, and future research will need to examine methods and additional signs and symptoms that have the potential to improve diagnostic precision.

TABLE 1

Enlarge table