The Risk of Gambling Problems in the General Population: A Reconsideration

See Gerstein et al. (1999), Williams and Wood (2004), Australian Productivity Commission (1999) and Abbott and Volberg (2000), respectively. Definitions of those most at risk of gambling-control problems vary across studies, for reasons we discuss in detail, but all are conventional in the extant literature.

Kessler and Pennell (2015, p. 144ff.) provide a valuable review of the historical evolution of survey research on mental disorders.

Examples of the many studies of the continuum of gambling disorders include Toce-Gerstein et al. (2003) and Blanco et al. (2006).

Harrison and Ng (2016) is an example of our general approach, applied to the problems of making decisions over an insurance product to evaluate the welfare cost to the individual of observed choices. That cost is measured by the foregone income-equivalent of the observed choice compared to the choice a latent structural theory predicts that the individual should have made. Calculating this income cost, which in the case of insurance arguably maps relatively straightforwardly onto welfare costs, requires a different set of data about the individual than one finds in surveys, but the end result is more usefully compared to non-binary measures of the severity of behavior.

The second wave of the NESARC was conducted in 2004/5, and was a longitudinal panel of 34,653 re-interviews from the first wave. The third wave was conducted in 2012/13, with a fresh sample of 36,309 individuals. Gambling prevalence questions were removed from waves 2 and 3 of the NESARC. Our analysis was prepared using a limited access data set obtained from the National Institute on Alcohol Abuse and Alcoholism (NIAAA) and does not reflect the opinions or views of NIAAA or the U.S. Government.

A comparable national survey that could also be evaluated in the same manner is the National Comorbidity Survey Replication conducted in the United States between 2001 and 2003 with a primary sample of 9282 individuals. We discuss the Canadian Community Health Survey of Mental Health and Well-Being of 2002 and the British Gambling Prevalence Survey of 2010 below.

Hernán et al. (2004) survey the many types of selection bias considered in epidemiology, and provide a general causal framework. The selection bias of concern here is a mixture of what they call “nonresponse bias/missing data bias,” “volunteer bias/self-selection bias,” and “health worker bias” (p. 618). Various statistical correction methods are discussed in major epidemiology texts, such as Rothman et al. (2012, ch. 19). To our knowledge, there are no applications of epidemiological corrections for these biases to general population surveys with trigger questions, although there are recognitions of their potential importance (Tam et al. 1996; Tam and Midanik 2000). Caetano (2001, p. 1543) editorialized on this issue in a clear fashion: “So, are survey respondents different from non-respondents in their use of alcohol and illicit drugs? The answer from the small number of studies mentioned above seems to be positive. But are my critics right in assuming that non-respondents are more likely than respondents to be drinkers, heavier drinkers or dependent on alcohol and illicit drugs? The evidence then suggests that, to use a common American expression, the ‘jury is still out.’ This is so partly because for the past 40 years those of us facing the critics have been complacent about the validity of survey research. The uncertainty regarding selective non-response should not justify the apparent lack of attention to the issue.”.

Their analysis, and most of those using the NESARC to study DG, suffers from an unfortunate coding error explained in Appendix A. There are in fact 207 respondents that meet the DSM-IV criteria, not the 195 used in most studies. The incorrectly coded classification had 21 respondents that should have been classified as DGs, and 9 that should not have been so classified. The effect is to change estimates slightly. We only use the correct DSM-IV classification of pathological gambling from the NESARC. None of our qualitative conclusions are affected by using the incorrect classification.

A constant term is always employed as well. This is what Petry et al. (2005) refer to as “model 1,” where there are no additional covariates added.

Epidemiologists often report “adjusted odds ratios,” which control for covariates. Typically the list of covariates is very small.

We know well the dangers of inferring causality from correlations, and indeed this concern is why many modern surveys of mental health take time to ask additional questions about “age of onset.” This information is particularly important when asking about incidence over lifetime frames, since the correlation might have any one of three temporal sequences (prior, simultaneous, and posterior). This is also why one-shot general population surveys are not the same as clinical evaluations that occur over several meetings, despite the attempt to ask questions about the clinical significance of symptoms. Moreover, it becomes difficult in general surveys to ask enough about the history of an individual to establish if a disorder is “substance-induced,” which is one exclusion criteria used for mood disorders, for example.

This is “model 3” of Petry et al. (2005).

The methods we use are full maximum likelihood. The “limited information” estimator of Heckman (1976, 1979) did not require all of the properties of the bivariate normal distribution. All that was required was that there be a linear relationship between the errors of the two equations, and that the error of the sample selection equation be marginally normal (so that one could calculate the inverse Mills ratio).

This SNP approach is computationally less intensive than comparable approaches based on the estimation of kernel densities. There is some evidence from Stewart (2005) and De Luca (2008) that this SNP approach has good finite sample performance when compared to conventional parametric alternatives and other SNP estimators. Stewart (2004; §3) provides an excellent discussion of the mild regularity conditions required for the SNP approximation to be valid, and the manner in which it is implemented so as to ensure that a special case is the parametric (ordered) probit specification. Appendix C presents the formal statistical model.

Criticisms were raised by Geneletti et al. (2011) in response to epidemiological applications of corrections for sample selection by Chaix et al. (2011) and Bärnighausen et al. (2011a).

Thus one finds comments such as: “Theoretically, we do not need such identifying variables, but without them, we depend on functional form to identify the model. It would be difficult for anyone to take such results seriously because the functional form assumptions have no firm basis in theory.” (StataCorp 2013; p. 782). A similar comment from Bärnighausen et al. (2011b, p. 446) in an epidemiological setting is that the “… performance of a Heckman-type model depends critically on the use of valid exclusion restrictions….” It is agreed that the functional form assumptions, including the bivariate normal error assumptions, have no firm basis in theory, but we make such assumptions all the time in other settings. If we can indeed test them, that would be ideal, but it is not clear why we should in this instance not use them if we have to. Our view is that these models should be viewed as statistical “canaries in the cave,” in the sense of pointing to potentially disastrous conditions that warrant immediate investigation. In other words, and to put the inferential shoe on the other foot, if some estimates show great sensitivity to sample selection corrections with these assumptions, and some decent effort to find good specifications, then one should not ignore that evidence because some of the parametric assumptions are untestable.

We undertake sample selection corrections for GD, but not for the other psychiatric conditions. Instead we use the NESARC determinations of diagnosis. An important extension of our approach would be to simultaneously undertake sample selection corrections for all conditions and then assess comorbidity with respect to the corrected diagnoses for all conditions.

Appendix B documents these covariates.

The intended interpretation of risk here is not prospective (the probability of developing GD at some point in the future). Rather, it is intended as the risk that the respondent would currently be diagnosed as a DG if he or she participated in a full clinical interview with more reliable discrimination.

Most of the DSM criteria include the requirement that the symptoms be “clinically significant.” This is normally identified by questions asking if the symptom(s) led to any contacts with medical professionals, use of medication more than once, or led to interference with “life or activities.” For reasons of survey efficiency, these questions are normally asked only if the respondent meets some threshold level of symptoms. Hence one must be careful to recognize that anyone that has met fewer than the threshold level of symptoms will not have been asked about clinical significance (and, more generally, that these thresholds can be applied differently across general surveys, leading to apparent discrepancies in prevalence estimates, as stressed by Narrow et al. 2002). There are no such criteria for GD evaluation in DSM 5 since the symptoms themselves are viewed as evidence of “clinically significant impairment or distress” (American Psychiatric Association 2013, p. 585). However, DSM-III, DSM-IV and DSM 5 all contain exceptions for anyone whose gambling behavior is not “better explained” by a manic episode. This exclusion criterion is also only asked in surveys if someone met the threshold level of symptoms. For NESARC there are only 25 (7) out of 68 respondents to this question who said that any (all) of the times they gambled happened “during a period when they felt extremely excited, extremely irritable or easily annoyed.” These respondents constitute only 0.042 (0.016) of a percentage point of the population. For consistency of interpretation across the hierarchy, we do not apply this exception.

Because the predicted fraction to be selected exceeds the observed fraction, one might just assume that the selection equation is mis-specified, and this is the simple explanation for our findings of a higher prevalence of individuals at risk. However, the predicted probability of being selected in the sample selection model is the predicted sample conditional on covariates plus an error term for that selection equation. In the usual parametric sample selection specification this error term is assumed to be zero, so these observed and predicted fractions should be more or less the same. However, the semi-nonparametric specification does not assume this error term to be zero, as emphasized by De Luca and Perotti (2011, p. 218). Hence the predicted fraction could be larger or smaller than the observed fraction. This point further illustrates how the sample selection model benefits from not having to impose a parametric stochastic structure.

The survey of gambling disorders in the Canadian Community Health Survey (CCHS) of Mental Health and Well-Being of 2002 illustrates this point perfectly. Their gateway questions resulted in only 1754 of 36,884 subjects being asked the full set of questions from the Canadian Problem Gambling Index (CPGI), the full clinical assessment protocol from which the PGSI short field screen is derived. In the raw data one observes 2.8%, 1.5% and 0.5% classified as Low Risk, Moderate Risk and Problem Gambler, respectively, using the categories defined by Statistics Canada for the CCHS. Thus 4.8% are classified as “at risk.” After sample selection corrections these become 0.6%, 1.7% and 2.3%, respectively, or 4.6% in total. So virtually the same fraction are classified as “at risk,” but the composition is more heavily weighted toward those at greatest risk for GD.

The percentile value is purely descriptive, as a summary statistic for 43,093 p-values. The p value is the inferential statistic.

The original DSM-III criteria stressed disruption of personal, family and employment activities. The revised criteria in DSM-III-R added physiological symptoms such as withdrawal problems.

The citation, strictly speaking, refers to the PGSI, the short scored field screen of the CPGI.

Sharp et al. (2012) further tried to encourage accurate responses by asking this question separately for each game the respondent reports playing. Since they know mean general house advantages for game types as set by South African regulations, this allows them to compute expected losses to the extent that subjects reported expenditures in the strict sense of that word. Of course this approach was profligate with subjects’ time, which can cause them to become impatient and consequently respond less accurately to questions in general.

The U.K. National Centre for Social Research, the Gambling Commission, and the UK Data Archive bear no responsibility for our analysis or interpretation of the BGPS. Figure B1 in Appendix B documents the claim about statistical insignificance of the differences.

Stone et al. (2015) also present results from the Swedish Longitudinal Gambling Survey. The data from that study are not available for replication or review (Ulla Romild, Public Health Agency of Sweden; personal communication, October 23, 2016).

For consistency we repeat the categories of gambling problems and risk used in the NESARC data analysis (Figs. 3, 4 and 5) rather than the categories reported by the original BGPS and VGS reports from the DSM-IV, PGSI, and NODS screens. In Fig. 6 the original category is “Problem Gambling,” with a DSM-IV score of 3 or more. In Fig. 7 the original categories are “Low Risk,” “Moderate Risk,” and “Problem Gambler,” respectively; as noted earlier, the PGSI uses “Problem Gambler” as synonymous with the DSM-IV’s “Pathological Gambler” (and, therefore, the DSM 5’s “Disordered Gambler.” In Fig. 8 the original categories are “At Risk,” “Problem Gambler,” and “Pathological Gambler,” respectively.

Figure B2 in Appendix B documents the claim about statistical insignificance of the differences.

Another example of additional threshold questions being used is the Canadian Community Health Survey of Mental Health and Well-Being of 2002. Of the sample of 36,984, 24.6% said that they had not engaged in any of 13 gambling activities in the past year. Then 46.3% of the total sample was not asked the full set of CPGI questions because they had only gambled between 1 and 5 times, at most, for each of the 13 activities. And then 24.0% of the sample was not asked the full set of CPGI questions because they said that they were a non-gambler on the first CPGI question. There were 98 subjects that refused to answer the initial questions about gambling activity, resulting in only 1759 being asked the full set of questions and having any chance of being scored as “at risk.” These deviations from the CPGI screen, and the PGSI index derived from it, were “approved by the authors of the scale” (Statistics Canada 2004, p. 19).

Figure B3 in Appendix B documents the claim about statistical significance of the differences.

To take a stark example, assume that gambling is illegal until one reaches a certain age of consent. Surveys of individuals who have just reached that age would show nobody at risk, but of course that says nothing about the future propensity of the individual to have gambling problems.

For an example from surveys, consider the follow-up to the longitudinal Movement to Opportunity (MTO) field experiment, in which 30% of the sample was randomly assigned to more intensive follow-up; see Orr et al. (2003; Exhibit B, §B1.3) and DiNardo et al. (2006). This randomized follow-up was in addition to the primary randomization to treatment: (1) a housing voucher with some strings attached and some counseling, (2) a housing voucher with no strings attached and no counseling, and (3) a control group. This additional randomization to more intensive follow-up had virtually no effect on results, since the effective response rates for the long-term MTO follow-up were around 90%, and similar across primary treatments. This methodological step was striking, since it provided some controlled basis for inferring sample attrition, which is formally identical to sample selection, albeit in the opposite direction (selecting out of the longitudinal sample). For an example from field experiments, see Harrison et al. (2014), where subjects were offered different non-risky incentives to participate and effects on measured risk aversion assessed after correcting for sample selection.

See Warner (1965) for the original idea, and Blaire et al. (2015) for a recent review and application of variants. There are other non-randomized survey methods for encouraging truthful responses to sensitive questions.

For example, Harrison (2017) applies the same approach to the population prevalence of nicotine dependence, which is DSM-IV code 305.10, and finds comparable biases in the United States using NESARC.