Results of the Brief Jail Mental Health Screen Across Repeated Jail Bookings

The prevalence of serious mental illnesses, including mood disorders, schizophrenia, and other psychotic disorders, is three to four times greater among jail detainees than in the general population (1); furthermore, estimates suggest that more than 2 million people with serious mental illness are booked into U.S. jails yearly (2). In an effort to improve clinical outcomes and, ultimately, reduce overrepresentation of serious mental illness in jails, the American Psychiatric Association and the National Commission on Correctional Health Care issued standards for mental health screening, referral, and evaluation for inmates in jails and prisons (3, 4). Specifically, the National Commission on Correctional Health Care requires a mental health screen within 14 days of booking, followed by an in-depth evaluation if the screen is positive (5). Accordingly, mental health screening has become commonplace in county jails across the United States (6), with the goal of identifying detainees at intake who may have serious mental illness and referring them for further, in-depth mental health evaluation. In this way, mental health screening may facilitate diagnosis and aid jail staff in making decisions regarding classification, treatment, and management (7).

Given the high volume of admissions, relatively low levels of staffing, and typically short periods of stay in U.S. county jails, mental health screening tools must be very short and easily administered. The Brief Jail Mental Health Screen (BJMHS) is one such instrument. Administered verbally by intake officers in about 5 minutes or less and requiring minimal training, the BJMHS queries the presence of current psychiatric symptoms, current use of psychiatric medication, and prior psychiatric hospitalization via eight yes-no questions. Detainees are referred for further mental health evaluation if they meet one or more of four decision rules (8). The BJMHS has been recommended by the Substance Abuse and Mental Health Services Administration for use as a jail mental health screening tool for nearly 20 years (9, 10), and it has been in use in jails across the United States since its creation (6).

Despite its widespread use, there has been relatively limited empirical investigation of the BJMHS. Of the handful of studies that have investigated the tool, the majority focus on its performance compared with “gold standard” mental health assessment tools, such as the Structured Clinical Interview for DSM-IV (SCID-IV; 8, 11) or the Mini International Neuropsychiatric Interview (12), or compared with other jail mental health screening tools, such as the Jail Screening Assessment Tool (13, 14). Taken together, evidence regarding its performance has been mixed; some studies have shown good concordance with the criterion (11), and others, less so (12–14). Beyond criterion validity, there remain questions regarding the functioning of the tool in practice. In particular, we are not aware of research that has examined BJMHS results over time, even though detainees may be assessed multiple times across repeated jail bookings. This is the focus of the current investigation.

Mental health screening results may change over time for reasons related to changes in mental health status. For example, following release from jail, individuals with serious mental illness may face significant challenges, including stress (15), housing instability, negative peer influences (16), employment difficulties (17), and limited access to treatment (18), that contribute to worsened illness trajectories. However, for many with serious mental illness, jail booking may represent the first point at which their symptoms are identified and they are referred to treatment (19). As a result, they may experience improved illness trajectories across repeated bookings (20–23). Alternatively, BJMHS results may change over time for reasons other than true changes in illness trajectories.

For instance, impaired insight or cognitive functioning could affect self-report at one screening but not at another (24). Furthermore, because the BJMHS is administered as a face-to-face interview, social desirability, impression management, and rapport also may affect results (25–27). A detainee may be more comfortable with certain staff or may become more comfortable with the process (and more likely to disclose information) over time. Staff may become familiar with an individual detainee over repeated admissions (28) and refer him or her for further evaluation, even in the absence of current symptoms. Moreover, staff beliefs, attitudes, and experiences may influence the way in which staff approach detainees (29), which may in turn affect detainee self-report and staff interpretation of that self-report. Finally, there are no training guidelines for the BJMHS (8), which may affect staff knowledge or understanding of the decision rules and, consequently, the fidelity with which the BJMHS is administered.

Although there are many reasons mental health screening results may change across repeated bookings, empirical evidence is lacking with respect to the BJMHS or any jail mental health screening tool, for that matter. Yet, mental health screening results have important implications for people with serious mental illness involved with the justice system. As such, understanding factors that may affect these screening results, including repeated administration, is critical. Our goal was to determine the odds of screening positive on the BJMHS across multiple jail bookings, after accounting for relevant sociodemographic (i.e., age, sex, race, and housing status) and other clinical (i.e., substance use) characteristics that may affect results. We used multilevel logistic modeling to examine changes in BJMHS results overall and for each of the four decision rules across repeated bookings in a population of detainees admitted to a large metropolitan jail.

Methods

All study procedures were approved by the university’s institutional review board. The sample comprised 12,531 detainees across 41,965 bookings into a large, urban county jail in the southeastern United States. This sample was drawn from the population of detainees (N=37,988) booked into the study jail between January 1, 2013, and June 30, 2016. Only those who were readmitted to the study jail at least one time (N=12,531) were included in this study because our interest was in examining change in screening results over time. Data were drawn from administrative and medical records.

The BJMHS was administered to all detainees in the context of routine practice by jail nurses at intake. As briefly described in the introduction, the BJMHS consists of eight self-report, dichotomous items (no=0, yes=1). These eight items are divided into two sections. The first section includes six items that query the presence of psychiatric symptoms in the past 6 months. The second section queries treatment history, including one item that addresses current use of psychotropic medication (item 7) and one item that addresses lifetime psychiatric hospitalization (item 8; 8). A positive screen occurs when a detainee meets at least one of the following decision rules: yes to at least two of items 1–6, yes to item 7, yes to item 8, or if the screener feels the detainee should be referred for any other reason (even in the absence of any yes responses). We examined the odds of meeting each of these four decision rules, as well as an overall positive screen (meeting at least one of the decision rules), as outcome variables.

An optional third section includes the screener’s comments or impressions regarding several extenuating circumstances (e.g., language barrier or inmate was under the influence of drugs or alcohol). This section does not bear on screening results and, thus, was not included as an outcome variable.

The predictor variable, booking number during the study period, allowed us to ascertain change in BJMHS across repeated jail bookings. Covariates included age, sex, race, homeless status, and substance use problems. Age (in years at the time of each booking), sex (0=female, 1=male), and race (0=non-Caucasian, 1=Caucasian) were obtained from the administrative records. Homelessness (0=no, 1=yes) was self-reported at intake. Substance use problems (0=no, 1=yes) was a composite measure composed of the optional third section of the BJMHS and four items from the jail intake medical screen. A person was coded “yes” for substance use problems if he or she met any of the following criteria: under the influence, signs of withdrawal, self-reported drug use, or self-reported problematic alcohol use (30).

We performed several multilevel logistic regression models using SAS PROC GLIMMIX, version 9.4, with bookings (level 1) nested within individuals (level 2). This technique allowed for examination of the rate of change in outcomes, incorporated all available data from each person, and controlled for the differences in timing of each repeated measure (time lapse between bookings). We ran one model examining odds of an overall positive screen followed by four models in which we examined each of the BJMHS decision rules separately. These five models were run once without covariates and once with covariates.

Results

Sample Characteristics

The mean±SD number of bookings per person in our analytic sample was 3.35±2.53 (range 2–65) during the study period, the mean length of stay was 21.42±50.59 days (range 0–1,049), and the mean time between bookings was 846.15±288.37 days (range 0–1,276). The sample was 34.8% (N=4,362) Caucasian, 65.2% (N=8,162) non-Caucasian, 65.1% (N=8,151) African American, and 0.1% (N=11) Asian. The mean age at first booking during the study period was 31.50±11.57 years (range 16–79). Most detainees (81.7%, N=10,232) were men. About 8.0% (N=1,002) of detainees reported homelessness, and 40.0% (N=4,994) screened positive for problematic substance use. At their initial booking, 62.9% (N=6,600) of the sample were charged with more serious offenses (typically felonies punishable by imprisonment for 1 year or more, such as assault, burglary, and sexual offenses), whereas 37.1% (N=3,886) were charged with less serious offenses (typically misdemeanors punishable by imprisonment for less than 1 year, for example, stolen property and disorderly conduct). The most common charges at initial booking included assault (21.2%, N=2,654), drugs or narcotics related (15.3%, N=1,916), and larceny or theft (11.5%, N=1,440).

Descriptive Statistics

In total, 23.3% (N=2,900) of detainees screened positive overall on the BJMHS at initial booking during the study period. Rates of positive screens for each decision rule at initial booking ranged from 2.7% (N=340) for current psychiatric symptoms to 12.7% (N=1,446) for current psychiatric medication. Visual exploration of the descriptive statistics suggested that the frequency of positive screens may be changing over time; however, rates and directions of change seemed to differ across items and decision rules (Figure 1). A breakdown of positive responses to individual BJMHS items and to BJMHS decision rules by booking number can be seen in Table 1.

Multilevel Logistic Models

Table 2 presents the estimates for each of the five multilevel logistic models. Results of the first multilevel logistic model confirmed that the odds of a positive screen overall (i.e., meeting at least one of the decision rules) increased with each additional booking into jail. Specifically, each additional booking was associated with an increase in the odds of a positive screen (odds ratio [OR]=1.02, 95% confidence interval [CI]=1.01–1.04). Results of the model including covariates showed the same pattern [see online supplement].

Results of the multilevel logistic models examining each of the decision rules showed that the odds changed across repeated bookings for three of the four rules (Table 2). The odds of screening positive for prior hospitalization and for reporting two or more current psychiatric symptoms decreased over time. With each additional booking, the odds of screening positive for prior hospitalization decreased significantly (OR=0.96, 95% CI=0.95–0.98) as did the odds of screening positive for current symptoms (OR=0.95, 95% CI=0.93–0.98). In contrast, detainees were more likely to be referred for any other reason (at the assessor’s discretion) with each additional booking into jail. For this decision rule, each additional booking was associated with an increase in the odds of referral (OR=1.03, 95% CI=1.02–1.04). Finally, the odds of screening positive for current use of psychiatric medication did not change across bookings. As before, results of the models including covariates showed the same patterns of results [see online supplement].

Discussion

Across the United States, the BJMHS is administered multiple times to detainees who cycle in and out of county jails; yet, little is known about how these mental health screening results may change across repeated jail bookings. Drawing data from bookings into a large, metropolitan county jail, we examined whether BJMHS results changed overall and by decision rule. Our findings demonstrate that BJMHS results changed across repeated jail bookings. In particular, we found that the odds of screening positive for prior hospitalization and current psychiatric symptoms decreased across bookings, whereas the odds of screening positive for current psychiatric medication remained unchanged. Odds of referral for any other reason and of screening positive overall increased across bookings. These patterns of results remained even after controlling for demographic variables, and substance use. In the sections that follow, we discuss these findings in more detail.

Summary of Findings

Detainees were less likely to screen positive for current psychiatric symptoms across repeated bookings. On the one hand, it is possible that a decrease in the reporting of symptoms reflects improvements in illness trajectories. Indeed, as discussed earlier, jails may represent a critical point of identification, referral, and initiation of psychiatric treatment for many adults with serious mental illness. On the other hand, changes in responses to the symptom items across bookings may not necessarily represent improved illness trajectories. Instead, different responses may reflect natural fluctuation, but not necessarily marked improvement, in clinical presentation (31). Furthermore, symptoms not queried by the BJMHS, such as symptoms associated with anxiety disorders, may have worsened over time and gone undetected by the BJMHS (32).

Our findings also may be attributable to problems associated with the self-report nature of the items, such as differences in insight or willingness to disclose information. Indeed, the question about hospitalization on the BJMHS queries lifetime hospitalization. The fact that detainees were less likely to screen positive for lifetime hospitalization across repeated bookings cannot indicate that lifetime hospitalization rates decreased. Rather, this indicates that reporting of hospitalization decreased among detainees who had previously reported being hospitalized; in other words, a small proportion of detainees (0.5%, N=63) recanted their prior hospitalization. This calls into question the reliability of this item across repeated administrations. Alternatively, it is possible that jail staff skipped this question at some bookings or forgot to record detainees’ responses. It is also possible that changes in this item are due to data-entry errors. These possibilities are important areas for further investigation.

To further illustrate the susceptibility of BJMHS items to reporting issues, research demonstrates that some people find repeated screening or assessment to be therapeutic, possibly increasing the likelihood that they will self-report symptoms, whereas others find it frustrating and suspicious (33). The jail mental health screening process is unlikely to confer positive therapeutic effects. Instead, it seems more realistic to conclude that having to answer the same questions across multiple bookings led detainees to feel that staff are not paying attention to their answers or that staff are trying to catch them giving conflicting answers.

Detainees may grow tired of repeating the same responses across multiple screens and may become less likely to report current symptoms or prior hospitalization. Furthermore, detainees may be fearful that disclosing symptoms will deny them pretrial release, lead to involuntary hospitalization, or affect their case in some way (33). Finally, detainees may be especially less likely to disclose mental health information if they feel that the disclosure of this information is not being used in a meaningful way (33), a strong possibility given the limited availability of treatment in jails generally (34) and the study jail, specifically.

In spite of the decreases in self-reported symptoms and hospitalization, the odds of a positive screen overall increased across bookings. This increase appears to be driven by increases in discretionary referral for “any other reason.” This may indicate that jail staff do not rely on the results of the other three decision rules when making referral decisions. In this study, the BJMHS detected current symptoms only in about 2.0% of detainees, whereas other studies have placed estimates of serious mental illness at 14.5% for men and 31.0% for women (2). Staff may not trust the screening results because of the incredibly low detection rate, which may explain why referral rates for staff overrides increased over time even though positive screens for the other three decision rules decreased or held steady. At the same time, however, this discretion is built into the BJMHS itself because one of the decision rules dictates that detainees should be referred “if [staff] feel it is necessary for any other reason” (8).

Thus, it may not be that staff do not trust the screening results but rather that they do not trust the detainee self-report. Such a “professional override” is included in many instruments designed for use in correctional settings (35). In this way, assessors are using the BJMHS as designed. Whether this rate of professional override is acceptable is questionable; no guidance or research exists to support the acceptable rate of professional override on this or any other instrument. This is an important avenue of investigation in relation to the BJMHS and more generally.

It is also possible that symptoms not included on the BJMHS increased across repeated bookings and were identified by jail staff, resulting in referral. Alternatively, staff may become familiar with a detainee across repeated bookings and refer on the basis of past rather than current clinical presentation. The familiarity hypothesis suggests that an evaluator will gain additional information with each subsequent administration of a test, which leads to improved assessment accuracy (28). However, this hypothesis has not always held when tested within the context of risk assessments (28, 36) and, thus, warrants further examination in the context of jail mental health screening. Indeed, confirmation bias suggests that such prior knowledge, or familiarity, may result in decreased screening accuracy over time.

It is also possible that changes in screening results across bookings reflect the limited training that jail staff receive before implementation of the BJMHS. There is no formal training required for the BJMHS, which may affect the fidelity with which the screen is administered and scored. Establishing a standardized training protocol, including booster training, should improve adherence to items and decision rules. Finally, changes in screening results may be due to poor reliability of the BJMHS items. A review of BJMHS studies demonstrated highly variable findings across studies (37), calling into question the tool’s accuracy and consistency in detecting mental health problems. Jails may be better served by other screening measures such as the Correctional Mental Health Screen or the Jail Screening Assessment Tool, which have been demonstrated in other studies to outperform the BJMHS in terms of detecting mental health problems among jail inmates (14, 38).

Limitations

A few limitations of the study design should be noted. First, we do not have information on the number of times a person was booked into this jail before our study period. As such, our first BJMHS results may not be the first time they were screened. Second, because the BJMHS is the only mental health screen administered in this jail and follow-up with an in-depth mental health evaluation was inconsistent, we were unable to compare results with another screening tool or against a criterion measure, such as the SCID-IV or clinical diagnosis. For these reasons, we cannot determine whether the observed changes reflect actual changes in mental health status, issues with fidelity of administration, issues of disclosure and reporting of symptoms, characteristics of the BJMHS itself, or otherwise. Third, we did not have access to treatment data and cannot speak to the treatment-related or other mechanisms through which BJMHS results changed. Fourth, we investigated changes in mental health screening results across repeated jail bookings using one tool in one jurisdiction; generalizability of findings must be examined in future research. Despite these limitations, the study’s strength lies in our longitudinal analysis of results of a widely used mental health screening tool in a large jail population. This study is the first of its kind.

Conclusions

Mental health screening tools may be administered repeatedly to individuals who cycle in and out of jails (6). Yet, there has been limited investigation of whether results of these jail mental health screens change over time, and the regulatory bodies that issue standards for screening and assessment of jail detainees have not provided guidance regarding screening protocols for repeat detainees (3, 4). Mental health screening results can have serious implications for jail detainees because they are administered at a critical intercept for identification, referral, and intervention (19). Given that jails have the highest volume of justice-involved individuals in the United States compared with other correctional settings, mental health screening tools used in jails have the potential to affect millions of people. For these reasons, it is imperative that research examines how mental health screening results change over time as well as the mechanisms of change. Indeed, examinations of test-retest reliability and measurement of mental health symptoms over time are recognized as crucial steps in the development of psychiatric measures (39).

Jail mental health screening tools must be examined in this way as well. Furthermore, the fourth decision rule, referral for any other reason (the professional override), was the only decision rule to increase over repeated bookings. Although it is positive that jail staff are attentive to the mental health needs of detainees beyond those captured by the BJMHS, it is critical to uncover the factors that jail staff are considering in making these override decisions. On the one hand, such research may elucidate factors that could improve the accuracy and utility of the BJMHS or other jail-based, mental health screening tools. On the other hand, such research may identify jail staff’s reliance on factors irrelevant to mental health but instead reflective of misconceptions or bias. The latter finding would represent issues of concern for implementation that may require training. Professional overrides are an understudied component of jail assessment tools. Our findings provide evidence of an increased use of professional overrides across repeated assessments and emphasize the need for research to understand the use of professional overrides in jail assessment tools.