Validity of self-reported criminal justice system involvement in substance abusing women at five-year follow-up

The setting was a 21-bed inpatient compulsory care residential care unit, Lunden, in Lund Sweden. The institution has 12 beds for adults and 9 for youth. The unit staff includes psychologists, psychiatrist, nurses, social workers, treatment attendants, and administration.

Women are treated under the Law on Compulsory Care for Substance Abusers (LVM, act 1988:870) or The Care of Young Persons Act (LVU, act 1990:52).

According to the LVU, "A care order is to be issued, if the young person exposes his health or development to a palpable risk of injury through the abuse of addictive substances, criminal activities, or some other socially degrading behaviour." (LVU, act 1990:52, section 3). Youth can also be taken into care under the LVU due to neglect or chaotic circumstances in the family.

Under Section 4 of the LVM, a court can order compulsory care for a person whose health is deemed to be at risk, or who may be placing others at risk, and who is considered to need assistance in order to discontinue substance use. The LVM and LVU acts are unrelated to penal code and laws of psychiatric care.

Patients are usually reported to courts by social welfare, or, more rarely, police, their family members or general practitioner. Within 8 days after report, an assessment of need for treatment must be completed, and court hearings proceed.

Care orders are implemented in specially certified LVM and LVU homes, under the authority of the National Board for Institutional Care (SiS).

The number of adults undergoing compulsory care was 1,029 persons in 2003, whereof 301 were women, and the number of youths was 1073, whereof 373 were women [8].

A consecutive sample of 132 female drug abusers treated in compulsory care at the residential centre Lunden in Lund, Sweden, from 1997-01-01 to 2000-12-31 were selected. Women were included if they had been psychiatrically assessed at request of the local municipal authorities.

The women were followed over five years after treatment. The treatment unit is reserved for treatment of women exclusively, with a focus on women's issues and drug addiction. Subjects went through a formal diagnostic procedure at intake, covering somatic complaints, psychiatric and psychological problems, and background factors using standardized instruments. Patients were requested to give written consent to participate in the study, allowing the researchers to use data for research purposes and to retrieve register data. At follow-up, subjects were requested to confirm consent.

At baseline assessment, subjects in the sample were assessed for psychiatric disorders (Axis I disorders assessed through SCID-I interview [9], and SCID-II interviews [10], were given a thorough medical examination, completed several personality tests, intelligence tests, and completed psychiatric self-rating scales. When indicated, women also underwent neuropsychological assessment.

The interview also included cross-sectional data, a Swedish adaptation of the Addiction Severity Index [11], standardized by a research team at the Swedish Board of Institutional Care (DOK) together with a longitudinal interview, an adapted version of the timeline follow-back interview covering the same seven problem areas and rating scales as in DOK. The Timeline Follow-Back [TLFB] interview is a structured format for assessing retrospective data that has been extensively validated [12]. The TLFB was conducted covering the entire period from discharge to follow-up (i.e., 60 months). Due to the long time covered, subjects were asked to indicate events in 6 month windows.

For the present study, we used only the part of the Timeline FollowBack Interview covering criminal charges and prison. We used four categories of legal problems: whether they had been charged with a drug offence, charged with theft, charged with violence, and whether they had been taken into an prison under the Swedish Prison and Probation Service.

The corresponding questions were: "have you been convicted of the following crimes: If yes, how many times?" The timeline then contained four categories in separate rows (drug related crimes, property crimes, violent crimes, and other crimes). Each column was then labelled with a year and letters indicating spring (V, for Swedish Vår), and fall (H for Swedish Høst). The years covered were 1997 to 2005.

Official records on all types of judicial sanctions were retrieved from The National Council for Crime Prevention (BRÅ), Stockholm, Sweden. The database has consistent penalty data on all persons having been in contact with the Swedish judicial system back to 1975 and up to the present day, and includes charges as well as sentences. We used data from the time of discharge for patients, and until the end of 2004. For each observation year, we recorded whether patients had been charged with a crime in one of the three categories, regardless of whether the charges lead to conviction or not.

At approximately 60 months post discharge, subjects were re-interviewed face-to-face by five independent clinical psychologists, and administered a battery of self-report questionnaires, partly the same as at index treatment admission.

The participants were traced trough social security and tax registers and information was added from significant others, staff in institutions, prisons and social bureaus, etc. At follow-up the diagnostic assessment was repeated and new areas added.

Since subjects were entered into the study over a 4-year period, the number of observations varied from year to year. The subjects who entered the study during 1999–2000 had some missing information from the database, as they were re-interviewed after 2004, and thus were interviewed about some years for which we did not have information from the criminal justice database.

Comparisons between the two sources of information were made for each observation year, and for the whole period. That is, one data file was constructed containing one observation for each subject, with self-reported and file-recorded outcome in each area as variables. A second data-file was constructed with several observations for each woman, one for each year for which the woman had been interviewed. For example, if a woman had been discharged in 1998 and re-interviewed in 2003, she will have 6 observations in the data-file for the years 1998–2003. If a woman had been discharged in 2000 and reinterviewed in 2005, she will have 5 observations in the data-file for the years 2000–2004 (as we did not have complete data for the year 2005). This second data-file was used to estimate the agreement within each year, and to conduct the regression analyses. The information collected from the interviews and the official records were essentially identical: simple yes/no responses to the questions: "was Y charged with theft/drug related offences/violence in the year 200X?", or " was Y sentenced to prison in the year 200X?"

For each category of outcome, a Maximum Likelihood random effects logistic regression was conducted to assess whether self-reported criminal justice system involvement in a given category was related to observed crime in the register. In order to assess the specificity of the relations between recorded and self-reported criminal justice system involvement, we conducted these analyses controlling for other types of crime (i.e., to control for the fact that some women may indiscriminately have reported several types of charges, regardless of whether they had actually faced all types of charges). While logistic regression analyses are not commonly reported as indicators of convergent validity, using the random effects regression allows us to take advantage of the fact that the same women were observed in several different years, controlling for repeated measurements of the same persons.

We also calculated the κ for agreement, and the false negative and false positive rate for every observation year. The κ reliability coefficient can vary from -1 to 1. Values between 0.40 and 0.59 are considered fair agreement, those between 0.60 and 0.74 reflect good agreement, and those above 0.74 indicate excellent agreement [13]. The frequency of false negatives and false positives was calculated, with false positives measured as the proportion of inaccurate reporting of an offence out of all women without a record, and false negatives measured as the proportion of inaccurate offenders out of all women with a record. These frequencies were calculated for each year, and for the total follow-up period.

In order to compare the frequency of events for self-report and official record, Cochran's Q was used [14]. This comparison was made for ever reporting an event, that is, a comparison of the likelihood that subjects would report an event at any point during follow-up vs. the likelihood that an official record would be found. Finally, we estimated the independent associations between official records and self-reported events, controlling for other events, in a maximum likelihood random effects logistic regression.

The original sample consisted of 132 women treated in compulsory care. A total of 6 women refused to give consent to participate in the study (all at baseline). At follow-up, 6 had died. In total, 109 were interviewed, and of these, 106 were administered the timeline follow-back interview, and could thus be included. A total of 11 subjects were lost to follow-up. Thus, 84% of living subjects could be included. The drugs used were stimulants, mainly amphetamine (51%), opiates, mainly heroin (35%), alcohol (7%), or other drugs (all<3%). A total of 78% were diagnosed with personality disorders according to SCID-II interview, and 60% had an axis I disorder not related to substance use.

The mean number of observation years that could be linked was 4.1 per subject (range = 2–6). The reason that the number of observation years is slightly lower than the number of follow-up years is that data from the criminal justice database were only available up until 2004, and most subjects were interviewed later than 2004.

The proportion of over-reporters and under-reporters and κ coefficients for the total period and for each year are shown in Table 1. Years are numbered, so that the year during which the interview was conducted is numbered 0, the previous year 1, and the year before that 2, etc. Results of ML random effects logistic regression are shown in Table 2.

For ever being charged with drug related offences, the percent agreement was 63, and κ was 0.24 (p = 0.006). The false positive rate was 35%, and the false negative rate was 40%. Subjects were no more or less likely to report drug offences than were the official records (48% in records vs. 50% in self-report, Q(1) = 0.90, p = 0.34). False negative and false positive rates varied from year to year.

The random effects logistic ML regression showed that self-reported drug offences were significantly related to observed record in the criminal offences register (OR = 6.66 for presence of record in the presence of self-reported drug record, Z = 3.76,s p < 0.001). After controlling for record of other offences than theft and prison, recorded drug offences remained significantly associated with self-reported drugs offences (OR = 5.17, Z = 3.33, p < 0.01), records of theft and violence was unrelated to self-reported drugs offences, but record of prison was related to self-reported drugs offences (OR = 4.24, z = 2.65, p < 0.05).

For ever being charged with theft, the percent agreement was 70%, and κ was 0.37 (p < 0.001). The false positive rate was 28, and the false negative rate was 34%. Subjects were no more or less likely to report theft than were the official records (42% vs. 36%, Q(1) = 1.12, p < 0.29).

The random effects logistic ML regression showed that self-reported theft was significantly related to observed record of theft in the criminal offences register (OR = 3.30 for self-report of theft in the presence of theft recorded, Z = 2.34, p < 0.05). After controlling for record of other offences and prison, self-reported theft was no longer significantly associated with a record of theft (OR = 1.96, Z = 1.31, NS). Self-reported theft was predicted by drugs offences (OR = 5.45, Z = 3.35, p < 0.01), and prison (OR = 4.80, Z = 2.44, p < 0.05).

For ever being charged with violence, the percent agreement was 74%, and κ was 0.23 (p < 0.01). The false positive rate was 9%, and the false negative rate was 71%. Subjects were significantly less likely to report violence than were the official records (14% vs. 27%, Q(1) = 6.26, p < 0.013).

The random effects logistic ML regression showed that self-reported violence was significantly related to observed record of violence in the criminal offences register (OR = 6.28 for self-report of violence in the presence of violence recorded, Z = 2.45, p < 0.001). After controlling for record of other offences and prison, self-reported violence remained significantly associated with a record of violence (OR = 5.59, Z = 2.00, p < 0.05).

For ever being in prison during the follow-up period, the percent agreement was 85%, and κ was 0.51 (p < 0.001). Subjects were not significantly more or less likely to report prison than were the official records (18% vs. 20%, Q(1) = 0.25, p < 0.62).

The random effects logistic ML regression showed that self-reported prison was significantly related to observed record of prison in the criminal offences register (OR = 70.60 for presence of prison record in the presence of self-reported prison, Z = 5.58, p < 0.001).

After controlling for record of specific offences, recorded prison sentence remained associated with self-reported prison (OR = 47.26, Z = 4.81, p < 0.001).

To test the influence of memory, we divided the time period up into two categories: year 0–3 and year 4–7. Within each time period, we analyzed agreement between self-reported criminal justice system involvement and criminal justice records. For drug related offences, the agreement for more recent events (past three years) was 0.48, and for earlier events it was 0.23. For theft the agreement for recent events was 0.32, and for more distant events, the agreement was 0.32. For violence, the agreement for more recent events was 0.09, and for earlier events was 0.31. And finally for prison, the agreement for more recent events was 0.53, and for earlier events was 0.38. Thus, the picture does not consistently show that failure to recall events that occurred several years ago is the main source of disagreement for these events, although for drug related events and prison, the findings favoured more recent events. For theft, no difference was found, and for violence the pattern was in the opposite direction.