Diagnosing comorbidity in psychiatric hospital: challenging the validity of administrative registers

Psychiatric comorbidity is a prevalent phenomenon and remains a challenge for the effective delivery of mental health services. Recent community surveys show that among those with a psychiatric disorder the lifetime prevalence of more than one diagnosis is about 50% [1]. The highest rates of comorbidity are observed between anxiety and affective disorders [1], and affective, anxiety and substance use disorder often occur together [2, 3]. The presence of comorbid disorders is associated with a significantly higher rate of help seeking [4‐6].

To what extent psychiatric case registers or administrative registers reflect this comorbidity is not known. Since the 1960s, psychiatric case registers have been regarded as important epidemiological research tools for estimating treated incidence, prevalence and patterns of care [7]. With the development of new and better information and communication technologies, their importance is expected to increase [8, 9]. Much of the utility of a psychiatric case register, however, will depend on the validity of the psychiatric diagnoses [10]. Byrne et al. [11] in their review conclude that relatively little high-quality work exists into systematically measuring the diagnostic data validity of registers for research purposes. Almost no studies (1 out of 14) performed anything else than case note reviews to assess validity. Only two of the studies reviewed stated that the register diagnoses were blinded to the researchers and inter-rater reliability testing was only performed in three of the studies.

Thus, there is a need for studies using more stringent methodological approaches to estimate the validity of case register diagnoses. In our study a structured diagnostic interview was performed on all new patients consecutively admitted to psychiatric hospital, comparing these diagnoses with those given by the clinicians. In a previous paper we have focused on the underdiagnosing of bipolar disorder [12]. In this paper we will present the results with regard to comorbid anxiety and substance use disorders.

This study casts doubt upon the validity of administrative registers concerning comorbidity of anxiety and substance use disorder. To diagnose anxiety in particular, seems to be neglected; the clinicians gave such a diagnosis to only 17% of those identified with an anxiety disorder (F40-41) by the expert. Concerning substance use disorders (F10-19) the picture is somewhat better with 31% identified by the clinicians. Regarding concurrent comorbidity of anxiety and substance use disorder, this was to a very small degree identified by the clinicians; only in two of 76 patients. Almost all these patients had a primary affective disorder.

Not many studies have looked at the quality of administrative registers with regard to comorbidity. Only one of the studies reviewed by Byrne et al. [11] did so, finding that comorbidity was under-reported as in our study [21]. In an investigation of the Danish Psychiatric Register, Hansen et al. [22] reported underdiagnosing of substance use disorder by nearly 50%. Further, several studies reported that comorbid anxiety are being missed in clinical practice [23, 24].

The question is raised whether comorbidity actually is an artefact of current diagnostic systems [25]. There is a marked symptom overlap between at first hand affective and anxiety disorders which could be a source of diagnostic unreliability and a dimensional classification system based on shared features of anxiety and mood disorders have been proposed [26]. In clinical practice a dimensional approach to treatment is common and a formal diagnosis is as a rule not given until discharge. It could be that the clinicians are aware of comorbid anxiety and substance use without diagnosing it formally [27]. One could imagine a kind of ethical considerations of not stigmatizing the patient with too many diagnoses giving only those who primarily brought the patient into hospital. However, Zimmerman and Chelminski [24] report underrecognition of anxiety disorders, for which the patients want treatment, in psychiatric outpatients with a principal diagnosis of major depression.

There could be several other reasons for this underdiagnosing of comorbidity. Treatment guidelines commonly only relate to specific diagnostic groups like affective disorders or anxiety disorders, and not to comorbid conditions [28]. This could make the clinicians less aware of comorbid conditions. The relation between co-occuring substance and anxiety disorders has not received much attention and is generally poorly understood. The diagnostic challenge in relation to individuals with current substance use disorders has been to devise diagnostic criteria and measurement techniques that differentiate between intoxication and withdrawal symptoms and the symptoms of psychiatric disorders. Many of the symptoms of intoxication and withdrawal from alcohol and other substances resemble the symptoms of mood and anxiety disorders [29]. Diagnostician using a structured interview routinely asks control questions that delineates between transient anxiety caused by substance withdrawal, and chronic anxiety symptoms. The results could also be due to the general phenomenon that clinicians rely on a limited number of heuristic principles which in some instances may lead to severe and systematic errors [30, 31]. We believe clinicians are more apt to use a heuristic top-down approach when they diagnose patients, i. e. not asking about other symptoms when the patient presents with depression. The expert employing data from a structured clinical interview, however, employs a bottom-up approach in the diagnostic process, i. e. asking questions which at first seem irrelevant. The risk of misclassification is supposed to be higher using the top-down diagnostic approach and relying on the diagnostic manual to confirm a clinical impression rather than to openly screen for alternative or additional diagnoses. Lack of relevant information in the patients’ records is shown to be a general phenomenon affecting all diagnostic groups [32].

The importance of correctly diagnosing comorbidity in clinical practice should be emphasized. Several studies have shown that psychiatric comorbidity is associated with a significantly increased probability of treatment and that comorbidity can be regarded as an index of a more severe course and outcome of mental disorders [1]. Comorbidity is associated with more severe psychiatric symptoms, more functional disability, longer illness duration, less social competence, and higher service utilization [33]. Furthermore, patients with affective disorder and comorbid anxiety and substance abuse, show less adherence to pharmacological treatment [34] and need more specialized treatment [35]. It is shown that such comorbidity is associated with suicidality in mood disorders [36]. This underpins the importance of correctly diagnosing comorbid conditions in the clinic. The psychiatric evaluation and diagnoses given at discharge from psychiatric hospital will follow the patient and is guiding for the treatment the patient will receive from GP’s and psychiatric personnel in the community.

Concerning administrative consequences, not diagnosing comorbidity represents an undercommunication of the burden these patients represent to the health care system and consequently gives the wrong signals concerning how to develop necessary services. Misleading medical statistics may cause spurious comparisons during the planning and evaluation of treatment for patients [1]. Further, our findings suggest that register diagnoses are dubious for research purposes when it comes to comorbid psychiatric diagnoses. This is in accordance with the investigations of Baca-Garcia et al. [37, 38] and McConville et al. [39].

Our study comprises only 250 patients, and only first time admissions, so the generalizability of the findings could be questioned. It could be argued that new patients are more difficult to diagnose making the diagnostic validity of registers including all patients, better. Compared with the studies reviewed by Byrne et al. [11] on the diagnostic validity of administrative registers in psychiatric research, our study has some advantages strengthening the validity of the results. First, a structured diagnostic interview was performed, adding information from the records when necessary, and the clinical diagnoses were blind to the expert. On the other hand, the expert never actually saw the patient. Thus the observations, scorings and case notes could have been evaluated otherwise if the expert had observed the patient directly. The greatest possible caveat here is that signs and symptoms may have been missed or misinterpreted. However the expert only scored a symptom as present if there was given a description of overt behaviour or citations from the patient in either the interview protocol or in the hospital records. Furthermore there is always a risk that diagnoses based on an interview which screens for all psychiatric symptoms may be overinclusive. This possible bias may result both from a “yes-saying” response style of the patient, and from a tendency of the interviewer to put weight on positive answers about signs and symptoms that are not clinically significant. Thus, there is a risk that the high number of diagnoses given by the expert is a result of response bias and scoring bias. However, we do not believe that this bias will disturb the main findings. Structured interviews are shown to be better than unstructured traditional diagnostic assessment [40, 41], and combining structured interviewing with a review of the medical records appears to produce more accurate primary diagnoses and to identify more secondary diagnoses than routine clinical methods or a structured interview alone [42]. The studies reviewed by Byrne et al. [11], where only case notes were checked and no new information added, should be regarded more as reliability studies than validity studies. Second, in our study the clinicians’ diagnoses were blind to the expert thus avoiding bias in either direction. A weakness in this study may be that formal inter-rater reliability testing was not done among the interviewers, however, there were organized discussions among them, also on selected videotaped cases. Inter-rater reliability can be low even if diagnoses are determined by researchers as found by Cheniaux et al. [43]. However, to counter this, diagnoses were not formulated by the interviewers, but by one experienced researcher.

Even if the nosological status of the comorbidity concept is by no means clear [25, 44] the importance of correctly diagnosing comorbidity should be emphasized and the results from this study tells us that much work needs to be done in the clinic in structuring the diagnostic process, and in developing treatment guidelines for comorbid conditions [41, 45, 46]. The validity of psychiatric administrative registers concerning comorbidity, seems dubious for research purposes as well as for administrative and clinical purposes.

In a previous paper we have reported that this is even the case especially for bipolar disorder [12]. When it comes to psychotic disorders such as substance use induced psychosis and schizophrenia, the validity seems satisfactory.