Psychometric properties of a sign language version of the Mini International Neuropsychiatric Interview (MINI)

Language and communication are fundamental for diagnosing most mental disorders. Most assessment instruments are designed for use with hearing individuals, and many deaf and severely hard-of-hearing people with psychiatric illness may receive incorrect diagnoses because of challenges in communication between patients and professionals [1‐3]. The resulting lack of diagnostic precision may have serious bearing on treatment adequacy and quality. In Norway and other countries, there is a need for valid and reliable instruments for assessment of mental disorders in deaf and severely hard-of-hearing persons who have sign language as their primary and preferred language (signers).

The Mini International Neuropsychiatric Interview (MINI) was developed to meet the need for a brief and reliable structured diagnostic interview in clinical practice as well as in research [4, 5]. In clinical practice, the MINI is considered a supplement and not a substitution for regular diagnostic intake interviews. It is a structured evaluation of most of the major psychiatric conditions [4, 5], and was therefore selected for translation into Norwegian Sign Language (NSL).

The MINI includes 23 disorders from the tenth revision of the International Classification of Diseases and Related Health Problems (ICD-10) [6] and from the Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition (DSM IV) [7]. It is organised in diagnostic sections with branching tree logic and two to four screening questions with yes or no responses for each disorder. Additional symptom questions are asked only when a screening question is endorsed [4, 8]. Excellent inter-rater and test-retest reliabilities have been reported for the English and French versions [9], as well as good to very good convergent validity relative to the Composite International Diagnostic Interview, CIDI [4] and the Structured Clinical Interview for Diagnostic and Statistical Manual, SCID [5]. In test-retest analyses, kappa values have indicated excellent agreement according to Fleiss [10] for six diagnoses, fair to good agreement for six diagnoses and poor agreement for seven diagnoses, whereas observed inter-rater agreement has been 75% or above [5].

When a psychiatric diagnostic instrument is translated into a new language, reassessment of validity and reliability is necessary. Wordings that seem identical in different languages may nonetheless be differently interpreted, especially when a written text is translated to a sign language [11, 12]. This is not just a matter of semantic divergences, but also of divergences related to cultural understanding of concepts, words (signs) and sentences. Development of expressions that both convey the question’s core content and appear meaningful to the patient is vital.

The MINI has been translated into 43 languages, and its validity and reliability have been explored not only for the original English and the French versions, but also for the Spanish [13], Italian [14], Japanese [15], Moroccan [16], Portuguese [17] and Norwegian versions [18] (see Table 1). No sign language version has been reported. Deaf and hard-of-hearing people have been excluded from most mental health research, including validation studies of the MINI. Exclusion criteria cited in such research include “language problems” [4, 14, 15], “patients who were hearing impaired, not fluent in English” [19] and “could not be interviewed due to language barriers” [18].

Sign languages are natural languages that have evolved through use by deaf and hard-of-hearing people. Utterances are produced with the hands and face and are visually received and decoded [20, 21]. Sign languages share many linguistic characteristics with spoken languages, but also have characteristics that are specific to the manual-visual modality [20, 21]. Most deaf adults use written language and consider it their second language; however, a considerable percentage of congenitally deaf people struggle with written texts [22, 23].

To reliably diagnose mental illness in signing persons, clinicians must have sufficient sign language competence and thorough knowledge of deaf culture. They must also have a deep understanding of the potential psychosocial consequences of profound hearing loss, the accompanying language and communication challenges, and the obstacles encountered by many deaf and hard-of-hearing people in society. If these requirements are not met, clinicians may misinterpret the patient’s utterances as symptoms of mental illness or thought disturbances, or may overlook symptoms of mental illness because they are not communicated in the expected manner.

The present study investigates the functionality of a NSL version of the MINI. The main research question is whether this version functions in the same manner with signing deaf and hard-of-hearing patients as do other versions of the MINI with hearing patients.

The present investigation is part of a comprehensive study of mental health in individuals referred to specialised psychiatric units for deaf and severely hard-of-hearing patients, and includes patients using different modes of communication and language.

The mean length of the MINI interview was 58 min (range 32–88). The most prevalent diagnoses assessed by expert opinion were major depressive disorder (n of diagnoses = 22; 54%) and anxiety disorder (n of diagnoses = 12; 29%). No participant met the criteria for mania, agoraphobia, obsessive compulsive disorder, anorexia nervosa or bulimia nervosa.

Twenty-eight participants (68%) received a diagnosis by both expert opinion and the MINI, with an average of 1.2 diagnoses according to expert opinion and 1.9 according to the MINI. Thirteen participants (32%) did not meet the criteria for any diagnosis on the MINI. Two of them did not receive a diagnosis by expert opinion—four were considered to have major depressive disorder—and seven were diagnosed with disorders of psychological development, mild mental retardation, personality disorder or disorder of severe stress and adjustment, which are included in ICD 10, but not in the MINI.

There was agreement about 49 (64%) of the 77 diagnoses given to the 41 participants.

In five conditions (6%) MINI gave no diagnosis and expert opinion assessed major depressive disorder, and in 13 conditions (17%) with a MINI diagnosis expert opinion assessed “no diagnosis”. Nine conditions (12%) were assigned a diagnosis included in the MINI by the MINI interview and a diagnosis not included in the MINI by the clinical interview. Last, there was major disagreement about one condition (1%), which was diagnosed as drug dependence by the MINI and as major depressive disorder by expert opinion (Additional file 1: Table S1).

Table 3 shows the concordance between the MINI and expert diagnoses. The participants’ conditions covered a wide range of diagnoses, and the two gross diagnostic categories of depressive disorders and anxiety disorders occurred in numbers sufficient to conduct kappa analyses. Four diagnoses were assigned to fewer than 5% of the participants. The point estimate of Cohen’s kappa was 0.46 for depressive disorders, indicating fair agreement according to Shrout [28], and 0.72 for anxiety disorders, indicating moderate agreement. The observed agreement between the raters was 73% and 88%, respectively (Table 4).

Twenty-six per cent of the participants underwent the two diagnostic interviews during the same day. The median number of days between the two interviews was 19 (range 0–115). The point estimate of Cohen’s kappa for depressive disorders was 0.46 for all participants, 0.50 for those who underwent the two interviews within 19 days, and 0.42 for those who waited longer than 19 days. For anxiety disorders the corresponding kappas were 0.72, 0.77, and 0.63.

Eleven participants who were diagnosed with a total of 17 conditions were used in the inter-rater reliability analysis of the diagnoses assessed by expert opinion; there was agreement about 11 conditions (65%). There was disagreement about six conditions (35%) concerning co-morbidity. For two conditions (12%), the initial expert opinion gave a diagnosis and the second rater did not; in another two conditions (12%), the reverse occurred. There was also major diagnostic disagreement about two additional conditions (12%) of co-morbidity. In some cases of disagreement, the second rater’s expert opinion diagnosis was in agreement with the MINI diagnosis and in conflict with the expert opinion. For instance, in one case, both the second rater and the MINI identified drug use, whereas the first expert diagnosed personality disorder (Additional file 2: Table S2).

Cohen’s kappa was computed for the two gross diagnostic categories “depressive disorders” and “anxiety disorders”. The point estimate of Cohen’s kappa in the analysis of the open clinical interview diagnoses was 0.81 for depressive disorders, indicating substantial agreement; the observed agreement was 91% (Table 5). For anxiety disorders, the point estimate of kappa was 0.44, indicating fair agreement, whereas the observed agreement was 73%.

In the inter-rater analysis of the diagnoses assessed by MINI, eight participants were diagnosed with 13 conditions; there was agreement about nine conditions (69%). In two conditions (15%) diagnosed by the initial MINI team as alcohol dependence and major depressive disorder, the second rater found no diagnosis. One condition (8%) was assessed as a dissocial personality disorder by the initial MINI team and as a psychotic disorder by the second rater, and one condition (8%) was diagnosed as major depressive disorder by the initial MINI team and as a dissocial personality disorder by the second rater (Additional file 3: Table S3).

For the MINI assessments, the point estimate of Cohen’s kappa was 0.71 for depressive disorders, indicating moderate agreement, and the observed agreement was 88% (Table 6). For anxiety disorders, the agreement was complete (kappa = 1.00, observed agreement = 100%). Four instances of inter-rater disagreement mirrored the disagreement between expert opinion and the MINI diagnoses in the validation analysis.

The MINI and expert diagnoses in deaf and severely hard-of-hearing signers referred for psychiatric evaluation show fair to substantial agreement. For validation against expert opinion, the kappa estimates in the present study are higher than those of Otsubo et al. [15] but lower than those of Kadri et al. [16]. Pinninti et al. [19] and Bobes [13] also used expert opinion for validation but did not offer kappa values.

The kappa scores were lower than those of studies that have validated MINI against SCID-P [5, 15, 17] and CIDI [4]. This is in line with other validation studies [13, 15, 19]. The kappa values in validations of the MINI against expert opinion were generally lower than when the MINI has been validated against a structured diagnostic instrument such as SCID-P or CIDI [4, 5, 15, 17]. Only Kadri et al. [16] report relatively high concordance with expert opinion. The open clinical interview and the structured MINI interview represent different approaches to assessing mental disorders, and complete agreement therefore cannot be expected. Structured psychiatric interviews are designed to elicit information about core symptoms of a set of mental disorders, and the concordance of two structured interviews is therefore likely to be higher than for a structured and an open interview. The diagnostic reflections of experts using an open diagnostic interview are likely more diverse than the focused questions and selected list of disorders in the structured interviews.

The concordance of diagnoses was examined by two different methods because the sample size limited the use of kappa statistics. A broad range of diagnoses was found, some with very low frequency. Cohen’s kappa is a conservative indicator and functions best with larger samples. It is unlikely to be reliable when a diagnosis is assessed in fewer than five per cent of a sample [30]. Only “depressive disorders” and “anxiety disorders” were identified frequently enough to assess reliable kappa scores. Matrixes were therefore drawn to show the distribution of diagnoses assessed with the two diagnostic methods and in the inter-rater reliability procedure. This concordance analysis indicates disagreement in approximately one-third of the diagnoses, with most disagreements related to the participants’ second diagnosis (co-morbidity). This finding applies to the validation of MINI versus expert opinion as well as to the inter-rater reliability of the expert diagnosis.

The main question in the present study is whether the NSL version of the MINI functions in the same manner as do other versions of the MINI when compared with expert opinion. Some of the lack of agreement between the two approaches may be caused by differences in diagnostic options. The MINI includes a limited number of diagnoses, and assessors will have to conform to those diagnoses or assign “no diagnosis”. Relying on the MINI for diagnostic assessment may therefore influence the direction of the evaluation, as clinicians attempt to comply with the diagnoses at hand. Clinicians who conduct an open interview have a wider range of diagnoses to consider, which again may shape the direction of their clinical line of thought and the resulting diagnosis. The fact that the kappa values in this study were in the lower range compared with other validation studies may also be partly explained by the presence of less severe disorders and fewer symptoms among the outpatients in this study than among the inpatients and acute ward patients assessed in most other studies (see Table 1).

The MINI indicated a larger number of co-morbid conditions than did the expert opinion. Similar results have been reported in other studies [15, 19] and suggest that the structure of the MINI may make the clinician more attentive to symptoms related to disorders additional to the main diagnosis, and that this attentional shift reduces diagnostic shadowing. This interpretation speaks in favour of including the MINI in routine assessment of signers, as is practiced for hearing patients. On the other hand, assessing severe and complex mental conditions by diagnostic categories exclusively may also increase the probability that symptoms are overlooked.

The MINI was originally developed to meet the need for a structured assessment instrument that could be administered in minimal time. In the present study, the average MINI assessment required almost three times the estimated “less than 20 minutes” [8]. There may be several reasons for this. The interviews were conducted by two therapists, a method requiring collaboration and coordination, which may account for part of the extra time used. Furthermore, as was pointed out by Black and Glickman [3], many deaf patients with mental disorders do not have full mastery of sign language and may struggle in expressing themselves and in comprehending what is being communicated to them, and this may lead to many repetitions. Finally, the interviewer cannot simultaneously take notes and communicate with the patient, and consequently the time needed for taking notes will add to the total interview time [12]. However, the reason for including the MINI as an assessment instrument for deaf and severely hard-of-hearing persons was to improve consistency, not to reduce assessment time.

The two most frequent diagnostic categories in the present study were “depressive disorders” and “anxiety disorders”. This result is similar to that of studies of psychiatric patient populations in general [8, 31, 32] as well as studies of mental disorders in signing deaf and hard-of-hearing patients [3, 33]. To ensure that the open clinical intake interview would serve as expert opinion for the validation of the NSL version of the MINI, these two categories were used to assess inter-rater reliability. A second rater re-diagnosed the participants from video-recordings of the original assessment interviews, a common procedure in reliability studies [30]. To our knowledge, this approach has not been applied before in research with deaf and severely hard-of-hearing persons. It poses quite a few challenges to the second rater, who is forced to follow “the diagnostic route” of the clinician who conducted the initial assessment. This is no problem if the second rater agrees with the first, but second raters cannot collect additional information or follow alternative diagnostic routes when they disagree with the assumption of the original assessor. We do not know whether this issue influenced the concordance estimates in the present study.

The average time span between the two assessments was longer than reported in other validation studies of the MINI. Twenty-six per cent of the participants went through both interviews the same day, which may seem optimal. However, participation was voluntary and cancellations and postponements of appointments were rather common. Furthermore, the participants’ symptoms may have influenced their daily functioning and ability to keep appointments. The centralised service for this geographically scattered patient population is likely to have bearing on the time between assessments. When participants lived far from the outpatient clinic and the assessments could not be made on the same or consecutive days, the result was sometimes a long interval between assessments. However, kappa values for assessments made with short and long intervals were quite similar.

The kappa scores in the present validation of the NSL version of the MINI and the observed agreements with expert opinion do not differ substantially from those of other validation studies. The NSL version appears to be a reliable diagnostic interview for assessing mental disorders in signing persons, provided that it is used by professionals with appropriate sign language skills and knowledge about the patient group. The results of the present study therefore encourage translation of the MINI into other sign languages.