The screen for cognitive impairment in psychiatry: diagnostic-specific standardization in psychiatric ill patients

Cognitive deficits are highly prevalent in psychotic disorders [1], including schizophrenia, bipolar disorder, and schizoaffective disorder [2‐6]. Numerous studies suggest that patients with severe psychiatric disorders have impaired sustained attention [7] and memory [8‐10]. A wide spectrum of executive deficits have also been described, including problems performing goal-oriented tasks, recognizing priority patterns, and planning [11, 12], along with diminished verbal fluency [13] and information processing speed [14, 15]. Increasing recognition that psychosocial prognosis is directly related to the severity of the cognitive impairments [16‐19], has resulted in a paradigm shift that may expand the targets for treatment beyond the mere symptom suppression and necessitate an integration of cognitive assessment into routine psychiatric practice.

The importance on the field of this study is emphasized by a long-standing initiative of the National Institute of Mental Health, known as MATRICS [20, 21], which has now been subdivided into three different programs: CNTRICS [22], TURNS [23], and TENETS (Treatment and Evaluation Network for Trials in Schizophrenia). The aim of these initiatives is to unify and standardize the types of deficits to be measured and the tests to use, with the final objective of developing effective new treatments for the neurocognitive deficits that occur in these patients. Recently, the MATRICS initiative has proposed a consensus battery that takes between 60 and 90 minutes to administer and is composed of 10 paper-and-pencil tests specifically for cognitive assessment of patients with schizophrenia – the MATRICS Consensus Cognitive Battery [24, 25]. Given the difficulties of performing an assessment lasting more than one hour in standard clinical practice, in the past few decades, considerable effort has been made to create brief cognitive batteries that facilitate an overall understanding of the individual’s cognitive status, without overly sacrificing the sensitivity and specificity of these new instruments. Some examples are the Cognistat, before 1995 known as the Neurobehavioral Cognitive Status Examination [26], the Repeatable Battery for the Assessment of Neuropsychological Status (RBANS) [27], the Woodcock-Johnson III Test of Cognitive Abilities (WJ III COG) [28], and the Brief Assessment of Cognition in Schizophrenia (BACS) [29]. These instruments have decreased the time it takes to assess patients to about 40–50 minutes, but even so they have a high cost in terms of time and economics due to time constraints on practitioners in their daily clinical practice.

More recently, other types of studies have focused on the development of cognitive screening tools – scales that do not require additional materials in order to be administered, tools that have different interchangeable versions, tools that are simple and easy to administer, and have an administration time that is appropriate and manageable in clinical practice, i.e., approximately 15 minutes. Some examples are the Brief Cognitive Assessment (BCA) [30], the Screen for Cognitive Impairment in Psychiatry (SCIP) [31], and the Brief Cognitive Assessment Tool for Schizophrenia (B-CATS) [32]. All of these have good psychometric properties [30‐35], but still no standardization data have been established for any of them.

A Spanish translation of the SCIP was recently introduced (SCIP-S) [36] which demonstrated appropriate psychometric properties both for patients with a diagnosis of schizophrenia [34] and those with bipolar I disorder [33], with regard to equivalence between parallel forms, internal consistency, temporal stability, dimensional structure, and convergent validity. Tentative cut-scores for identification of significant cognitive impairment irrespective of diagnosis are available [35], but the resulting binary classification is insufficient for description of the severity of identified impairment relative to a patient’s clinical cohort after adjustment for age, gender, and education. Guidelines for the interpretation of the SCIP-S would thus represent a step forward in the clinical utility of this instrument and add valuable information on its proper use.

Normative data represent performance on a measure or test by a standardization sample against which other performances on the measure can be compared [37]. Lack of normative data limits the interpretation of scores in individual cases as well as in treatment outcome research (as we cannot know if a score is typical, high or low for the population being studied) [38]. This has implications for our ability to assess the clinical significance of a score (or change in a score). Norm scores can assist clinicians in providing quantitative labels for the degree to which a raw score is to be considered average, elevated, or extreme and might be useful for diagnostic purposes, clinical decision making, or evaluation of treatment effects [37]. A traditional approach to deriving norm scores is to compare an individual’s raw score to a reference group with the same condition matched for background variables such as age and gender. In addition, clinicians using norms for comparison can more readily interpret a patient’s performance on a number of relevant self-report dimensions as well. This should assist in the determination of whether or not an individual’s responses are unusual for someone experiencing, in this case, cognitive deficits. In turn, this may suggest possible courses of action, such as further investigation or treatment (whose outcome can be evaluated against the normative dataset) [37, 38].

When evaluating cognitive function in routine practice, clinicians usually compare the patient score against the norms in the general healthy population to ascertain whether the patient cognitive function is preserved or impaired. In such case, comparison allows to determine the distance to what a particular patient separate from the mean score. Nevertheless, in many cases the practitioner refers the patient to a specialist for formal recognition when his/her performance is unusually low compared with patients with same condition. Particularly when additional etiologies (in addition or replacement of schizophrenia) responsible for the high cognitive impairment observed is suspected [39]. Concerning this point, patient score should be compared with the norms belonging to subjects with the same health condition to assess how the different is the patient scoring related to his/her population of reference. As stated by Irvison et al. [40], this information can improve the clinician´s understanding of patient´s cognitive strength and weakness, put a patient’s cognitive abilities into perspective for their diagnosis, and facilitate multidisciplinary treatment decisions.

In this context, to date there are no standardization data for the SCIP-S scale in psychiatric patients that allow the examiner to interpret the patient scores relative to the cognitive performance of their peers. Thus, the objective of this study is to provide the first clinical normative data for the SCIP-S in patients with functional psychosis, and specifically with schizophrenia-spectrum disorder or bipolar I disorder.

The clinical value of a screening tool is directly related to either considering cognitive impairment a key aspect of schizophrenic psychopathology and, according to the proposed DSM-V revisions, recommending it as one key dimension to be measured in all patients with a psychotic disorder, or including cognitive deficit as one of the diagnostic criteria for psychoses as suggested by some authors [46, 47]. The practical utility of the administered tests should not be forgotten when conducting a neuropsychological assessment, and since there is a large number of psychiatric patients (accounting for around 2% of the general population) who require diagnosis, there is a growing need for cost-effective and highly efficient diagnostic tools. In this regard, the creation of the SCIP-S precisely had these two objectives. Previous studies [33, 34] have shown that the SCIP-S takes approximately 15 minutes to administer, compared to a mean of around 75 minutes for the administration of a full neuropsychological battery or between 60–90 minutes for the MCCB, and it has good validity and reliability. Furthermore, Rojo et al. [35] reported the good sensitivity and specificity of the test and its high diagnostic value for appropriately distinguishing cognitively preserved from cognitively impaired individuals.

This study goes a step farther and presents the diagnostic-specific norms and performance for the SCIP-S according to the age and educational level of subjects in a large sample of patients with schizophrenia and bipolar disorder. Some studies report differences in neuropsychological performance between subjects with different educational levels [48, 49], and such differences were also found in this study, which is why in the SCIP-S standardization data the sample has been divided based on educational level. It should be pointed out that, although there is not always a direct correspondence between educational level and years of education, we may consider that in the vast majority of cases a less than secondary education implies fewer than 12 years of education, while a secondary or higher educational level implies at least 12 years of education.

Another aspect known to influence cognitive performance is age, as over the years a series of cortical changes occurs resulting in a loss of brain volume [50, 51] associated with a decrease in cognitive performance in the general population [52, 53]. Our norms take this aspect into account by dividing the sample according to age and limiting patient age to 55 years in order not to introduce bias due to patients whose performance could be affected by early onset of a picture of dementia.

One item that bears emphasizing refers to that fact that a certain pattern was observed to repeat in the various subtests and total SCIP-S score. Specifically, the median score in the two age groups is similar (a maximum difference of 1 point) when the patients have a high educational level, while differences of up to 8 points are found in the groups with a primary or lower educational level. This may be explained by an interaction between age and educational level and the possibility that age at onset of illness plays an important role, since some studies show that both verbal intelligence and impairment of verbal memory and executive functioning could be affected in patients before they experience their first psychotic episode [54‐56], such that the earlier the onset of illness, the greater the potential for limiting the patient’s ability to normally pursue an education. Therefore, the effects of age and educational level and their interaction were explored by adding age at onset of illness as a covariate. Such interaction was not statistically significant in any SCIP-S score (all p > .01).

One of the aspects highlighted by this study is that of all the tests mentioned above that have been developed for the purposes of cognitively assessing psychiatric patients, we find diagnosis-specific standardization data for patients with schizophrenia only for the RBANS [39, 40]. And as stated by Iverson et al. [40], being able to describe a patient’s cognitive performance in terms of expectation for their peer group is more useful to multidisciplinary treatment teams than just comparing them to a healthy population. So the present study provides the tools necessary to interpret the score obtained by a patient with functional psychosis relative to other patients after administration of the SCIP-S scale. As an example, let us apply the SCIP-S to an imaginary 38 year old college graduate diagnosed with schizophrenia who obtains a direct score of 13 on the WMT subtest. After determining their performance relative to healthy controls (healthy control norms are under elaboration), the clinician interest could move to answer the question: Where do we situate that individual with respect to other patients? Looking at Table 4, we see that, based on his age and educational level, this patient is in the 9th percentile, has a z-score of −1.45, a T-score of 36, and an IQ of 78. This tells us that only 9% of his comparison group has obtained a score below his and that his working memory score of 13 is situated approximately 1.5 standard deviations below the other patients.

The SCIP and the SCIP-S provides a quick and convenient cognitive diagnosis and, in that regard, its usefulness extends to areas such as detection, cognitive assessment of large samples, epidemiological and screening diagnostic studies more than to specific cognitive domains or type of impairment in patients with functional psychosis. In that way, it is a complementary test that is not intended to replace complete neuropsychological batteries. Future studies should explore how performance on the SCIP relates to other cognitive domains that it does not measure directly (e.g., problem solving, social cognition, etc.).

A study that could continue this one should be perform standardization data for patients over 55 years of age, since, at a cognitive level, that is a critical age where the SCIP-S could help us reach a differential diagnosis between onset of dementia versus cognitive dysfunction associated with functional psychosis. Future research with this scale should also incorporate the development of guidelines for interpreting the scoring according to results of treatment of patients. Additionally, last evidences in schizophrenic and bipolar patients from recent studies have suggested that the history of psychosis explain part of the neurocognitive performance [57], thus in future studies with cognitive screening tools would be interesting to take this variable into account.

In short, this study presents the first jointly developed diagnostic-specific norms for the SCIP for functional psychosis (schizophrenia and bipolar disorder), providing increased information about their cognitive abilities.