Introduction

Patients with schizophrenia experience deficits across a variety of cognitive domains (Elvevag & Goldberg, 2000; Gold & Weinberger, 1995). It has been shown that these cognitive impairments have a negative impact on patients’ ability to function (Bowie et al., 2008; Green, 1996), contributing to schizophrenia’s status as one of the leading causes of disability in the United States (Ormel et al., 2008). Therefore, cognition in schizophrenia has emerged as an important target for treatment development (Gray & Roth, 2007; McGurk, Twamley, Sitzer, McHugo, & Mueser, 2007). In response to this need, various stakeholder groups, including researchers, the NIMH, industry, and the FDA, started two initiatives, both of which resulted in batteries of cognitive paradigms that assess multiple domains of cognition: the Measurement and Treatment Research to Improve Cognition in Schizophrenia (MATRICS) battery (Kern et al., 2008; Nuechterlein et al., 2008) and the Cognitive Neuroscience Test Reliability and Clinical Applications for Schizophrenia (CNTRACS) consortium battery (Barch et al., 2012; Henderson et al., 2012; Ragland et al., 2012; Silverstein et al., 2012) that followed from the Cognitive Neuroscience Treatment Research to Improve Cognition in Schizophrenia (CNTRICS) initiative. To provide researchers with more information about the characteristics and utility of tasks from the CNTRACS and the MATRICS batteries, the goals of the present study were to assess the interrelationships among tasks from these batteries, to determine the degree to which tasks from each battery capture unique variance in cognitive dysfunction in schizophrenia, and to examine the ability of tasks from each battery to predict functional outcome.

We are very pleased to put forth these results from the CNTRACS work as part of this special issue honoring Ed Smith. Ed Smith played an important role in helping to spawn the CNTRICS initiative. He was part of the RAND panel that helped to select tasks to be included in the MATRICS battery and expressed his frustration that few paradigms developed as part of modern cognitive neuroscience could be considered for the MATRICS battery because of the absence of psychometric data and the lack of standardization and optimization for use in clinical populations. During this panel, he clearly voiced the need for the field to put effort into the translation and psychometric development of paradigms developed as part of cutting edge cognitive neuroscience research, which helped spur Deanna Barch, Cam Carter, and others to start the CNTRICS initiative. A perhaps little known fact about Ed Smith is that he started and ended his career with a focus on schizophrenia, bringing full circle an extraordinary career spent elucidating the psychological and neural mechanisms that allow humans to control their thoughts and actions. Given that schizophrenia centrally involves deficits in cognitive control, working memory, and episodic memory, it is fitting that the final stages of Ed Smith’s career involved applying the insights he developed to help us understand how to characterize and treat cognitive deficits in schizophrenia.

Relationship of constructs assessed by CNTRACS tasks to MATRICS battery

The MATRICS and CNTRACS batteries were each developed through a consensus process, but with somewhat different conceptual approaches. The MATRICS battery needed to be developed in a short time frame and focused on identifying tasks with already established psychometric properties (Kern et al., 2008; Nuechterlein et al., 2008). This necessitated the inclusion of more traditional and primarily neuropsychological tasks, such as the Brief Assessment of Cognition in Schizophrenia (BACS) Symbol-Coding test, the Hamilton Verbal Learning Test, and a maze task, among others. In contrast, the CNTRICS (Carter & Barch, 2007) initiative focused on utilizing a contemporary cognitive neuroscience-based approach, with the goal of identifying and standardizing cognitive paradigms that were well-validated measures of specific cognitive mechanisms that had, or were seen as amenable to, expression in animal models that could be used in drug discovery and that could be used in both early phase clinical trials and pathophysiology studies. Also, the CNTRICS process included recognition that many of the most promising tasks would need further modification, standardization, and optimization of their psychometric properties before widespread use in clinical trials of schizophrenia patients, a process that was not needed in MATRICS since standardized clinical neuropsychology instruments were used. Therefore, following on CNTRICS, the CNTRACS consortium developed a battery that assesses four component processes from three cognitive domains: goal maintenance (cognitive control), episodic memory (long-term memory), visual integration (perception), and gain control in visual processing (perception). Briefly, goal maintenance is the ability to actively maintain contextual information that is important for an ongoing task, such as task rules or previous stimuli. It is distinct from short-term memory, in that the individual must maintain a context, as opposed to the identity of a stimulus (Servan-Schreiber, Cohen, & Steingard, 1996). Long-term memory is the ability to store and appropriately retrieve previously presented information (Ragland et al., 2012). Visual integration, or perceptual organization, is a process in which pieces of visual information are integrated into a whole scene or object and occurs one step beyond the registration of color, orientation, motion, and depth (Silverstein et al., 2012). Gain control is also a perceptual process and it refers to “processes that amplify or attenuate overall levels of neural activity to optimize operation of systems with limited dynamic signaling range” (Barch et al., 2012, p. 135).

The motivation for the CNTRACS tasks was that they would isolate specific perceptual or cognitive functions that may be conflated in more traditional batteries, such as the MATRICS, where task scores reflect the contribution of multiple cognitive processes (e.g., visual scanning, attention, memory, problem solving, etc.). Therefore, the first goal of the present study was to examine the relationships between cognitive tasks from the CNTRACS and MATRICS batteries and to assess the success of the CNTRACS paradigms in isolating specific cognitive functions. Specifically, we hypothesized that we would see relatively small correlations with tasks from the MATRICS battery, and we should see that the CNTRACS tasks can identify variance in cognitive domains in schizophrenia that may not be apparent on the MATRICS tasks (and potentially, vice versa).

Relationship of CNTRACS and MATRICS tasks to function in schizophrenia

As was described above, it has been shown that cognitive impairments in schizophrenia have a negative impact on patients’ ability to function (Bowie et al., 2008; Green, 1996). Thus, one goal in developing the MATRICS battery was for it to predict functional status and for changes in MATRICS task performance during treatment trials to predict change in function. Consistent with these goals, the MATRICS composite score correlates strongly with the UCSD Performance-Based Skills Assessment–Brief (UPSA–B), a performance-based measure of functional outcome (Keefe, Fox, Harvey, Cucchiaro, Siu, & Loebel, 2011; Nuechterlein et al., 2008), and also shows sensitivity to work status in schizophrenia patients, with a “poor work group” performing significantly worse on the battery than a “good work group” (August, Kiwanuka, McMahon, & Gold, 2012). In the original development study, the MATRICS measures showed relatively modest associations to self-reported function (Nuechterlein et al., 2008), although this is perhaps not surprising, given the evidence of limited validity for self-reports of function in schizophrenia (Bowie et al., 2007). One of the goals in developing the CNTRACS battery was to test mechanisms of cognitive impairment and mechanisms of change. However, it is also important to know whether the CNTRACS tasks relate to function, to determine whether paradigms derived from basic cognitive neuroscience are also relevant to understanding aspects of, and variability in, real-world functioning in schizophrenia. Clarification of these issues is also important for addressing FDA requirements for functional “co-primary” measures accompanying the use of cognitive task performance as end points in registration studies. In a previous study, we showed that the CNTRACS tasks assessing goal maintenance and episodic memory correlated with the UPSA–B (Gold et al., 2012). Furthermore, the CNTRACS goal maintenance task was also correlated with informant reports of patient function, although not with self-reported function.

The results of these prior studies with the MATRICS and CNTRACS suggest that both sets of tasks (or at least a subset) show relationships to performance-based measures of functional outcome and some relationships to self-reported or informant-reported outcome. Importantly, however, the magnitude of the reported relationships between functional outcome measures and the cognitive batteries differ, with the MATRICS composite score showing a stronger relationship with performance-based functioning than do the individual CNTRACS tasks, in addition to slightly stronger relationships with self-reported functioning (Gold et al., 2012; Keefe et al., 2011; Nuechterlein et al., 2008). Although this pattern suggests that MATRICS tasks are better at predicting functional outcome, this is difficult to compare across different patient groups in different studies. As such, a second goal of the present study was to examine the relationships of the CNTRACS tasks and a subset of the MATRICS tasks to performance-based function and both self- and informant-rated functional outcome in a new sample of patients. In addition, we asked whether tasks from either battery correlated with function after accounting for performance on the other task battery, to test the hypothesis that the tasks from each battery would uniquely account for variance in function.

Common versus specific deficits in relationship to function in schizophrenia

An ongoing debate in the literature is whether patients suffer only from a single broad impairment that affects cognition in schizophrenia (Dickinson, Iannone, Wilk, & Gold, 2004; Dickinson, Ragland, Gold, & Gur, 2008) or whether there are additional deficits in specific domains that provide insight into the illness (Chapman & Chapman, 1978; Fornito, Yoon, Zalesky, Bullmore, & Carter, 2011; Lesh, Niendam, Minzenberg, & Carter, 2011; Repovs, Csernansky, & Barch, 2011). Prior work has shown strong intercorrelations among all the MATRICS tasks (August et al., 2012), while only the goal maintenance and episodic memory tasks from the CNTRACS were moderately intercorrelated (Gold et al., 2012). Thus, the literature clearly suggests that there is a common deficit contributing to performance across tasks in the MATRICS battery, with more modest evidence for such a common factor in the CNTRACS tasks—supporting the view that the CNTRACS tasks are more process specific than those in the MATRICS battery (for reasons noted above). However, to examine this more closely, the final goal of the present study was to determine whether such a common factor of shared variance correlates with functional capacity or observer or self-reported function.

Method

Subjects

Subjects were recruited from five study sites: Washington University in St. Louis (controls, n = 22; patients, n = 16), Maryland Psychiatric Research Center at the University of Maryland (controls, n = 25; patients, n = 20), University of California–Davis (controls, n = 24; patients, n = 18), Rutgers University (controls, n = 38; patients, n = 32), and University of Minnesota–Twin Cities (controls, n = 23; patients, n = 18). Each site received approval from their respective institutional review boards, and all subjects signed an informed consent document before beginning the study.

In total, 132 healthy control subjects and 104 schizophrenia subjects participated in the study. All subjects were interviewed using the Structured Clinical Interview for the Diagnostic and Statistical Manual–IV (SCID; First, Spitzer, Gibbon, & Williams, 1995). Individuals were excluded if they had experienced a serious head injury, endorsed a neurological disease, or had a history of intellectual disability or other pervasive developmental disorder. Subjects were also excluded if they endorsed substance dependence in the last 6 months and/or substance abuse in the past month. All subjects were native English speakers and scored at least a six on the Wechsler Test of Adult Reading (WTAR), a measure of premorbid IQ (Weschler, 2001). All subjects received a drug and alcohol screen on the day of testing.

In addition to the above criteria, healthy control subjects were excluded if they had a history of schizophrenia, bipolar disorder, or any other psychotic disorder, were currently experiencing major depression, or were currently taking psychotropic or cognition-enhancing medications. Patients were included only if they met criteria for a diagnosis of schizophrenia or schizoaffective disorder. In addition, patients were included only if they did not anticipate any medication changes within a month and had stable outpatient or partial hospital status. Demographic characteristics for each group are shown in Table 1.

Table 1 Demographic and clinical variables
Full size table

Procedure

All subjects were assessed across four study sessions. During session one, subjects completed the SCID (First et al., 1995), WTAR (Weschler, 2001), two cognitive measures from the MATRICS battery, the UPSA–B (Mausbach, Harvey, Goldman, Jeste, & Patterson, 2007), an alcohol screen via breathalyzer, and a drug screen using a test card sensitive to the presence of street drugs in a urine sample; patients completed three additional measures of functional outcome (the Specific Levels of Functioning Scale–Self, the Specific Levels of Functioning Scale–Informant, and the Multidimensional Scale of Independent Functioning) (Jaeger, Berns, & Czobor, 2003; Schneider & Struening, 1983). Session two, which was planned within 2 weeks of session one, included an alcohol and drug screen and five cognitive paradigms developed by the CNTRACS Consortium: the Dot Probe Expectancy Task (DPX), the AX-Continuous Performance Task (AX-CPT), the Contrast-Contrast Effect (CCE), the Jittered Orientation Visual Integration Task (JOVI), and the Relational and Item-Specific Encoding Task (RISE), all of which are described further below. Subjects also repeated the CNTRACS tasks at two additional sessions to assess test–retest reliability. However, the focus of the present study was just on the baseline assessment. All subjects performed the tasks in the order listed above, although the AX-CPT and DPX tasks were counterbalanced, with one occurring at the beginning and the other at the end. Importantly, although the CCE was administered to all subjects, our prior work suggested that abnormal CCE performance could not be isolated from attention lapse errors (Barch et al., 2012) and that the critical CCE indices have relatively low reliability (Strauss et al., 2013). Therefore, data from the CCE were not included in any of the present analyses.

Measures of cognition

Performance was measured across three domains of cognitive ability, using the four CNTRACS tasks mentioned above. These computer-based paradigms have been previously reported in great detail, so only brief summaries of each task are provided below. In addition, these tasks are openly available to investigators at http://CNTRACS.ucdavis.edu/task.

  1. 1.

    Goal maintenance was assessed through the DPX paradigm (Henderson et al., 2012) and the letter-version of the expectancy AX-CPT task (MacDonald, Pogue-Geile, Johnson, & Carter, 2003; Schneider & Struening, 1983). Both tasks require subjects to view a series of cue and probe sequences, one stimulus at a time, and make a buttonpress indicating whether the stimulus seen on the screen does or does not complete a target stimulus pair. In the AX-CPT version, this target pair is AX, meaning that when a subject sees an A (cue) followed by an X (probe), they should indicate that the X completes the target pair. This is contrasted with a BX trial, during which an X probe follows an invalid cue (“B,” which is any letter other than “A”), so subjects must respond nontarget to X. Critically, most trials (70%) are AX trials, creating two expectations. The first is that an A cue will be followed by an X probe, at least in people who use the cue to prepare prospectively for upcoming probes. If so, subjects are more likely to false alarm on “AY” trials (with “Y” indicating any letter other than “X”) or to be slow to respond correctly on “AY” trials. In addition, the second expectation is that the majority of the X trials are target probes (i.e., A followed by X), resulting in a response bias for indicating that the X is a target. Thus, “BX” trials require using the goal information provided by the “B” cue to overcome this prepotent response tendency to respond target to “X.” This discrimination between AX and BX trials therefore requires maintenance of task-relevant goals. Accordingly, performance on both the DPX and AX-CPT tasks are measured using d’context, a variable that indexes the hit rate for AX trials, relative to false alarms for BX trials. The procedure, numbers of trials of each type, expectations, and interpretation of results are identical for the DPX paradigm, with the only difference being the stimuli. In the DPX paradigm, instead of responding to letters, subjects must respond to dot arrays, which are visual renderings of Braille versions of the AX-CPT stimuli. Subjects are trained before beginning the task to ensure that they can identify which dot array represents the probe and which represents the cue for the target pair.

  2. 2.

    Visual integration was assessed using the JOVI (Silverstein et al., 2012). On each trial of this task, a display containing Gabor elements against a gray background is presented, with a subset of these elements forming a leftward- or rightward-pointing oval (for stimuli examples, see Silverstein et al., 2012). The subject’s task is to indicate on each trial, via a keypress, whether the target stimulus is pointing to the right or left. Task difficulty is manipulated by jittering the elements of the contour by various degrees across several conditions. At 0° jitter, a smooth shape is formed. As jitter increases to ±7°, 9°, 11°, 13°, or 15° for each element of a single contour, discrimination of the overall shape created by the elements becomes increasingly difficult. Performance was first characterized as accuracy (proportion correct) for each individual jitter level, which was then fit to a sigmoidal (cumulative logistic) function, which could vary in shape along the parameters of threshold, slope, and upper asymptote (Wichmann & Hill, 2001). Threshold corresponds to the level of jitter at which a subject reaches a level of accuracy that is halfway between the upper asymptote (~100% correct) and chance (50%). Therefore, the higher an individual’s threshold, the better he or she was at visually integrating the shape at a higher jitter magnitude. Threshold was used as the dependent variable for this task.

  3. 3.

    Episodic memory encoding and retrieval was assessed through the RISE, a task that includes two types of encoding for visually presented objects: item-specific and relational encoding (Ragland et al., 2012). During item-specific encoding, subjects view 36 objects and must decide whether the object is living or nonliving. During relational encoding, subjects view 18 pairs of objects and must decide whether one of the objects could fit inside of the other object. Subjects then perform two different retrieval tasks: item recognition and associative recognition. During item recognition, subjects view all previously seen objects, mixed with an equal number of foils that are similar (in color, size, etc.), and must determine whether the object is old or new. During associative recognition, subjects view pairs of objects, all of which are old but half of which are foil pairs, and must decide whether those objects had been previously paired together. Due to concerns about practice effects across the three administration time points, three psychometrically similar versions of the RISE were used with different stimuli. Performance on this task was determined by overall recognition accuracy (hit rate − false alarm rate) (1) for item recognition for items following item-specific encoding (RISE IRIE), (2) for item recognition for items following relational encoding (RISE IRAE), and (3) for associative recognition for item pairs from the relational encoding condition (RISE AR).

In addition to the above paradigms from the CNTRACS battery, subjects were also tested on two paradigms from the MATRICS Consensus Cognitive Battery (MCCB): the Symbol-Coding Test from the BACS (Keefe, Goldberg, Harvey, Gold, Poe, & Coughenour, 2004), which is a MATRICS measure of speed of processing, and the Hopkins Verbal Learning Task–Revised (HVLT–R; Brandt, 1991), which is the MATRICS measure of verbal learning and memory (Kern et al., 2011). Although it would have been ideal to administer the entire MCCB, time constraints would not allow it. As such, we selected the BACS Symbol Coding and HVLT–R, on the basis of their large effect sizes for deficits in schizophrenia (Dickinson, Ramsey, & Gold, 2007; Kern et al., 2011). Going forward, we will use the acronym MATRICSsub when discussing the BACS Symbol Coding and the HVLT–R, as a reminder that not all MATRICS tasks were included in the present study. We will also use the acronym BACSsc when discussing the symbol-coding task from the BACS, as a reminder that only this subtest from the BACS was used.

Functional capacity and status

To assess patients’ ability to perform everyday skills, we administered the UPSA–B, which is a well-validated measure requiring subjects to engage in simulated life skills (Patterson, Goldman, McKibbin, Hughs, & Jeste, 2001; Twamley et al., 2002). This measure is particularly useful because it does not depend on self-report of perceived functioning and, instead, tests these skills directly. We also included the Multidimensional Scale of Independent Functioning (MSIF; Jaeger et al., 2003), which is a self-report scale that includes measures of functioning within work, education, and residential environments, as well as assessing function in terms of the role position the patients have within each environment, the level of support they require, and their general performance. For the purposes of the presented analyses, we used the global rating of function, which combines scores from all of those domains. Unlike the other outcome measures used, high scores on the MSIF indicate poorer functioning. We also administered the Specific Levels of Functioning Scale (SLOF; Schneider & Struening, 1983), which measures the individual’s interpersonal relationships, participation in community activities, and work skills. This scale was completed by the patients themselves; however, due to previous work showing that patient reports alone may not provide a sufficiently valid assessment of real-world function (Keefe, Poe, Walker, Kang, & Harvey, 2006), this scale was also filled out by an informant. The informants included family members, case workers, and therapists who felt that they had sufficient knowledge to speak to the patient’s functioning. This informant measure was obtained from 71 of the 104 subjects with schizophrenia. Finally, we used the WTAR (Weschler, 2001) as a measure of premorbid IQ.

Clinical symptoms

All schizophrenia subjects were assessed using the 24-item Brief Psychiatric Rating Scale (BPRS; Overall & Gorham, 1962)), which includes subscales for positive symptoms, negative symptoms, disorganized symptoms, manic symptoms, and depressed mood. All items are measured on a 7-point scale, ranging in severity from not present (1) to extremely severe (7). Scores were derived through a semistructured interview with a certified clinical rater. Total scores for each subscale were calculated by summing all items within that scale, and mean subscale scores were substituted for any missing items. To ensure consistent ratings across sites, raters were trained by teleconference, during which ratings and anchor points for all scales were discussed and six training videos were completed. For at least six interviews, raters had to achieve agreement based on a “gold” standard (those of the trainers who were highly skilled clinicians from either the St. Louis or Maryland sites), in order to be certified. Agreement was defined as no more than two items with a difference of more than 1 rating point from the gold standard trainers. In order to maintain reliability across sites throughout the study, videotaped interviews were rated every 2–4 weeks, with all raters participating in a teleconference to resolve discrepancies. We conducted analyses on the relationship between task performance and clinical symptoms, as well as on the relationship between functional capacity/status measures and clinical symptoms. However, because these were not the focus of the present study, they are presented in Supplemental Materials .

Data analysis

In order to assess relatedness of cognitive performance between the six different measures of the CNTRACS and the two tasks from the MATRICS battery, bivariate intercorrelations were calculated, separately for each group. Correlations with CNTRACS tasks, MATRICSsub tasks, measures of functional outcome, and measures of symptom severity were also calculated for the patient group. Correlations between cognitive tasks and the four functional outcome measures were corrected for multiple comparisons within each task measure, with significance defined as p < 0.0125; corrections were also made for correlations with the five symptom measures, with significance defined as p < 0.01.

To determine whether tasks from one battery accounted for unique variance in schizophrenia impairments after accounting for performance on the other battery, we performed a series of regressions. For the MATRICSsub tasks, we entered BACSsc and HVLT–R as dependent variables (one regression for each task) and used three core CNTRACS tasks (DPX, JOVI, and RISE IRAE) as predictor variables. For predictor variables we selected the DPX task (as opposed to the AX-CPT), due to a recent study by our group showing better reliability for the DPX (Strauss et al., 2013); we chose to include the RISE IRAE variable (as opposed to the other RISE variables) because it is the most specific variable for assessing a relational processing deficit (Ragland et al., 2012). We then saved the residuals, which quantified the amount of variance remaining within the MATRICSsub tasks after accounting for the variance shared by the CNTRACS tasks. This was then done for each of the six CNTRACS tasks, but using the BACSsc and HVLT–R as predictors. This process yielded residual scores for each of the eight cognitive tasks, taking into account the variance shared by the other cognitive battery. These residual scores were then compared between the patient and control groups in an independent samples t-test and were also correlated with symptom and function measures.

To address the question of shared variance even more conservatively, we performed another series of linear regressions in which variance shared by all other tasks was regressed out and the residuals for each individual task were saved, leaving only unique task variance (when predicting DPX, the AX-CPT was not included as an independent variable, and the other RISE measures were not used when predicting RISE). These residuals were then compared between groups and also correlated with functioning and symptom measures.

In addition, to understand how the common or shared variance itself related to function, we performed an unrotated principal axis factor analysis, again for all subjects, which included the three core CNTRACS tasks, BACSsc, and HVLT–R. Only factors with eigenvalues greater than one were retained, and these factor scores were saved as variables and correlated with symptom and function measures within the patient group.

Results

Demographic and clinical characteristics

Schizophrenia patients and healthy control subjects did not significantly differ on age, gender, race, parental socioeconomic status (SES), or parental education. Schizophrenia patients had significantly lower personal education and personal SES, as compared with healthy control subjects (see Table 1). Additionally, there were no significant differences in clinical or demographic variables between schizophrenia patients who did and did not have a completed SLOF Informant report, except for a trend toward higher maternal education for the patients who had SLOF Informant data, t(96) = −1.99, p = 0.05.

What is the relationship among the CNTRACS tasks and among the MATRICSsub tasks?

As is shown in Table 2, the structure of the correlations within the patient group among the CNTRACS tasks was similar to that in our prior study (Gold et al., 2012). Specifically, the three RISE measures were significantly intercorrelated (rs range from 0.32 to 0.91, p < 0.01). In addition the AX-CPT and DPX measures were significantly intercorrelated, r = 0.65, p < 0.001, and were also correlated with the RISE measures (rs range from 0.25 to 0.39, p < 0.05). In contrast, JOVI threshold was not correlated with any other task score. The pattern of intercorrelations was broadly similar for the RISE and AX-CPT/DPX tasks for healthy control subjects (see Table 2), although correlation coefficients were generally smaller. In controls, the JOVI showed modest correlations with two of the RISE measures and the AX-CPT. From the MATRICSsub, the HVLT–R and the BACSsc were correlated in both the patients, r = 0.48, p < 0.001, and controls, r = 0.48, p < 0.001.

Table 2 Correlations among the CNTRACS tasks in controls and individuals with schizophrenia
Full size table

What is the relationship between the CNTRACS and MATRICSsub tasks?

As is shown in Table 3, the HVLT–R showed consistent correlations with both the AX-CPT/DPX and RISE measures for both patients and controls, with little correlation with the JOVI. The BACSsc also showed consistent correlations with the AX-CPT/DPX and RISE, although the correlations were somewhat weaker among patients for the RISE item recognition measures. The BACSsc did show somewhat more evidence of association with the JOVI than did the HVLT–R, although the magnitudes of these correlations were not significantly different.

Table 3 Correlations between the CNTRACS tasks, MATRICSsub tasks, and WTAR
Full size table

Do controls and patients differ on MATRICSsub, CNTRACS, and function measures?

Group differences in CNTRACS task performance in this sample have been reported previously (Strauss et al., 2013). However, for clarity, we report group differences for the CNTRACS tasks here as well. Schizophrenia patients performed significantly worse than healthy controls on all CNTRACS and MATRICSsub tasks (Table 4). Average effect size for CNTRACS tasks was 0.83 (range: Cohen’s d = 0.31–1.05), and average effect size for the MATRICSsub tasks was 1.04 (BACSsc, d = 0.88; HVLT–R, d = 1.21). Schizophrenia patients had a significantly lower premorbid IQ than did healthy controls, t(234) = 2.03, p = 0.044, and also had significantly poorer functioning, as measured by the UPSA–B, t(233) = 6.3, p < 0.001. Patients who did and did not have completed SLOF Informant data did not significantly differ in performance on any cognitive task or on any other function measure.

Table 4 Task performance in controls and individuals with schizophrenia
Full size table

Do controls and patients differ on MATRICSsub or CNTRACS tasks after accounting for variance shared by the other task battery?

As was described in the Method section, we calculated residual scores for each of the task measures, after regressing out the variance shared by measures from the other battery. After taking out variance shared with the BACSsc and HVLT–R, all three RISE measures remained significantly different between groups [RISE AR, t(224) = 3.11, p < 0.01; RISE IRAE, t(224) = 4.51, p < 0.001; RISE IRIE, t(224) = 3.77, p < 0.001]. The DPX and AX-CPT also remained significant, t(219) = 2.03, p < 0.05, and t(223) = 2.05, p < 0.05, respectively. However JOVI performance no longer differed between groups, t(216) = −0.85, p = 0.40. After accounting for variance shared with the CNTRACS measures, BACSsc and HVLT–R performance continued to be significantly worse in patients than in controls, t(203) = 2.55, p = 0.01, and t(203) = 3.9, p < 0.001, respectively.

Do controls and patients differ on MATRICSsub or CNTRACS tasks after accounting for variance shared by all other tasks, even on the same battery?

We next calculated residuals for each task, after including all other tasks as independent variables. We found that performance for the RISE IRIE, t(202) = 3.20, p < 0.01, RISE IRAE, t(202) = 3.94, p < 0.001, RISE AR, t(202) = 2.71, p < 0.01, and HVLT–R, t(202) = 2.82, p < 0.01, continued to be significantly different between patients and controls. The BACSsc, JOVI, DPX, and AX-CPT residuals no longer showed significant group differences.

How well does cognitive performance predict functional outcome?

Next, in our patient group, we performed correlations between the cognitive measures and the measures of functional outcome (Table 5). Although all functional outcome assessments were designed to measure related constructs, they themselves were not all intercorrelated. The UPSA–B was significantly associated only with the SLOF Informant report, r = 0.31, p = 0.01, the MSIF was significantly associated only with the SLOF Patient report, r = −0.33, p = 0.001, and the SLOF Patient and Informant reports were significantly associated with each other, r = 0.31, p = 0.01.

Table 5 Functional and cognitive correlations for patients
Full size table

We found that the UPSA–B was significantly correlated with the AX-CPT, DPX, RISE IRAE, and BACSsc even after corrections for multiple comparisons, with trends at a nominal p-value of 0.05 for RISE IRIE, RISE AR, and HVLT–R. MSIF Global scores were significantly correlated with the HVLT–R, with a trend for the RISE IRAE. The SLOF Patient report was significantly correlated with the HVLT–R. The SLOF Informant report did not significantly correlate with any of the tasks after correction for multiple comparisons. However, the DPX was correlated with SLOF Informant reports at a nominal p-value of 0.05.

Does cognitive performance continue to predict functional outcome after accounting for variance shared by the other task battery?

When residual scores from regressing out the alternative task battery were correlated with measures of functioning, we found that the DPX residual was significantly correlated with scores on the UPSA–B, r = 0.27, p < 0.01, with trends for the AX-CPT, r = 0.21, p < 0.05, and the RISE IRIE, r = 0.21, p < 0.05. The HVLT–R residual was significantly correlated with the SLOF Patient report, r = 0.28, p < 0.01, and also trended toward significance for the MSIF Global, r = −0.25, p = 0.02. There were trends for the DPX, r = 0.24, p = 0.06, and the RISE IRAE, r = 0.21, p = 0.10, residual to correlate with SLOF Informant reports.

Does cognitive performance continue to predict functional outcome after accounting for variance shared by all other tasks?

When residual scores from regressing out all other tasks were correlated with measures of functioning, we found that the DPX, r = 0.26, p < 0.05, was still correlated with the UPSA–B at a nominal p-value, although it did not pass corrections for multiple comparisons. The residual for the HVLT–R remained significantly correlated with scores on the MSIF, r = −0.33, p < 0.01, and the SLOF Patient report, r = 0.29, p < 0.01.

Does common task variance predict functional outcome?

The residual score analysis presented above indicates that there was unique variance associated with both the CNTRACS tasks and the MATRICSsub tasks that showed significant group differences and relationships to measures of function. However, it is clear that there is also shared variance across the tasks, and we wanted to understand how that shared variance related to function. Thus, we performed a nonrotated principal axis factor analysis that included the three CNTRACS tasks (DPX, JOVI, and RISE IRAE) and both MATRICSsub tasks and included both groups. The first factor, which was the only factor with an eigenvalue of >1, explained 47% of the overall variance. The four measures of “higher cognitive” functions (DPX, RISE, HVLT–R, and BACSsc) loaded most strongly on this factor (all > 0.56), and the JOVI, a measure of perceptual integration, had a relatively low load (−0.28). This factor correlated significantly with performance on the UPSA–B, r = 0.48, p < 0.001), and trended toward significantly correlating with scores on the SLOF Informant report, r = 0.31, p = 0.02, and SLOF Patient report, r = 0.24, p = 0.03, after correction for multiple comparisons.

Discussion

Overview

We assessed cognitive ability using paradigms put forth by both the CNTRACS and MATRICS initiatives in a large group of schizophrenia patients and healthy control subjects and explored how cognitive ability related to functional outcome. For all cognitive paradigms, we evaluated the specificity of each task by examining how robustly it correlated with other tasks designed to measure different cognitive functions. In general, we found that a number of cognitive paradigms from both the CNTRACS and MATRICSsub batteries were intercorrelated at a somewhat higher level than in our prior study (Gold et al., 2012). However, these intercorrelations varied greatly, ranging from 0.03 to 0.91, with the highest correlations for similar measures (e.g., DPX and AX-CPT) and with the perceptual paradigm from the CNTRACS yielding the lowest intercorrelations with other tasks, which measured higher cognitive functions. In addition, we assessed the contribution of both unique and common variance for each task to group differences and relationships to function. A number of the CNTRACS tasks, as well as the HVLT–R, continued to identify group differences and to predict function even after accounting for variance associated with other tasks. Each of these results is discussed in more detail below.

Relationships among the CNTRACS tasks

We saw significant intercorrelations among many of the CNTRACS tasks within both patients and controls. Importantly, as in our prior work (Gold et al., 2012), these intercorrelations were primarily between the RISE, DPX, and AX-CPT tasks, which are all measures of higher cognitive functions (episodic memory and goal maintenance) (Henderson et al., 2012; Ragland et al., 2012). In contrast, the JOVI, which is a task designed to measure the early perceptual function of visual integration (Silverstein et al., 2012), was much more weakly correlated with performance on these episodic memory and goal maintenance tasks, in both patients and controls. This pattern provides evidence for a distinction between perceptual and high-level cognitive functions that is consistent with the hypothesis of different neurobiological mechanisms (Pylyshyn, 1999). Other support for this distinction comes from prior research in which schizophrenia patients showed a significant deficit on measures of contextual, but not perceptual, control, when compared with healthy control subjects (Chambon et al., 2008). Given that cognitive control functions supported by prefrontal cortex functions may contribute to performance on both episodic encoding (Ranganath, Minzenberg, & Ragland, 2008; Rizio & Dennis, 2013; Spaniol et al., 2009) and goal maintenance tasks (for a review, see Lesh et al., 2011), it is not surprising that the RISE and AX-CPT/DPX tasks were significantly related and suggests that they share both common and unique processes.

While this pattern of intercorrelations is not surprising on the basis of theories of cognitive control (Barch, Csernansky, Conturo, & Snyder, 2002; Van Snellenberg, 2009), these relationships were somewhat stronger than expected, given our prior findings (Gold et al., 2012). In the first CNTRACS study, we found minimal intercorrelations among the CNTRACS tasks, such that the DPX did not correlate significantly with any other CNTRACS task (the AX-CPT was not in the prior study), suggesting that each paradigm measured fairly discrete cognitive processes. One possible contribution to the stronger intertask correlations in the present study is a difference in the design of the two studies. In the Gold et al. study, subjects completed the tasks on different days, whereas in the present study, all tasks were completed on the same day. There are many variables that can affect performance on cognitive tests, including nutrition (Dye, Lluch, & Blundell, 2000), mood (Ashby, Isen, & Turken, 1999), and fatigue (Durmer & Dinges, 2005). If tested on the same day, these factors would be likely to influence performance on tasks in a consistent manner, thereby inducing some degree of correlation. In contrast, these state factors may not influence performance across tasks when testing subjects across multiple days. Considering that most studies will test subjects only on a single day, data from the present study are likely more reflective of the level of intercorrelation among CNTRACS tasks that most investigators would observe.

Relationship between CNTRACS and MATRICSsub tasks

We felt that it was important to compare the relationships between the CNTRACS and MATRICSsub tasks within the same patient group, as well as their relative relationships to measures of function. Interestingly, we found that both MATRICSsub tasks were at least moderately correlated with the RISE and AX-CPT/DPX measures from the CNTRACS, although less strongly correlated (or not significantly correlated) with performance on the JOVI. A relationship between the HVLT–R and the RISE measures is consistent with both tasks measuring aspects of episodic memory. The correlations between the HVLT–R and the AX-CPT/DPX are somewhat more surprising. However, as was described above, cognitive control functions can contribute to episodic memory performance. The BACSsc is described as a measure of processing speed in the MATRICSsub battery. However, good performance on the BACSsc is facilitated by being able to maintain the symbols and digit pairings in working memory and/or by memorizing them via episodic memory function. In this framework, significant correlations between the BACSsc and the DPX/AX-CPT and RISE are less surprising.

Common versus specific deficits

A pervasive topic in schizophrenia research is that of a generalized cognitive deficit, which is a hypothesized explanation for the impairments observed across multiple cognitive domains in schizophrenia (Dickinson & Harvey, 2009; Dickinson et al., 2008; Keefe, Bilder, Harvey, Davis, Palmer, Gold & Lieberman, 2006a). When variability shared by the alternative battery was removed, performance remained significantly worse between patients and controls for all tasks other than the JOVI, suggesting that impaired performance on these tasks could not be explained solely by common variance shared with the other cognitive battery. Taking this one step further, we removed the variance shared by all other cognitive tasks, leaving only unique task variance for each of the CNTRACS and MATRICSsub paradigms. We continued to find a significant deficit for the entire RISE measure and the HVLT–R. It is interesting to note that the tasks continuing to show group differences in unique variance are measures of episodic memory, a domain previously hypothesized to be one of the core cognitive deficits in schizophrenia (Hill, Beers, Kmiec, Keshavan, & Sweeney, 2004; Saykin et al., 1991). It should also be noted, however, that scores on measures of visual integration, including the JOVI (Silverstein & Keane, 2009; Uhlhaas, Phillips, & Silverstein, 2005), as well as those on some other perceptual tasks (Keane, Silverstein, Wang, & Papathomas, 2013; Silverstein et al., 2013), are highly sensitive to clinical state, with normalization of performance occurring as patients move from the acute to stabilization to stable phases of illness. Since our sample consisted of clinically stable patients who were all past the acute phase, we may not have captured the potential for impaired visual integration as well as we did performance impairment on other tasks that measure impairments that are less state related.

These data again support the hypotheses that both common and specific deficits are present in schizophrenia (Fornito et al., 2011). Furthermore, these findings suggest that the CNTRACS and MATRICSsub batteries measure both unique and common cognitive deficits, with the percentage of shared variance (47%) similar to that seen in previous studies of cognition in schizophrenia (Dickinson et al., 2004; Keefe et al., 2006b). Although the generalized or common deficit is sometimes discussed as it if it were a “nuisance” variable, in the present study it is picking up on important variance related to function in schizophrenia, since the common variance was strongly correlated with multiple measures of function. It is interesting to speculate on the process represented by this common deficit. It has been suggested that this common factor might reflect “processing speed,” which could be contributing to performance in many domains (Schatz, 1998). The fact that the BACSsc loaded highly on the principal component is consistent with this hypothesis. However, all four of the tasks measuring “higher cognitive” functions loaded highly on this factor, raising the possibility that part of this common deficit reflects impairments in cognitive control that may cut across a number of different domains. The latter account might bring new psychological and neurobiological specificity to the concept of a general cognitive deficit in schizophrenia, since cognitive control is made up of a well-characterized set of cognitive processes linked to the function of a common frontal-parietal neural network that is recruited when task demands increase across a wide range of cognitive domains.

Associations with function

All of the cognitive tasks were moderately correlated with the UPSA–B. The BACSsc, DPX, AX-CPT, and RISE IRAE showed the strongest relationship with the UPSA–B. In addition, we found that, after regressing out the MATRICSsub tasks, the DPX residual continued to significantly correlate with the UPSA–B, and the HVLT–R residual correlated with the MSIF and SLOF Patient report after regressing out the CNTRACS tasks. These findings suggest that there is unique variance associated with each task battery that predicts function, although possibly different aspects of functioning for the CNTRACS and the MATRICSsub batteries. The CNTRACS task residuals correlated with the UPSA–B, which is a performance-based measure of functioning, as well as showing trends to correlate with observer rating measures of function (i.e., the DPX at a nominally significant p-value). Thus, these data demonstrate that paradigms translated from basic cognitive neuroscience meaningfully predict variance in functional capacity. The MATRICSsub task residuals correlated with the MSIF and SLOF patient, which are measurements based on self-report. However, comparing the correlation coefficients of the MATRICSsub tasks and functional outcome measures that we obtained with those reported in other studies, we notice some variations. For example, Nuechterlein et al. (2008) found a weaker relationship between the HVLT–R and self-reported functioning (rs ranged from −0.06 for social functioning to 0.20 for independent living, with r = 0.14 for the composite), while Keefe et al. (2006a) and Burton et al. (2013) reported slightly higher correlation coefficients between the HVLT–R, BACSsc, and UPSA–B (rs > 0.30). It is difficult to determine why these effect sizes vary across studies, particularly given that most studies using the MCCB report relationships only between functional outcome and the composite score, not individual tasks, providing few comparisons. Perhaps importantly, within our sample, scores on the MSIF and SLOF patient report were not significantly correlated with scores on the UPSA–B, suggesting that these performance-based and report-based tools measure slightly different aspects of functioning, which is, in turn, reflected in their differential association with cognitive tasks. Although this does not address why UPSA–B and self-report measures have different relationships with tasks across studies, it does help explain why studies have found distinct relationships between cognitive tasks, performance-based functioning, and self-reported functioning.

Limitations

We had hoped that the SLOF Informant report would provide an externally assessed picture of patient functioning. However, we were unable to obtain the report for every subject. Additionally, the informants who provided data were primarily family members, case workers, and therapists. There is evidence that ratings from this range of informants are less associated with patient behavior than are reports from only high-contact clinicians, which were not available for all subjects in our study (Sabbag et al., 2011). This may have limited our ability to detect a relationship between functioning and cognitive performance, using the SLOF. Additionally, our patient group was relatively high functioning, which limits the generalizability of our results and also could have limited our ability to detect relationships with functional outcome and clinical symptom measures. Finally, as in most studies of schizophrenia patients, our patient sample was medicated. The impact of medication on cognitive performance is still largely unclear, but it is important to note that this could have impacted our data.

Conclusions

We observed a pattern of significant intercorrelations between cognitive tasks from both the CNTRACS and MATRICS initiatives, in both schizophrenia and healthy control subjects. These intercorrelations were strongest among tasks that measured higher cognitive functions, such as memory, processing speed, and goal maintenance, providing evidence for a shared cognitive factor contributing to performance in both groups and also providing evidence that a measure of early visual perception is subserved by different processes. Ongoing tasks using fMRI and the CNTRACS tasks in schizophrenia patients and controls will shed light on the nature and neural underpinnings of these common factors and allow us to test the hypothesis that it may be related to neural circuitry supporting cognitive control. Further analysis of a common deficit revealed unique task variance that remained sensitive to group differences across all higher-order cognitive tasks, although a more conservative analysis showed that unique variance was most robustly related to group differences for tasks measuring episodic memory. In line with the goal of both the CNTRACS and MATRICS initiatives, we found significant relationships between all cognitive tasks and measures of functional outcome. Notably, CNTRACS tasks were more related to a performance-based and observer-rated measure of function, while MATRICSsub tasks were more related to self-reported functioning, suggesting that tasks from these batteries are picking up on slightly different aspects of the measurement of functional outcome. Most important, our data indicate that measures derived from the cognitive neuroscience literature can show meaningful relationships to functional capacity and status and can identify deficits among individuals with schizophrenia even when variance associated with a common deficit is removed.