Meta-analysis of cognitive functioning in patients with psychotic disorders and obsessive–compulsive symptoms

The aims of the current study are to [1] examine the association of comorbid OCS in psychotic disorders and performance on different cognitive domains and to [2] examine which patient or study characteristics might explain heterogeneity of result between studies on cognitive function in patients with comorbid OCS. Our hypothesis is that patients with comorbid OCS show heightened cognitive impairment compared to those without comorbid OCS, specifically in the executive domain. Increased knowledge on the type or extent of cognitive dysfunction in patients with comorbid OCS, might have implications for classification and treatment of this patient group.

The search was performed in cooperation with a clinical librarian (JD). We searched EMBASE, MEDLINE, Web of Science and PsychInfo on 27-5-2019. A set of reference articles and conference abstracts was used to refine the search strategy (supplemental material Sect. 1). In addition, we hand-searched the reference lists of all included articles. The search roughly had three components, namely ‘psychosis’, ‘OCS/OCD’, and ‘cognition’. The full search terms can be found in the supplemental material (Sect. 1).

References were screened by two researchers (LD + FS) and added to the initial selection of articles if the title or abstract (1) mentioned obsessive–compulsive symptoms or disorder in a population with psychotic illness, (2) mentioned a cognitive function or cognitive test, (3) mentioned that the study was not a case report, expert opinion, editorial or review, and (4) if the full text was in English, French, Dutch, Spanish, or German. Any discrepancies in selected articles were solved by consensus in a meeting between both researchers. We used the Rayyan app to screen articles and facilitate the comparison of articles between researchers [28]. Studies that were included in the meta-analysis after full-text review had to meet the following criteria: (1) the study evaluated patients with a psychotic illness and determined presence of OCS or OCD, (2) cognitive domains were assessed with neuropsychological tests, and (3) means and standard deviations for the neuropsychological test outcomes were reported in the paper or were made available upon request. Articles were also eligible for inclusion if a sample size and correlations between obsessive–compulsive symptoms and neuropsychological test outcomes were available.

Data were extracted by one investigator (LD) using a standardized data extraction form, that was developed and tested by two researchers (LD + FS). We emailed authors in case data on neurocognitive test outcomes or confounders were missing from their publication, with at least 3 attempts per publication. The full data extraction form can be accessed in the supplemental material (Sect. 2). In cases, where multiple studies were published by the same author or research group, studies were checked for potential duplicate data. In cases of duplicate data, the largest sample was used. As both cross-sectional and prospective studies were included, only baseline data were extracted from prospective studies.

First, the tests used in each study were classified into a cognitive (sub)domain according to the Strauss and the Lezak compendium of neuropsychological assessments [29, 30]. In addition, we used the classification of the MATRICS Consensus Cognitive Battery initiative, as this battery was specifically designed for people with schizophrenia [31]. Details on how tests were aggregated can be found in the supplemental material (Sect. 3). We subdivided immediate verbal memory into two outcome measures: score of the first trial in a word learning task (‘trial 1 verbal memory’) and sum of immediate verbal memory trials in a word learning task (‘sum of trials verbal memory’), as the second outcome measure contains a learning element and thus measures something distinctly different from the first. In general, meta-analyses were performed in case a minimum of 4 studies were available per domain.

In cases, where authors provided us with full data sets of correlational or unpublished data, we used a YBOCS score of 8 as the cut-off to define the comorbid (OCS +) group, since this was a commonly used cut-off value in the references we included [32‐36]. Analyses were restricted to the per study level,for instance, reported subgroups based on disease duration were combined into one OCS + and one OCS− group [37]. In case multiple subgroups based on YBOCS severity were compared, these were combined into one OCS + and one OCS− group. For example, in the study by Ongur et al. [38] we categorized the YBOCS scores up to 11 as the OCS- group and calculated weighted means and SD for this aggregated control group. For the study of Michalopoulou et al. [39] we left out the Stroop task in the cognitive inhibition domain, as it was not clear to us what the values reported in the original study meant.

We used Cohen’s d (standardized mean difference) as the primary measure to evaluate and compare effect sizes. To account for the expected heterogeneity between studies, a random-effects model was used for meta-analyses. The presence of heterogeneity was further evaluated by calculating the I² metric.

We examined whether the applied cognitive test within a domain had an effect on the outcome in cases were multiple tests were compiled into one domain, by applying post-hoc sensitivity analysis for each outcome measure that appeared at least 6 times in a domain.

We aimed to perform meta-regression analyses for mean age, gender, mean PANSS positive score and clozapine use in the whole sample. In addition, meta-regression was done for mean YBOCS score of the comorbid group and the categorial variable full diagnoses of OCD vs. presence of OCS as the criterion for the comorbid group. According to methodological guidelines a minimum of ten studies per covariate was assumed appropriate for meta-regression analyses [40].

Comprehensive Meta-analysis software (CMA) version 3 was used for all analyses [41].

Included studies were assessed for quality using an adapted version of the National Institute of Health (NIH) Quality Assessment Tool for Observational Cohort and Cross-sectional studies by two researchers (LD and FS) and discrepancies were discussed and resolved.

Publication bias was assessed using Rosenthal’s fail safe N and a funnel plot. The above mentioned methods are generally more reliable when applied in a larger number of studies [42, 43]. We did not use statistical tests for funnel plot asymmetry, as these are known to be insensitive in cases, where there are less than ten studies in a meta-analysis [44].

The search yielded a total of 2365 references from MEDLINE, EMBASE, PsychInfo, and Web of Science after removing duplicates using Endnote and Rayyan. In addition, we found three publications on the website ‘freefullpdf.com’ that were not indexed in the abovementioned databases [45‐47]. Three studies were excluded, because the cognitive test used could not be grouped into a domain and one was excluded for poor quality reporting [47‐50]. After full text review, aggregation of cognitive tests, and efforts to obtain data from authors, 32 records were included in the synthesis. Four publications presented data of two overlapping samples, but reporting different outcome measures, so these were combined. The Schulte et al. [51] conference abstract was combined with the corresponding full-text article in the synthesis referred to as ‘Veerman 2016’ [52]. The study of Schirmbeck et al. [33] and Mier et al. [36] were combined and now refer to the ‘Mannheim study’. Finally, access to baseline data of a larger GROUP sample, allowed us to recalculated earlier reported outcomes based on this larger sample [34, 53]. This resulted in a total of 30 studies represented in the meta-analysis. A flow-chart of the search and screening process can be found in Fig. 1.

Most studies had a cross-sectional design and included schizophrenia patients in an outpatient setting. Four studies used consecutive sampling and eight used matched sampling, most studies did not report recruitment or sampling strategies. OCS definition was heterogeneous, with a variety of YBOCS cut-off scores or a full DSM diagnosis of OCD. Overall, a total of 2738 patients were included in the presented meta-analyses. Table 1 lists the included studies and their relevant characteristics. Study quality was generally fair and full ratings can be found in the supplemental material (Sect. 4).

Meta-analysis was possible for 17 individual domains. As shown in Table 2, meta-analyses were done for attention (with subdomains processing speed and sustained attention), memory (with subdomains working memory, immediate and delayed visual memory, and immediate and delayed verbal memory), executive function (fluency, cognitive inhibition, cognitive flexibility, set-shifting, abstract thinking, planning, and reasoning), facial affect recognition, and visual spatial ability. None of the meta-analyses showed significant results. However, when examining the forest plots, the studies showed a wide spread in effect sizes and even effect directions. Figure 2 shows the condensed forest plot for one of the largest meta-analysis, working memory, as an example of the heterogeneity in effect sizes and directions. Most studies also had a medium to high I² statistic [54]. Table 2 shows a summary of the results of the meta-analyses and the corresponding forest plots can be found in the supplemental material (Sect. 5).

Post-hoc sensitivity analyses were conducted on more homogenous outcome measures within specific domains (see supplemental material Sect. 5a). Sensitivity analyses on processing speed only including not purely reaction time based outcome measures resulted in significantly worse performance of the OCS + group (SMD = − 0.190, p = 0.029) (Fig. 3). No other post-hoc analyses were significant.

Due to a limited number of studies and missing information on confounders in the individual studies, we were only able to conduct meta-regression analyses with the following covariates: YBOCS score for the comorbid group, PANSS positive score for the whole sample, whether a full OCD diagnoses or OCS symptoms were used as the criterion to define the OCS + group, and the mean age for the whole sample. We did not have sufficient studies to examine more than one confounder in a single meta-regression. This resulted in a total of 19 meta-regressions. Meta-regression revealed advanced age was significantly correlated at the α = 0.05 level with worse performance of the comorbid group for processing speed (R² = 0.7, p = 0.029), working memory (R² = 0.11, p = 0.031), cognitive inhibition (R² = 0.59, p = 0.024), and cognitive flexibility (R² = 0.34, p = 0.015). We found that an OCD diagnosis (n = 7) instead of OCS (n = 3) was associated with better cognitive inhibition of the comorbid group (R² = 0.63, p = 0.017). Neither PANSS positive score nor severity of OCS (reflected in the mean YBOCS scores) were associated with performance in the cognitive domains in which meta-regression was possible. The results of all meta-regressions can be found in the supplemental material (Sect. 8).

Publication bias upon visual inspection of the funnel plots was generally low. Rosenthal’s N showed varying degrees of publication bias, but this method is considered to be unreliable for small meta-analyses. Section 7 in the supplemental material shows the publication bias statistics. All included studies were rated for quality and all except one [45] received a rating of ‘fair’ quality. Section 4 in the supplemental material shows the quality assessments for each study.

To the best of our knowledge this is the first meta-analysis investigating the association between comorbid OCS and functioning in multiple cognitive domains and the first on this subject to do subsequent meta-regression analyses. Furthermore, detailed search strategies enabled additional inclusions of publications and extensive attempts to obtain unpublished data made it possible to include data that had not been presented in the literature on cognition and OCS in psychotic disorders before.

This meta-analysis has several limitations. Firstly, we combined the results of a variety of neuropsychological tests within domains to increase the power of the meta-analyses, by which we consequently could have introduced heterogeneity within these domains. Over 60 different tests were used in the included studies and in addition, studies sometimes varied in applied scoring systems of these tests. This severely complicates attempts to compare and replicate results. However, where possible we performed subsequent sensitivity analyses with more homogeneous outcome measures (e.g., only using TMTB for cognitive flexibility) that showed comparable results. A second limitation was that some of the meta-analyses only included a small number of studies, which warrants cautious interpretation of the results as meta-analyses with small numbers of studies are less reliable than those with larger numbers of studies. In addition, analyses of multiple outcome measures, performing multiple sensitivity analyses and meta-regressions on the same data might impact the validity of meta-analytical results. As there is an ongoing debate on how to correct for multiplicity, while at the same time being cautious not to decrease power, we did not correct for alpha inflation [86]. However, we acknowledge that statistical significance testing needs cautious interpretation and clinical relevance of the results should rather be interpreted based on the average effects-sizes and confidence intervals. Thirdly, the included studies often only reported a select number of confounders and factors such as depressive symptoms, ethnicity, medication, and education status were often not reported. As mentioned above, some possibly important moderators or confounders which might explain the observed large heterogeneity between studies received very little attention in primary studies. In addition, many included studies used substance (ab)use as an exclusion criterion for selection of participants. This is, therefore, not a likely explanation for the observed heterogeneity in this meta-analysis, but as substance use is highly prevalent in people with psychotic illness [87, 88], current findings are probably not representative of the actual population. Apart from the limitations of the included studies, subgroup analyses and meta-regression can only be applied on the per study level, and factors that might vary between the OCS + and OCS− group or on the per person level, such as for example disease severity or ethnicity, cannot be investigated [40]. Finally, the combination of a small number of studies and missing data on confounders meant that the results of the meta-regression should also be interpreted cautiously. We could only enter one covariate at once due to the small number of studies and we could, therefore, not assess the interaction between covariates such as between age and symptom severity.

In light of our findings, it seems that the way forward for research on cognitive function in people with a psychotic disorder and comorbid OCS is to focus on unveiling the cause for the large heterogeneity in results.

Adopting a dimensional approach could be more suitable to examine the association between cognitive performance and co-occurring OCS. As a variety of factors, among which the severity of OCS, might be at play, creating a dichotomy in people with and without OCS hampers the investigation of OCS severity and other moderating factors. Although some of the studies evaluated the association between dimensional measures [24, 50, 61, 62, 66], most used a categorical approach. Future studies should aim at using methods that allows to capture the complexity of this issue, for which a dimensional approach appears to be the most suitable option available. In this context the YBOCS should be used as the standard instrument as it has been validated in patients with psychotic disorders [89] and would ensure better comparability between individual studies. In addition, more prospective studies should be considered, as they could help shed light not only on the association between comorbid OCS and cognition, but also on the course and nature of the association. As most studies to date have been cross-sectional, causal conclusions cannot be drawn.

Finally, more uniformity should be sought in the type of cognitive tests used in this patient group. A possible solution could be the use of the MATRICS Consensus Cognitive Battery, which was especially developed for people with schizophrenia [31, 90, 91]. This battery has the additional advantage of being relatively fast to administer (65 min), which might make it more suitable for people with more severe psychotic symptoms and which could allow for using it on larger samples. This battery could be supplemented by tasks that test domains that are impaired in primary OCD, such as the Wisconsin Card Sorting Test and the Trail Making Test part B for cognitive inflexibility [20].