Post-Stroke Working Memory Dysfunction: A Meta-Analysis and Systematic Review

Electronic database Pubmed was searched for relevant studies; last search was performed on 10–03-2019. The following search terms were used: “stroke”, “post-stroke”, “cva”, “cerebrovascular accident”, “cerebral vascular accident”, “brain infarct*”, “cerebral infarct*”, “brain lesion”, “ischemic lesion”, “cerebral ischemia”, “tia”, “transient ischemic attack” AND “memory disorders”, “memory”, “cognitive domain*”. Reference lists of selected articles were searched for potential missed articles.

A two-step approach was used to select articles. Firstly, titles and abstracts of all search results were screened for the following characteristics by one reviewer (S.L.): (1) original article published in English, (2) participants are adults (> 18 years of age), (3) study concerning stroke or transient ischemic attack (TIA) patients, (4) sample size of at least 10 patients, as single-case studies or case series often concern rare cases whose behaviour might not be representative for the larger stroke patient population (5) outcome measures or descriptives include at least one working memory task; in case the abstract only mentioned memory function in general, the article was selected for full-text evaluation, (6) studies that only included patients with (subjective) memory complaints were excluded.

Secondly, 543 full-text articles were obtained from the selected studies and were reviewed on the following inclusion criteria: (7) a stroke-free control group was included as comparison, (8) clinical stroke patients in the sample (9) working memory function measured with a formal test or clearly described experimental paradigm, (10) treatment studies are included when baseline measures are reported. When two or more publications referring to the same sample were available, we extracted data only from the publication presenting the most accurate estimate, either because of sample size or outcome assessment. Two authors (S.L. and N.A.L.) independently performed the second step of the selection process. A meeting was held in case of disagreement which in all cases led to consensus.

Studies that included TIA patients were included in the meta-analysis as post-TIA cognitive impairment is often reported. A systematic review by Van Rooij, Kessels, Richard, De Leeuw, & van Dijk (2016) including 13 studies with data from 1,318 TIA patients, concluded that mild cognitive impairment is present in over a third of the TIA patients. When a mixed etiology lesion population was tested, the study was only included if separate results for the stroke patients could be retrieved.

Performance on working memory tests as compared to a healthy control group were extracted, as were participant characteristics, specific in- and exclusion criteria, and timing of assessment. First, overall performance on working memory tests was compared between stroke patients and healthy controls. Second, performances on low-load and high-load tasks were compared with a distinction between verbal and non-verbal tasks. Tests were considered low-load if they rely on remembering a limited amount of information over a short time, such as forward digit or spatial span tasks. In the working memory model of Baddeley and Hitch (1974) this is based on the phonological loop for verbal information and the visuo-spatial sketchpad for non-verbal information. Tasks that were considered high-load required some form of manipulation or updating, such as backward span or sequencing tasks, based on the central executive of the model. Tasks in which the stimuli (either visually or auditory) were digits, words, sentences or stories were categorized as verbal in nature. Tests were nonverbal in nature if stimuli were pictures of objects, scenes, line drawings or abstract figures. Third, a sub-analysis was conducted to examine the effect of timing of assessment (i.e. duration post stroke). Assessment within the first three months after stroke was considered sub-acute. Assessment after three months was considered as chronic. Qualitative synthesis of study results was performed with attention to lesion location. Working memory performance was compared between studies with specific inclusion criteria based on lesion location or imaging analyses relating lesion location to working memory performance. Working memory performance was compared within studies that selected groups based on lesion location.

Risk of bias assessment was performed with the Research Triangle Institute (RTI) item bank, a tool to evaluate the bias and quality of observational studies (Viswanathan & Berkman, 2012; Viswanathan, Berkman, Dryden, & Hartling, 2013). As suggested by the RTI developers, slight adjustments were made to match the designs of the included studies (Appendix 2). Four items were dropped and items were reformulated to fit a case–control design. Ten items that assessed the selection bias, detection bias, attrition bias, selective outcome reporting and confounding were selected.

For the meta-analysis, sample sizes, means and standard deviations from the working memory tests were extracted from the studies. If necessary, corresponding authors were contacted to obtain these statistics based on the raw data. Summary statistics (Hedges’ g) were calculated based on sample sizes (N), means of patients and controls (M) and standard deviations (SD) with the following formula: g = StdDiff × J. StdDiff was calculated using the following formula: (M1—M2) / SD_pooled, with SD_pooled = √(((N1-1) × SD1^2 + (N2 -1) × SD2^2) / (N1 + N2 -2)). The correction factor for different sample sizes is J, that was calculated as: 1—(3 / (4 × df—1)), where df = N1 + N2—2. Pooled variance was calculated by: √(1 / N1 + 1 / N2) × SD_pooled. In case more than one outcome measure was reported, the average ES was calculated for the overall analysis. By using the mean of different outcome measures to calculate the effect size of the study we assume a correlation of 1 between different measures. This is a conservative approach as the actual correlation is probably less than 1 and the variance is lower than what we assume. The alternative is treating every outcome measure as fully independent, assuming a correlation of 0, which results in under-estimation of variance of the summary effect size. The actual overall effect size might therefore be slightly higher than our estimate. For the mixed-effect analyses of the effect of load and modality it was necessary to assume independence of outcome measures. Effect sizes were interpreted according to Cohen’s (1992) convention of small (0.20), medium (0.50), and large (0.80) effects for positive and negative values. Performance of stroke patients is lower compared to controls if the effect size (ES) is negative. Bias due to small sample sizes was corrected for by including sample size as a weighting factor (Hedges & Olkin, 1985). Random-effects models were used because of heterogeneity in populations studied and in outcome measures. Additionally, the goal is to generalize the results beyond the observed studies (Borenstein, Hedges, Higgins, & Rothstein, 2011). Heterogeneity was checked for by the use of the chi-square homogeneity test (Q). The fail-safe N was calculated for each study and a funnel plot was made (Rosenthal, 1991). The fail-safe N must to be larger than (5 × k) + 10, where k refers to the number of studies included in the meta-analysis (Clark-Carter, 2010). This measure gives an indication of how many studies with null-results should be unpublished due to publication bias to nullify the effect. All analyses were performed using Comprehensive Meta-Analysis version 2.0 (Engelwood, NJ, USA, 2005).

The literature search resulted in 4,318 articles after removal of duplicates. Seven additional studies where identified by manually checking of reference lists of selected papers. In total, 553 were selected for full text screening. Seventy-five articles published between 1992 and early 2019 were eligible for inclusion. The eventual meta-analysis includes data of 3,084 stroke patients from 50 studies. An additional 25 studies of which the necessary statistics could not be retrieved or studies with overlapping samples but with relevant subgroup analyses were included in the systematic review. Figure 1 shows the flowchart of the literature search. Table S1 (online supplemental materials) shows the details of the studies included, these are: sample size and specific inclusion criteria, stroke type, age, interval between stroke and assessment, inclusion of prior stroke patients, inclusion of patients with pre-existing dementia, task load, task, Hedges g, and variance, and number of effect sizes for each primary study.

Ischemic stroke was the inclusion criterion for 30.6% (k = 23) of the studies. A quarter of the studies (24.0%, k = 18) included both patients with haemorrhagic and ischemic stroke. Concerning studies that included TIA patients and minor stroke, 5.3% (k = 4) included both stroke and TIA patients, 2.7% (k = 2) reported on patients with minor stroke,¹ and 2.7% (k = 2) included only patients with transient ischemic attack (TIA). One study (1.3%, k = 1) included only patients with haemorrhagic stroke. A third (33.3%, k = 25) did not specify stroke type. A majority of the studies (61.3%, k = 46) did not select patients based on stroke location. Some studies (14.7%, k = 11) included only patients with subcortical lesions. Only one community-based study was fulfilled the inclusion criteria, all other studies were hospital- or rehabilitation center-based. Twenty percent (k = 10) studies included in the meta-analysis reported only one working memory measure. The mean number of outcome measures per study was 3.56. The maximum F number of outcome measures per study was 14. Comparisons between left and right hemisphere stroke were made in 16% of the studies (k = 12). Few studies included patients with stroke in one specific hemisphere (left: 6.7%, k = 5, right: 9.3%, k = 7). A total of 50 authors were approached to obtain additional information and necessary statistics from tasks separately (42.0%, k = 21), from stroke patients separately (12.0%, k = 6), means and standard deviations that were not reported (32%, k = 16), clarification on tasks or population (14%, k = 7). This resulted in 16 studies that could be included in the meta-analysis. Of the 34 articles of which the authors did not respond, 22 were included in the qualitative analyses.

Risk of bias assessment of individual studies based on the RTI items (see Appendix 2 and online supplemental materials Table S2) showed an unclear or high risk of confounding, as in more than two-third (72.0%, k = 54) of the studies details on possible prior strokes or pre-existing dementia were not specified, and in almost half (56.7%, k = 35) of the studies, in- and exclusion criteria for healthy controls were unclear. Although not invalidating the results of individual studies, it increased between-study heterogeneity. A majority of the studies (72.0%, k = 54) did include an age and education matched healthy control group. A second source of heterogeneity is the large variability in the intervals between stroke and assessment, both between and within studies. Almost all studies bear the risk of confirmation bias; only 5% of the studies (k = 4) reported that assessors were blinded to the status of the participant (patient or control). As deficits are often prominent, it is unsurprising that other studies did not report assessors to be naive to the participants group. The funnel plot shows the relation between the sample size and the effect size (Fig. 2). The plot shows some asymmetry, which may partly be due to heterogeneity in outcome measures and partly be due to publication bias. Especially the lower right corner is empty, which indicates that there are no small studies with small effect sizes. The summary statistics of the meta-analysis as shown in Table 1 includes the Fail-safe N for each analysis as the number of studies with an effect size of zero that should be added to lose the significant result. All analyses have a Fail-safe N larger than the pre-set criterion, indicating that it is unlikely that the effect is only significant because of publication bias.

Analysis of data from 50 studies including 3,084 patients and 2,898 healthy controls on working memory averaged across tasks resulted in an overall moderate ES of -0.65 ([− 0.80 to − 0.51], p < 0.001) for lower working memory performance in stroke patients. Thirty-nine studies (78%) included a low-load working memory task. The analysis showed a moderate ES of -0.58 ([-0.77 to -0.40], p < 0.001). Analysis of forty-one studies (82%) with a high-load working memory task resulted in a comparable ES of -0.59 ([-0.73 to -0.45]), p < 0.001). Figures 3 and 4 show the effect sizes per study grouped by modality under high and low-load conditions separately. Analyses show medium effect sizes (< -0.50) in the verbal domain for high-load (-0.63) and low-load (-0.53) tasks and for low-load (-0.62) tasks in the non-verbal domain. For high-load non-verbal tasks the effect size was slightly lower (-0.43). In order to compare the effect of low- and high-load we needed to assume independence of outcome measures. A mixed-effects meta-analysis shows a Q statistic for the difference between the effect-sizes of 0.009 (p = 0.922), indicating that there were neither differences in effect sizes between low- and high-load tasks, nor differences in effect sizes for non-verbal and verbal tasks (high-load Q = 2.16, p = 0.142; low-load Q = 0.24, p = 0.626). To examine the effect of interval between stroke onset and assessment, a secondary analysis was performed. For this analysis we excluded studies that analysed patients in the sub-acute and chronic stage as one group. Fourteen studies (28%) included patients in the sub-acute stage. The analysis showed a moderate ES of -0.43 ([-0.68 to -0.19], p < 0.001). For patients in the chronic stage (k = 22, 44%) the ES was large, -0.90 ([-1.15 to -0.65], p < 0.001, Fig. 5). A mixed-effects analysis showed that this difference in effect-sizes is statistically significant, Q = 6.88, p = 0.009. Patients in the chronic stage show more decrement in working memory performance compared to healthy controls than patients in the sub-acute stage. The heterogeneity indices (Q) were all statistically significant (p < 0.05) for all analyses, indicating variation in study outcomes. The between study variance (τ ²) of all studies is estimated as 0.21. The I² of 82.98 indicates that most of the observed variance reflects differences in study effect rather than sampling error. Results of the overall analysis and sub-analyses are presented in Table 1. Exclusion of studies that included patients with TIA did not lead to different results (Table S3a online supplemental materials). To minimize heterogeneity due to task variation, we reran the analysis including only studies with Digit Span or Spatial Span as working memory measure. Effect sizes were highly similar to those of the analysis including all studies for the overall, and low- and high-load analyses. The sub-analysis with patients in the sub-acute stage included only eight studies and yielded a lower effect size (Table S3b online supplemental materials).

This qualitative assessment includes 75 studies, 50 from the meta-analysis and 25 additional studies that were not taken into account in the meta-analysis due to missing statistics or overlapping samples. Sixteen percent of the studies (k = 12) compared working memory performance of patients with a left hemisphere stroke to patients with a right hemisphere stroke. Two-third of them (66.7%, k = 8) did not report a statistically significant difference in performance between left and right hemisphere stroke patients. Twenty-five percent (k = 3) reported a worse performance in left hemisphere stroke patients compared to right hemisphere stroke patients and controls on immediate serial recall tasks (Ho, Kong, & Koon, 2018), on letter-number sequencing (Andrews, Halford, Shum, Maujean, Chappell, & Birney, 2014), and on a visually presented digit span forward and backward task (Low, Crewther, Perre, Ong, Laycock, & Wijeratne, 2016). One study (8.3%) reported no difference in performance of left hemisphere stroke patients and controls but impaired performance in right hemisphere stroke patients on a backward spatial span task (Van der Ham, van Wezel, Oleksiak, van Zandvoort, Frijns, Kappelle, & Postma, 2012).

A second comparison made in studies is between patients with an anterior and posterior lesion. Studies showed performance more strongly affected in patients with frontal lesions compared to posterior lesions on different forward span tasks (Roussel, Dujardin, Hénon, & Godefroy, 2012), on digit span backward (but not forward, Leskelä et al., 1999), and on high-load n-back tasks (Andrews, Halford, Shum, Maujean, Chappell, & Birney, 2013). In contrast, two studies reported the opposite; one reported performance in the posterior group to be inferior to patients with anterior lesions on digit span forward, with no differences on spatial span (Beeson, Bayles, Rubens, & Kaszniak, 1993). The other study reported lower performance in patients with inferior parietal lesions compared to inferior frontal lesions on several forward span tasks (Baldo & Dronkers, 2006).

Whereas 59 studies used neuroimaging to confirm stroke, to check for exclusion criteria and to describe the sample or to create subgroups, only seven studies related specific lesion locations to working memory performance. Spatial working memory performance was associated with lesions in the right posterior parietal and right dorsolateral prefrontal cortex and bilaterally in the hippocampal formation (Van Asselen, Kessels, Neggers, Kappelle, Frijns, & Postma, 2006). Both parietal white matter and insula lesions were associated with spatial working memory deficits in neglect patients (Malhotra, Jäger, Parton, Greenwood, Playford, Brown, & Husain, 2005). Not only spatial, but also verbal short-term memory was associated with parietal lesions. Lesions in the insula were in the same study associated with a lower performance in a musical working memory task (Hirel, Nighoghossian, Lévêque, Hannoun, Fornoni, Daligault, & Caclin, 2017). Another study demonstrated that high-load working memory tasks were associated with lesions in both the frontobasal and posterior centrum semi-ovale regions (Roussel et al., 2012). A study that only included patients with frontal lesions reported that the posterior part of the left middle frontal gyrus is significant for high-load but not for low-load working memory tasks (Volle, Kinkingnéhun, Pochon, Mondon, Thiebaut de Schotten, Seassau, & Levy, 2008). A study with patients with cerebellar lesions attributed filtering of information in working memory tasks, but not working memory capacity, to specific areas of the cerebellum, such as the tonsil, the inferior semilunar lobule, and parts of the vermal pyramid (Baier, Müller, & Dieterich, 2014). Finally, one study reported no predictive effect of lesion topography on memory. However, this study used a combined measure of spatial and verbal recall, recognition and working memory (Ramsey, Siegel, Lang, Strube, Shulman, & Corbetta, 2017). Figure 6 shows how the results of these neuroimaging studies relate to each other.

Concerning timing of post-stroke working memory assessment, only four studies employed a longitudinal design. Three of these did not find any difference in performance of patients between the different time points. Two of these studies assessed the patients in the first week after stroke, with a follow up at three and six months respectively (Su, Guo, Zhang, Zhou, Chen, Zhou, & He, 2018; Van Zandvoort, De Haan, & Kappelle, 2001a, b). The third study performed the first assessment between three and six months, with a three-year follow-up (Sachdev, Chen, Brodaty, Thompson, Altendorf, & Wen, 2009). The fourth study reported improved performance measured over three intervals; at two weeks, three months, and 12 months (Ramsey et al., 2017).