Retrieval practice improves memory in patients with schizophrenia: new perspectives for cognitive remediation

Schizophrenia is associated with severe cognitive deficits, particularly episodic memory impairment. They interfere with patients’ socio-professional functioning [1, 2] and are correlated with occupational engagement, residential independence, and self-care [3]. Patients’ cognitive deficits can be improved by cognitive remediation therapies, being either restorative, compensatory or environmental, with immediate post-treatment effects on global cognition and durable effects on functioning [4, 5]. Studies have shown that cognitive remediation is more effective in association with a psychiatric rehabilitation program with a strategic approach [6]. Thus, understanding the kind of strategic approaches that are most effective in patients with schizophrenia is of major importance in building relevant or improving existing cognitive remediation programs. At present, most cognitive remediation programs in schizophrenia use either drill-and-practice or strategy-based techniques [5, 6]. While drill-and-practice entails repetitive practice of cognitive exercises that become gradually more difficult as performance improves, strategy-based techniques include a specific focus on strategies to be used. For instance, different strategies have been identified in the general population to improve memory through a semantic task—grouping items by category, visual repetition, counting words or letters, contextual encoding or sentence generation [7]. The use of these strategies varies with age [8‐10], education and cognitive impairments [11‐14]. In patients with schizophrenia, several studies have demonstrated a deficient use of encoding strategies that can be at least partially reversed when the efficient strategy used is explicitly provided to patients [15‐18].

Retrieval practice (RP) is a well-established memory strategy that produces robust long-term episodic memory improvements in both healthy [19] and clinical populations with memory deficits (see Table 1 for a review of the results observed in clinical populations) [20‐25]. It has been shown that taking a test on material has a greater positive effect on future retention than spending an equivalent amount of time restudying it: this effect is named the testing effect [26]. Roediger & Karpicke [26] described it in students by comparing the effect of repetitive testing (either taking a test in classrooms or self-testing by students) to repetitive study on the retention of various materials (e.g., word lists, prose passages) and with various modes of final recall (e.g., free recall, cued recall, multiple-choice). Their results convincingly demonstrated that testing is a relevant method to improve learning. Rowland et al. [19] later demonstrated that some parameters fostered testing effect, such as the material to be learnt (prose or paired associates), the kind of initial test (cued or free recall), the presence or absence of feedback during the initial test, and the retention interval (1 day or more). Studies on clinical populations with memory deficits were mostly conducted by Sumowski and collaborators [20‐25]. The tests comprised weakly associated word pairs as a stimulus, a cued-recall method at both the initial and final tests, feedback during the initial test, and both short- and long-retention intervals (from 30 min to 30 days). They confirmed that RP improved memory performance in final recall in patients with multiple sclerosis, traumatic brain injury or HIV. The authors highlighted that RP could be effective for future rehabilitation methods.

To our knowledge, no study has ever investigated testing effect in patients with schizophrenia to evaluate its relevance for cognitive remediation. However, previous studies in schizophrenia have explored retrieval practice using a different procedure called retrieval-induced forgetting (RIF). This paradigm refers to the fact that, under particular conditions, episodic memory recall can suppress the accessibility of semantically related information [27‐30]. Thereby, Nestor et al. showed that learning words through RP (RP+) induced the forgetting of other related words (i.e., belonging to the same category) for which RP was not practiced (RP-), in comparison to unpracticed words from another category (NRP) [28]. Their results confirmed that (1) RP improves recall at the final test in comparison to unpracticed material (RP- and NRP conditions), and that (2) the inhibitory mechanism leading to the loss of retrieval access to the unpracticed related items is preserved in patients with schizophrenia. However, the procedure used to assess RIF differs in several points from those used to assess testing effect: first, it does not compare RP to another learning condition, such as restudy; secondly, the retention interval is quite short (5–20 min), which may reduce the efficacy of RP on episodic memory (see Table 2).

In fact, studies on RP in healthy subjects produced larger testing effects when the retention interval was at least over 1 day [19]. Consequently, studies implementing optimal procedures to verify that RP yields noteworthy memory improvements in patients with schizophrenia are still lacking. Our study is the first to compare RP to restudy in schizophrenia. Because patients with schizophrenia show an impaired ability to spontaneously use effortful encoding strategies but have a tendency to use rote repetition to learn new information such as weakly associated word pairs [31], RP may help them to engage in an efficient encoding strategy and improve their episodic memory performances. In these circumstances, we hypothesise that RP will efficiently improve episodic memory performances in patients with schizophrenia and then represent a relevant tool for cognitive remediation.

Cognitive performances were significantly lower in patients than in controls in almost all neuropsychological tests (except for phonological fluency) (see Table 3).

Regarding the initial test, the number of pairs recalled directly after the learning phase (considered as baseline performance) were lower in patients than in controls (43.4 and 56.5% respectively, Pr(patients > controls) = 0.008). However, the comparison between baseline scores and scores at the final test indicated that the improvement of performance observed (improvement of 3.6 and 7.3% of word-pairs recalled at the final test respectively, OR = 1.076, CI95%:1.026 – 1.128; Pr(final > baseline) = 0.998)² was comparable in both groups (the interaction between the group and time (baseline test vs. final test) was not relevant (Pr = 0.179). Nevertheless, despite this improvement, patients did not reach the “baseline performance” level of controls during the final test.

Univariate analyses showed that word-pairs were better recalled through RP than through RS (56.1% vs 35.7%, respectively; OR = 2.29, CI95%:1.67 – 3.12, Pr(RP > RS) > 0.999) and when a weak semantic association was present (64.7% vs 27.1%, respectively; OR = 5.45, CI95%:4.12 – 7.15, Pr(weak > no) > 0.999). Patients’ performances were lower than controls’ performances (37.4% vs 53.1%, respectively; OR = 0.56, CI95%:0.33 – 0.93, Pr(patients>controls) = 0.014) both in the weak and in the no association conditions (see Fig. 2 or Additional file 1: Table S2).

Multivariate analyses confirmed the relevant effects of group, condition and association and showed that RP was significantly more effective for word-pairs with weak semantic association as reflected by a relevant interaction between condition and semantic association (Pr = 0.998). In contrast, neither the interactions between group and condition or semantic association nor the interaction between all predictors were relevant (all Pr between 0.2 and 0.9) (see Additional file 1: Table S3). Importantly, the visual inspection of individual data indicated that all patients but one presented with a testing effect. For readers who are more familiar with frequentist statistic, results and effect sizes are reported in Additional file 1: Table S4. They indicate that all main effects and interactions were also significant using classical frequentist statistical analyses.

Sensitivity analyses were performed firstly using non-informative priors and secondly using pessimistic priors (i.e. priors in the opposite direction of the expected effects). The estimated coefficients remained unchanged (see Additional file 1: Table S5), suggesting that the estimations of coefficients were mostly driven by the data we collected and not by the expected results.

In order to investigate whether baseline cognition or performance at the initial test (baseline performance) had an impact on RP, we performed complementary analyses in which each of the following cognitive measures (fNART, Mill-Hill, WMS III total recall, working memory indirect span, semantic fluency, Ruff speed and accuracy) was added separately and successively to the initial Beta regression analysis. In all cases, main effects of condition and semantic link as well as all interactions remained unchanged. However, although the initial effect of group (Pr(patients > controls) = 0.03) was not influenced by the number of years of schooling, Mill Hill score, Ruff accuracy, and semantic fluency, this effect was smaller when working memory and Ruff speed were taken into account (Pr(patients > controls)s = 0.07), and disappeared when fNART or baseline performance were taken into account (Pr(patients > controls) = 0.25 and Pr(patients > controls) = 0.28, respectively).

Our study showed that retrieval practice can better improve long-term episodic memory performance than restudy in patients with schizophrenia, and that this so-called testing effect is similar to that reported in control participants. To our knowledge, only six previous studies demonstrated that retrieval practice improves memorisation in clinical populations with cognitive difficulties (traumatic brain injury, multiple sclerosis and HIV), while no study has shown its superiority to restudy in patients with schizophrenia. Moreover, all but one patient benefitted from RP in our study, indicating that this method should be helpful in a large proportion of patients with schizophrenia.

Although our results are very encouraging, they raise several methodological issues. First, as procedural similarities exist between the initial and final testing, one may argue that improvement of cognitive performances with RP only rely on a practice effect, that is, the increase in test scores that occurs upon repeating neuropsychological testing. To highlight practice effect, Goldberg et al. (2007) compared the test scores at t0 and t1 in patients with first-episode schizophrenia receiving antipsychotic medication to those of healthy controls (HC) [48]. They concluded that cognitive enhancement between t0 and t1 resulted mostly from practice effect, as the effect sizes for performance improvements were generally equal in patients and HC. In parallel, the transfer appropriate processing theory of testing effect explicitly calls attention to the importance of the similarity between the initial and final testing conditions. This theory suggests that testing effect derives from the overlap in material processing that occurs during the initial and final testing [19]. However, some results clearly challenge this theory: Carpenter & DeLosh (2006) fully crossed both the initial and final test types (free recall, cued recall, and recognition) and found that regardless of the format of the final test, the free recall initial testing yielded the best performance [49]. Further studies with similar methodology in patients with schizophrenia would shed light on the impact of practice effect in RP. In addition, Goldberg et al. showed a gain of 0.36 in effect size (Cohen’s d) upon repeated neuropsychological testing in first-episode psychosis patients treated with second-generation antipsychotics, while in our sample of patients, RP caused a gain of 1.04 in effect size in comparison to restudy. Though our results do not apply to global cognition, and though the comparison of effect-sizes must lead to careful conclusions, they suggest that it is quite unlikely that the superiority of RP over restudy at a later cued recall only stems from a practice effect.

Our study shows that in the retrieval practice condition, the items with weak cue-to-target association were better recalled than the items with no semantic association (in both the patient and control groups). These results are consistent with previous findings in healthy populations and clinical populations with memory deficits, where RP was regularly shown as fostering better final recall for weakly associated semantic items compared to strongly or non-associated items [19, 22]. Contrary to our findings, when using RP in patients with schizophrenia, previous studies have shown that patients did not improve their performance at final recall when weak categories-exemplar pairs were used. For example, using a RIF paradigm, Allen et al. showed that patients recalled as many weak category-exemplar pairs in the RP+ conditions as in the NRP conditions (36.7% in both conditions), while the patients improved their performance with RP for strong categories-exemplar pairs (61.7% recalled in the RP+ condition vs. 45% in the NRP condition) [27]. These results may be explained by the significant differences between our procedures and those of Allen et al. First, feedback was not provided in the previous studies (that is, unsuccessfully retrieved items were not presented again during the initial test). However, the meta-analysis by Rowland et al. clearly showed that feedback has a substantial effect on the final memory test scores (Hedge’s g = 0.39 without feedback vs. 0.73 with feedback) [19]. Moreover, in studies without feedback, a reliable testing effect of medium effect size (Hedge’s g = 0.56) was found when the initial retrieval success was greater than 75%. In contrast, when the initial test performances were scored between 51 and 75%, a testing effect of small effect size was yielded (Hedge’s g = 0.29) [19]. Such differences in the initial cued recall were observed by AhnAllen et al.: scores at the initial cued recall for strong and weak associates were 92 and 82%, respectively, in controls, and 80 and 53%, respectively, in patients [29]. Actually, as scores at the initial cued recall for weak associates were low in the patients with schizophrenia, the absence of feedback may have prevented the favourable effect of RP in this condition. These results, compared to ours, confirmed that feedback improves cued recall in patients with schizophrenia as much as in the control participants. In addition, studies of healthy subjects receiving feedback with low initial test performances (≤ 50%) yielded larger effect sizes (g = 0.99) than those with moderate (51–75%; g = 0.68) and high (> 75%; g = 0.40) initial test performances [19]. In our study, as patients demonstrated lower initial performances than controls (43.4% vs. 56.5% of words recalled at the initial phase respectively), feedback may have fostered the occurrence of a testing effect in both our samples. Secondly, retention delays in studies based on RIF procedure were short (5–20 min) and may have prevented the occurrence of a testing effect [27‐30]. Instead, our protocol set a retention interval at 2 days, based on previous findings indicating that longer retention intervals (≥ 1 day) yielded larger testing effects. As testing-effect appears to grow with the duration of the retention interval and persists with very long retention intervals in healthy subjects (at least several months) [19, 26], demonstrating a similar pattern in patients with schizophrenia would confirm its interest for cognitive remediation.

Our results also show that patients’ scores at the final cued-recall test did not reach the baseline performance of the healthy controls. These findings might be explained by the theory of generalised deficit in schizophrenia [50], and they align with previous research investigating practice effect in these patients. For example, Goldberg and collaborators indicated that despite the improvement due to repeated testing, patients started and ended lower than the controls [51]. However, if we base our hypotheses on the generalised deficit theory, we would also expect a lower improvement using RP in the patients with schizophrenia in comparison to the controls. On the contrary, we found that the patients benefitted from RP as much as healthy controls (the group by condition interaction was not relevant), supporting the idea that the cognitive mechanisms underlying testing effect are preserved in patients with schizophrenia. This interpretation is further supported by the semantic association effect in our study. We know from the literature regarding semantic memory that the patients’ impairment is due to difficulties in self-initiating effective semantic encoding strategies, and that the patients’ performances can improve or even normalise when effective strategies are explicitly given to them [52‐54]. Yet, our study did not use explicit instructions, but may have rather stimulated a specific encoding process (RP) that implicitly produced semantic encoding strategies. This interpretation is consistent with a previous study that concluded that the reduced processing of semantic relationships during encoding is less explained by the implicit activation and retrieval of the patients’ semantic lexicon than by the reduced ability for the patients to implement explicit relational processing strategies [53]. However, further studies using RP with explicit in comparison to implicit instructions are needed to confirm this hypothesis using a similar procedure as ours. Apart from this, as a family of theories (namely retrieval effort theories) supposes that testing effect relies on both the quality and intensity of processing that is induced by the initial retrieval attempt, this ability to process semantic information may be maintained, though untrained in patients with schizophrenia. For instance, Bjork and Bjork assume that difficult tests foster storage strength of a memory (i.e., the memory is durably established to a greater degree), thus leading to effective memory retrieval. Support for this family of theories comes from the studies showing that difficult initial tests and increased delay between the initial study and the initial test both produce larger testing effect. However, further studies manipulating the difficulty of the task are needed to examine whether the retrieval practice effect observed in our sample of patients can be explained by this effect.

Our interpretation of the results underscores a preserved ability to retrieve during the final test the existing semantic associations which were encoded at study. We have to acknowledge that this statement may be at odds with the previous literature showing that associative or relational memory processes are especially impaired in schizophrenia [55]. In our study, we chose to compare weakly and non-associated pairs as our aim was to demonstrate that RP could be a relevant cognitive therapy tool and as we considered non-associated word pairs being of greater interest in a cognitive remediation perspective (items to learn in daily life are in fact often unrelated: e.g., name and address, shopping list, password and login). However, as RP was regularly shown as fostering a greater testing effect for weakly associated semantic items in comparison to strongly-associated items in healthy populations, our hypotheses might be challenged in a further study comparing strongly and weakly associated word pairs. A similar result in patients would support the idea of a preserved ability to retrieve previously encoded semantic associations in schizophrenia, whereas other patterns of results (a stronger testing effect with the strongly associated pairs, or no difference between the strongly and weakly associated word pairs) would confirm semantic memory impairments in schizophrenia. The elaborative retrieval hypothesis, developed by Carpenter, tries to explain this greater improvement in memory recall with the weakly-associated pairs in comparison to the strongly associated pairs. This theory considers that engaging in a retrieval attempt produces semantic associations with the target, thus effectively guiding its retrieval (e.g., with the cue-target pair “PINEAPPLE-ISLAND”, a participant may generate plausible but incorrect candidates [e.g., fruit, exotic, etc.] before reaching the target [island]) [56]. As the strongly associated items require little elaboration to be retrieved, restudying these items generates some accurate answers in itself. In contrast, restudying weakly associated items, which require much elaboration to be retrieved, does not activate effortful semantic elaboration and is less efficient than RP, the latter fostering semantic associations. Following this theory, our results suggest that patients with schizophrenia are not only able to retrieve previously learned material through basic rote repetition, but also to elaborate semantic associations at an initial test and to retrieve these associations at a final cued-recall. Moreover, this ability was not affected by the neurocognitive dysfunction revealed during the initial neuropsychological examination of our patients.

Some limitations of our study must be acknowledged. First, our sample size was small, yet similar to sample sizes of previous studies on retrieval practice performed in clinical populations with memory deficits (which included between 10 and 73 participants) [21, 24] and in patients with schizophrenia (between 10 and 30 participants) [27‐30]. Moreover, the Bayesian analysis method we used made it possible to test whether our data are sensitive to priors that may counter our conclusions. The analysis clearly showed that the results remained unchanged with non-informative priors and with pessimistic priors minoring possible effects. Secondly, though the patients recalled significantly more words at the final cued-recall test in the RP condition, the generalisation of benefits in episodic memory to other memory tasks remains to be demonstrated. A protocol including various cognitive tasks in verbal and/or visual episodic memory, before and after RP, might resolve this issue [57]. It is worth stressing here that although the patients exhibited poorer cognitive performance during our neuropsychological assessments, taking global cognitive performance into account did not influence the effect of retrieval practice or the interaction between group and retrieval practice, suggesting that retrieval practice might benefit patients whatever their level of cognitive deficits.

We think that our findings open new perspectives in cognitive remediation strategies, which are a pillar of recovery, in schizophrenia. RP has been previously considered as a compensatory approach to memory rehabilitation in clinical populations with memory deficits [20, 21]. In restorative approaches, RP could be integrated into computerised CT programs to train episodic memory. Indeed, as Carpenter showed evidence for testing effect transferability in terms of temporal contexts, test formats and knowledge domains, this training might foster the use of RP by patients in daily life situations [58]. However, though most cognitive remediation techniques to date rely on a computerised drill-and-practice approach alone, they appear to yield stronger effect sizes when strategy coaching is added [5, 59]. Thus, RP could also be taught in compensatory cognitive training among mnemonic strategies for remembering new information. For example, patients may engage in self-testing when they need to remember associated items in a shopping list (e.g., cheese-bread) or non-associated items (e.g., the name of a person with a photo or an address). RP might also be an effective complementary strategy to improve patients’ medication management, as episodic memory deficits predict patients’ lower medication adherence [60]. For example, patients’ relatives or health care providers may foster RP in patients by testing them about their next appointment date or their medication regimen, rather than repeating it twice or more. Furthermore, the results reported by previous studies on CT showed a great heterogeneity, depending on the patients and cognitive remediation programs [61, 62]. In our study, as RP was found effective in every patient but one, it appears to be a promising method of cognitive remediation. However, since the patients’ final cued recall scores using RP did not reach the baseline performance of the healthy controls, further investigations are necessary in order to explore the factors which might contribute to robustly improve patients’ performance while using RP. Then, future studies on cognitive remediation programs that include RP might show how long-term RP training can improve episodic memory and the quality of life in patients with schizophrenia.

Retrieval practice is an efficient long-term episodic memory strategy in healthy subjects. Our study is the first to demonstrate that retrieval practice is also superior to restudy in improving later recall in patients with schizophrenia presenting with episodic memory impairment. While semantic relational memory deficits in schizophrenia mostly rely on a difficulty to self-initiate semantic memory strategies, RP may foster semantic elaboration, leading to memory performance enhancement. As memory impairment severely affects patients’ daily lives, retrieval practice, integrated into cognitive remediation programs, should be considered as a valuable tool to address these deficits and to help reduce disability. Further investigations are needed to explore the factors that promote the effectiveness of RP in patients in the context of cognitive remediation.