Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

  • Brief Communication
  • Published:

Linking pattern completion in the hippocampus to predictive coding in visual cortex

Nature Neuroscience volume 19, pages 665–667 (2016)Cite this article

Subjects

Abstract

Models of predictive coding frame perception as a generative process in which expectations constrain sensory representations. These models account for expectations about how a stimulus will move or change from moment to moment, but do not address expectations about what other, distinct stimuli are likely to appear based on prior experience. We show that such memory-based expectations in human visual cortex are related to the hippocampal mechanism of pattern completion.

This is a preview of subscription content, access via your institution

Access options

Buy this article

  • Purchase on Springer Link
  • Instant access to full article PDF
Buy now

Prices may be subject to local taxes which are calculated during checkout

Figure 1: Analysis approach.
Figure 2: Decoding performance.
Figure 3: Hippocampal–visual relationship.

Similar content being viewed by others

References

  1. Rao, R.P.N. & Ballard, D.H. Nat. Neurosci. 2, 79–87 (1999).

    Article  CAS  Google Scholar 

  2. Friston, K. Phil. Trans. R. Soc. Lond. B 360, 815–836 (2005).

    Article  Google Scholar 

  3. Clark, A. Behav. Brain Sci. 36, 181–204 (2013).

    Article  Google Scholar 

  4. Smith, F.W. & Muckli, L. Proc. Natl. Acad. Sci. USA 107, 20099–20103 (2010).

    Article  CAS  Google Scholar 

  5. Alink, A., Schwiedrzik, C.M., Kohler, A., Singer, W. & Muckli, L. J. Neurosci. 30, 2960–2966 (2010).

    Article  CAS  Google Scholar 

  6. Schapiro, A.C., Kustner, L.V. & Turk-Browne, N.B. Curr. Biol. 22, 1622–1627 (2012).

    Article  CAS  Google Scholar 

  7. Hawkins, J. & Blakeslee, S. On Intelligence (Times Books, 2004).

  8. Kok, P., Jehee, J.F. & de Lange, F.P. Neuron 75, 265–270 (2012).

    Article  CAS  Google Scholar 

  9. Marr, D. Phil. Trans. R. Soc. B. 262, 23–81 (1971).

    Article  CAS  Google Scholar 

  10. Cohen, N.J. & Eichenbaum, H. Memory, Amnesia, and the Hippocampal System (MIT Press, 1993).

  11. Leutgeb, S. & Leutgeb, J.K. Learn. Mem. 14, 745–757 (2007).

    Article  Google Scholar 

  12. Ji, D. & Wilson, M.A. Nat. Neurosci. 10, 100–107 (2007).

    Article  CAS  Google Scholar 

  13. Bosch, S.E., Jehee, J.F., Fernández, G. & Doeller, C.F. J. Neurosci. 34, 7493–7500 (2014).

    Article  CAS  Google Scholar 

  14. Hindy, N.C. & Turk-Browne, N.B. Cereb. Cortex 10.1093/cercor/bhv030 (2015).

  15. O'Doherty, J. et al. Science 304, 452–454 (2004).

    Article  CAS  Google Scholar 

  16. Bar, M. et al. Proc. Natl. Acad. Sci. USA 103, 449–454 (2006).

    Article  CAS  Google Scholar 

  17. Dale, A.M. Hum. Brain Mapp. 8, 109–114 (1999).

    Article  CAS  Google Scholar 

  18. Yushkevich, P.A. et al. Hum. Brain Mapp. 36, 258–287 (2015).

    Article  Google Scholar 

  19. Aly, M. & Turk-Browne, N.B. Cereb. Cortex 26, 783–796 (2016).

    PubMed  Google Scholar 

  20. Duvernoy, H.M. The Human Hippocampus: Functional Anatomy, Vascularization and Serial Sections with MRI (Springer: 2005).

  21. Carr, V.A., Rissman, J. & Wagner, A.D. Neuron 65, 298–308 (2010).

    Article  CAS  Google Scholar 

  22. Treves, A., Tashiro, A., Witter, M.P. & Moser, E.I. Neuroscience 154, 1155–1172 (2008).

    Article  CAS  Google Scholar 

  23. Ketz, N., Morkonda, S.G. & O'Reilly, R.C. PLoS Comput. Biol. 9, 1–9 (2013).

    Article  Google Scholar 

  24. Nakashiba, T. et al. Cell 149, 188–201 (2012).

    Article  CAS  Google Scholar 

  25. Duncan, K., Tompary, A. & Davachi, L. J. Neurosci. 34, 11188–11198 (2014).

    Article  CAS  Google Scholar 

  26. Bakker, A., Kirwan, C.B., Miller, M. & Stark, C.E. Science 319, 1640–1642 (2008).

    Article  CAS  Google Scholar 

  27. Fischl, B. et al. Cereb. Cortex 18, 1973–1980 (2008).

    Article  Google Scholar 

  28. Hinds, O.P. et al. Neuroimage 39, 1585–1599 (2008).

    Article  Google Scholar 

  29. Dale, A.M., Fischl, B. & Sereno, M.I. Neuroimage 9, 179–194 (1999).

    Article  CAS  Google Scholar 

  30. Wang, L., Mruczek, R.E., Arcaro, M.J. & Kastner, S. Cereb. Cortex 25, 3911–3931 (2014).

    Article  Google Scholar 

  31. Kriegeskorte, N., Goebel, R. & Bandettini, P. Proc. Natl. Acad. Sci. USA 103, 3863–3868 (2006).

    Article  CAS  Google Scholar 

  32. Debas, K. et al. Neuroimage 99, 50–58 (2014).

    Article  Google Scholar 

  33. Gabitov, E., Manor, D. & Karni, A. J. Cogn. Neurosci. 27, 736–751 (2015).

    Article  Google Scholar 

  34. Smith, S.M. et al. Neuroimage 23 (suppl. 1), S208–S219 (2004).

    Article  Google Scholar 

  35. Greve, D.N. & Fischl, B. Neuroimage 48, 63–72 (2009).

    Article  Google Scholar 

  36. Jenkinson, M., Bannister, P., Brady, M. & Smith, S. Neuroimage 17, 825–841 (2002).

    Article  Google Scholar 

  37. Andersson, J.L., Jenkinson, M. & Smith, S. FMRIB Tech. Rep. TR07JA2 (2007).

  38. O'Reilly, R.C. & Rudy, J.W. Psychol. Rev. 108, 311–345 (2001).

    Article  CAS  Google Scholar 

  39. Komorowski, R.W., Manns, J.R. & Eichenbaum, H. J. Neurosci. 29, 9918–9929 (2009).

    Article  CAS  Google Scholar 

  40. Hindy, N.C., Solomon, S.H., Altmann, G.T.M. & Thompson-Schill, S.L. Cereb. Cortex 25, 884–894 (2015).

    Article  Google Scholar 

  41. Efron, B. & Tibshirani, R. Stat. Sci. 1, 54–75 (1986).

    Article  Google Scholar 

Download references

Acknowledgements

This work was supported by NIH grants F32 EY023162, S10 OD016277, and R01 EY021755. The authors thank P. Kok and A. Schapiro for helpful comments on an earlier draft.

Author information

Authors and Affiliations

  1. Princeton Neuroscience Institute, Princeton University, Princeton, New Jersey, USA

    Nicholas C Hindy & Nicholas B Turk-Browne

  2. Department of Psychology, Princeton University, Princeton, New Jersey, USA

    Felicia Y Ng & Nicholas B Turk-Browne

Authors
  1. Nicholas C Hindy
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Felicia Y Ng
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Nicholas B Turk-Browne
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

N.C.H., F.Y.N., and N.B.T.-B. designed the experiment, reviewed the analyses, and discussed the results. N.C.H. and F.Y.N. collected the data. N.C.H. performed the analyses. N.C.H. and N.B.T.-B. wrote and revised the paper.

Corresponding author

Correspondence to Nicholas C Hindy.

Ethics declarations

Competing interests

The authors declare no competing financial interests.

Integrated supplementary information

Supplementary Figure 1 Task procedure

(a) The stimuli consisted of 12 fractal-like images. (b) During initial training and scanning, full-sequence trials began with a cue at fixation. A double-headed arrow prompted participants to press a button with their choice of left or right hand, at which point an outcome replaced the cue. Cue+action trials contained the same structure, but a blank screen appeared for 1000 ms instead of the outcome. Outcome-only trials contained just an outcome for 1000 ms without a preceding cue or action. (c) Behavioral tests were conducted to assess learning at different stages. Each test trial involved a verbal “top” or “bottom” response to select which of the two outcomes associated with the cue and action seemed most probable.

Supplementary Figure 2 Resampled decoding performance

Subject-level bootstrap resampling41 was used to confirm the random-effects significance of classification for each ROI, without the assumptions of parametric tests. (a) Sequence decoding was reliable in CA2–CA3–DG (P = 0.008) and CA1 (P = 0.004), but not in subiculum, V1, or V2 (P > 0.77). (b) Outcome decoding was reliable in V1 (P = 0.0005) and V2 (P = 0.01), but not in CA2–CA3–DG, CA1, or subiculum (P > 0.55). Error bars indicate 95% confidence intervals. *P < 0.05, **P < 0.01, ***P < 0.001

Supplementary Figure 3 Unpredictable trials

No reliable effects were obtained in a control condition where actions did not predict outcomes, providing evidence that predictive actions were required to observe sequence decoding and outcome decoding. (a) Sequence decoding for unpredictable trials was not reliable in either CA–DG (t23 = 0.85, P = 0.40) or V1–V2 (t23 = 0.13, P = 0.90). (b) Outcome decoding for unpredictable trials was also not reliable in either CA–DG (t23 = 0.61, P = 0.55) or V1–V2 (t23 = 0.60, P = 0.56). Error bars depict ±1 s.e.m.

Supplementary Figure 4 Action decoding

Predictable and unpredictable trials were used to examine action information in the hippocampus and the right putamen. (a) Classifiers trained to distinguish predictable full-sequence trials with left vs. right actions for one cue (subscript 1) could not reliably decode predictable full-sequence trials with corresponding left vs. right actions for the other cue (subscript 2) in either CA–DG (t23 = -0.39, P = 0.70) or the right putamen (t23 = 1.50, P = 0.15). (b) These classifiers also could not reliably decode predictable cue+action trials with left vs. right actions for another cue in CA–DG (t23 = 1.66, P = 0.11), but could marginally decode them in the right putamen (t23 =2.05, P = 0.05). (c) Classifiers trained to distinguish unpredictable full-sequence trials with left vs. right actions (orthogonal to outcomes) could not reliably decode unpredictable cue+action trials with corresponding left vs. right actions in CA–DG (t23 = 0.11, P = 0.92), but could decode them in the right putamen (t23 = 2.85, P = 0.009). Error bars depict ±1 s.e.m. ~P < 0.10, **P < 0.01

Supplementary Figure 5 Within-action classification

Classifiers trained to distinguish predictable full-sequence trials with different cues and outcomes but identical left or right actions reliably decoded predictable cue+action trials with different cues but identical actions in CA–DG (t23 = 2.20, P = 0.04), but not in the right putamen (t23 = -0.65, P = 0.52). Error bars depict ±1 s.e.m. *P < 0.05

Supplementary Figure 6 Brain–behavior correlations

For the predictable condition, voice RT in the behavioral tests (a) had a marginally negative correlation across participants with sequence decoding in CA–DG (r22 = -0.36, P = 0.09) and (b) had a reliably negative correlation across participants with outcome decoding in V1–V2 (r22 = -0.54, P = 0.007). That is, better outcome decoding and to some extent better sequence decoding were associated with faster outcome identification at test. For the unpredictable condition, test RT was not correlated with either (c) sequence decoding in CA–DG (r22 = -0.10, P = 0.63) or (d) outcome decoding in V1–V2 (r22 = -0.02, P = 0.92). ~ P < 0.10, ** P < 0.01

Supplementary Figure 7 Cross-classification

Consistent with our hypotheses that full-sequence trials would most effectively target conjunctive representations in the hippocampus and that outcome-only trials would most effectively target outcome representations in early visual cortex, cross-classification in CA–DG was reliable only when full-sequence trials were part of either the training or testing set and cross-classification in V1–V2 was reliable only when outcome-only trials were part of either the training or testing set. (a) Cross-classification from full-sequence to cue+action trials (sequence decoding elsewhere) was reliable in CA–DG (t23 = 2.64, P = 0.01), but not in V1–V2 (t23 = -0.97, P = 0.34). In contrast, cross-classification from full-sequence to outcome-only trials was reliable in both CA–DG (t23 = 2.29, P = 0.03) and V1–V2 (t23 = 3.00, P = 0.006). (b) Cross-classification from outcome-only to cue+action trials (outcome decoding elsewhere) was reliable in V1–V2 (t23 = 2.99, P = 0.007), but not in CA–DG (t23 = 0.09, P = 0.93), and cross-classification from outcome-only to full-sequence trials was likewise reliable in V1–V2 (t23 = 2.35, P = 0.03), but not in CA–DG (t23 = 1.60, P = 0.12). (c) Similar to sequence decoding, cross-classification from cue+action to full-sequence trials was reliable in CA–DG (t23 = 2.69, P = 0.01), but not in V1–V2 (t23 = -0.03, P = 0.97). Unlike outcome decoding, cross-classification from cue+action to outcome-only trials was not reliable in V1–V2 (t23 = 1.24, P = 0.23), and still not in CA–DG (t23 = 1.14, P = 0.27). The difference in V1–V2 cross-classification for [outcome-only → cue+action] vs. [cue+action → outcome-only] may be due to worse classifier training with cue+action trials: there were more outcome-only than cue+action training examples in the design, cue+action trials contained an additional uninformative stimulus (the cue), and the outcome representation on cue+action trials may have been weaker because it reflected an internal expectation rather than an external stimulus. Error bars depict ±1 s.e.m. *P < 0.05, **P < 0.01

Supplementary Figure 8 Resampled hippocampal–visual relationships

Subject-level bootstrap resampling41 was used to confirm the random-effects significance of within- and across-participant relationships between sequence decoding in the hippocampus and outcome decoding in early visual cortex, without the assumptions of parametric tests. (a) Outcome decoding in V1–V2 was more reliable (P = 0.02) on trials where sequence decoding in CA–DG was correct (vs. 50% chance: P = 0.0006) vs. incorrect (P = 0.28). (b) Individual differences in V1–V2 outcome decoding could be predicted from CA–DG sequence decoding (P = 0.004). Error bars and bands indicate 95% confidence intervals. *P < 0.05, **P < 0.01, ***P < 0.001

Supplementary Figure 9 Timecourse analysis

The temporal precedence of CA–DG sequence decoding vs. V1–V2 outcome decoding might provide tentative evidence about the directionality of the relationship between these regional processes. (a) If sequence information in the hippocampus precedes outcome information in visual cortex, then CA–DG sequence decoding at TR 3 should be predictive of V1–V2 outcome decoding on TR 4 in a multinomial regression (purple). (b) If outcome information in visual cortex precedes sequence information in the hippocampus, then V1–V2 outcome decoding at TR 3 should be predictive of CA–DG sequence decoding at TR 4 (green). (c) Classifiers for CA–DG were trained on patterns of GLM beta parameters from the full-sequence trials, whereas classifiers for V1–V2 were trained on patterns of GLM beta parameters from the outcome-only trials. All classifiers were tested on raw activity patterns from cue+action trials that were z-scored and extracted from timepoints around the peak response (TRs 3 and 4 after trial onset). CA–DG sequence decoding at TR 3 predicted V1–V2 outcome decoding at TR 4 (purple; t23 = 2.30, P = 0.03), whereas V1–V2 outcome decoding at TR 3 did not reliably predict CA–DG sequence decoding at TR 4 (green; t23 = 0.84, P = 0.41). In contrast, CA–DG sequence decoding did not reliably predict V1–V2 outcome decoding (gray) within either TR 3 (t23 = 1.50, P = 0.15) or TR 4 (t23 = -0.31, P = 0.76). CA–DG sequence decoding at TR 3 was more predictive of V1–V2 outcome decoding at TR 4 than was CA–DG sequence decoding at TR 4 (t23 = 2.21, P = 0.04). CA–DG sequence decoding at TR 3 did not predict V1–V2 outcome decoding at TR 4 more reliably than V1–V2 outcome decoding at TR 3 (t23 = 1.25, P = 0.23), although the difference was in the same direction. Mean parameter estimates are shown in bold font, with s.e.m. in parentheses. *P < 0.05

Supplementary information

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Hindy, N., Ng, F. & Turk-Browne, N. Linking pattern completion in the hippocampus to predictive coding in visual cortex. Nat Neurosci 19, 665–667 (2016). https://doi.org/10.1038/nn.4284

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1038/nn.4284

This article is cited by

Search

Advanced search

Quick links

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing