Randall C. O'Reilly's Online Publications

Copyright Notice

The documents distributed here have been provided as a means to ensure timely dissemination of scholarly and technical work on a noncommercial basis. Copyright and all rights therein are maintained by the authors or by other copyright holders, notwithstanding that they have offered their works here electronically. It is understood that all persons copying this information will adhere to the terms and constraints invoked by each author's copyright. These works may not be reposted without the explicit permission of the copyright holder. ( Notice borrowed from Dave Plaut ).

Other papers listed in my Curriculum Vitae may be available by sending an email request to anti-spam email addr img .

Most documents here are available in PDF format.

Last updated: 04/15/08. Back to my home page.

Papers With Abstracts, Listed by Date

[PDF] Pauli, W.M. & O'Reilly, R.C. (2007). Attentional control of associative learning -- a possible role of the central cholinergic system. Brain Research.
Abstract:
How does attention interact with learning? Kruschke [Kruschke, J.K. (2001). Toward a unified Model of Attention in Associative Learning. J. Math. Psychol. 45, 812--863.] proposed a model (EXIT) that captures Mackintosh's [Mackintosh, N.J. (1975). A theory of attention: Variations in the associability of stimuli with reinforcement. Psychological Review, 82(4), 276--298.] framework for attentional modulation of associative learning. We developed a computational model that showed analogous interactions between selective attention and associative learning, but is significantly simplified and, in contrast to EXIT, is motivated by neurophysiological findings. Competition among input representations in the internal representation layer, which increases the contrast between stimuli, is critical for simulating these interactions in human behavior. Furthermore, this competition is modulated in a way that might be consistent with the phasic activation of the central cholinergic system, which modulates activity in sensory cortices. Specifically, phasic increases in acetylcholine can cause increased excitability of both pyramidal excitatory neurons in cortical layers II/III and cortical GABAergic inhibitory interneurons targeting the same pyramidal neurons. These effects result in increased attentional contrast in our model. This model thus represents an initial attempt to link human attentional learning data with underlying neural substrates.
[PDF] Frank, M., Santamaria, A., O'Reilly, R. & Willcutt, E. (2007). Testing Computational Models of Dopamine and Noradrenaline Dysfunction in Attention Deficit/Hyperactivity Disorder. Neuropsychopharmacology, 32, 1583-99.
Abstract:
We test our neurocomputational model of fronto-striatal dopamine (DA) and noradrenaline (NA) function for understanding cognitive and motivational deficits in attention deficit / hyperactivity disorder (ADHD). Our model predicts that low striatal DA levels in ADHD should lead to deficits in ``Go'' learning from positive reinforcement, which should be alleviated by stimulant medications, as observed with DA manipulations in other populations. Indeed, while non-medicated adult ADHD participants were impaired at both positive (Go) and negative (NoGo) reinforcement learning, only the former deficits were ameliorated by medication. We also found evidence for our model's extension of the same striatal DA mechanisms to working memory, via interactions with prefrontal cortex. In a modified AX-continuous performance task, ADHD participants showed reduced sensitivity to working memory contextual information, despite no global performance deficits, and were more susceptible to the influence of distractor stimuli presented during the delay. These effects were reversed with stimulant medications. Moreover, the tendency for medications to improve Go relative to NoGo reinforcement learning was predictive of their improvement in working memory in distracting conditions, suggestive of common DA mechanisms and supporting a unified account of DA function in ADHD. However, other ADHD effects such as erratic trial-to-trial switching and reaction time variability are not accounted for by model DA mechanisms, and are instead consistent with cortical noradrenergic dysfunction and associated computational models. Accordingly, putative NA deficits were correlated with each other and independent of putative DA-related deficits. Taken together, our results demonstrate the usefulness of computational approaches for understanding cognitive deficits in ADHD.
[Main PDF] [Supp PDF] Hazy, T.E., Frank, M.J. & O'Reilly, R.C. (2007). Towards an executive without a homunculus: computational models of the prefrontal cortex/basal ganglia system. Philosophical Transactions of the Royal Society B, 362, 1601-1613.
Abstract:
The prefrontal cortex (PFC) has long been thought to serve as an "executive" that controls the selection of actions and cognitive functions more generally. However, the mechanistic basis of this executive function has not been clearly specified often amounting to a homunculus. This paper reviews recent attempts to deconstruct this homunculus by elucidating the precise computational and neural mechanisms underlying the executive functions of the PFC. The overall approach builds upon existing mechanistic models of the basal ganglia (BG) and frontal systems known to play a critical role in motor control and action selection, where the BG provide a "Go" versus "NoGo" modulation of frontal action representations. In our model, the BG modulate working memory representations in prefrontal areas to support more abstract executive functions. We have developed a computational model of this system that is capable of developing human-like performance on working memory and executive control tasks through trial-and-error learning. This learning is based on reinforcement learning mechanisms associated with the midbrain dopaminergic system and its activation via the BG and amygdala. Finally, we briefly describe various empirical tests of this framework.
[PDF] O'Reilly, R.C., Frank, M.J., Hazy, T.E. & Watz, B. (2007). PVLV: The Primary Value and Learned Value Pavlovian Learning Algorithm. Behavioral Neuroscience, 121, 31-49.
Abstract:
The authors present their primary value learned value (PVLV) model for understanding the reward-predictive firing properties of dopamine (DA) neurons as an alternative to the temporal-differences (TD) algorithm. PVLV is more directly related to underlying biology and is also more robust to variability in the environment. The primary value (PV) system controls performance and learning during primary rewards, whereas the learned value (LV) system learns about conditioned stimuli. The PV system is essentially the Rescorla-Wagner/delta-rule and comprises the neurons in the ventral striatum/nucleus accumbens that inhibit DA cells. The LV system comprises the neurons in the central nucleus of the amygdala that excite DA cells. The authors show that the PVLV model can account for critical aspects of the DA firing data, making a number of clear predictions about lesion effects, several of which are consistent with existing data. For example, first- and second-order conditioning can be anatomically dissociated, which is consistent with PVLV and not TD. Overall, the model provides a biologically plausible framework for understanding the neural basis of reward learning.
[PDF] Atallah, H.E., Lopez-Paniagua, D., Rudy, J.W. & O'Reilly, R. (2007). Separate neural substrates for skill learning and performance in the ventral and dorsal striatum. Nature Neuroscience, 10, 126-131.
Abstract:
It is widely accepted that the striatum of the basal ganglia is a primary substrate for the learning and performance of skills. We provide evidence that two regions of the rat striatum, ventral and dorsal, play distinct roles in instrumental conditioning (skill learning), with the ventral striatum being critical for learning and the dorsal striatum being important for performance but, notably, not for learning. This implies an actor (dorsal) versus director (ventral) division of labor, which is a new variant of the widely discussed actor-critic architecture. Our results also imply that the successful performance of a skill can ultimately result in its establishment as a habit outside the basal ganglia.
[PDF] OReilly, R.C. (2006). Biologically Based Computational Models of High-Level Cognition. Science, 314, 91-94.
Abstract:
Computer models based on the detailed biology of the brain can help us understand the myriad complexities of human cognition and intelligence. Here, we review models of the higher level aspects of human intelligence, which depend critically on the prefrontal cortex and associated subcortical areas. The picture emerging from a convergence of detailed mechanistic models and more abstract functional models represents a synthesis between analog and digital forms of computation. Specifically, the need for robust active maintenance and rapid updating of information in the prefrontal cortex appears to be satisfied by bistable activation states and dynamic gating mechanisms. These mechanisms are fundamental to digital computers and may be critical for the distinctive aspects of human intelligence.
[PDF] Frank, M.J., OReilly, R.C. & Curran, T. (2006). When memory fails, intuition reigns: Midazolam enhances implicit inference in humans. Psychological Science, 17, 700-707.
Abstract:
People often make logically sound decisions using explicit reasoning strategies, but sometimes it pays to rely on more implicit ``gut-level'' intuition. The transitive inference paradigm has been widely used as a test of explicit logical reasoning in animals and humans, but it can also be solved in a more implicit manner. Some have argued that the hippocampus supports relational memories required for making logical inferences. Here we show that the benzodiazepene midazolam, which inactivates the hippocampus, causes profound explicit memory deficits in healthy participants, but actually enhances their ability in making implicit transitive inferences. These results are consistent with neurocomputational models of the basal ganglia/dopamine system that learn to make decisions based on positive and negative reinforcement. We suggest that disengaging the hippocampal explicit memory system can be advantageous for this more implicit form of learning.
[Main PDF] [Supp PDF] Frank, M.J. & O'Reilly, R.C. (2006). A mechanistic account of striatal dopamine function in human cognition: Psychopharmacological studies with cabergoline and haloperidol. Behavioral Neuroscience, 120, 497-517.
Abstract:
We test a neurocomputational model of dopamine function in cognition by administering to healthy participants low doses of D2 agents cabergoline and haloperidol. Our model suggests that dopamine dynamically modulates the balance of ``Go'' and ``NoGo'' basal ganglia pathways during cognitive learning and performance. Cabergoline impaired, while haloperidol enhanced, Go learning from positive reinforcement, consistent with presynaptic drug effects. Cabergoline also caused an overall bias toward Go responding, consistent with postsynaptic action. These same effects extended to working memory and attentional domains, supporting the idea that the basal ganglia / dopamine system modulates the updating of prefrontal representations. Drug effects interacted with baseline working memory span in all tasks. Taken together, our results support a unified account of the role of dopamine in modulating cognitive processes that depend on the basal ganglia.
[PDF] Hazy, T.E., Frank, M.J. & O'Reilly, R.C. (2006). Banishing the Homunculus: Making Working Memory Work. Neuroscience, 139, 105-118.
Abstract:
The prefrontal cortex (PFC) has long been thought to subserve both working memory and ``executive'' function, but the mechanistic basis of their integrated function has remained poorly understood, often amounting to a homunculus. This paper reviews the progress in our lab and others pursuing a long-term research agenda to deconstruct this homunculus by elucidating the precise computational and neural mechanisms underlying these phenomena. We outline six key functional demands underlying working memory, and then describe the current state of our computational model of the PFC and associated systems in the basal ganglia (BG). The model, called PBWM (prefrontal-cortex, basal-ganglia working memory model), relies on actively maintained representations in the PFC, which are dynamically updated/gated by the BG. It is capable of developing human-like performance largely on its own by taking advantage of powerful reinforcement learning mechanisms, based on the midbrain dopaminergic system and its activation via the BG and amygdala. These learning mechanisms enable the model to learn to control both itself and other brain areas in a strategic, task-appropriate manner. The model can learn challenging working memory tasks, and has been corroborated by several important empirical studies.
[PDF] Cer, D.M. & O'Reilly, R.C. (in press). Neural mechanisms of binding in the hippocampus and neocortex: Insights from computational models. H.D. Zimmer, A. Mecklinger & U. Lindenberger (Eds) Binding in Memory, Oxford: Oxford University Press.
Abstract:
An account of the neurological mechanisms that underlie binding is given which is characterized by the decomposition of the binding problem into three distinct subproblems. Each subproblem is then supported by anatomically specialized brain regions. The posterior cortex employs coarse-coded distributed representations of low-order conjunctions to resolve binding ambiguities, while also supporting systematic generalization to novel stimuli and situations. These representations are slowly acquired over experience. The hippocampus can more rapidly bind higher-order conjunctions of information such as episodes or locations. Finally, the prefrontal cortex supports transient, actively maintained bindings that are used in the service of working memory. We argue that this approach to the binding problem compares favorably with those based on temporal synchrony binding.
[PDF] O'Reilly, R.C. (in press). The Division of Labor Between the Neocortex and Hippocampus. G. Houghton (Ed) Connectionist Modeling in Cognitive Psychology, : Psychology Press.
Abstract:
This chapter presents an overview of a computational approach towards understanding the different contributions of the neocortex and hippocampus in learning and memory. The approach is based on a set of principles derived from converging biological, psychological, and computational constraints. The most central principles are that the neocortex employs a slow learning rate and overlapping distributed representations to extract the general statistical structure of the environment, while the hippocampus learns rapidly using separated representations to encode the details of specific events while suffering minimal interference. Additional principles concern the nature of learning (error-driven and Hebbian), and recall of information via pattern completion. These principles are demonstrated through neocortical and hippocampal models of a well-known memory task, the AB-AC paired associates task. The results of applying these principles to a wide range of phenomena in conditioning, habituation, contextual learning, recognition memory, recall, and retrograde amnesia, are also summarized.
[PDF] O'Reilly, R.C. (in press). Modeling Integration and Dissociation in Brain and Cognitive Development. Y. Munakata & M.H. Johnson (Eds) Processes of Change in Brain and Cognitive Development: Attention and Performance XXI., Oxford: Oxford University Press.
Abstract:
Over the course of development, brain areas can become increasingly dissociated in their functions, or increasingly integrated. Computational models can provide insights into how and why these opposing effects happen. This paper presents a computational framework for understanding the specialization of brain functions across the hippocampus, neocortex, and basal ganglia. This framework is based on computational tradeoffs that arise in neural network models, where achieving one type of learning function requires very different parameters from those necessary to achieve another form of learning. For example, we dissociate the hippocampus from cortex with respect to general levels of activity, learning rate, and level of overlap between activation patterns. Similarly, the frontal cortex and associated basal ganglia system have important neural specializations not required of the posterior cortex system. Taken together, these brain areas form an overall cognitive architecture, which has been implemented in functioning computational models, provides a rich and often subtle means of explaining a wide range of behavioral and cognitive neuroscience data. The developmental implications of this framework, and other computational mechanisms of dissociation and integration, are reviewed.
[PDF] O'Reilly, R.C. & Frank, M.J. (2006). Making Working Memory Work: A Computational Model of Learning in the Frontal Cortex and Basal Ganglia. Neural Computation, 18, 283-328.
Abstract:
The prefrontal cortex has long been thought to subserve both working memory (the holding of information online for processing) and ``executive'' functions (deciding how to manipulate working memory and perform processing). Although many computational models of working memory have been developed, the mechanistic basis of executive function remains elusive, often amounting to a homunculus. This paper presents an attempt to deconstruct this homunculus through powerful learning mechanisms that allow a computational model of the prefrontal cortex to control both itself and other brain areas in a strategic, task-appropriate manner. These learning mechanisms are based on subcortical structures in the midbrain, basal ganglia and amygdala, which together form an actor/critic architecture. The critic system learns which prefrontal representations are task-relevant and trains the actor, which in turn provides a dynamic gating mechanism for controlling working memory updating. Computationally, the learning mechanism is designed to simultaneously solve the temporal and structural credit assignment problems. The model's performance compares favorably with standard backpropagation-based temporal learning mechanisms on the challenging 1-2-AX working memory task, and other benchmark working memory tasks.
[PDF] Herd, S.A., Banich, M.T. & O'Reilly, R.C. (2006). Neural Mechanisms of Cognitive Control: An integrative Model of Stroop Task Performance and fMRI data. Journal of Cognitive Neuroscience, 18, 22-32.
Abstract:
We address the connection between conceptual knowledge and cognitive control using a neural network model. This model extends a widely held theory of cognitive control [Cohen, J. D., Dunbar, K., & McClelland, J. L. On the control of automatic processes: A parallel distributed processing model of the Stroop effect. Psychological Review, 97, 332-361, 1990] so that it can explain new empirical findings. Leveraging other computational modeling work, we hypothesize that representations used for task control are recruited from preexisting representations for categories, such as the concept of color relevant to the Stroop task we model here. This hypothesis allows the model to account for otherwise puzzling fMRI results, such as increased activity in brain regions processing to-be-ignored information. In addition, biologically motivated changes in the model��s pattern of connectivity show how global competition can arise when inhibition is strictly local, as it seems to be in the cortex. We also discuss the potential for this theory to unify models of task control with other forms of attention.
[PDF] Herd, S.A. & O'Reilly, R.C. (2005). Serial visual search from a parallel model. Vision Research, 45, 2987-2992.
Abstract:
We tested a parallel neural network model of visual search, and found that it located targets more quickly when allowed to take several fast guesses. We suggest that this serially iterated parallel search may be the mode used by the visual system, in accord with theories such as the Guided Search model. Furthermore, in our model the most efficient mode of processing varied with the type of search. If the nature of visual search varies with task demands, seemingly contradictory findings can be reconciled.
[PDF] Frank, M.J., Rudy, J.W., Levy, W.B. & O'Reilly, R.C. (2005). When Logic Fails: Implicit Transitive Inference in Humans. Memory and Cognition, 33, 742-750.
Abstract:
Transitive inference (TI) in animals (e.g., choosing A over C based on knowing that A is better than B and B is better C) has been interpreted by some as reflecting a declarative, logical inference process. We invert this anthropomorphic interpretation by providing evidence that humans can exhibit TI-like behavior based on simpler associative mechanisms that underly many theories of animal learning. In this study, human participants were trained on a five-pair TI problem (A+B-, B+C-, C+D-, D+E-, E+F-), and, unlike in previous human TI studies, were prevented from becoming explicitly aware of the logical hierarchy, so they could not employ logical reasoning. They were then tested with three problems: B vs D, B vs. E, and C vs. E. Participants only reliably chose B over E, whereas the other test conditions yielded chance performance. This result is inconsistent with the use of logical reasoning, and is instead consistent with an account developed to explain earlier TI studies with rats that found the same pattern of results. In this account, choice performance is based on differential associative strengths across the stimulus items that develop over training, despite equal overt reinforcement.
[PDF] Rudy, J.W., Biedenkapp, J.C. & O'Reilly, R.C. (2005). Prefrontal Cortex and the Organization of Recent and Remote Memories: An Alternative View. Learning and Memory, 12, 445-446.
Abstract:
Commentary suggesting that interactions between the prefrontal cortex/anterior cingulate and hippocampus may be more about cognitive control needed to recall weak memories from the hippocampus, than about consolidation of memories from hippocampus to prefrontal cortex and anterior cingulate cortex.
[Main PDF] [Supp PDF] Rougier, N.P., Noelle, D., Braver, T.S., Cohen, J.D. & O'Reilly, R.C. (2005). Prefrontal Cortex and the Flexibility of Cognitive Control: Rules Without Symbols. Proceedings of the National Academy of Sciences, 102, 7338-7343.
Abstract:
Human cognitive control is uniquely flexible, and has been shown to depend on prefrontal cortex (PFC). But exactly how the biological mechanisms of the PFC support flexible cognitive control remains a profound mystery. Existing theoretical models have posited powerful task-specific PFC representations, but not how these develop. We show how this can occur when a set of PFC-specific neural mechanisms interact with breadth of experience to self-organize abstract, rule-like PFC representations that support flexible generalization in novel tasks. The same model is shown to apply to benchmark PFC tasks (Stroop and Wisconsin card sorting), accurately simulating the behavior of neurologically intact and frontally-damaged people.
[Main PDF] [Supp PDF] Frank, M.J., Seeberger, L. & O'Reilly, R.C. (2004). By carrot or by stick: Cognitive reinforcement learning in Parkinsonism. Science, 306, 1940-1943.
Abstract:
To what extent do we learn from the positive versus negative outcomes of our decisions? The neuromodulator dopamine plays a key role in these reinforcement learning processes. Patients with Parkinson's disease, who have depleted dopamine in the basal ganglia, are impaired in tasks that require learning from trial and error. Here we show, using two cognitive procedural learning tasks, that Parkinson's patients off medication are better at learning to avoid choices that lead to negative outcomes than they are at learning from positive outcomes. Dopamine medication reverses this bias, making patients more sensitive to positive than negative outcomes. This pattern was predicted by our biologically-based computational model of basal ganglia/dopamine interactions in cognition, which has separate pathways for ``Go'' and ``NoGo'' responses that are differentially modulated by positive and negative reinforcement.
[PDF] Atallah, H.E., Frank, M.J. & O'Reilly, R.C. (2004). Hippocampus, cortex and basal ganglia: Insights from computational models of complementary learning systems. Neurobiology of Learning and Memory, 82/3, 253-67.
Abstract:
We present a framework for understanding how the hippocampus, neocortex, and basal ganglia work together to support cognitive and behavioral function in the mammalian brain. This framework is based on computational tradeoffs that arise in neural network models, where achieving one type of learning function requires very different parameters from those necessary to achieve another form of learning. For example, we dissociate the hippocampus from cortex with respect to general levels of activity, learning rate, and level of overlap between activation patterns. Similarly, the frontal cortex and associated basal ganglia system have important neural specializations not required of the posterior cortex system. Taken together, this overall cognitive architecture, which has been implemented in functioning computational models, provides a rich and often subtle means of explaining a wide range of behavioral and cognitive neuroscience data. Here, we summarize recent results in the domains of recognition memory, contextual fear conditioning, effects of basal ganglia lesions on stimulus-response and place learning, and flexible responding.
[PDF] Munakata, Y. & O'Reilly, R.C. (2003). Developmental and Computational Neuroscience Approaches to Cognition: The Case of Generalization. Cognitive Studies, 10, 76-92.
Abstract:
The ability to generalize --- to abstract regularities from our experiences that can be applied to new experiences --- is fundamental to human cognition and our abilities to flexibly adapt to changing situations. However, the generalization abilities of children and adults are far from perfect, with many clear demonstrations of failures to generalize in situations that would otherwise appear to lend themselves to generalization. It seems that people require extensive experience with a domain to demonstrate good generalization, and that their generalization abilities are best when dealing with relatively concrete, familiar situations. In this paper, we argue that people's successes and failures in generalization are well characterized by neural network models. Networks of neurons connected by synaptic weights are naturally predisposed to encode information in a highly specific fashion, which does not support generalization (as has been seized upon by critics of such models). However, with sufficient experience and appropriate architectural properties, such models can develop abstract representations that support good generalization. Implications for the neural bases and development of generalization abilities are discussed.
[PDF] Jilk, D.J., Cer, D.M. & O'Reilly, R.C. (2003). Learning Rules Generated by a Biophysical Model of Synaptic Plasticity. Computational Neuroscience Conference, 2003,
Abstract:
We describe our initial attempts to reconcile powerful neural network learning rules derived from computational principles with learning rules derived bottom-up from biophysical mechanisms. Using a biophysical model of synaptic plasticity (Shouval, Bear, and Cooper, 2002), we generated numerical synaptic learning rules and compared them to the performance of a Hebbian learning rule in a previously studied neural network model of self-organized learning. In general, the biophysically derived learning rules did not perform as well as the analytic rule, but their performance could be improved by adjusting various aspects of the biophysical model. These results show that some progress has been made in integrating our understanding of biological and artificial neural networks.
[Main PDF] [Supp PDF] Norman, K.A. & O'Reilly, R.C. (2003). Modeling Hippocampal and Neocortical Contributions to Recognition Memory: A Complementary Learning Systems Approach. Psychological Review, 110, 611-646.
Abstract:
We present a computational neural network model of how hippocampus and medial temporal lobe cortex (MTLC) contribute to recognition memory. The hippocampal component contributes by recalling studied details. The MTLC component can not support recall, but one can extract a scalar familiarity signal from MTLC that tracks how well the test item matches studied items. We present simulations that establish key differences in the operating characteristics of the hippocampal recall and MTLC familiarity signals, and we identify several manipulations (e.g., target-lure similarity, interference) that differentially affect the two signals. We also use the model to address the stochastic relationship between recall and familiarity and the effects of partial vs. complete hippocampal lesions on recognition.
[PDF] VanElzakker, M., O'Reilly, R.C. & Rudy, J.W. (2003). Transitivity, Flexibility, Conjunctive Representations and the Hippocampus: I. An Empirical Analysis. Hippocampus, 13, 334-340.
Abstract:
Following training on a set of 4 ordered, simultaneous, odor discrimination problems: A+B-; B+C-; C+D-; D+E-, intact rats display transitivity: When tested on the novel combination BD they choose B. Rats with damage to the hippocampus, however, do not show transitivity (Dusek & Eichenbaum, 1997). These results have been interpreted as support for the idea that the hippocampus is a relational memory storage system that enables the subject to make comparisons among representations of the individual problems and choose based on inferential logic. We provide evidence for a simpler explanation: Specifically, subjects make their choices based on the absolute excitatory value of the individual stimuli. This value determines the ability of that stimulus to attract a response. This conclusion emerged because following training on a 5 problem set A+B-; B+C-; C+D-; D+E-, E+F-, rats preferred B when tested with BE but not when tested with BD. The implication of these results for how to conceptualize the role of the hippocampus in transitive-like phenomena is discussed.
[PDF] Frank, M.J., Rudy, J.W. & O'Reilly, R.C. (2003). Transitivity, Flexibility, Conjunctive Representations and the Hippocampus: II. A Computational Analysis. Hippocampus, 13, 341-354.
Abstract:
Following training on a set of 4 ordered, simultaneous, odor discrimination problems: A+B-; B+C-; C+D-; D+E-, intact rats display transitivity: When tested on the novel combination BD they choose B. Rats with damage to the hippocampus, however, do not show transitivity (Dusek & Eichenbaum, 1997). These results have been interpreted as support for the idea that the hippocampus is a relational memory storage system that enables the subject to make comparisons among representations of the individual problems and choose based on inferential logic. We provide evidence for a simpler explanation: Specifically, subjects make their choices based on the absolute excitatory value of the individual stimuli. This value determines the ability of that stimulus to attract a response. This conclusion emerged because following training on a 5 problem set A+B-; B+C-; C+D-; D+E-, E+F-, rats preferred B when tested with BE but not when tested with BD. The implication of these results for how to conceptualize the role of the hippocampus in transitive-like phenomena is discussed.
[PDF] Huber, D.E. & O'Reilly, R.C. (2003). Persistence and accommodation in short-term priming and other perceptual paradigms: Temporal segregation through synaptic depression. Cognitive Science, 27, 403-430.
Abstract:
In everyday situations, perceptual input constantly changes in an unpredictable fashion. However, our perceptual systems are somewhat sluggish, requiring adequate processing time in order to integrate information. This integration results in persistent activation across stimuli, which can lead to confusion regarding the source of a given activation signal (i.e., source confusion). We propose that activity dependent neural accommodation naturally limits this source confusion by suppressing items once they have been identified. We review behavioral paradigms from different literatures that measure the correlates of persistence and accommodation. Of the various accommodation mechanisms, we focus on synaptic depression for its well-specified dynamic characteristics. We derive a rate-coded version of synaptic depression that can be used to produce accommodating dynamics in any neural network with real valued activation. We implement this expression in a hierarchical model of perception termed, a neural mechanism for responding optimally with unknown sourced of evidence (nROUSE). This model is similar to the ROUSE model of Huber et al. (2001), which produces accommodated levels of feature evidence through an optimal calculation. Important parallels are drawn between the models and nROUSE is applied to three short-term priming experiments that manipulated prime duration.
[PDF] O'Reilly, R.C., Busby, R.S. & Soto, R. (2003). Three Forms of Binding and their Neural Substrates: Alternatives to Temporal Synchrony. A. Cleeremans (Ed) The Unity of Consciousness: Binding, Integration, and Dissociation, 168-192, Oxford: Oxford University Press.
Abstract:
This paper presents three different ways of addressing the binding problem in different brain areas: generic neocortex, hippocampus, and prefrontal cortex. None of these approaches involve the popular mechanism of temporal synchrony. The first two involve conjunctive representations that bind by ensuring that different neural units are activated for different combinations of input features. Specifically, we think the cortex constructs low-order conjunctions using coarse-coded distributed representations to avoid the combinatorial explosion usually associated with conjunctive solutions to the binding problem. We present a model that learns these representations in a challenging relational binding task, and furthermore is capable of considerable generalization to novel inputs. Next, we review the idea that the hippocampus performs conjunctive binding in long term memory through the use of higher-order conjunctions that are much more specific to particular events than those in the cortex. Finally, we present a model of a very different form of binding that involves the phonological loop --- a mechanism for maintaining arbitrary sequences of phonemes in active memory. This phonological system can be used to bind by continuously repeating the to-be-bound information (e.g., ``press left key for green X's,...''). In total, this work suggests that instead of one simple and generic solution to the binding problem, the brain has developed a number of specialized mechanisms that build on the strengths of existing neural hardware in different brain areas.
[PDF] O'Reilly, R.C. & Munakata, Y. (2003). Computational Neuroscience and Cognitive Modeling. L. Nadel (Ed) Encyclopedia of Cognitive Sciences, London: Macmillan.
Abstract:
This article describes computer models that simulate the neural networks of the brain, with the goal of understanding how cognitive functions (perception, memory, thinking, language, etc) arise from their neural basis. Many neural network models have been developed over the years, focused at many different levels of analysis from engineering to relatively low-level biology to cognition. Here, we consider models that try to span the gap between biology and cognition, such that they deal with real cognitive data, using mechanisms that are related to the underlying biology.
[PDF] O'Reilly, R.C. & Norman, K.A. (2002). Hippocampal and Neocortical Contributions to Memory: Advances in the Complementary Learning Systems Framework. Trends in Cognitive Sciences, 6, 505-510.
Abstract:
The complementary learning systems framework provides a simple set of principles, derived from converging biological, psychological, and computational constraints, for understanding the differential contributions of the neocortex and hippocampus to learning and memory. The most central principles are that the neocortex employs a small learning rate and overlapping distributed representations to extract the general statistical structure of the environment, while the hippocampus learns rapidly using separated representations to encode the details of specific events while minimizing interference. In recent years, we and our collaborators have instantiated these principles in working computational models, and we have used these models to address detailed patterns of human and animal learning and memory findings, across a wide range of domains and paradigms. Here, we review a few representative applications of our models, focusing on two domains: recognition memory and animal learning in the fear conditioning paradigm. In both domains, the models have generated novel predictions that have been tested and confirmed.
[PDF] Rudy, J.W., Barrientos, R.M. & OReilly, R.C. (2002). Hippocampal Formation Supports Conditioning to Memory of a Context. Behavioral Neuroscience, 116, 530-538.
Abstract:
It has been proposed that contextual fear conditioning depends on two processes: (a) the construction of a conjunctive representation of the features that make up the context, and (b) associating that representation with shock. Support for this view comes from studies indicating that prior exposure to the conditioning context facilitates contextual fear conditioning supported by immediate shock. An implication of this result is that conditioning produced by immediate shock is to the memory representation of the preexposed context, which is activated by retrieval cues associated with the preexposed context. Our experiments support this interpretation and indicate that this process depends on an intact hippocampal formation. These results support the hypothesis that the hippocampal formation supports contextual fear conditioning by storing a conjunctive representation of context.
[PDF] Barrientos, R.M., OReilly, R.C. & Rudy, J.W. (2002). Memory for context is impaired by injecting anisomycin into dorsal hippocampus following context exploration. Behavioural Brain Research, 134, 299-306.
Abstract:
Pre-exposure to the context facilitates the small amount of contextual fear conditioning that is normally produced by immediate shock. This context pre-exposure facilitation effect provides a convenient way to study the rat's learning about context. We recently reported that anterograde damage to dorsal hippocampus prevents this facilitation. The present experiments strengthen this conclusion by showing that the protein synthesis inhibitor, anisomycin, injected bilaterally into the dorsal hippocampus following context pre-exposure also significantly reduces the facilitation effect. The same treatment given immediately after immediate shock, however, had no effect on facilitation. These results support theories that assume that, (a) contextual fear involves two processes, acquiring and storing a conjunctive representation of a context and associating that representation with fear; and (b) the hippocampus contributes to contextual fear by participating in the storage of the memory representation of the context.
[PDF] Holdstock, J.S., Mayes, A.R., Roberts, N., Cezayirli, E., Isaac, C.L., O'Reilly, R.C. & Norman, K.A. (2002). Under What Conditions is Recognition Spared Relative to Recall After Selective Hippocampal Damage in Humans?. Hippocampus, 12, 341-351.
Abstract:
The claim that recognition memory is spared relative to recall after focal hippocampal damage has been disputed in the literature. We examined this claim by investigating object and object location recall and recognition memory in a patient, YR, who has adult-onset selective hippocampal damage. Our aim was to identify the conditions under which recognition was spared relative to recall in this patient. She showed unimpaired forced-choice objectre cognition but clearly impaired recall, even when her control subjects found the object recognition task to be numerically harder than the object recall task. However, on two other recognition tests, YR's performance was not relatively spared. First, she was clearly impaired at an equivalently difficult yes/no object recognition task, but only when targets and foils were very similar. Second, YR was clearly impaired at forced-choice recognition of object location associations. This impairment was also unrelated to difficulty because this task was no more difficult than the forced-choice object recognition task for control subjects. The clear impairment of yes/no, but not of forced-choice, object recognition after focal hippocampal damage, when targets and foils are very similar, is predicted by the neural network-based Complementary Learning Systems model of recognition. This model postulates that recognition is mediated by hippocampally dependent recollection and cortically dependent familiarity; thus hippocampal damage should not impair item familiarity. The model postulates that familiarity is ineffective when very similar targets and foils are shown one at a time and subjects have to identify which items are old (yes/no recognition). In contrast, familiarity is effective in discriminating which of similar targets and foils, seen together, is old (forced-choice recognition). Independent evidence from the remember/know procedure also indicates that YR's familiarity is normal. The Complementary Learning Systems model can also accommodate the clear impairment of forced-choice object location recognition memory if it incorporates the view that the most complete convergence of spatial and object information, represented in different cortical regions, occurs in the hippocampus.
[PDF] Rougier, N.P. & O`Reilly, R.C. (2002). Learning representations in a gated prefrontal cortex model of dynamic task switching. Cognitive Science, 26, 503-520.
Abstract:
The prefrontal cortex is widely believed to play an important role in facilitating people's ability to switch performance between different tasks. We present a biologically-based computational model of prefrontal cortex (PFC) that explains its role in task switching in terms of the greater flexibility conferred by activation-based working memory representations in PFC, as compared with more slowly adapting weight-based memory mechanisms. Specifically we show that PFC representations can be rapidly updated when a task switches via a dynamic gating mechanism based on a temporal-differences reward-prediction learning mechanism. Unlike prior models of this type, the present model develops all of its internal representations via learning mechanisms as shaped by the demands of continuous periodic task switching. This advance opens up a new domain of research into the interactions between working memory task demands and the representations that develop to meet them. Results on a version of the Wisconsin Card Sorting task are presented for the full model and a number of comparison networks that test the importance of various model features. Furthermore, we show that a lesioned model produces perseverative errors like those seen in frontal patients.
[PDF] O'Reilly, R.C. & Soto, R. (2002). A Model of the Phonological Loop: Generalization and Binding. Advances in Neural Information Processing Systems (NIPS) 14,T.G. Dietterich, S. Becker & Z. Ghahramani (Eds), Cambridge, MA; MIT Press
Abstract:
We present a neural network model that shows how the prefrontal cortex, interacting with the basal ganglia, can maintain a sequence of phonological information in activation-based working memory (i.e., the phonological loop). The primary function of this phonological loop may be to transiently encode arbitrary bindings of information necessary for tasks --- the combinatorial expressive power of language enables very flexible binding of essentially arbitrary pieces of information. Our model takes advantage of the closed-class nature of phonemes, which allows different neural representations of all possible phonemes at each sequential position to be encoded. To make this work, we suggest that the basal ganglia provide a region-specific update signal that allocates phonemes to the appropriate sequential coding slot. To demonstrate that flexible, arbitrary binding of novel sequences can be supported by this mechanism, we show that the model can generalize to novel sequences after moderate amounts of training.
[PDF] O'Reilly, R.C. & Busby, R.S. (2002). Generalizable Relational Binding from Coarse-coded Distributed Representations. Advances in Neural Information Processing Systems (NIPS) 14,T.G. Dietterich, S. Becker & Z. Ghahramani (Eds), Cambridge, MA; MIT Press
Abstract:
We present a model of binding of relationship information in a spatial domain (e.g., square above triangle) that uses low-order coarse-coded conjunctive representations instead of more popular temporal synchrony mechanisms. Supporters of temporal synchrony argue that conjunctive representations lack both efficiency (i.e., combinatorial numbers of units are required) and systematicity (i.e., the resulting representations are overly specific and thus do not support generalization to novel exemplars). To counter these claims, we show that our model: a) uses far fewer hidden units than the number of conjunctions represented, by using coarse-coded, distributed representations where each unit has a broad tuning curve through high-dimensional conjunction space, and b) is capable of considerable generalization to novel inputs.
[PDF] O'Reilly, R.C. & Munakata, Y. (2002). Psychological Function in Computational Models of Neural Networks. M. Gallagher & R. Nelson (Eds) Handbook of Psychology, Vol 3, Biological Psychology, New York: Wiley.
Abstract:
The overarching goal of cognitive neuroscience is to understand how the brain gives rise to thought. Toward this goal, researchers employ various methods to measure neural variables while people and other animals think. A complementary method, computer models of neural networks, allows unparalleled levels of control and supports the further understanding of the relation between brain and mind. Using these models, one can simulate a network of interacting neurons, and measure cognitive function in these networks at the same time. Furthermore, many variables in these networks can be manipulated, so that their effects on cognitive processes can be observed. In this chapter, we provide an up-to-date review of some of the core principles and prominent applications of computational models in cognitive neuroscience, based on our recent textbook on this topic (O'Reilly & Munakata, 2000). We begin with a summary of some of the basic questions confronting computational modelers in cognitive neuroscience. We then discuss provisional answers to these questions, showing how they apply to a range of empirical data. Throughout, and in closing, we discuss challenges to neural network models. We will see how some network models can have possibly problematic properties, often driven by constraints from biology or cognition, but the models can nonetheless help to advance the field of cognitive neuroscience.
[PDF] O'Reilly, R.C., Noelle, D., Braver, T.S. & Cohen, J.D. (2002). Prefrontal cortex and dynamic categorization tasks: Representational organization and neuromodulatory control. Cerebral Cortex, 12, 246-257.
Abstract:
We present a computational model of the intradimensional / extradimensional (ID/ED) task (a variant of the Wisconsin card sorting task) that simulates the performance of intact and frontally-lesioned monkeys on three different kinds of rule changes (Dias, Robbins, & Roberts, 1997). Although Dias et al. interpret the lesion data as supporting a model in which prefrontal cortex is organized into different processing functions, our model suggests an alternative account based on representational content. A key aspect of the model is that prefrontal cortex representations are organized according to different levels of abstraction, with orbital areas encoding more specific featural information and dorsolateral areas encoding more abstract dimensional information. This representational scheme of the model is integrated with two additional key elements: (a) activation-based working memory representations controlled by a dynamic gating mechanism that simulates the hypothesized phasic actions of dopaminergic neuromodulation in prefrontal cortex, which acts to stabilize or destabilize frontal representations based on success in the task; and (b) a weight-based associative learning system simulating posterior cortex and other subcortical areas, where the stimulus-response mappings are encoded. Frontal cortex contributes to the task via top-down activation-based biasing of task-appropriate features and dimensions in this posterior cortex system this top-down biasing is specifically important for overcoming prepotent associations after a sorting rule reverses. The ability of the model to capture the double-dissociation observed by Dias et al. with orbital versus dorsolateral lesions supports the validity of these principles, many of which have also been useful in accounting for other frontal phenomena.
[PDF] O'Reilly, R.C. & McClelland, J.L. (2001). The Importance of Modeling for the Future of Molecular Studies of Learning and Memory. A.J. Silva (Ed) Molecular Studies of Learning and Memory, : Unpublished.
Abstract:
Computational models are important for guiding and interpreting molecular studies of learning and memory because they provide a bridge between biological and behavioral levels of analysis. These models facilitate the identification of central underlying principles that span different levels, and they can accommodate the many complexities of each of these levels and their interrelationships. We present an overviewof our computational approach towards understanding the different contributions of the neocortex and hippocampus in learning and memory. The approach is based on a set of principles derived from converging biological, psychological, and computational constraints. This framework provides insight into: (a) which behavioral paradigms are most appropriate for dissociating cortical and hippocampal learning contributions; (b) effects of selective impairments on hippocampal areas; (c) effects of synaptic modification (LTP/LTD) impairments on various pathways within the hippocampal system; (d) effects of manipulations of hippocampal activation parameters; and (e) effects of manipulations that selectively impair different components of learning (Hebbian versus error-driven). We hope that these insights provide fruitful avenues for further research using molecular techniques to inform our understanding of exactly how learning and memory phenomena emerge from their biological basis.
[PDF] Rudy, J.W. & O'Reilly, R.C. (2001). Conjunctive Representations, the Hippocampus, and Contextual Fear Conditioning. Cognitive, Affective, and Behavioral Neuroscience, 1, 66-82.
Abstract:
The context in which events occur can be represented either as (a) a set of independent features, the feature representation view, or (b) the features can be bound into a unitary representation of their co-occurrence, the conjunction representation view. Extrahippocampal (e.g., neocortical) areas provide a basis for feature representations, but the hippocampus makes an essential contribution to the automatic storage of conjunctive representations. We develop this dual representation view and explore its implications for hippocampal contributions to contextual fear conditioning processes. To this end, we discuss how our framework can resolve some of the conflicts in the recent literature relating the hippocampus to contextual fear conditioning. We also present new data supporting the role of a key mechanism afforded by conjunctive representations, pattern completion (the ability of a subset of a memory pattern to activate the complete memory), in contextual fear conditioning. As implied by this mechanism, we report that fear can be conditioned to the memory representation of a context that is not actually present at the time of shock. Moreover, this result is predicted by our computational model of cortical and hippocampal function. We suggest that pattern completion demonstrated in animals and by our model provides a mechanistic bridge to human declarative memory.
[PDF] O'Reilly, R.C. (2001). Generalization in Interactive Networks: The Benefits of Inhibitory Competition and Hebbian Learning. Neural Computation, 13, 1199-1242.
Abstract:
Computational models in cognitive neuroscience should ideally use biological properties and powerful computational principles to produce behavior consistent with psychological findings. Error-driven backpropagation is computationally powerful, and has proven useful for modeling a range of psychological data, but is not biologically plausible. Several approaches to implementing backpropagation in a biologically plausible fashion converge on the idea of using bidirectional activation propagation in interactive networks to convey error signals. This paper demonstrates two main points about these error-driven interactive networks: (a) they generalize poorly due to attractor dynamics that interfere with the network's ability to systematically produce novel combinatorial representations in response to novel inputs; and (b) this generalization problem can be remedied by adding two widely used mechanistic principles, inhibitory competition and Hebbian learning, that can be independently motivated for a variety of biological, psychological and computational reasons. Simulations using the Leabra algorithm, which combines the generalized recirculation (GeneRec) biologically-plausible error-driven learning algorithm with inhibitory competition and Hebbian learning, show that these mechanisms can result in good generalization in interactive networks. These results support the general conclusion that cognitive neuroscience models that incorporate the core mechanistic principles of interactivity, inhibitory competition, and error-driven and Hebbian learning satisfy a wider range of biological, psychological and computational constraints than models employing a subset of these principles.
[PDF] Frank, M.J., Loughry, B. & O'Reilly, R.C. (2001). Interactions between the frontal cortex and basal ganglia in working memory: A computational model. Cognitive, Affective, and Behavioral Neuroscience, 1, 137-160.
Abstract:
The frontal cortex and basal ganglia interact via a relatively well-understood and elaborate system of interconnections. In the context of motor function, these interconnections can be understood as disinhibiting or ``releasing the brakes'' on frontal motor action plans --- the basal ganglia detect appropriate contexts for performing motor actions, and enable the frontal cortex to execute such actions at the appropriate time. We build on this idea in the domain of working memory through the use of computational neural network models of this circuit. In our model, the frontal cortex exhibits robust active maintenance, while the basal ganglia contribute a selective, dynamic gating function that enables frontal memory representations to be rapidly updated in a task-relevant manner. We apply the model to a novel version of the continuous performance task (CPT) that requires subroutine-like selective working memory updating, and compare and contrast our model with other existing models and theories of frontal cortex--basal ganglia interactions.
[PDF] O'Reilly, R.C. & Rudy, J.W. (2001). Conjunctive Representations in Learning and Memory: Principles of Cortical and Hippocampal Function. Psychological Review, 108, 311-345.
Abstract:
We present a theoretical framework for understanding the roles of the hippocampus and neocortex in learning and memory. This framework incorporates a theme found in many theories of hippocampal function, that the hippocampus is responsible for developing conjunctive representations binding together stimulus elements into a unitary representation that can later be recalled from partial input cues. This idea appears problematic, however, because it is contradicted by the fact that hippocampally lesioned rats can learn nonlinear discrimination problems that require conjunctive representations. Our framework accommodates this finding by establishing a principled division of labor between the cortex and hippocampus, where the cortex is responsible for slow learning that integrates over multiple experiences to extract generalities, while the hippocampus performs rapid learning of the arbitrary contents of individual experiences. This framework shows that nonlinear discrimination problems are not good tests of hippocampal function, and suggests that tasks involving rapid, incidental conjunctive learning are better. We implement this framework in a computational neural network model, and show that it can account for a wide range of data in animal learning, thus validating our theoretical ideas, and providing a number of insights and predictions about these learning phenomena.
[PDF] Munakata, Y., Santos, L.R., Spelke, E.S., Hauser, M.D. & O'Reilly, R.C. (2001). Visual representation in the wild: How rhesus monkeys parse objects. Journal of Cognitive Neuroscience, 13, 44-58.
Abstract:
Visual object representation was studied in free-ranging rhesus monkeys. To facilitate comparison with humans, and to provide a new tool for neurophysiologists, we used a looking time procedure originally developed for studies of human infants. Monkeys looking times were measured to displays with one or two distinct objects, separated or together, stationary or moving. Results indicate that rhesus monkeys used featural information to parse the displays into distinct objects, and they found events in which distinct objects moved together more novel or unnatural than events in which distinct objects moved separately. These findings show both commonalities and contrasts with those obtained from human infants. We discuss their implications for the development and neural mechanisms of higher-level vision.
[PDF] O'Reilly, R.C. & Rudy, J.W. (2000). Computational Principles of Learning in the Neocortex and Hippocampus. Hippocampus, 10, 389-397.
Abstract:
We present an overview of our computational approach towards understanding the different contributions of the neocortex and hippocampus in learning and memory. The approach is based on a set of principles derived from converging biological, psychological, and computational constraints. The most central principles are that the neocortex employs a slow learning rate and overlapping distributed representations to extract the general statistical structure of the environment, while the hippocampus learns rapidly using separated representations to encode the details of specific events while suffering minimal interference. Additional principles concern the nature of learning (error-driven and Hebbian), and recall of information via pattern completion. We summarize the results of applying these principles to a wide range of phenomena in conditioning, habituation, contextual learning, recognition memory, recall, and retrograde amnesia, and point to directions of current development.
[Main PDF] [Supp PDF] Norman, K.A., O'Reilly, R.C. & Huber, D.E. (2000). Modeling Neocortical Contributions to Recognition Memory. The Cognitive Neuroscience Meeting, 2000,
Abstract:
In this poster, we present a biologically-based dual-process model of recognition memory. Dual-process models posit that recognition judgments are based on: - recollection of specific details, and - nonspecific feelings of familiarity Recollection depends on the hippocampus. Recent data suggest that medial temporal neocortical regions (MTLC) play an important role in supporting familiarity-based recognition (for a review, see Aggleton & Brown, 1999). We seek to understand, in mechanistic detail, how MTLC and the hippocampus contribute to recognition memory, by constructing neural network models of these structures, and using them to simulate recognition data from lesioned and intact subjects.
[PDF] Vecera, S.P. & O'Reilly, R.C. (2000). Graded Effects in Hierarchical Figure-Ground Organization: Reply To Peterson (1999). Journal of Experimental Psychology: Human Perception and Performance, 26, 1221-1230.
Abstract:
An important issue in vision research concerns the order of visual processing. S. P. Vecera and R. C. O'Reilly (1998) presented an interactive, hierarchical model that placed figure-ground segregation prior to object recognition. M. A. Peterson (1999) critiqued this model, arguing that because it used ambiguous stimulus displays, figure-ground processing did not precede object processing. In the current article, the authors respond to Peterson's (1999) interpretation of ambiguity in the model and her interpretation of what it means for figure-ground processing to come before object recognition. The authors argue that complete stimulus ambiguity is not critical to the model and that figure-ground precedes object recognition architecturally in the model. The arguments are supported with additional simulation results and an experiment, demonstrating that top-down inputs can influence figure-ground organization in displays that contain stimulus cues.
[PDF] Rudy, J.W. & O'Reilly, R.C. (1999). Contextual Fear Conditioning, Conjunctive Representations, Pattern Completion, and the Hippocampus. Behavioral Neuroscience, 113, 867-880.
Abstract:
Impaired contextual fear conditioning produced by damage to the hippocampus has been attributed to the loss of a conjunctive representation of the features of the context. There is, however, no direct evidence that conjunctive representations contribute to contextual fear conditioning. Our experiments addressed this issue and support the conjunctive representation view. Two results make this point: (a) preexposure to the conditioning context, but not to itsseparable features, facilitates contextual fear conditioning, and (b) generalization of fear conditioning to similar contexts is enhanced by preexposure to the context used to test for generalization --- we interpret this as pattern completion to the preexposed context during the conditioning episode. These results support the view that a conjunctive representation of context plays an important role in contextual fear conditioning, and that the impairments produced by damage to the hippocampus result from the loss of this conjunctive contribution.
[PDF] O'Reilly, R.C., Braver, T.S. & Cohen, J.D. (1999). A Biologically Based Computational Model of Working Memory. A. Miyake & P. Shah (Eds) Models of Working Memory: Mechanisms of Active Maintenance and Executive Control., 375-411, New York: Cambridge University Press.
Abstract:
We define working memory as controlled processing involving active maintenance and/or rapid learning, where controlled processing is an emergent property of the dynamic interactions of multiple brain systems, but the prefrontal cortex (PFC) and hippocampus (HCMP) are especially influential due to their specialized processing abilities and their privileged locations within the processing hierarchy (both the PFC and HCMP are well connected with a wide range of brain areas, allowing them to influence behavior at a global level). The specific features of our model include: 1. A PFC specialized for active maintenance of internal contextual information that is dynamically updated and self-regulated, allowing it to bias (control) ongoing processing according to maintained information (e.g., goals, instructions, partial products, etc). 2. A HCMP specialized for rapid learning of arbitrary information, which can be recalled in the service of controlled processing, while the posterior perceptual and motor cortex (PMC) exhibits slow, long-term learning that can efficiently represent accumulated knowledge and skills. 3. Control that emerges from interacting systems (PFC, HCMP and PMC). 4. Dimensions that define continua of specialization in different brain systems: e.g., robust active maintenance, fast vs slow learning. 5. Integration of biological and computational principles.
[PDF] O'Reilly, R.C., Mozer, M., Munakata, Y. & Miyake, A. (1999). Discrete Representations in Working Memory: A Hypothesis and Computational Investigations. The Second International Conference on Cognitive Science,183-188, Tokyo; Japanese Cognitive Science Society
Abstract:
We present a novel hypothesis concerning the nature and development of working memory representations, and some initial computational investigations of this hypothesis. Working memory refers to the active maintenance of information in the service of complex cognition, such as language comprehension, spatial thinking, and problem solving (Miyake & Shah, 1999). We propose that the unique demands placed on the working memory system shape its representations over learning and development, affecting the use of working memory by the cognitive system as a whole. Our primary source of insight into this process comes from a computational analysis, which is used to integrate and explore relevant findings from neurobiology as well as developmental and adult cognition.
[PDF] O'Reilly, R.C. & Farah, M.J. (1999). Simulation and Explanation in Neuropsychology and Beyond. Cognitive Neuropsychology, 16, 49-72.
Abstract:
This paper is a reply to a critique by Young & Burton (same issue) of our earlier prosopagnosia model (Farah, O'Reilly & Vecera, 1993, Psychological Review). We provide a number of new extensions to our original model, and address general issues in the debate between distributed, learning-based models and localist, hand-wired models.
[PDF] O'Reilly, R.C. (1998). Six Principles for Biologically-Based Computational Models of Cortical Cognition. Trends in Cognitive Sciences, 2, 455-462.
Abstract:
This paper describes and motivates six principles for computational cognitive neuroscience models: biological realism, distributed representations, inhibitory competition, bidirectional activation propagation, error-driven task learning, and Hebbian model learning. Although these principles are supported by a number of cognitive, computational, and biological motivations, the prototypical neural network model (a feedforward backpropagation network) incorporates only two of them, and no widely used model incorporates all of them. This paper argues that these principles should be integrated into a coherent overall framework, and discusses some potential synergies and conflicts in doing so.
[PDF] O'Reilly, R.C., Norman, K.A. & McClelland, J.L. (1998). A Hippocampal Model of Recognition Memory. M.I. Jordan, M.J. Kearns & S.A. Solla (Eds) Advances in Neural Information Processing Systems 10, 73-79, Cambridge, MA: MIT Press.
Abstract:
A rich body of data exists showing that recollection of specific information contributes to recognition memory. We present a model, based largely on known features of hippocampal anatomy and physiology, that accounts for both the {\em high-threshold} nature of this recollection process (i.e., the fact that only studied items are recollected, although nonstudied items sometimes trigger recollection of similar studied items), and the fact that increasing interference leads to less recollection but apparently does not compromise the {\em quality} of recollection (i.e., the extent to which recollection veridically reflects events that occurred at study). Both of these properties of recollection pose problems for existing computational and mathematical models.
[PDF] Vecera, S.P. & O'Reilly, R.C. (1998). Figure-Ground Organization and Object Recognition Processes: An Interactive Account. Journal of Experimental Psychology: Human Perception and Performance, 24, 441-462.
Abstract:
Traditional theories of visual processing have assumed that figure-ground organization must precede object representation and identification. Such a view seems logically necessary: How can one recognize an object before the visual system knows which region should be the figure? However, a number of behavioral studies have shown that subjects are more likely to call a familiar region ``figure'' relative to a less familiar region, a finding inconsistent with the traditional accounts of visual processing. To explain these results, Peterson and colleagues have proposed an additional ``prefigural'' object recognition process that operates before any figure-ground organization (M. A. Peterson, 1994). We propose a more parsimonious interactive account of figure-ground organization in which partial results of figure-ground processes interact with object representations in a hierarchical system similar to that envisioned by traditional theories. We present a computational model that embodies this graded, interactive approach and show that this model can account several behavioral results, including orientation effects, exposure duration effects, and the combination of multiple cues. Finally, these principles of graded, interactive processing offer the possibility of providing a more general information processing framework for visual and higher-cognitive systems.
[PDF] Cohen, J.D. & O'Reilly, R.C. (1996). A Preliminary Theory of the Interactions Between Prefrontal Cortex and Hippocampus that Contribute to Planning and Prospective Memory. M. Brandimonte, G.O. Einstein & M.A. McDaniel (Eds) Prospective Memory: Theory and Applications, 267-296, Mahwah, New Jersey: Lawrence Erlbaum Associates.
Abstract:
This chapter addresses the neurobiological mechanisms that may underlie prospective memory. In our work, we have exploited the use of computational modeling techniques to help identify the role that specific brain structures play in cognition. In particular, we have used such techniques to characterize the function of prefrontal cortex and hippocampus in terms of specific processing mechanisms. This work suggests that an important function of prefrontal cortex is the representation and maintenance of contextual information -- information that must be held in mind in such a form that it can be used to mediate an appropriate behavioral response. At the same time, our work supports the idea that an important function of the hippocampus is to rapidly establish novel associations, that can also be used to guide behavior. In our view, prospective memory reflects the interaction between these two systems, allowing established sequences of behavior to be associated with new conditions -- in effect, providing a mechanism for planning.
[PDF] Cohen, J.D., Braver, T.S. & O'Reilly, R.C. (1996). A Computational Approach to Prefrontal Cortex, Cognitive Control, and Schizophrenia: Recent Developments and Current Challenges. Philosophical Transactions of the Royal Society (London) {B}, 351, 1515-1527.
Abstract:
In this chapter we consider the mechanisms involved in cognitive control --- from botha computational and a neurobiological perspective --- and how these might be impaired in schizophrenia. By 'control' we mean the ability of the cognitive system to flexibly adapt its behaviour to the demands of particular tasks, favouring the processing of task-relevant information over other sources of competing information, and mediating task-relevant behaviour over habitual, or otherwise prepotent responses. There is a large body of evidence to suggest that the prefrontal cortex (PFC) plays a critical role in cognitive control. In previous work, we have used a computational framework to understand and develop explicit models of this function of PFC, and its impairment in schizophrenia. This work has lead to the hypothesis that PFC houses a mechanism for representing and maintainging context information. We have demonstrated that this mechanism can account for the behavioural inhibition and active memory functions commonly ascribed to PFC, and for human performance in simple attention, language and memory tasks that draw upon these functions for cognitive control. Furthermore, we have used our models to simulate detailed patterns of cognitive deficit observed in schizophrenia, an illness associated with marked distrubances in cognitive control, and well established deficits of PFC. Here, we review results of recent empirical studies that test predictions made by our models regarding schizophrenic performance in tasks designed specifically to probe the processing of context. These results showed selective schizophrenic deficits in task conditions that placed the greatest demands on memory and inhibition, both of which we have argued rely on the processing of context. Furthermore, we observed predicted patterns of deterioration in first episode vs multi-episode patients. We also discuss recent developments in our computational work, that have led to refinements of the models that allow us to simulate more detailed aspects of task performance, such as reaction time data and manipulations of task parameters such as interstimulus delay. These refined models make several provocative new predictions, including conditions in which schizophrenics and control subjects are expected to show similar reaction time performance, and we provide preliminary data in support of these predictions. These successes notwhithstanding, our theory of PFC function and its impairment in schizophrenia is still in an early stage of development. We conclude by presenting some of the challenges to the theory in its current form, and new directions that we have begun to take to meet these challenges. In particular, we focus on refinements concerning the mechanisms underlying active maintenance of representations within PFC, and the characteristics of these representations that allow them to support the flexibility of cognitive control exhibited by normal human behaviour. Taken in toto, we believe that this work illustrates the value of a computational approach for understanding the mechanisms responsible for cognitive control, at both the neural and psychological levels, and the specific manner in which they break down in schizophrenia.
[PDF] O'Reilly, R.C. (1996). Biologically Plausible Error-driven Learning using Local Activation Differences: The Generalized Recirculation Algorithm. Neural Computation, 8, 895-938.
Abstract:
The error backpropagation learning algorithm (BP) is generally considered biologically implausible because it does not use locally available, activation-based variables. A version of BP that can be computed locally using bi-directional activation recirculation (Hinton & McClelland, 1988) instead of backpropagated error derivatives is more biologically plausible. This paper presents a generalized version of the recirculation algorithm (GeneRec), which overcomes several limitations of the earlier algorithm by using a generic recurrent network with sigmoidal units that can learn arbitrary input/output mappings. However, the contrastive-Hebbian learning algorithm (CHL), a.k.a. DBM or mean field learning) also uses local variables to perform error-driven learning in a sigmoidal recurrent network. CHL was derived in a stochastic framework (the Boltzmann machine), but has been extended to the deterministic case in various ways, all of which rely on problematic approximations and assumptions, leading some to conclude that it is fundamentally flawed. This paper shows that CHL can be derived instead from within the BP framework via the GeneRec algorithm. CHL is a symmetry-preserving version of GeneRec which uses a simple approximation to the midpoint or second-order accurate Runge-Kutta method of numerical integration, which explains the generally faster learning speed of CHL compared to BP. Thus, all known fully general error-driven learning algorithms that use local activation-based variables in deterministic networks can be considered variations of the GeneRec algorithm (and indirectly, of the backpropagation algorithm). GeneRec therefore provides a promising framework for thinking about how the brain might perform error-driven learning. To further this goal, an explicit biological mechanism is proposed which would be capable of implementing GeneRec-style learning. This mechanism is consistent with available evidence regarding synaptic modification in neurons in the neocortex and hippocampus, and makes further predictions.
[PDF] O'Reilly, R.C. (1996). The Leabra Model of Neural Interactions and Learning in the Neocortex. Phd Thesis, Carnegie Mellon University, Pittsburgh, PA
Abstract:
There is evidence that the specialized neural processing systems in the neocortex, which are responsible for much of human cognition, arise from the action of a relatively general-purpose learning mechanism. I propose that such a neocortical learning mechanism can be best understood as the combination of error-driven and self-organizing (Hebbian associative) learning. This model of neocortical learning, called LEABRA (local, error-driven and associative, biologically realistic algorithm), is computationally powerful, has important implications for psychological models, and is biologically feasible. The thesis begins with an evaluation of the strengths and limitations of current neural network learning algorithms as models of a neocortical learning mechanism according to psychological, biological, and computational criteria. I argue that error-driven (e.g., backpropagation) learning is a reasonable computational and psychological model, but it is biologically implausible. I show that backpropagation can be implemented in a biologically plausible fashion by using interactive (bi-directional, recurrent) activation flow, which is known to exist in the neocortex, and has been important for accounting for psychological data. However, the interactivity required for biological and psychological plausibility significantly impairs the ability to respond systematically to novel stimuli, making it still a bad psychological model (e.g., for nonword reading). I propose that the neocortex solves this problem by using inhibitory activity regulation and Hebbian associative learning, the computational properties of which have been explored in the context of self-organizing learning models. I show that by introducing these properties into an interactive (biologically plausible) error-driven network, one obtains a model of neocortical learning that: 1) provides a clear computational role for a number of biological features of the neocortex; 2) behaves systematically on novel stimuli, and exhibits transfer to novel tasks; 3) learns rapidly in networks with many hidden layers; 4) provides flexible access to learned knowledge; 5) shows promise in accounting for psychological phenomena such as the U-shaped curve in over-regularization of the past-tense inflection; 6) has a number of other nice properties.
[PDF] McClelland, J.L., McNaughton, B.L. & O'Reilly, R.C. (1995). Why There are Complementary Learning Systems in the Hippocampus and Neocortex: Insights from the Successes and Failures of Connectionist Models of Learning and Memory. Psychological Review, 102, 419-457.
Abstract:
The use of information from prior experience in explicit memory tasks is drastically affected by damage to the hippocampal system. However, if the hippocampus and related structures are left intact for some time after an experience, resistance to later disruption from hippocampal damage develops. This process, called consolidation, can span several years in humans. The data are consistent with the view that the initial memory trace consists of changes to connections among neurons in the hippocampal system; that incoming and outgoing pathways allow these changes to mediate the reinstatement of representations of recent experiences in the neocortex; and that consolidation results from cumulative effects of small changes to neocortical connections that occur each time a representation is reinstated. Two questions arise: 1) Why are plastic changes made initially in the hippocampus, if the ultimate substrate of consolidated memory is in the neocortex? 2) Why does consolidation span an extended time period? Connectionist models of learning and memory provide possible answers to these questions. These models offer procedures for assigning weights to the connections among simple processing units so that a network of such units may capture the structure in ensembles of experiences. The success of these procedures depends on making small weight changes on each learning trial. Rapid acquisition of new data is incompatible with gradual discovery of structure and can lead to catastrophic interference. This suggests that the neocortex may be optimized for discovery of shared structure, and that the hippocampal system provides rapid acquisition of new information without interference with previously discovered regularities. After initial acquisition, the hippocampal system serves as teacher to the rest of the brain: It reinstates representations of events so that they may be gradually learned by the cortical system, interleaved with other experiences. We equate interleaved learning with consolidation and suggest that it is slow to allow new material to become integrated into structured systems of knowledge in the neocortex.
[PDF] O'Reilly, R.C. & McClelland, J.L. (1994). Hippocampal Conjunctive Encoding, Storage, and Recall: Avoiding a Tradeoff. Hippocampus, 4, 661-682.
Abstract:
The hippocampus and related structures are thought to be capable of: 1) representing cortical activity in a way that minimizes overlap of the representations assigned to different cortical patterns (pattern separation); and 2) modifying synaptic connections so that these representations can later be reinstated from partial or noisy versions of the cortical activity pattern that was present at the time of storage (pattern completion). We point out that there is a tradeoff between pattern separation and completion, and propose that the unique anatomical and physiological properties of the hippocampus might serve to minimize this tradeoff. We use analytical methods to determine parameterized models of the hippocampus. These estimates are then used to evaluate the role of various properties and of the hippocampus, such as the activity levels seen in different hippocampal regions, synaptic potentiation and depression, the multi-layer connectivity of the system, and the relatively focused and strong mossy fiber projections. This analysis is focused on the feedforward pathways from the Entorhinal Cortex (EC) to the Dentate Gyrus (DG) and region CA3. Among our results are the following: 1) Hebbian synaptic modification (LTP) facilitates completion but reduces separation, unless the strengths of synapses from inactive presynaptic units to active postsynaptic units are reduced (LTD). 2) Multiple layers, as in EC to DG to CA3, allow the compounding of pattern separation, but not pattern completion. 3) The variance of the input signal carried by the mossy fibers is important for separation, not the raw strength, which may explain why the mossy fiber inputs are few and relatively strong, rather than many and relatively weak like the other hippocampal pathways. 4) The EC projects to CA3 both directly and indirectly via the DG, which suggests that the two-stage pathway may dominate during pattern separation and the one-stage pathway may dominate during completion; methods the hippocampus may use to enhance this effect are discussed.
[PDF] O'Reilly, R.C. & Johnson, M.H. (1994). Object Recognition and Sensitive Periods: A Computational Analysis of Visual Imprinting. Neural Computation, 6, 357-389.
Abstract:
Evidence from a variety of methods suggests that a localized portion of the domestic chick brain, the Intermediate and Medial Hyperstriatum Ventrale (IMHV), is critical for filial imprinting. Data further suggest that IMHV is performing the object recognition component of imprinting, as chicks with IMHV lesions are impaired on other tasks requiring object recognition. We present a neural network model of translation invariant object recognition developed from computational and neurobiological considerations that incorporates some features of the known local circuitry of IMHV. In particular, we propose that the recurrent excitatory and lateral inhibitory circuitry in the model, and observed in IMHV, produces hysteresis on the activation state of the units in the model and the principal excitatory neurons in IMHV. Hysteresis, when combined with a simple Hebbian covariance learning mechanism, has been shown in earlier work to produce translation invariant visual representations. To test the idea that IMHV might be implementing this type of object recognition algorithm, we have used a simple neural network model to simulate a variety of different empirical phenomena associated with the imprinting process. These phenomena include reversibility, sensitive periods, generalization, and temporal contiguity effects observed in behavioral studies of chicks. In addition to supporting the notion that these phenomena, and imprinting itself, result from the IMHV properties captured in the simplified model, the simulations also generate several predictions and clarify apparent contradictions in the behavioral data.
[PDF] O'Reilly, R.C. (1994). Temporally Local Unsupervised Learning: The MaxIn Algorithm for Maximizing Input Information. Proceedings of the 1993 Connectionist Models Summer School,M.C. Mozer, P. Smolensky & A.S. Weigend (Eds), Hillsdale, NJ; Lawrence Erlbaum Associates
Abstract:
There are many appealing aspects of self-organizing learning rules, among them the notion that they are more ``biologically plausible'' than supervised learning algorithms. This plausibility usually derives from the ability to compute the algorithm with variables available locally to the unit or ``neuron'', typically using some variant of a Hebbian learning rule. Ironically, however, the locality in time of the variables that determine the learning is often ignored. This temporal non-locality presents a problem both from a biological and a psychological standpoint. In this paper, I present an alternative objective function for self-organizing algorithms that is local in both space and time, and results in a simple learning rule that can be implemented with properties of neuronal synaptic modification.
[PDF] Farah, M.J., O'Reilly, R.C. & Vecera, S.P. (1993). Dissociated Overt and Covert Recognition as an Emergent Property of a Lesioned Neural Network. Psychological Review, 100, 571-588.
Abstract:
Covert recognition of faces in prosopagnosia, in which patients cannot overtly recognize faces but nevertheless mainfest recognition when tested in certain indirect ways, has been interpreted as the functioning of an intact visual face recognition system deprived of access to other brain systems necessary for consciousness. The authors proposa an alternative hypothesis: that the visual face recognition system is damaged but not obliterated in these patients and that damaged neural networks will mainfest their residual knowledge in just the kinds of tasks used to measure overt recognition. To test this, a simple model of face recognition is lesioned in the parts of the model corresponding to visual processing. The model demonstrates covert recognition in 3 qualitatively different tasks. Implications for the nature of prosopagnosia, and for other types of dissociations between conscious and unconscious perception, are discussed.
[PDF] O'Reilly, R.C. & McClelland, J.L. (1992). The Self-Organization of Spatially Invariant Representations. Technical Report {PDP.CNS.92.5} Carnegie Mellon University, Department of Psychology.
Abstract:
The problem of computing object-based visual representations can be construed as the development of invariancies to visual dimensions irrelevant for object identity. This view, when implemented in a neural network, suggests a different set of algorithms for computing object-based visual representations than the ``traditional'' approach pioneered by Marr (1980). A biologically plausible self-organizing neural network model that develops spatially invariant representations is presented. There are four features of the self-organizing algorithm that contribute to the development of spatially invariant representations: temporal continuity of environmental stimuli, hysteresis of the activation state (via recurrent activation loops and lateral inhibition in an interactive network), Hebbian learning, and a split pathway between ``what'' and ``where'' representations. These constraints are tested with a backprop network, which allows for the evaluation of the individual contributions of each constraint on the development of spatially invariant representations. Subsequently, a complete model embodying a modified Hebbian learning rule and interactive connectivity is developed from biological and computational considerations. The activational stability and weight function maximization properties of this interactive network are analyzed using a Lyapunov function approach. The model is tested first on the same simple stimuli used in the backprop simulation, and then with a more complex environment consisting of right and left diagonal lines. The results indicate that the hypothesized constraints, implemented in a Hebbian network, were capable of producing spatially invariant representations. Further, evidence for the gradual integration of both featural complexity and spatial invariance over increasing layers in the network, thought to be important for real-world applications, was obtained. As the approach is generalizable to other dimensions such as orientation and size, it could provide the basis of a more complete biologically plausible object recognition system. Indeed, this work forms the basis of a recent model of object recognition in the domestic chick (O'Reilly & Johnson, 1994).
[PDF] O'Reilly, R.C., Kosslyn, S.M., Marsolek, C.J. & Chabris, C.F. (1990). Receptive Field Characteristics that Allow Parietal Lobe Neurons to Encode Spatial Properties of Visual Input: A computational investigation. Journal of Cognitive Neuroscience, 2, 141-155.

Back to my home page.