Randall C. O'Reilly's Online Publications

Please visit the NEW wiki for Online Papers: Online Papers

The content on this page has not been updated since 2013

Copyright Notice

The documents distributed here have been provided as a means to ensure timely dissemination of scholarly and technical work on a noncommercial basis. Copyright and all rights therein are maintained by the authors or by other copyright holders, notwithstanding that they have offered their works here electronically. It is understood that all persons copying this information will adhere to the terms and constraints invoked by each author's copyright. These works may not be reposted without the explicit permission of the copyright holder. ( Notice borrowed from Dave Plaut ).

Other papers listed in my Curriculum Vitae may be available by sending an email request to anti-spam email addr img .

Most documents here are available in PDF format.

Last updated: -n 08/22/13 . Back to my home page.

Papers With Abstracts, Listed by Date

[PDF] [Web] O'Reilly, R.C. (2013). Commentary: Individual differences in cognitive flexibility. Biological Psychiatry, 74, 78-79.
[PDF] [Web] Ketz, N., O'Reilly, R.C. & Curran, T. (2013). Classification aided analysis of oscillatory signatures in controlled retrieval. NeuroImage.
Abstract:
Control processes are critical for both facilitating and suppressing memory retrieval, but these processes are not well understood. The current work, inspired by a similar fMRI design (Detre et al., in press), used a modified Think/No-Think(TNT) paradigm to investigate the neural signatures of volition over enhancing and suppressing memory retrieval. Previous studies have shown memory enhancement when well-learned stimulus pairs are restudied in cued recall ("Recall or think of studied pair item"), and degradation when restudied with cued suppression ("Avoid thinking of studied pair item"). We used category-based (faces vs. scenes) multivariate classification of electroencephalography signals to determine if individual target items were successfully retrieved or suppressed. A logistic regression based on classifier output determined that retrieval activation during the cued recall/suppression period was a predictor for subsequent memory. Labeling trials with this internal measure, as opposed to their nominal Think vs. No-Think condition, revealed the classic TNT pattern of enhanced memory for successful cued-retrieval and degraded memory for cued-suppression. This classification process enabled a more selective investigation into the time-frequency signatures of control over retrieval. Comparing controlled retrieval vs. controlled suppression, results showed more prominent Theta oscillations (3 to 8Hz) in controlled retrieval. Beta oscillations (12 to 30Hz) were involved in high levels of both controlled retrieval and suppression, suggesting it may have a more general control-related role. These results suggest unique roles for these frequency bands in retrieval processes.
[PDF] [Web] Herd, S.A., Krueger, K.A., Kriete, T.E., Huang, T. & O'Reilly, R.C. (2013). Strategic Cognitive Sequencing: A Computational Cognitive Neuroscience Approach. Computational Intelligence and Neuroscience, 2013, 149329.
Abstract:
We address strategic cognitive sequencing, the "outer loop" of human cognition: how the brain decides what cognitive process to apply at a given moment to solve complex, multistep cognitive tasks. We argue that this topic has been neglected relative to its importance for systematic reasons but that recent work on how individual brain systems accomplish their computations has set the stage for productively addressing how brain regions coordinate over time to accomplish our most impressive thinking. We present four preliminary neural network models. The first addresses how the prefrontal cortex (PFC) and basal ganglia (BG) cooperate to perform trial-and-error learning of short sequences; the next, how several areas of PFC learn to make predictions of likely reward, and how this contributes to the BG making decisions at the level of strategies. The third models address how PFC, BG, parietal cortex, and hippocampus can work together to memorize sequences of cognitive actions from instruction (or "self-instruction"). The last shows how a constraint satisfaction process can find useful plans. The PFC maintains current and goal states and associates from both of these to find a "bridging" state, an abstract plan. We discuss how these processes could work together to produce strategic cognitive sequencing and discuss future directions in this area.
[PDF] [Web] O'Reilly, R.C., Wyatte, D., Herd, S., Mingus, B. & Jilk, D.J. (2013). Recurrent Processing during Object Recognition. Frontiers in Psychology, 4, 124.
Abstract:
How does the brain learn to recognize objects visually, and perform this difficult feat robustly in the face of many sources of ambiguity and variability? We present a computational model based on the biology of the relevant visual pathways that learns to reliably recognize 100 different object categories in the face of naturally occurring variability in location, rotation, size, and lighting. The model exhibits robustness to highly ambiguous, partially occluded inputs. Both the unified, biologically plausible learning mechanism and the robustness to occlusion derive from the role that recurrent connectivity and recurrent processing mechanisms play in the model. Furthermore, this interaction of recurrent connectivity and learning predicts that high-level visual representations should be shaped by error signals from nearby, associated brain areas over the course of visual learning. Consistent with this prediction, we show how semantic knowledge about object categories changes the nature of their learned visual representations, as well as how this representational shift supports the mapping between perceptual and conceptual knowledge. Altogether, these findings support the potential importance of ongoing recurrent processing throughout the brain's visual system and suggest ways in which object recognition can be understood in terms of interactions within and between processes over time.
[PDF] [Web] Ketz, N., Morkonda, S.G. & O'Reilly, R.C. (2013). Theta coordinated error-driven learning in the hippocampus. PLoS computational biology, 9, e1003067.
Abstract:
The learning mechanism in the hippocampus has almost universally been assumed to be Hebbian in nature, where individual neurons in an engram join together with synaptic weight increases to support facilitated recall of memories later. However, it is also widely known that Hebbian learning mechanisms impose significant capacity constraints, and are generally less computationally powerful than learning mechanisms that take advantage of error signals. We show that the differential phase relationships of hippocampal subfields within the overall theta rhythm enable a powerful form of error-driven learning, which results in significantly greater capacity, as shown in computer simulations. In one phase of the theta cycle, the bidirectional connectivity between CA1 and entorhinal cortex can be trained in an error-driven fashion to learn to effectively encode the cortical inputs in a compact and sparse form over CA1. In a subsequent portion of the theta cycle, the system attempts to recall an existing memory, via the pathway from entorhinal cortex to CA3 and CA1. Finally the full theta cycle completes when a strong target encoding representation of the current input is imposed onto the CA1 via direct projections from entorhinal cortex. The difference between this target encoding and the attempted recall of the same representation on CA1 constitutes an error signal that can drive the learning of CA3 to CA1 synapses. This CA3 to CA1 pathway is critical for enabling full reinstatement of recalled hippocampal memories out in cortex. Taken together, these new learning dynamics enable a much more robust, high-capacity model of hippocampal learning than was available previously under the classical Hebbian model.
[PDF] [Web] Huang, T., Hazy, T.E., Herd, S.A. & O'Reilly, R.C. (2013). Assembling old tricks for new tasks: A neural model of instructional learning and control. Journal of Cognitive Neuroscience, 25, 843-851.
Abstract:
We can learn from the wisdom of others to maximize success. However, it is unclear how humans take advice to flexibly adapt behavior. On the basis of data from neuroanatomy, neurophysiology, and neuroimaging, a biologically plausible model is developed to illustrate the neural mechanisms of learning from instructions. The model consists of two complementary learning pathways. The slow-learning parietal pathway carries out simple or habitual stimulus-response (S-R) mappings, whereas the fast-learning hippocampal pathway implements novel S-R rules. Specifically, the hippocampus can rapidly encode arbitrary S-R associations, and stimulus-cued responses are later recalled into the basal ganglia-gated pFC to bias response selection in the premotor and motor cortices. The interactions between the two model learning pathways explain how instructions can override habits and how automaticity can be achieved through motor consolidation.
[PDF] O'Reilly, R.C., Hazy, T.E. & Herd, S.A. (in press). The Leabra Cognitive Architecture: How to Play 20 Principles with Nature and Win!. S. Chipman (Ed) Oxford Handbook of Cognitive Science, Oxford: Oxford University Press.
Abstract:
This chapter provides a synthetic review of a long-term effort to produce an internally consistent theory of the neural basis of human cognition, the Leabra cognitive architecture, which explains a great deal of brain and behavioral data. In a highly influential commentary, Allen Newell (1973) first issued a call for a more comprehensive, principled approach to studying cognition. ``You can't play 20 questions with nature, and win,'' he said, alluding to the old parlor guessing game involving 20 yes or no questions. His point was that cognition, and the brain that gives rise to it, is just too complex and multidimensional a system to ever hope that a series of narrowly framed experiments and/or models would ever be able to characterize it. Instead, a single cognitive architecture should be used to simulate a wide range of data at many levels in a cumulative manner. However, these cognitive architectures tend to be complex and difficult to fully comprehend. In an attempt to most clearly and simply present the Leabra biologically-based cognitive architecture, we articulate 20 principles that motivate its design, at multiple levels of analysis.
[PDF] [Web] Stocco, A., Lebiere, C., O'Reilly, R.C. & Anderson, J.R. (2012). Distinct contributions of the caudate nucleus, rostral prefrontal cortex, and parietal cortex to the execution of instructed tasks. Cognitive, Affective and Behavioral Neuroscience.
Abstract:
When we behave according to rules and instructions, our brains interpret abstract representations of what to do and transform them into actual behavior. In order to investigate the neural mechanisms behind this process, we devised an fMRI experiment that explicitly isolated rule interpretation from rule encoding and execution. Our results showed that a specific network of regions (including the left rostral prefrontal cortex, the caudate nucleus, and the bilateral posterior parietal cortices) is responsible for translating rules into executable form. An analysis of activation patterns across conditions revealed that the posterior parietal cortices represent a mental template for the task to perform, that the inferior parietal gyrus and the caudate nucleus are responsible for instantiating the template in the proper context, and that the left rostral prefrontal cortex integrates information across complex relationships.
[PDF] [Web] Frank, G.K.W., Reynolds, J.R., Shott, M.E., Jappe, L., Yang, T.T., Tregellas, J.R. & O'Reilly, R.C. (2012). Anorexia nervosa and obesity are associated with opposite brain reward response. Neuropsychopharmacology, 37, 2031-2046.
Abstract:
Anorexia nervosa (AN) is a severe psychiatric disorder associated with food avoidance and malnutrition. In this study, we wanted to test whether we would find brain reward alterations in AN, compared with individuals with normal or increased body weight. We studied 21 underweight, restricting-type AN (age M 22.5, SD 5.8 years), 19 obese (age M 27.1, SD 6.7 years), and 23 healthy control women (age M 24.8, SD 5.6 years), using blood oxygen level-dependent functional magnetic resonance brain imaging together with a reward-conditioning task. This paradigm involves learning the association between conditioned visual stimuli and unconditioned taste stimuli, as well as the unexpected violation of those learned associations. The task has been associated with activation of brain dopamine reward circuits, and it allows the comparison of actual brain response with expected brain activation based on established neuronal models. A group-by-task condition analysis (family-wise-error-corrected P<0.05) indicated that the orbitofrontal cortex differentiated all three groups. The dopamine model reward-learning signal distinguished groups in the anteroventral striatum, insula, and prefrontal cortex (P<0.001, 25 voxel cluster threshold), with brain responses that were greater in the AN group, but lesser in the obese group, compared with controls. These results suggest that brain reward circuits are more responsive to food stimuli in AN, but less responsive in obese women. The mechanism for this association is uncertain, but these brain reward response patterns could be biomarkers for the respective weight state.
[PDF] [Web] O'Reilly, R.C., Bhattacharyya, R., Howard, M.D. & Ketz, N. (2011). Complementary Learning Systems. Cognitive Science Epub ahead of print.
Abstract:
This paper reviews the fate of the central ideas behind the complementary learning systems (CLS) framework as originally articulated in McClelland, McNaughton, and O'Reilly (1995). This framework explains why the brain requires two differentially specialized learning and memory systems, and it nicely specifies their central properties (i.e., the hippocampus as a sparse, pattern-separated system for rapidly learning episodic memories, and the neocortex as a distributed, overlapping system for gradually integrating across episodes to extract latent semantic structure). We review the application of the CLS framework to a range of important topics, including the following: the basic neural processes of hippocampal memory encoding and recall, conjunctive encoding, human recognition memory, consolidation of initial hippocampal learning in cortex, dynamic modulation of encoding versus recall, and the synergistic interactions between hippocampus and neocortex. Overall, the CLS framework remains a vital theoretical force in the field, with the empirical data over the past 15 years generally confirming its key principles.
[PDF] [Web] Pauli, W.M., Hazy, T.E. & O'Reilly, R.C. (2012). Expectancy, ambiguity, and behavioral flexibility: separable and complementary roles of the orbital frontal cortex and amygdala in processing reward expectancies. Journal of Cognitive Neuroscience, 24, 351-366.
Abstract:
Appetitive goal-directed behavior can be associated with a cue-triggered expectancy that it will lead to a particular reward, a process thought to depend on the OFC and basolateral amygdala complex. We developed a biologically informed neural network model of this system to investigate the separable and complementary roles of these areas as the main components of a flexible expectancy system. These areas of interest are part of a neural network with additional subcortical areas, including the central nucleus of amygdala, ventral (limbic) and dorsomedial (associative) striatum. Our simulations are consistent with the view that the amygdala maintains Pavlovian associations through incremental updating of synaptic strength and that the OFC supports flexibility by maintaining an activation-based working memory of the recent reward history. Our model provides a mechanistic explanation for electrophysiological evidence that cue-related firing in OFC neurons is nonselectively early after a contingency change and why this nonselective firing is critical for promoting plasticity in the amygdala. This ambiguous activation results from the simultaneous maintenance of recent outcomes and obsolete Pavlovian contingencies in working memory. Furthermore, at the beginning of reversal, the OFC is critical for supporting responses that are no longer inappropriate. This result is inconsistent with an exclusive inhibitory account of OFC function.
[PDF] [Web] Pauli, W.M., Clark, A.D., Guenther, H., O'Reilly, R.C. & Rudy, J.W. (2012). Inhibiting PKMzeta Reveals Dorsal Lateral and Dorsal Medial Striatum Store the Different Memories Needed to Support Adaptive Behavior. Learning and Memory, 19, 307-14.
Abstract:
Evidence suggests that two regions of the striatum contribute differential support to instrumental response selection. The dorsomedial striatum (DMS) is thought to support expectancy-mediated actions, and the dorsolateral striatum (DLS) is thought to support habits. Currently it is unclear whether these regions store task-relevant information or just coordinate the learning and retention of these solutions by other brain regions. To address this issue, we developed a two-lever concurrent variable-interval reinforcement operant conditioning task and used it to assess the trained rat's sensitivity to contingency shifts. Consistent with the view that these two regions make different contributions to actions and habits, injecting the NMDA antagonist DL-AP5 into the DMS just prior to the shift impaired the rat's performance but enhanced performance when injected into the DLS. To determine if these regions support memory content, we first trained rats on a biased concurrent schedule (Lever 1: VI 40" and Lever 2: VI 10"). With the intent of "erasing" the memory content stored in striatum, after this training we inhibited the putative memory-maintenance protein kinase C isozyme protein kinase M-zeta (PKMzeta). Infusing zeta inhibitory peptide (ZIP) into the DLS enhanced the rat's ability to adapt to the contingency shift 2 d later, whereas injecting it into the DMS had the opposite effect. Infusing GluR2(3Y) into the DMS 1 h before ZIP infusions prevented ZIP from impairing the rat's sensitivity to the contingency shift. These results support the hypothesis that the DMS stores information needed to support actions and the DLS stores information needed to support habits.
[PDF] [Web] Wyatte, D., Herd, S., Mingus, B. & O'Reilly, R. (2012). The Role of Competitive Inhibition and Top-Down Feedback in Binding during Object Recognition. Frontiers in psychology, 3, 182.
Abstract:
How does the brain bind together visual features that are processed concurrently by different neurons into a unified percept suitable for processes such as object recognition? Here, we describe how simple, commonly accepted principles of neural processing can interact over time to solve the brain's binding problem. We focus on mechanisms of neural inhibition and top-down feedback. Specifically, we describe how inhibition creates competition among neural populations that code different features, effectively suppressing irrelevant information, and thus minimizing illusory conjunctions. Top-down feedback contributes to binding in a similar manner, but by reinforcing relevant features. Together, inhibition and top-down feedback contribute to a competitive environment that ensures only the most appropriate features are bound together. We demonstrate this overall proposal using a biologically realistic neural model of vision that processes features across a hierarchy of interconnected brain areas. Finally, we argue that temporal synchrony plays only a limited role in binding - it does not simultaneously bind multiple objects, but does aid in creating additional contrast between relevant and irrelevant features. Thus, our overall theory constitutes a solution to the binding problem that relies only on simple neural principles without any binding-specific processes.
[PDF] [Web] Wyatte, D., Curran, T. & O'Reilly, R. (2012). The limits of feedforward vision: Recurrent processing promotes robust object recognition when objects are degraded. Journal of Cognitive Neuroscience, 24, 2248--2261.
Abstract:
Everyday vision requires robustness to a myriad of environmental factors that degrade stimuli. Foreground clutter can occlude objects of interest, and complex lighting and shadows can decrease the contrast of items. How does the brain recognize visual objects despite these low-quality inputs? On the basis of predictions from a model of object recognition that contains excitatory feedback, we hypothesized that recurrent processing would promote robust recognition when objects were degraded by strengthening bottom-up signals that were weakened because of occlusion and contrast reduction. To test this hypothesis, we used backward masking to interrupt the processing of partially occluded and contrast reduced images during a categorization experiment. As predicted by the model, we found significant interactions between the mask and occlusion and the mask and contrast, such that the recognition of heavily degraded stimuli was differentially impaired by masking. The model provided a close fit of these results in an isomorphic version of the experiment with identical stimuli. The model also provided an intuitive explanation of the interactions between the mask and degradations, indicating that masking interfered specifically with the extensive recurrent processing necessary to amplify and resolve highly degraded inputs, whereas less degraded inputs did not require much amplification and could be rapidly resolved, making them less susceptible to masking. Together, the results of the experiment and the accompanying model simulations illustrate the limits of feedforward vision and suggest that object recognition is better characterized as a highly interactive, dynamic process that depends on the coordination of multiple brain areas.
[PDF] [Web] Reynolds, J.R., O'Reilly, R.C., Cohen, J.D. & Braver, T.S. (2012). The function and organization of lateral prefrontal cortex: a test of competing hypotheses. PloS One, 7, e30284.
Abstract:
The present experiment tested three hypotheses regarding the function and organization of lateral prefrontal cortex (PFC). The first account (the information cascade hypothesis) suggests that the anterior-posterior organization of lateral PFC is based on the timing with which cue stimuli reduce uncertainty in the action selection process. The second account (the levels-of-abstraction hypothesis) suggests that the anterior-posterior organization of lateral PFC is based on the degree of abstraction of the task goals. The current study began by investigating these two hypotheses, and identified several areas of lateral PFC that were predicted to be active by both the information cascade and levels-of-abstraction accounts. However, the pattern of activation across experimental conditions was inconsistent with both theoretical accounts. Specifically, an anterior area of mid-dorsolateral PFC exhibited sensitivity to experimental conditions that, according to both accounts, should have selectively engaged only posterior areas of PFC. We therefore investigated a third possible account (the adaptive context maintenance hypothesis) that postulates that both posterior and anterior regions of PFC are reliably engaged in task conditions requiring active maintenance of contextual information, with the temporal dynamics of activity in these regions flexibly tracking the duration of maintenance demands. Activity patterns in lateral PFC were consistent with this third hypothesis: regions across lateral PFC exhibited transient activation when contextual information had to be updated and maintained in a trial-by-trial manner, but sustained activation when contextual information had to be maintained over a series of trials. These findings prompt a reconceptualization of current views regarding the anterior-posterior organization of lateral PFC, but do support other findings regarding the active maintenance role of lateral PFC in sequential working memory paradigms.
[PDF] [Web] Chatham, C.H., Herd, S.A., Brant, A.M., Hazy, T.E., Miyake, A., O'Reilly, R. & Friedman, N.P. (2011). From an executive network to executive control: a computational model of the n-back task. Journal of cognitive neuroscience, 23, 3598-3619.
Abstract:
A paradigmatic test of executive control, the n-back task, is known to recruit a widely distributed parietal, frontal, and striatal "executive network," and is thought to require an equally wide array of executive functions. The mapping of functions onto substrates in such a complex task presents a significant challenge to any theoretical framework for executive control. To address this challenge, we developed a biologically constrained model of the n-back task that emergently develops the ability to appropriately gate, bind, and maintain information in working memory in the course of learning to perform the task. Furthermore, the model is sensitive to proactive interference in ways that match findings from neuroimaging and shows a U-shaped performance curve after manipulation of prefrontal dopaminergic mechanisms similar to that observed in studies of genetic polymorphisms and pharmacological manipulations. Our model represents a formal computational link between anatomical, functional neuroimaging, genetic, behavioral, and theoretical levels of analysis in the study of executive control. In addition, the model specifies one way in which the pFC, BG, parietal, and sensory cortices may learn to cooperate and give rise to executive control.
[PDF] [Web] Munakata, Y., Herd, S.A., Chatham, C.H., Depue, B.E., Banich, M.T. & O'Reilly, R.C. (2011). A unified framework for inhibitory control. Trends in Cognitive Sciences, 15, 453-459.
Abstract:
Inhibiting unwanted thoughts, actions and emotions figures centrally in daily life, and the prefrontal cortex (PFC) is widely viewed as a source of this inhibitory control. We argue that the function of the PFC is best understood in terms of representing and actively maintaining abstract information, such as goals, which produces two types of inhibitory effects on other brain regions. Inhibition of some subcortical regions takes a directed global form, with prefrontal regions providing contextual information relevant to when to inhibit all processing in a region. Inhibition within neocortical (and some subcortical) regions takes an indirect competitive form, with prefrontal regions providing excitation of goal-relevant options. These distinctions are crucial for understanding the mechanisms of inhibition and how they can be impaired or improved.
[PDF] [Web] Frank, G.K.W., Reynolds, J.R., Shott, M.E. & O'Reilly, R.C. (2011). Altered temporal difference learning in bulimia nervosa. Biological psychiatry, 70, 728-735.
Abstract:
The neurobiology of bulimia nervosa (BN) is poorly understood. Recent animal literature suggests that binge eating is associated with altered brain dopamine (DA) reward function. In this study, we wanted to investigate DA-related brain reward learning in BN. METHODS: Ill BN (n = 20, age: mean = 25.2, SD = 5.3 years) and healthy control women (CW) (n = 23, age: mean = 27.2, SD = 6.4 years) underwent functional magnetic resonance brain imaging together with application of a DA-related reward learning paradigm, the temporal difference (TD) model. That task involves association learning between conditioned visual and unconditioned taste stimuli, as well as unexpected violation of those learned associations. Study participants also completed the Sensitivity to Reward and Punishment Questionnaire. RESULTS: Bulimia nervosa individuals showed reduced brain response compared with CW for unexpected receipt and omission of taste stimuli, as well as reduced brain regression response to the TD computer model generated reward values, in insula, ventral putamen, amygdala, and orbitofrontal cortex. Those results were qualitatively similar in BN individuals who were nondepressed and unmedicated. Binge/purge frequency in BN inversely predicted reduced TD model response. Bulimia nervosa individuals showed significantly higher Sensitivity to Reward and Punishment compared with CW. CONCLUSIONS: This is the first study that relates reduced brain DA responses in BN to the altered learning of associations between arbitrary visual stimuli and taste rewards. This attenuated response is related to frequency of binge/purge episodes in BN. The brain DA neurotransmitter system could be an important treatment target for BN.
[PDF] [Web] Snyder, H.R., Hutchison, N., Nyhus, E., Curran, T., Banich, M.T. & Munakata, Y. (2010). Neural inhibition enables selection during language processing. Proceedings of the National Academy of Sciences, 107, 16483-16488.
Abstract:
Whether grocery shopping or choosing words to express a thought, selecting between options can be challenging, especially for people with anxiety. We investigate the neural mechanisms supporting selection during language processing and its breakdown in anxiety. Our neural network simulations demonstrate a critical role for competitive, inhibitory dynamics supported by GABAergic interneurons. As predicted by our model, we found that anxiety (associated with reduced neural inhibition) impairs selection among options and associated prefrontal cortical activity, even in a simple, nonaffective verb-generation task, and the GABA agonist midazolam (which increases neural inhibition) improves selection, whereas retrieval from semantic memory is unaffected when selection demands are low. Neural inhibition is key to choosing our words.
[PDF] [Web] O'Reilly, R.C. (2010). The What and How of prefrontal cortical organization. Trends in neurosciences, 33, 355-361.
Abstract:
How is the prefrontal cortex (PFC) organized such that it is capable of making people more flexible and in control of their behavior? Is there any systematic organization across the many diverse areas that comprise the PFC, or is it uniquely adaptive such that no fixed representational structure can develop? Going against the current tide, this paper argues that there is indeed a systematic organization across PFC areas, with an important functional distinction between ventral and dorsal regions characterized as processing What versus How information, respectively. This distinction has implications for the rostro-caudal and medial-lateral axes of organization as well. The resulting large-scale functional map of PFC could prove useful in integrating diverse data, and in generating novel predictions.
[PDF] [Web] O'Reilly, R.C., Herd, S.A. & Pauli, W.M. (2010). Computational models of cognitive control. Current opinion in neurobiology, 20, 257--261.
Abstract:
Cognitive control refers to the ability to perform task-relevant processing in the face of other distractions or other forms of interference, in the absence of strong environmental support. It depends on the integrity of the prefrontal cortex and associated biological structures (e.g., the basal ganglia). Computational models have played an influential role in developing our understanding of this system, and we review current developments in three major areas: dynamic gating of prefrontal representations, hierarchies in the prefrontal cortex, and reward, motivation, and goal-related processing in prefrontal cortex. Models in these and other areas are advancing the field further forward.
[PDF] [Web] Hazy, T.E., Frank, M.J. & O'Reilly, R.C. (2010). Neural mechanisms of acquired phasic dopamine responses in learning. Neuroscience and biobehavioral reviews, 34, 701--720.
Abstract:
What biological mechanisms underlie the reward-predictive firing properties of midbrain dopaminergic neurons, and how do they relate to the complex constellation of empirical findings understood as Pavlovian and instrumental conditioning? We previously presented PVLV, a biologically inspired Pavlovian learning algorithm accounting for DA activity in terms of two interrelated systems: a primary value (PV) system, which governs how DA cells respond to a US (reward) and; a learned value (LV) system, which governs how DA cells respond to a CS. Here, we provide a more extensive review of the biological mechanisms supporting phasic DA firing and their relation to the spate of Pavlovian conditioning phenomena and their sensitivity to focal brain lesions. We further extend the model by incorporating a new NV (novelty value) component reflecting the ability of novel stimuli to trigger phasic DA firing, providing "novelty bonuses" which encourages exploratory working memory updating and in turn speeds learning in trace conditioning and other working memory-dependent paradigms. The evolving PVLV model builds upon insights developed in many earlier computational models, especially reinforcement learning models based on the ideas of Sutton and Barto, biological models, and the psychological model developed by Savastano and Miller. The PVLV framework synthesizes these various approaches, overcoming important shortcomings of each by providing a coherent and specific mapping to much of the relevant empirical data at both the micro- and macro-levels, and examines their relevance for higher order cognitive functions..
[PDF] [Web] Reynolds, J.R. & O'Reilly, R.C. (2009). Developing PFC representations using reinforcement learning. Cognition, 113, 281-292.
Abstract:
From both functional and biological considerations, it is widely believed that action production, planning, and goal-oriented behaviors supported by the frontal cortex are organized hierarchically (Fuster, 1990, Koechlin, Ody, \& Kouneiher, 2003, \& Miller, Galanter, \& Pribram, 1960) However, the nature of the different levels of the hierarchy remains unclear, and little attention has been paid to the origins of such a hierarchy. We address these issues through biologically-inspired computational models that develop representations through reinforcement learning. We explore several different factors in these models that might plausibly give rise to a hierarchical organization of representations within the PFC, including an initial connectivity hierarchy within PFC, a hierarchical set of connections between PFC and subcortical structures controlling it, and differential synaptic plasticity schedules. Simulation results indicate that architectural constraints contribute to the segregation of different types of representations, and that this segregation facilitates learning. These findings are consistent with the idea that there is a posterior-anterior directionality to a hierarchy in PFC, as captured in our earlier computational models of PFC function and a growing body of empirical data.
[PDF] [Web] Huber, D.E., Tian, X., Curran, T., O'Reilly, R.C. & Woroch, B. (2008). The dynamics of integration and separation: ERP MEG and neural network studies of immediate repetition effects. Journal of experimental psychology, 34, 1389--1416.
Abstract:
This article presents data and theory concerning the fundamental question of how the brain achieves a balance between integrating and separating perceptual information over time. This theory was tested in the domain of word reading by examining brain responses to briefly presented words that were either new or immediate repetitions. Critically, the prime that immediately preceded the target was presented either for 150 ms or 2,000 ms, thus examining a situation of perceptual integration versus one of perceptual separation. Electrophysiological responses during the first 200 ms following presentation of the target word were assessed using electroencephalography (EEG) and magnetoencephalography (MEG) recordings. As predicted by a dynamic neural network model with habituation, repeated words produced less of a perceptual response, and this effect diminished with increased prime duration. Using dynamics that best accounted for the behavioral transition from positive to negative priming with increasing prime duration, the model correctly predicted the time course of the event-related potential (ERP) repetition effects under the assumption that letter processing is the source of observed P100 repetition effects and word processing is the source of observed N170 repetition effects.
[PDF] Jilk, D.J., Lebiere, C., O'Reilly, R.C. & Anderson, J.R. (2008). SAL: An explicitly pluralistic cognitive architecture. Journal of Experimental and Theoretical Artificial Intelligence, 20, 197-218.
Abstract:
The SAL cognitive architecture is a synthesis of two well-established constituents: ACT-R, a hybrid symbolic-subsymbolic cognitive architecture, and Leabra, a neural architecture. These component architectures have vastly different origins yet suggest a surprisingly convergent view of the brain, the mind and behaviour. Furthermore, both of these architectures are internally pluralistic, recognising that models at a single level of abstraction cannot capture the required richness of behaviour. In this article, we offer a brief principled defence of epistemological pluralism in cognitive science and artificial intelligence, and elaborate on the SAL architecture as an example of how pluralism can be highly effective as an approach to research in cognitive science.
[PDF] [Web] Aisa, B., Mingus, B. & O'Reilly, R. (2008). The emergent neural modeling system. Neural networks, 21, 1146--1152.
Abstract:
Emergent (http://grey.colorado.edu/emergent) is a powerful tool for the simulation of biologically plausible, complex neural systems that was released in August 2007. Inheriting decades of research and experience in network algorithms and modeling principles from its predecessors, PDP++ and PDP, Emergent has been redesigned as an efficient workspace for academic research and an engaging, easy-to-navigate environment for students. The system provides a modern and intuitive interface for programming and visualization centered around hierarchical, tree-based navigation and drag-and-drop reorganization. Emergent contains familiar, high-level simulation constructs such as Layers and Projections, a wide variety of algorithms, general-purpose data handling and analysis facilities and an integrated virtual environment for developing closed-loop cognitive agents. For students, the traditional role of a textbook has been enhanced by wikis embedded in every project that serve to explain, document, and help newcomers engage the interface and step through models using familiar hyperlinks. For advanced users, the software is easily extensible in all respects via runtime plugins, has a powerful shell with an integrated debugger, and a scripting language that is fully symmetric with the interface. Emergent strikes a balance between detailed, computationally expensive spiking neuron models and abstract, Bayesian or symbolic systems. This middle level of detail allows for the rapid development and successful execution of complex cognitive models while maintaining biological plausibility.
[PDF] [Web] Bayley, P.J., O'Reilly, R.C., Curran, T. & Squire, L.R. (2008). New semantic learning in patients with large medial temporal lobe lesions. Hippocampus, 18, 575-583.
Abstract:
Two patients with large lesions of the medial temporal lobe were given four tests of semantic knowledge that could only have been acquired after the onset of their amnesia. In contrast to previous studies of postmorbid semantic learning, correct answers could be based on a simple, nonspecific sense of familiarity about single words, faces, or objects. According to recent computational models (for example, Norman and O'Reilly (2003) Psychol Rev 110:611--646), this characteristic should be optimal for detecting the kind of semantic learning that might be supported directly by the neocortex. Both patients exhibited some capacity for new learning, albeit at a level substantially below control performances. Notably, the correct answers appeared to reflect declarative memory. It was not the case that the correct answers simply popped out in some automatic way in the absence of any additional knowledge about the items. Rather, the few correct choices made by the patients tended to be accompanied by additional information about the chosen items, and the available knowledge appeared to be similar qualitatively to the kind of factual knowledge that healthy individuals gradually acquire over the years. The results are consistent with the idea that neocortical structures outside the medial temporal lobe are able to support some semantic learning, albeit to a very limited extent. Alternatively, the small amount of learning detected in the present study could depend on tissue within the posterior medial temporal lobe that remains intact in both patients.
[PDF] Atallah, H.E., Rudy, J.W. & O'Reilly, R.C. (2008). The role of the dorsal striatum and dorsal hippocampus in probabilistic and deterministic odor discrimination tasks. Learning and Memory, 15, 294-298.
Abstract:
Three experiments explored the contribution of the cortico-striatal system and the hippocampus system to the acquisition of solutions to simultaneous instrumental odor discriminations. Inactivation of the dorsal striatum after rats had reached criterion on a three problem probabilistic set of discriminations -- A (80%) vs. B (20%), C (67%) vs. D (33%), E(67%) vs. F(33%) -- impaired test performance and disrupted performance when the rats were tested with novel cue combinations (C vs. F and E vs. D), where control animals chose C and F. In contrast, inactivating the dorsal hippocampus enhanced performance on this task and on a deterministic discrimination A (100%) vs. B (0%). These results are consistent with the complementary learning systems view, which assumes that the cortico-striatal and hippocampal system capture information in parallel. How this information combines to influence task performance depends on the compatibility of the content captured by each system. These results suggest that the trial-specific information captured by the hippocampal system can be incompatible with the across-trial integration of trial outcomes captured by the cortico-striatal system.
[PDF] Pauli, W.M. & O'Reilly, R.C. (2008). Attentional control of associative learning -- a possible role of the central cholinergic system. Brain Research, 1202, 43-53.
Abstract:
How does attention interact with learning? Kruschke [Kruschke, J.K. (2001). Toward a unified Model of Attention in Associative Learning. J. Math. Psychol. 45, 812--863.] proposed a model (EXIT) that captures Mackintosh's [Mackintosh, N.J. (1975). A theory of attention: Variations in the associability of stimuli with reinforcement. Psychological Review, 82(4), 276--298.] framework for attentional modulation of associative learning. We developed a computational model that showed analogous interactions between selective attention and associative learning, but is significantly simplified and, in contrast to EXIT, is motivated by neurophysiological findings. Competition among input representations in the internal representation layer, which increases the contrast between stimuli, is critical for simulating these interactions in human behavior. Furthermore, this competition is modulated in a way that might be consistent with the phasic activation of the central cholinergic system, which modulates activity in sensory cortices. Specifically, phasic increases in acetylcholine can cause increased excitability of both pyramidal excitatory neurons in cortical layers II/III and cortical GABAergic inhibitory interneurons targeting the same pyramidal neurons. These effects result in increased attentional contrast in our model. This model thus represents an initial attempt to link human attentional learning data with underlying neural substrates.
[PDF] Frank, M., Santamaria, A., O'Reilly, R. & Willcutt, E. (2007). Testing Computational Models of Dopamine and Noradrenaline Dysfunction in Attention Deficit/Hyperactivity Disorder. Neuropsychopharmacology, 32, 1583-99.
Abstract:
We test our neurocomputational model of fronto-striatal dopamine (DA) and noradrenaline (NA) function for understanding cognitive and motivational deficits in attention deficit / hyperactivity disorder (ADHD). Our model predicts that low striatal DA levels in ADHD should lead to deficits in ``Go'' learning from positive reinforcement, which should be alleviated by stimulant medications, as observed with DA manipulations in other populations. Indeed, while non-medicated adult ADHD participants were impaired at both positive (Go) and negative (NoGo) reinforcement learning, only the former deficits were ameliorated by medication. We also found evidence for our model's extension of the same striatal DA mechanisms to working memory, via interactions with prefrontal cortex. In a modified AX-continuous performance task, ADHD participants showed reduced sensitivity to working memory contextual information, despite no global performance deficits, and were more susceptible to the influence of distractor stimuli presented during the delay. These effects were reversed with stimulant medications. Moreover, the tendency for medications to improve Go relative to NoGo reinforcement learning was predictive of their improvement in working memory in distracting conditions, suggestive of common DA mechanisms and supporting a unified account of DA function in ADHD. However, other ADHD effects such as erratic trial-to-trial switching and reaction time variability are not accounted for by model DA mechanisms, and are instead consistent with cortical noradrenergic dysfunction and associated computational models. Accordingly, putative NA deficits were correlated with each other and independent of putative DA-related deficits. Taken together, our results demonstrate the usefulness of computational approaches for understanding cognitive deficits in ADHD.
[Main PDF] [Supp PDF] Hazy, T.E., Frank, M.J. & O'Reilly, R.C. (2007). Towards an executive without a homunculus: computational models of the prefrontal cortex/basal ganglia system. Philosophical Transactions of the Royal Society B, 362, 1601-1613.
Abstract:
The prefrontal cortex (PFC) has long been thought to serve as an "executive" that controls the selection of actions and cognitive functions more generally. However, the mechanistic basis of this executive function has not been clearly specified often amounting to a homunculus. This paper reviews recent attempts to deconstruct this homunculus by elucidating the precise computational and neural mechanisms underlying the executive functions of the PFC. The overall approach builds upon existing mechanistic models of the basal ganglia (BG) and frontal systems known to play a critical role in motor control and action selection, where the BG provide a "Go" versus "NoGo" modulation of frontal action representations. In our model, the BG modulate working memory representations in prefrontal areas to support more abstract executive functions. We have developed a computational model of this system that is capable of developing human-like performance on working memory and executive control tasks through trial-and-error learning. This learning is based on reinforcement learning mechanisms associated with the midbrain dopaminergic system and its activation via the BG and amygdala. Finally, we briefly describe various empirical tests of this framework.
[PDF] O'Reilly, R.C., Frank, M.J., Hazy, T.E. & Watz, B. (2007). PVLV: The Primary Value and Learned Value Pavlovian Learning Algorithm. Behavioral Neuroscience, 121, 31-49.
Abstract:
The authors present their primary value learned value (PVLV) model for understanding the reward-predictive firing properties of dopamine (DA) neurons as an alternative to the temporal-differences (TD) algorithm. PVLV is more directly related to underlying biology and is also more robust to variability in the environment. The primary value (PV) system controls performance and learning during primary rewards, whereas the learned value (LV) system learns about conditioned stimuli. The PV system is essentially the Rescorla-Wagner/delta-rule and comprises the neurons in the ventral striatum/nucleus accumbens that inhibit DA cells. The LV system comprises the neurons in the central nucleus of the amygdala that excite DA cells. The authors show that the PVLV model can account for critical aspects of the DA firing data, making a number of clear predictions about lesion effects, several of which are consistent with existing data. For example, first- and second-order conditioning can be anatomically dissociated, which is consistent with PVLV and not TD. Overall, the model provides a biologically plausible framework for understanding the neural basis of reward learning.
[PDF] Atallah, H.E., Lopez-Paniagua, D., Rudy, J.W. & O'Reilly, R. (2007). Separate neural substrates for skill learning and performance in the ventral and dorsal striatum. Nature Neuroscience, 10, 126-131.
Abstract:
It is widely accepted that the striatum of the basal ganglia is a primary substrate for the learning and performance of skills. We provide evidence that two regions of the rat striatum, ventral and dorsal, play distinct roles in instrumental conditioning (skill learning), with the ventral striatum being critical for learning and the dorsal striatum being important for performance but, notably, not for learning. This implies an actor (dorsal) versus director (ventral) division of labor, which is a new variant of the widely discussed actor-critic architecture. Our results also imply that the successful performance of a skill can ultimately result in its establishment as a habit outside the basal ganglia.
[PDF] OReilly, R.C. (2006). Biologically Based Computational Models of High-Level Cognition. Science, 314, 91-94.
Abstract:
Computer models based on the detailed biology of the brain can help us understand the myriad complexities of human cognition and intelligence. Here, we review models of the higher level aspects of human intelligence, which depend critically on the prefrontal cortex and associated subcortical areas. The picture emerging from a convergence of detailed mechanistic models and more abstract functional models represents a synthesis between analog and digital forms of computation. Specifically, the need for robust active maintenance and rapid updating of information in the prefrontal cortex appears to be satisfied by bistable activation states and dynamic gating mechanisms. These mechanisms are fundamental to digital computers and may be critical for the distinctive aspects of human intelligence.
[PDF] Frank, M.J., OReilly, R.C. & Curran, T. (2006). When memory fails, intuition reigns: Midazolam enhances implicit inference in humans. Psychological Science, 17, 700-707.
Abstract:
People often make logically sound decisions using explicit reasoning strategies, but sometimes it pays to rely on more implicit ``gut-level'' intuition. The transitive inference paradigm has been widely used as a test of explicit logical reasoning in animals and humans, but it can also be solved in a more implicit manner. Some have argued that the hippocampus supports relational memories required for making logical inferences. Here we show that the benzodiazepene midazolam, which inactivates the hippocampus, causes profound explicit memory deficits in healthy participants, but actually enhances their ability in making implicit transitive inferences. These results are consistent with neurocomputational models of the basal ganglia/dopamine system that learn to make decisions based on positive and negative reinforcement. We suggest that disengaging the hippocampal explicit memory system can be advantageous for this more implicit form of learning.
[Main PDF] [Supp PDF] Frank, M.J. & O'Reilly, R.C. (2006). A mechanistic account of striatal dopamine function in human cognition: Psychopharmacological studies with cabergoline and haloperidol. Behavioral Neuroscience, 120, 497-517.
Abstract:
We test a neurocomputational model of dopamine function in cognition by administering to healthy participants low doses of D2 agents cabergoline and haloperidol. Our model suggests that dopamine dynamically modulates the balance of ``Go'' and ``NoGo'' basal ganglia pathways during cognitive learning and performance. Cabergoline impaired, while haloperidol enhanced, Go learning from positive reinforcement, consistent with presynaptic drug effects. Cabergoline also caused an overall bias toward Go responding, consistent with postsynaptic action. These same effects extended to working memory and attentional domains, supporting the idea that the basal ganglia / dopamine system modulates the updating of prefrontal representations. Drug effects interacted with baseline working memory span in all tasks. Taken together, our results support a unified account of the role of dopamine in modulating cognitive processes that depend on the basal ganglia.
[PDF] Hazy, T.E., Frank, M.J. & O'Reilly, R.C. (2006). Banishing the Homunculus: Making Working Memory Work. Neuroscience, 139, 105-118.
Abstract:
The prefrontal cortex (PFC) has long been thought to subserve both working memory and ``executive'' function, but the mechanistic basis of their integrated function has remained poorly understood, often amounting to a homunculus. This paper reviews the progress in our lab and others pursuing a long-term research agenda to deconstruct this homunculus by elucidating the precise computational and neural mechanisms underlying these phenomena. We outline six key functional demands underlying working memory, and then describe the current state of our computational model of the PFC and associated systems in the basal ganglia (BG). The model, called PBWM (prefrontal-cortex, basal-ganglia working memory model), relies on actively maintained representations in the PFC, which are dynamically updated/gated by the BG. It is capable of developing human-like performance largely on its own by taking advantage of powerful reinforcement learning mechanisms, based on the midbrain dopaminergic system and its activation via the BG and amygdala. These learning mechanisms enable the model to learn to control both itself and other brain areas in a strategic, task-appropriate manner. The model can learn challenging working memory tasks, and has been corroborated by several important empirical studies.
[PDF] Cer, D.M. & O'Reilly, R.C. (in press). Neural mechanisms of binding in the hippocampus and neocortex: Insights from computational models. H.D. Zimmer, A. Mecklinger & U. Lindenberger (Eds) Binding in Memory, Oxford: Oxford University Press.
Abstract:
An account of the neurological mechanisms that underlie binding is given which is characterized by the decomposition of the binding problem into three distinct subproblems. Each subproblem is then supported by anatomically specialized brain regions. The posterior cortex employs coarse-coded distributed representations of low-order conjunctions to resolve binding ambiguities, while also supporting systematic generalization to novel stimuli and situations. These representations are slowly acquired over experience. The hippocampus can more rapidly bind higher-order conjunctions of information such as episodes or locations. Finally, the prefrontal cortex supports transient, actively maintained bindings that are used in the service of working memory. We argue that this approach to the binding problem compares favorably with those based on temporal synchrony binding.
[PDF] O'Reilly, R.C. (in press). The Division of Labor Between the Neocortex and Hippocampus. G. Houghton (Ed) Connectionist Modeling in Cognitive Psychology, : Psychology Press.
Abstract:
This chapter presents an overview of a computational approach towards understanding the different contributions of the neocortex and hippocampus in learning and memory. The approach is based on a set of principles derived from converging biological, psychological, and computational constraints. The most central principles are that the neocortex employs a slow learning rate and overlapping distributed representations to extract the general statistical structure of the environment, while the hippocampus learns rapidly using separated representations to encode the details of specific events while suffering minimal interference. Additional principles concern the nature of learning (error-driven and Hebbian), and recall of information via pattern completion. These principles are demonstrated through neocortical and hippocampal models of a well-known memory task, the AB-AC paired associates task. The results of applying these principles to a wide range of phenomena in conditioning, habituation, contextual learning, recognition memory, recall, and retrograde amnesia, are also summarized.
[PDF] O'Reilly, R.C. (in press). Modeling Integration and Dissociation in Brain and Cognitive Development. Y. Munakata & M.H. Johnson (Eds) Processes of Change in Brain and Cognitive Development: Attention and Performance XXI., Oxford: Oxford University Press.
Abstract:
Over the course of development, brain areas can become increasingly dissociated in their functions, or increasingly integrated. Computational models can provide insights into how and why these opposing effects happen. This paper presents a computational framework for understanding the specialization of brain functions across the hippocampus, neocortex, and basal ganglia. This framework is based on computational tradeoffs that arise in neural network models, where achieving one type of learning function requires very different parameters from those necessary to achieve another form of learning. For example, we dissociate the hippocampus from cortex with respect to general levels of activity, learning rate, and level of overlap between activation patterns. Similarly, the frontal cortex and associated basal ganglia system have important neural specializations not required of the posterior cortex system. Taken together, these brain areas form an overall cognitive architecture, which has been implemented in functioning computational models, provides a rich and often subtle means of explaining a wide range of behavioral and cognitive neuroscience data. The developmental implications of this framework, and other computational mechanisms of dissociation and integration, are reviewed.
[PDF] O'Reilly, R.C. & Frank, M.J. (2006). Making Working Memory Work: A Computational Model of Learning in the Frontal Cortex and Basal Ganglia. Neural Computation, 18, 283-328.
Abstract:
The prefrontal cortex has long been thought to subserve both working memory (the holding of information online for processing) and ``executive'' functions (deciding how to manipulate working memory and perform processing). Although many computational models of working memory have been developed, the mechanistic basis of executive function remains elusive, often amounting to a homunculus. This paper presents an attempt to deconstruct this homunculus through powerful learning mechanisms that allow a computational model of the prefrontal cortex to control both itself and other brain areas in a strategic, task-appropriate manner. These learning mechanisms are based on subcortical structures in the midbrain, basal ganglia and amygdala, which together form an actor/critic architecture. The critic system learns which prefrontal representations are task-relevant and trains the actor, which in turn provides a dynamic gating mechanism for controlling working memory updating. Computationally, the learning mechanism is designed to simultaneously solve the temporal and structural credit assignment problems. The model's performance compares favorably with standard backpropagation-based temporal learning mechanisms on the challenging 1-2-AX working memory task, and other benchmark working memory tasks.
[PDF] Herd, S.A., Banich, M.T. & O'Reilly, R.C. (2006). Neural Mechanisms of Cognitive Control: An integrative Model of Stroop Task Performance and fMRI data. Journal of Cognitive Neuroscience, 18, 22-32.
Abstract:
We address the connection between conceptual knowledge and cognitive control using a neural network model. This model extends a widely held theory of cognitive control [Cohen, J. D., Dunbar, K., & McClelland, J. L. On the control of automatic processes: A parallel distributed processing model of the Stroop effect. Psychological Review, 97, 332-361, 1990] so that it can explain new empirical findings. Leveraging other computational modeling work, we hypothesize that representations used for task control are recruited from preexisting representations for categories, such as the concept of color relevant to the Stroop task we model here. This hypothesis allows the model to account for otherwise puzzling fMRI results, such as increased activity in brain regions processing to-be-ignored information. In addition, biologically motivated changes in the model's pattern of connectivity show how global competition can arise when inhibition is strictly local, as it seems to be in the cortex. We also discuss the potential for this theory to unify models of task control with other forms of attention.
[PDF] Herd, S.A. & O'Reilly, R.C. (2005). Serial visual search from a parallel model. Vision Research, 45, 2987-2992.
Abstract:
We tested a parallel neural network model of visual search, and found that it located targets more quickly when allowed to take several fast guesses. We suggest that this serially iterated parallel search may be the mode used by the visual system, in accord with theories such as the Guided Search model. Furthermore, in our model the most efficient mode of processing varied with the type of search. If the nature of visual search varies with task demands, seemingly contradictory findings can be reconciled.
[PDF] Frank, M.J., Rudy, J.W., Levy, W.B. & O'Reilly, R.C. (2005). When Logic Fails: Implicit Transitive Inference in Humans. Memory and Cognition, 33, 742-750.
Abstract:
Transitive inference (TI) in animals (e.g., choosing A over C based on knowing that A is better than B and B is better C) has been interpreted by some as reflecting a declarative, logical inference process. We invert this anthropomorphic interpretation by providing evidence that humans can exhibit TI-like behavior based on simpler associative mechanisms that underly many theories of animal learning. In this study, human participants were trained on a five-pair TI problem (A+B-, B+C-, C+D-, D+E-, E+F-), and, unlike in previous human TI studies, were prevented from becoming explicitly aware of the logical hierarchy, so they could not employ logical reasoning. They were then tested with three problems: B vs D, B vs. E, and C vs. E. Participants only reliably chose B over E, whereas the other test conditions yielded chance performance. This result is inconsistent with the use of logical reasoning, and is instead consistent with an account developed to explain earlier TI studies with rats that found the same pattern of results. In this account, choice performance is based on differential associative strengths across the stimulus items that develop over training, despite equal overt reinforcement.
[PDF] Rudy, J.W., Biedenkapp, J.C. & O'Reilly, R.C. (2005). Prefrontal Cortex and the Organization of Recent and Remote Memories: An Alternative View. Learning and Memory, 12, 445-446.
Abstract:
Commentary suggesting that interactions between the prefrontal cortex/anterior cingulate and hippocampus may be more about cognitive control needed to recall weak memories from the hippocampus, than about consolidation of memories from hippocampus to prefrontal cortex and anterior cingulate cortex.
[Main PDF] [Supp PDF] Rougier, N.P., Noelle, D., Braver, T.S., Cohen, J.D. & O'Reilly, R.C. (2005). Prefrontal Cortex and the Flexibility of Cognitive Control: Rules Without Symbols. Proceedings of the National Academy of Sciences, 102, 7338-7343.
Abstract:
Human cognitive control is uniquely flexible, and has been shown to depend on prefrontal cortex (PFC). But exactly how the biological mechanisms of the PFC support flexible cognitive control remains a profound mystery. Existing theoretical models have posited powerful task-specific PFC representations, but not how these develop. We show how this can occur when a set of PFC-specific neural mechanisms interact with breadth of experience to self-organize abstract, rule-like PFC representations that support flexible generalization in novel tasks. The same model is shown to apply to benchmark PFC tasks (Stroop and Wisconsin card sorting), accurately simulating the behavior of neurologically intact and frontally-damaged people.
[Main PDF] [Supp PDF] Frank, M.J., Seeberger, L. & O'Reilly, R.C. (2004). By carrot or by stick: Cognitive reinforcement learning in Parkinsonism. Science, 306, 1940-1943.
Abstract:
To what extent do we learn from the positive versus negative outcomes of our decisions? The neuromodulator dopamine plays a key role in these reinforcement learning processes. Patients with Parkinson's disease, who have depleted dopamine in the basal ganglia, are impaired in tasks that require learning from trial and error. Here we show, using two cognitive procedural learning tasks, that Parkinson's patients off medication are better at learning to avoid choices that lead to negative outcomes than they are at learning from positive outcomes. Dopamine medication reverses this bias, making patients more sensitive to positive than negative outcomes. This pattern was predicted by our biologically-based computational model of basal ganglia/dopamine interactions in cognition, which has separate pathways for ``Go'' and ``NoGo'' responses that are differentially modulated by positive and negative reinforcement.
[PDF] Atallah, H.E., Frank, M.J. & O'Reilly, R.C. (2004). Hippocampus, cortex and basal ganglia: Insights from computational models of complementary learning systems. Neurobiology of Learning and Memory, 82/3, 253-67.
Abstract:
We present a framework for understanding how the hippocampus, neocortex, and basal ganglia work together to support cognitive and behavioral function in the mammalian brain. This framework is based on computational tradeoffs that arise in neural network models, where achieving one type of learning function requires very different parameters from those necessary to achieve another form of learning. For example, we dissociate the hippocampus from cortex with respect to general levels of activity, learning rate, and level of overlap between activation patterns. Similarly, the frontal cortex and associated basal ganglia system have important neural specializations not required of the posterior cortex system. Taken together, this overall cognitive architecture, which has been implemented in functioning computational models, provides a rich and often subtle means of explaining a wide range of behavioral and cognitive neuroscience data. Here, we summarize recent results in the domains of recognition memory, contextual fear conditioning, effects of basal ganglia lesions on stimulus-response and place learning, and flexible responding.
[PDF] Munakata, Y. & O'Reilly, R.C. (2003). Developmental and Computational Neuroscience Approaches to Cognition: The Case of Generalization. Cognitive Studies, 10, 76-92.
Abstract:
The ability to generalize --- to abstract regularities from our experiences that can be applied to new experiences --- is fundamental to human cognition and our abilities to flexibly adapt to changing situations. However, the generalization abilities of children and adults are far from perfect, with many clear demonstrations of failures to generalize in situations that would otherwise appear to lend themselves to generalization. It seems that people require extensive experience with a domain to demonstrate good generalization, and that their generalization abilities are best when dealing with relatively concrete, familiar situations. In this paper, we argue that people's successes and failures in generalization are well characterized by neural network models. Networks of neurons connected by synaptic weights are naturally predisposed to encode information in a highly specific fashion, which does not support generalization (as has been seized upon by critics of such models). However, with sufficient experience and appropriate architectural properties, such models can develop abstract representations that support good generalization. Implications for the neural bases and development of generalization abilities are discussed.
[PDF] Jilk, D.J., Cer, D.M. & O'Reilly, R.C. (2003). Learning Rules Generated by a Biophysical Model of Synaptic Plasticity. Computational Neuroscience Conference, 2003,
Abstract:
We describe our initial attempts to reconcile powerful neural network learning rules derived from computational principles with learning rules derived bottom-up from biophysical mechanisms. Using a biophysical model of synaptic plasticity (Shouval, Bear, and Cooper, 2002), we generated numerical synaptic learning rules and compared them to the performance of a Hebbian learning rule in a previously studied neural network model of self-organized learning. In general, the biophysically derived learning rules did not perform as well as the analytic rule, but their performance could be improved by adjusting various aspects of the biophysical model. These results show that some progress has been made in integrating our understanding of biological and artificial neural networks.
[Main PDF] [Supp PDF] Norman, K.A. & O'Reilly, R.C. (2003). Modeling Hippocampal and Neocortical Contributions to Recognition Memory: A Complementary Learning Systems Approach. Psychological Review, 110, 611-646.
Abstract:
We present a computational neural network model of how hippocampus and medial temporal lobe cortex (MTLC) contribute to recognition memory. The hippocampal component contributes by recalling studied details. The MTLC component can not support recall, but one can extract a scalar familiarity signal from MTLC that tracks how well the test item matches studied items. We present simulations that establish key differences in the operating characteristics of the hippocampal recall and MTLC familiarity signals, and we identify several manipulations (e.g., target-lure similarity, interference) that differentially affect the two signals. We also use the model to address the stochastic relationship between recall and familiarity and the effects of partial vs. complete hippocampal lesions on recognition.
[PDF] VanElzakker, M., O'Reilly, R.C. & Rudy, J.W. (2003). Transitivity, Flexibility, Conjunctive Representations and the Hippocampus: I. An Empirical Analysis. Hippocampus, 13, 334-340.
Abstract:
Following training on a set of 4 ordered, simultaneous, odor discrimination problems: A+B-; B+C-; C+D-; D+E-, intact rats display transitivity: When tested on the novel combination BD they choose B. Rats with damage to the hippocampus, however, do not show transitivity (Dusek & Eichenbaum, 1997). These results have been interpreted as support for the idea that the hippocampus is a relational memory storage system that enables the subject to make comparisons among representations of the individual problems and choose based on inferential logic. We provide evidence for a simpler explanation: Specifically, subjects make their choices based on the absolute excitatory value of the individual stimuli. This value determines the ability of that stimulus to attract a response. This conclusion emerged because following training on a 5 problem set A+B-; B+C-; C+D-; D+E-, E+F-, rats preferred B when tested with BE but not when tested with BD. The implication of these results for how to conceptualize the role of the hippocampus in transitive-like phenomena is discussed.
[PDF] Frank, M.J., Rudy, J.W. & O'Reilly, R.C. (2003). Transitivity, Flexibility, Conjunctive Representations and the Hippocampus: II. A Computational Analysis. Hippocampus, 13, 341-354.
Abstract:
Following training on a set of 4 ordered, simultaneous, odor discrimination problems: A+B-; B+C-; C+D-; D+E-, intact rats display transitivity: When tested on the novel combination BD they choose B. Rats with damage to the hippocampus, however, do not show transitivity (Dusek & Eichenbaum, 1997). These results have been interpreted as support for the idea that the hippocampus is a relational memory storage system that enables the subject to make comparisons among representations of the individual problems and choose based on inferential logic. We provide evidence for a simpler explanation: Specifically, subjects make their choices based on the absolute excitatory value of the individual stimuli. This value determines the ability of that stimulus to attract a response. This conclusion emerged because following training on a 5 problem set A+B-; B+C-; C+D-; D+E-, E+F-, rats preferred B when tested with BE but not when tested with BD. The implication of these results for how to conceptualize the role of the hippocampus in transitive-like phenomena is discussed.
[PDF] Huber, D.E. & O'Reilly, R.C. (2003). Persistence and accommodation in short-term priming and other perceptual paradigms: Temporal segregation through synaptic depression. Cognitive Science, 27, 403-430.
Abstract:
In everyday situations, perceptual input constantly changes in an unpredictable fashion. However, our perceptual systems are somewhat sluggish, requiring adequate processing time in order to integrate information. This integration results in persistent activation across stimuli, which can lead to confusion regarding the source of a given activation signal (i.e., source confusion). We propose that activity dependent neural accommodation naturally limits this source confusion by suppressing items once they have been identified. We review behavioral paradigms from different literatures that measure the correlates of persistence and accommodation. Of the various accommodation mechanisms, we focus on synaptic depression for its well-specified dynamic characteristics. We derive a rate-coded version of synaptic depression that can be used to produce accommodating dynamics in any neural network with real valued activation. We implement this expression in a hierarchical model of perception termed, a neural mechanism for responding optimally with unknown sourced of evidence (nROUSE). This model is similar to the ROUSE model of Huber et al. (2001), which produces accommodated levels of feature evidence through an optimal calculation. Important parallels are drawn between the models and nROUSE is applied to three short-term priming experiments that manipulated prime duration.
[PDF] O'Reilly, R.C., Busby, R.S. & Soto, R. (2003). Three Forms of Binding and their Neural Substrates: Alternatives to Temporal Synchrony. A. Cleeremans (Ed) The Unity of Consciousness: Binding, Integration, and Dissociation, 168-192, Oxford: Oxford University Press.
Abstract:
This paper presents three different ways of addressing the binding problem in different brain areas: generic neocortex, hippocampus, and prefrontal cortex. None of these approaches involve the popular mechanism of temporal synchrony. The first two involve conjunctive representations that bind by ensuring that different neural units are activated for different combinations of input features. Specifically, we think the cortex constructs low-order conjunctions using coarse-coded distributed representations to avoid the combinatorial explosion usually associated with conjunctive solutions to the binding problem. We present a model that learns these representations in a challenging relational binding task, and furthermore is capable of considerable generalization to novel inputs. Next, we review the idea that the hippocampus performs conjunctive binding in long term memory through the use of higher-order conjunctions that are much more specific to particular events than those in the cortex. Finally, we present a model of a very different form of binding that involves the phonological loop --- a mechanism for maintaining arbitrary sequences of phonemes in active memory. This phonological system can be used to bind by continuously repeating the to-be-bound information (e.g., ``press left key for green X's,...''). In total, this work suggests that instead of one simple and generic solution to the binding problem, the brain has developed a number of specialized mechanisms that build on the strengths of existing neural hardware in different brain areas.
[PDF] O'Reilly, R.C. & Munakata, Y. (2003). Computational Neuroscience and Cognitive Modeling. L. Nadel (Ed) Encyclopedia of Cognitive Sciences, London: Macmillan.
Abstract:
This article describes computer models that simulate the neural networks of the brain, with the goal of understanding how cognitive functions (perception, memory, thinking, language, etc) arise from their neural basis. Many neural network models have been developed over the years, focused at many different levels of analysis from engineering to relatively low-level biology to cognition. Here, we consider models that try to span the gap between biology and cognition, such that they deal with real cognitive data, using mechanisms that are related to the underlying biology.
[PDF] O'Reilly, R.C. & Norman, K.A. (2002). Hippocampal and Neocortical Contributions to Memory: Advances in the Complementary Learning Systems Framework. Trends in Cognitive Sciences, 6, 505-510.
Abstract:
The complementary learning systems framework provides a simple set of principles, derived from converging biological, psychological, and computational constraints, for understanding the differential contributions of the neocortex and hippocampus to learning and memory. The most central principles are that the neocortex employs a small learning rate and overlapping distributed representations to extract the general statistical structure of the environment, while the hippocampus learns rapidly using separated representations to encode the details of specific events while minimizing interference. In recent years, we and our collaborators have instantiated these principles in working computational models, and we have used these models to address detailed patterns of human and animal learning and memory findings, across a wide range of domains and paradigms. Here, we review a few representative applications of our models, focusing on two domains: recognition memory and animal learning in the fear conditioning paradigm. In both domains, the models have generated novel predictions that have been tested and confirmed.
[PDF] Rudy, J.W., Barrientos, R.M. & OReilly, R.C. (2002). Hippocampal Formation Supports Conditioning to Memory of a Context. Behavioral Neuroscience, 116, 530-538.
Abstract:
It has been proposed that contextual fear conditioning depends on two processes: (a) the construction of a conjunctive representation of the features that make up the context, and (b) associating that representation with shock. Support for this view comes from studies indicating that prior exposure to the conditioning context facilitates contextual fear conditioning supported by immediate shock. An implication of this result is that conditioning produced by immediate shock is to the memory representation of the preexposed context, which is activated by retrieval cues associated with the preexposed context. Our experiments support this interpretation and indicate that this process depends on an intact hippocampal formation. These results support the hypothesis that the hippocampal formation supports contextual fear conditioning by storing a conjunctive representation of context.
[PDF] Barrientos, R.M., OReilly, R.C. & Rudy, J.W. (2002). Memory for context is impaired by injecting anisomycin into dorsal hippocampus following context exploration. Behavioural Brain Research, 134, 299-306.
Abstract:
Pre-exposure to the context facilitates the small amount of contextual fear conditioning that is normally produced by immediate shock. This context pre-exposure facilitation effect provides a convenient way to study the rat's learning about context. We recently reported that anterograde damage to dorsal hippocampus prevents this facilitation. The present experiments strengthen this conclusion by showing that the protein synthesis inhibitor, anisomycin, injected bilaterally into the dorsal hippocampus following context pre-exposure also significantly reduces the facilitation effect. The same treatment given immediately after immediate shock, however, had no effect on facilitation. These results support theories that assume that, (a) contextual fear involves two processes, acquiring and storing a conjunctive representation of a context and associating that representation with fear; and (b) the hippocampus contributes to contextual fear by participating in the storage of the memory representation of the context.
[PDF] Holdstock, J.S., Mayes, A.R., Roberts, N., Cezayirli, E., Isaac, C.L., O'Reilly, R.C. & Norman, K.A. (2002). Under What Conditions is Recognition Spared Relative to Recall After Selective Hippocampal Damage in Humans?. Hippocampus, 12, 341-351.
Abstract:
The claim that recognition memory is spared relative to recall after focal hippocampal damage has been disputed in the literature. We examined this claim by investigating object and object location recall and recognition memory in a patient, YR, who has adult-onset selective hippocampal damage. Our aim was to identify the conditions under which recognition was spared relative to recall in this patient. She showed unimpaired forced-choice objectre cognition but clearly impaired recall, even when her control subjects found the object recognition task to be numerically harder than the object recall task. However, on two other recognition tests, YR's performance was not relatively spared. First, she was clearly impaired at an equivalently difficult yes/no object recognition task, but only when targets and foils were very similar. Second, YR was clearly impaired at forced-choice recognition of object location associations. This impairment was also unrelated to difficulty because this task was no more difficult than the forced-choice object recognition task for control subjects. The clear impairment of yes/no, but not of forced-choice, object recognition after focal hippocampal damage, when targets and foils are very similar, is predicted by the neural network-based Complementary Learning Systems model of recognition. This model postulates that recognition is mediated by hippocampally dependent recollection and cortically dependent familiarity; thus hippocampal damage should not impair item familiarity. The model postulates that familiarity is ineffective when very similar targets and foils are shown one at a time and subjects have to identify which items are old (yes/no recognition). In contrast, familiarity is effective in discriminating which of similar targets and foils, seen together, is old (forced-choice recognition). Independent evidence from the remember/know procedure also indicates that YR's familiarity is normal. The Complementary Learning Systems model can also accommodate the clear impairment of forced-choice object location recognition memory if it incorporates the view that the most complete convergence of spatial and object information, represented in different cortical regions, occurs in the hippocampus.
[PDF] Rougier, N.P. & O`Reilly, R.C. (2002). Learning representations in a gated prefrontal cortex model of dynamic task switching. Cognitive Science, 26, 503-520.
Abstract:
The prefrontal cortex is widely believed to play an important role in facilitating people's ability to switch performance between different tasks. We present a biologically-based computational model of prefrontal cortex (PFC) that explains its role in task switching in terms of the greater flexibility conferred by activation-based working memory representations in PFC, as compared with more slowly adapting weight-based memory mechanisms. Specifically we show that PFC representations can be rapidly updated when a task switches via a dynamic gating mechanism based on a temporal-differences reward-prediction learning mechanism. Unlike prior models of this type, the present model develops all of its internal representations via learning mechanisms as shaped by the demands of continuous periodic task switching. This advance opens up a new domain of research into the interactions between working memory task demands and the representations that develop to meet them. Results on a version of the Wisconsin Card Sorting task are presented for the full model and a number of comparison networks that test the importance of various model features. Furthermore, we show that a lesioned model produces perseverative errors like those seen in frontal patients.
[PDF] O'Reilly, R.C. & Soto, R. (2002). A Model of the Phonological Loop: Generalization and Binding. Advances in Neural Information Processing Systems (NIPS) 14,T.G. Dietterich, S. Becker & Z. Ghahramani (Eds), Cambridge, MA; MIT Press
Abstract:
We present a neural network model that shows how the prefrontal cortex, interacting with the basal ganglia, can maintain a sequence of phonological information in activation-based working memory (i.e., the phonological loop). The primary function of this phonological loop may be to transiently encode arbitrary bindings of information necessary for tasks --- the combinatorial expressive power of language enables very flexible binding of essentially arbitrary pieces of information. Our model takes advantage of the closed-class nature of phonemes, which allows different neural representations of all possible phonemes at each sequential position to be encoded. To make this work, we suggest that the basal ganglia provide a region-specific update signal that allocates phonemes to the appropriate sequential coding slot. To demonstrate that flexible, arbitrary binding of novel sequences can be supported by this mechanism, we show that the model can generalize to novel sequences after moderate amounts of training.
[PDF] O'Reilly, R.C. & Busby, R.S. (2002). Generalizable Relational Binding from Coarse-coded Distributed Representations. Advances in Neural Information Processing Systems (NIPS) 14,T.G. Dietterich, S. Becker & Z. Ghahramani (Eds), Cambridge, MA; MIT Press
Abstract:
We present a model of binding of relationship information in a spatial domain (e.g., square above triangle) that uses low-order coarse-coded conjunctive representations instead of more popular temporal synchrony mechanisms. Supporters of temporal synchrony argue that conjunctive representations lack both efficiency (i.e., combinatorial numbers of units are required) and systematicity (i.e., the resulting representations are overly specific and thus do not support generalization to novel exemplars). To counter these claims, we show that our model: a) uses far fewer hidden units than the number of conjunctions represented, by using coarse-coded, distributed representations where each unit has a broad tuning curve through high-dimensional conjunction space, and b) is capable of considerable generalization to novel inputs.
[PDF] O'Reilly, R.C. & Munakata, Y. (2002). Psychological Function in Computational Models of Neural Networks. M. Gallagher & R. Nelson (Eds) Handbook of Psychology, Vol 3, Biological Psychology, New York: Wiley.
Abstract:
The overarching goal of cognitive neuroscience is to understand how the brain gives rise to thought. Toward this goal, researchers employ various methods to measure neural variables while people and other animals think. A complementary method, computer models of neural networks, allows unparalleled levels of control and supports the further understanding of the relation between brain and mind. Using these models, one can simulate a network of interacting neurons, and measure cognitive function in these networks at the same time. Furthermore, many variables in these networks can be manipulated, so that their effects on cognitive processes can be observed. In this chapter, we provide an up-to-date review of some of the core principles and prominent applications of computational models in cognitive neuroscience, based on our recent textbook on this topic (O'Reilly & Munakata, 2000). We begin with a summary of some of the basic questions confronting computational modelers in cognitive neuroscience. We then discuss provisional answers to these questions, showing how they apply to a range of empirical data. Throughout, and in closing, we discuss challenges to neural network models. We will see how some network models can have possibly problematic properties, often driven by constraints from biology or cognition, but the models can nonetheless help to advance the field of cognitive neuroscience.
[PDF] O'Reilly, R.C., Noelle, D., Braver, T.S. & Cohen, J.D. (2002). Prefrontal cortex and dynamic categorization tasks: Representational organization and neuromodulatory control. Cerebral Cortex, 12, 246-257.
Abstract:
We present a computational model of the intradimensional / extradimensional (ID/ED) task (a variant of the Wisconsin card sorting task) that simulates the performance of intact and frontally-lesioned monkeys on three different kinds of rule changes (Dias, Robbins, & Roberts, 1997). Although Dias et al. interpret the lesion data as supporting a model in which prefrontal cortex is organized into different processing functions, our model suggests an alternative account based on representational content. A key aspect of the model is that prefrontal cortex representations are organized according to different levels of abstraction, with orbital areas encoding more specific featural information and dorsolateral areas encoding more abstract dimensional information. This representational scheme of the model is integrated with two additional key elements: (a) activation-based working memory representations controlled by a dynamic gating mechanism that simulates the hypothesized phasic actions of dopaminergic neuromodulation in prefrontal cortex, which acts to stabilize or destabilize frontal representations based on success in the task; and (b) a weight-based associative learning system simulating posterior cortex and other subcortical areas, where the stimulus-response mappings are encoded. Frontal cortex contributes to the task via top-down activation-based biasing of task-appropriate features and dimensions in this posterior cortex system this top-down biasing is specifically important for overcoming prepotent associations after a sorting rule reverses. The ability of the model to capture the double-dissociation observed by Dias et al. with orbital versus dorsolateral lesions supports the validity of these principles, many of which have also been useful in accounting for other frontal phenomena.
[PDF] O'Reilly, R.C. & McClelland, J.L. (2001). The Importance of Modeling for the Future of Molecular Studies of Learning and Memory. A.J. Silva (Ed) Molecular Studies of Learning and Memory, : Unpublished.
Abstract:
Computational models are important for guiding and interpreting molecular studies of learning and memory because they provide a bridge between biological and behavioral levels of analysis. These models facilitate the identification of central underlying principles that span different levels, and they can accommodate the many complexities of each of these levels and their interrelationships. We present an overviewof our computational approach towards understanding the different contributions of the neocortex and hippocampus in learning and memory. The approach is based on a set of principles derived from converging biological, psychological, and computational constraints. This framework provides insight into: (a) which behavioral paradigms are most appropriate for dissociating cortical and hippocampal learning contributions; (b) effects of selective impairments on hippocampal areas; (c) effects of synaptic modification (LTP/LTD) impairments on various pathways within the hippocampal system; (d) effects of manipulations of hippocampal activation parameters; and (e) effects of manipulations that selectively impair different components of learning (Hebbian versus error-driven). We hope that these insights provide fruitful avenues for further research using molecular techniques to inform our understanding of exactly how learning and memory phenomena emerge from their biological basis.
[PDF] Rudy, J.W. & O'Reilly, R.C. (2001). Conjunctive Representations, the Hippocampus, and Contextual Fear Conditioning. Cognitive, Affective, and Behavioral Neuroscience, 1, 66-82.
Abstract:
The context in which events occur can be represented either as (a) a set of independent features, the feature representation view, or (b) the features can be bound into a unitary representation of their co-occurrence, the conjunction representation view. Extrahippocampal (e.g., neocortical) areas provide a basis for feature representations, but the hippocampus makes an essential contribution to the automatic storage of conjunctive representations. We develop this dual representation view and explore its implications for hippocampal contributions to contextual fear conditioning processes. To this end, we discuss how our framework can resolve some of the conflicts in the recent literature relating the hippocampus to contextual fear conditioning. We also present new data supporting the role of a key mechanism afforded by conjunctive representations, pattern completion (the ability of a subset of a memory pattern to activate the complete memory), in contextual fear conditioning. As implied by this mechanism, we report that fear can be conditioned to the memory representation of a context that is not actually present at the time of shock. Moreover, this result is predicted by our computational model of cortical and hippocampal function. We suggest that pattern completion demonstrated in animals and by our model provides a mechanistic bridge to human declarative memory.
[PDF] O'Reilly, R.C. (2001). Generalization in Interactive Networks: The Benefits of Inhibitory Competition and Hebbian Learning. Neural Computation, 13, 1199-1242.
Abstract:
Computational models in cognitive neuroscience should ideally use biological properties and powerful computational principles to produce behavior consistent with psychological findings. Error-driven backpropagation is computationally powerful, and has proven useful for modeling a range of psychological data, but is not biologically plausible. Several approaches to implementing backpropagation in a biologically plausible fashion converge on the idea of using bidirectional activation propagation in interactive networks to convey error signals. This paper demonstrates two main points about these error-driven interactive networks: (a) they generalize poorly due to attractor dynamics that interfere with the network's ability to systematically produce novel combinatorial representations in response to novel inputs; and (b) this generalization problem can be remedied by adding two widely used mechanistic principles, inhibitory competition and Hebbian learning, that can be independently motivated for a variety of biological, psychological and computational reasons. Simulations using the Leabra algorithm, which combines the generalized recirculation (GeneRec) biologically-plausible error-driven learning algorithm with inhibitory competition and Hebbian learning, show that these mechanisms can result in good generalization in interactive networks. These results support the general conclusion that cognitive neuroscience models that incorporate the core mechanistic principles of interactivity, inhibitory competition, and error-driven and Hebbian learning satisfy a wider range of biological, psychological and computational constraints than models employing a subset of these principles.
[PDF] Frank, M.J., Loughry, B. & O'Reilly, R.C. (2001). Interactions between the frontal cortex and basal ganglia in working memory: A computational model. Cognitive, Affective, and Behavioral Neuroscience, 1, 137-160.
Abstract:
The frontal cortex and basal ganglia interact via a relatively well-understood and elaborate system of interconnections. In the context of motor function, these interconnections can be understood as disinhibiting or ``releasing the brakes'' on frontal motor action plans --- the basal ganglia detect appropriate contexts for performing motor actions, and enable the frontal cortex to execute such actions at the appropriate time. We build on this idea in the domain of working memory through the use of computational neural network models of this circuit. In our model, the frontal cortex exhibits robust active maintenance, while the basal ganglia contribute a selective, dynamic gating function that enables frontal memory representations to be rapidly updated in a task-relevant manner. We apply the model to a novel version of the continuous performance task (CPT) that requires subroutine-like selective working memory updating, and compare and contrast our model with other existing models and theories of frontal cortex--basal ganglia interactions.
[PDF] O'Reilly, R.C. & Rudy, J.W. (2001). Conjunctive Representations in Learning and Memory: Principles of Cortical and Hippocampal Function. Psychological Review, 108, 311-345.
Abstract:
We present a theoretical framework for understanding the roles of the hippocampus and neocortex in learning and memory. This framework incorporates a theme found in many theories of hippocampal function, that the hippocampus is responsible for developing conjunctive representations binding together stimulus elements into a unitary representation that can later be recalled from partial input cues. This idea appears problematic, however, because it is contradicted by the fact that hippocampally lesioned rats can learn nonlinear discrimination problems that require conjunctive representations. Our framework accommodates this finding by establishing a principled division of labor between the cortex and hippocampus, where the cortex is responsible for slow learning that integrates over multiple experiences to extract generalities, while the hippocampus performs rapid learning of the arbitrary contents of individual experiences. This framework shows that nonlinear discrimination problems are not good tests of hippocampal function, and suggests that tasks involving rapid, incidental conjunctive learning are better. We implement this framework in a computational neural network model, and show that it can account for a wide range of data in animal learning, thus validating our theoretical ideas, and providing a number of insights and predictions about these learning phenomena.
[PDF] Munakata, Y., Santos, L.R., Spelke, E.S., Hauser, M.D. & O'Reilly, R.C. (2001). Visual representation in the wild: How rhesus monkeys parse objects. Journal of Cognitive Neuroscience, 13, 44-58.
Abstract:
Visual object representation was studied in free-ranging rhesus monkeys. To facilitate comparison with humans, and to provide a new tool for neurophysiologists, we used a looking time procedure originally developed for studies of human infants. Monkeys looking times were measured to displays with one or two distinct objects, separated or together, stationary or moving. Results indicate that rhesus monkeys used featural information to parse the displays into distinct objects, and they found events in which distinct objects moved together more novel or unnatural than events in which distinct objects moved separately. These findings show both commonalities and contrasts with those obtained from human infants. We discuss their implications for the development and neural mechanisms of higher-level vision.
[PDF] O'Reilly, R.C. & Rudy, J.W. (2000). Computational Principles of Learning in the Neocortex and Hippocampus. Hippocampus, 10, 389-397.
Abstract:
We present an overview of our computational approach towards understanding the different contributions of the neocortex and hippocampus in learning and memory. The approach is based on a set of principles derived from converging biological, psychological, and computational constraints. The most central principles are that the neocortex employs a slow learning rate and overlapping distributed representations to extract the general statistical structure of the environment, while the hippocampus learns rapidly using separated representations to encode the details of specific events while suffering minimal interference. Additional principles concern the nature of learning (error-driven and Hebbian), and recall of information via pattern completion. We summarize the results of applying these principles to a wide range of phenomena in conditioning, habituation, contextual learning, recognition memory, recall, and retrograde amnesia, and point to directions of current development.
[Main PDF] [Supp PDF] Norman, K.A., O'Reilly, R.C. & Huber, D.E. (2000). Modeling Neocortical Contributions to Recognition Memory. The Cognitive Neuroscience Meeting, 2000,
Abstract:
In this poster, we present a biologically-based dual-process model of recognition memory. Dual-process models posit that recognition judgments are based on: - recollection of specific details, and - nonspecific feelings of familiarity Recollection depends on the hippocampus. Recent data suggest that medial temporal neocortical regions (MTLC) play an important role in supporting familiarity-based recognition (for a review, see Aggleton & Brown, 1999). We seek to understand, in mechanistic detail, how MTLC and the hippocampus contribute to recognition memory, by constructing neural network models of these structures, and using them to simulate recognition data from lesioned and intact subjects.
[PDF] Vecera, S.P. & O'Reilly, R.C. (2000). Graded Effects in Hierarchical Figure-Ground Organization: Reply To Peterson (1999). Journal of Experimental Psychology: Human Perception and Performance, 26, 1221-1230.
Abstract:
An important issue in vision research concerns the order of visual processing. S. P. Vecera and R. C. O'Reilly (1998) presented an interactive, hierarchical model that placed figure-ground segregation prior to object recognition. M. A. Peterson (1999) critiqued this model, arguing that because it used ambiguous stimulus displays, figure-ground processing did not precede object processing. In the current article, the authors respond to Peterson's (1999) interpretation of ambiguity in the model and her interpretation of what it means for figure-ground processing to come before object recognition. The authors argue that complete stimulus ambiguity is not critical to the model and that figure-ground precedes object recognition architecturally in the model. The arguments are supported with additional simulation results and an experiment, demonstrating that top-down inputs can influence figure-ground organization in displays that contain stimulus cues.
[PDF] Rudy, J.W. & O'Reilly, R.C. (1999). Contextual Fear Conditioning, Conjunctive Representations, Pattern Completion, and the Hippocampus. Behavioral Neuroscience, 113, 867-880.
Abstract:
Impaired contextual fear conditioning produced by damage to the hippocampus has been attributed to the loss of a conjunctive representation of the features of the context. There is, however, no direct evidence that conjunctive representations contribute to contextual fear conditioning. Our experiments addressed this issue and support the conjunctive representation view. Two results make this point: (a) preexposure to the conditioning context, but not to itsseparable features, facilitates contextual fear conditioning, and (b) generalization of fear conditioning to similar contexts is enhanced by preexposure to the context used to test for generalization --- we interpret this as pattern completion to the preexposed context during the conditioning episode. These results support the view that a conjunctive representation of context plays an important role in contextual fear conditioning, and that the impairments produced by damage to the hippocampus result from the loss of this conjunctive contribution.
[PDF] O'Reilly, R.C., Braver, T.S. & Cohen, J.D. (1999). A Biologically Based Computational Model of Working Memory. A. Miyake & P. Shah (Eds) Models of Working Memory: Mechanisms of Active Maintenance and Executive Control., 375-411, New York: Cambridge University Press.
Abstract:
We define working memory as controlled processing involving active maintenance and/or rapid learning, where controlled processing is an emergent property of the dynamic interactions of multiple brain systems, but the prefrontal cortex (PFC) and hippocampus (HCMP) are especially influential due to their specialized processing abilities and their privileged locations within the processing hierarchy (both the PFC and HCMP are well connected with a wide range of brain areas, allowing them to influence behavior at a global level). The specific features of our model include: 1. A PFC specialized for active maintenance of internal contextual information that is dynamically updated and self-regulated, allowing it to bias (control) ongoing processing according to maintained information (e.g., goals, instructions, partial products, etc). 2. A HCMP specialized for rapid learning of arbitrary information, which can be recalled in the service of controlled processing, while the posterior perceptual and motor cortex (PMC) exhibits slow, long-term learning that can efficiently represent accumulated knowledge and skills. 3. Control that emerges from interacting systems (PFC, HCMP and PMC). 4. Dimensions that define continua of specialization in different brain systems: e.g., robust active maintenance, fast vs slow learning. 5. Integration of biological and computational principles.
[PDF] O'Reilly, R.C., Mozer, M., Munakata, Y. & Miyake, A. (1999). Discrete Representations in Working Memory: A Hypothesis and Computational Investigations. The Second International Conference on Cognitive Science,183-188, Tokyo; Japanese Cognitive Science Society
Abstract:
We present a novel hypothesis concerning the nature and development of working memory representations, and some initial computational investigations of this hypothesis. Working memory refers to the active maintenance of information in the service of complex cognition, such as language comprehension, spatial thinking, and problem solving (Miyake & Shah, 1999). We propose that the unique demands placed on the working memory system shape its representations over learning and development, affecting the use of working memory by the cognitive system as a whole. Our primary source of insight into this process comes from a computational analysis, which is used to integrate and explore relevant findings from neurobiology as well as developmental and adult cognition.
[PDF] O'Reilly, R.C. & Farah, M.J. (1999). Simulation and Explanation in Neuropsychology and Beyond. Cognitive Neuropsychology, 16, 49-72.
Abstract:
This paper is a reply to a critique by Young & Burton (same issue) of our earlier prosopagnosia model (Farah, O'Reilly & Vecera, 1993, Psychological Review). We provide a number of new extensions to our original model, and address general issues in the debate between distributed, learning-based models and localist, hand-wired models.
[PDF] O'Reilly, R.C. (1998). Six Principles for Biologically-Based Computational Models of Cortical Cognition. Trends in Cognitive Sciences, 2, 455-462.
Abstract:
This paper describes and motivates six principles for computational cognitive neuroscience models: biological realism, distributed representations, inhibitory competition, bidirectional activation propagation, error-driven task learning, and Hebbian model learning. Although these principles are supported by a number of cognitive, computational, and biological motivations, the prototypical neural network model (a feedforward backpropagation network) incorporates only two of them, and no widely used model incorporates all of them. This paper argues that these principles should be integrated into a coherent overall framework, and discusses some potential synergies and conflicts in doing so.
[PDF] O'Reilly, R.C., Norman, K.A. & McClelland, J.L. (1998). A Hippocampal Model of Recognition Memory. M.I. Jordan, M.J. Kearns & S.A. Solla (Eds) Advances in Neural Information Processing Systems 10, 73-79, Cambridge, MA: MIT Press.
Abstract:
A rich body of data exists showing that recollection of specific information contributes to recognition memory. We present a model, based largely on known features of hippocampal anatomy and physiology, that accounts for both the {\em high-threshold} nature of this recollection process (i.e., the fact that only studied items are recollected, although nonstudied items sometimes trigger recollection of similar studied items), and the fact that increasing interference leads to less recollection but apparently does not compromise the {\em quality} of recollection (i.e., the extent to which recollection veridically reflects events that occurred at study). Both of these properties of recollection pose problems for existing computational and mathematical models.
[PDF] Vecera, S.P. & O'Reilly, R.C. (1998). Figure-Ground Organization and Object Recognition Processes: An Interactive Account. Journal of Experimental Psychology: Human Perception and Performance, 24, 441-462.
Abstract:
Traditional theories of visual processing have assumed that figure-ground organization must precede object representation and identification. Such a view seems logically necessary: How can one recognize an object before the visual system knows which region should be the figure? However, a number of behavioral studies have shown that subjects are more likely to call a familiar region ``figure'' relative to a less familiar region, a finding inconsistent with the traditional accounts of visual processing. To explain these results, Peterson and colleagues have proposed an additional ``prefigural'' object recognition process that operates before any figure-ground organization (M. A. Peterson, 1994). We propose a more parsimonious interactive account of figure-ground organization in which partial results of figure-ground processes interact with object representations in a hierarchical system similar to that envisioned by traditional theories. We present a computational model that embodies this graded, interactive approach and show that this model can account several behavioral results, including orientation effects, exposure duration effects, and the combination of multiple cues. Finally, these principles of graded, interactive processing offer the possibility of providing a more general information processing framework for visual and higher-cognitive systems.
[PDF] Cohen, J.D. & O'Reilly, R.C. (1996). A Preliminary Theory of the Interactions Between Prefrontal Cortex and Hippocampus that Contribute to Planning and Prospective Memory. M. Brandimonte, G.O. Einstein & M.A. McDaniel (Eds) Prospective Memory: Theory and Applications, 267-296, Mahwah, New Jersey: Lawrence Erlbaum Associates.
Abstract:
This chapter addresses the neurobiological mechanisms that may underlie prospective memory. In our work, we have exploited the use of computational modeling techniques to help identify the role that specific brain structures play in cognition. In particular, we have used such techniques to characterize the function of prefrontal cortex and hippocampus in terms of specific processing mechanisms. This work suggests that an important function of prefrontal cortex is the representation and maintenance of contextual information -- information that must be held in mind in such a form that it can be used to mediate an appropriate behavioral response. At the same time, our work supports the idea that an important function of the hippocampus is to rapidly establish novel associations, that can also be used to guide behavior. In our view, prospective memory reflects the interaction between these two systems, allowing established sequences of behavior to be associated with new conditions -- in effect, providing a mechanism for planning.
[PDF] Cohen, J.D., Braver, T.S. & O'Reilly, R.C. (1996). A Computational Approach to Prefrontal Cortex, Cognitive Control, and Schizophrenia: Recent Developments and Current Challenges. Philosophical Transactions of the Royal Society (London) {B}, 351, 1515-1527.
Abstract:
In this chapter we consider the mechanisms involved in cognitive control --- from botha computational and a neurobiological perspective --- and how these might be impaired in schizophrenia. By 'control' we mean the ability of the cognitive system to flexibly adapt its behaviour to the demands of particular tasks, favouring the processing of task-relevant information over other sources of competing information, and mediating task-relevant behaviour over habitual, or otherwise prepotent responses. There is a large body of evidence to suggest that the prefrontal cortex (PFC) plays a critical role in cognitive control. In previous work, we have used a computational framework to understand and develop explicit models of this function of PFC, and its impairment in schizophrenia. This work has lead to the hypothesis that PFC houses a mechanism for representing and maintainging context information. We have demonstrated that this mechanism can account for the behavioural inhibition and active memory functions commonly ascribed to PFC, and for human performance in simple attention, language and memory tasks that draw upon these functions for cognitive control. Furthermore, we have used our models to simulate detailed patterns of cognitive deficit observed in schizophrenia, an illness associated with marked distrubances in cognitive control, and well established deficits of PFC. Here, we review results of recent empirical studies that test predictions made by our models regarding schizophrenic performance in tasks designed specifically to probe the processing of context. These results showed selective schizophrenic deficits in task conditions that placed the greatest demands on memory and inhibition, both of which we have argued rely on the processing of context. Furthermore, we observed predicted patterns of deterioration in first episode vs multi-episode patients. We also discuss recent developments in our computational work, that have led to refinements of the models that allow us to simulate more detailed aspects of task performance, such as reaction time data and manipulations of task parameters such as interstimulus delay. These refined models make several provocative new predictions, including conditions in which schizophrenics and control subjects are expected to show similar reaction time performance, and we provide preliminary data in support of these predictions. These successes notwhithstanding, our theory of PFC function and its impairment in schizophrenia is still in an early stage of development. We conclude by presenting some of the challenges to the theory in its current form, and new directions that we have begun to take to meet these challenges. In particular, we focus on refinements concerning the mechanisms underlying active maintenance of representations within PFC, and the characteristics of these representations that allow them to support the flexibility of cognitive control exhibited by normal human behaviour. Taken in toto, we believe that this work illustrates the value of a computational approach for understanding the mechanisms responsible for cognitive control, at both the neural and psychological levels, and the specific manner in which they break down in schizophrenia.
[PDF] O'Reilly, R.C. (1996). Biologically Plausible Error-driven Learning using Local Activation Differences: The Generalized Recirculation Algorithm. Neural Computation, 8, 895-938.
Abstract:
The error backpropagation learning algorithm (BP) is generally considered biologically implausible because it does not use locally available, activation-based variables. A version of BP that can be computed locally using bi-directional activation recirculation (Hinton & McClelland, 1988) instead of backpropagated error derivatives is more biologically plausible. This paper presents a generalized version of the recirculation algorithm (GeneRec), which overcomes several limitations of the earlier algorithm by using a generic recurrent network with sigmoidal units that can learn arbitrary input/output mappings. However, the contrastive-Hebbian learning algorithm (CHL), a.k.a. DBM or mean field learning) also uses local variables to perform error-driven learning in a sigmoidal recurrent network. CHL was derived in a stochastic framework (the Boltzmann machine), but has been extended to the deterministic case in various ways, all of which rely on problematic approximations and assumptions, leading some to conclude that it is fundamentally flawed. This paper shows that CHL can be derived instead from within the BP framework via the GeneRec algorithm. CHL is a symmetry-preserving version of GeneRec which uses a simple approximation to the midpoint or second-order accurate Runge-Kutta method of numerical integration, which explains the generally faster learning speed of CHL compared to BP. Thus, all known fully general error-driven learning algorithms that use local activation-based variables in deterministic networks can be considered variations of the GeneRec algorithm (and indirectly, of the backpropagation algorithm). GeneRec therefore provides a promising framework for thinking about how the brain might perform error-driven learning. To further this goal, an explicit biological mechanism is proposed which would be capable of implementing GeneRec-style learning. This mechanism is consistent with available evidence regarding synaptic modification in neurons in the neocortex and hippocampus, and makes further predictions.
[PDF] O'Reilly, R.C. (1996). The Leabra Model of Neural Interactions and Learning in the Neocortex. Phd Thesis, Carnegie Mellon University, Pittsburgh, PA
Abstract:
There is evidence that the specialized neural processing systems in the neocortex, which are responsible for much of human cognition, arise from the action of a relatively general-purpose learning mechanism. I propose that such a neocortical learning mechanism can be best understood as the combination of error-driven and self-organizing (Hebbian associative) learning. This model of neocortical learning, called LEABRA (local, error-driven and associative, biologically realistic algorithm), is computationally powerful, has important implications for psychological models, and is biologically feasible. The thesis begins with an evaluation of the strengths and limitations of current neural network learning algorithms as models of a neocortical learning mechanism according to psychological, biological, and computational criteria. I argue that error-driven (e.g., backpropagation) learning is a reasonable computational and psychological model, but it is biologically implausible. I show that backpropagation can be implemented in a biologically plausible fashion by using interactive (bi-directional, recurrent) activation flow, which is known to exist in the neocortex, and has been important for accounting for psychological data. However, the interactivity required for biological and psychological plausibility significantly impairs the ability to respond systematically to novel stimuli, making it still a bad psychological model (e.g., for nonword reading). I propose that the neocortex solves this problem by using inhibitory activity regulation and Hebbian associative learning, the computational properties of which have been explored in the context of self-organizing learning models. I show that by introducing these properties into an interactive (biologically plausible) error-driven network, one obtains a model of neocortical learning that: 1) provides a clear computational role for a number of biological features of the neocortex; 2) behaves systematically on novel stimuli, and exhibits transfer to novel tasks; 3) learns rapidly in networks with many hidden layers; 4) provides flexible access to learned knowledge; 5) shows promise in accounting for psychological phenomena such as the U-shaped curve in over-regularization of the past-tense inflection; 6) has a number of other nice properties.
[PDF] McClelland, J.L., McNaughton, B.L. & O'Reilly, R.C. (1995). Why There are Complementary Learning Systems in the Hippocampus and Neocortex: Insights from the Successes and Failures of Connectionist Models of Learning and Memory. Psychological Review, 102, 419-457.
Abstract:
The use of information from prior experience in explicit memory tasks is drastically affected by damage to the hippocampal system. However, if the hippocampus and related structures are left intact for some time after an experience, resistance to later disruption from hippocampal damage develops. This process, called consolidation, can span several years in humans. The data are consistent with the view that the initial memory trace consists of changes to connections among neurons in the hippocampal system; that incoming and outgoing pathways allow these changes to mediate the reinstatement of representations of recent experiences in the neocortex; and that consolidation results from cumulative effects of small changes to neocortical connections that occur each time a representation is reinstated. Two questions arise: 1) Why are plastic changes made initially in the hippocampus, if the ultimate substrate of consolidated memory is in the neocortex? 2) Why does consolidation span an extended time period? Connectionist models of learning and memory provide possible answers to these questions. These models offer procedures for assigning weights to the connections among simple processing units so that a network of such units may capture the structure in ensembles of experiences. The success of these procedures depends on making small weight changes on each learning trial. Rapid acquisition of new data is incompatible with gradual discovery of structure and can lead to catastrophic interference. This suggests that the neocortex may be optimized for discovery of shared structure, and that the hippocampal system provides rapid acquisition of new information without interference with previously discovered regularities. After initial acquisition, the hippocampal system serves as teacher to the rest of the brain: It reinstates representations of events so that they may be gradually learned by the cortical system, interleaved with other experiences. We equate interleaved learning with consolidation and suggest that it is slow to allow new material to become integrated into structured systems of knowledge in the neocortex.
[PDF] O'Reilly, R.C. & McClelland, J.L. (1994). Hippocampal Conjunctive Encoding, Storage, and Recall: Avoiding a Tradeoff. Hippocampus, 4, 661-682.
Abstract:
The hippocampus and related structures are thought to be capable of: 1) representing cortical activity in a way that minimizes overlap of the representations assigned to different cortical patterns (pattern separation); and 2) modifying synaptic connections so that these representations can later be reinstated from partial or noisy versions of the cortical activity pattern that was present at the time of storage (pattern completion). We point out that there is a tradeoff between pattern separation and completion, and propose that the unique anatomical and physiological properties of the hippocampus might serve to minimize this tradeoff. We use analytical methods to determine parameterized models of the hippocampus. These estimates are then used to evaluate the role of various properties and of the hippocampus, such as the activity levels seen in different hippocampal regions, synaptic potentiation and depression, the multi-layer connectivity of the system, and the relatively focused and strong mossy fiber projections. This analysis is focused on the feedforward pathways from the Entorhinal Cortex (EC) to the Dentate Gyrus (DG) and region CA3. Among our results are the following: 1) Hebbian synaptic modification (LTP) facilitates completion but reduces separation, unless the strengths of synapses from inactive presynaptic units to active postsynaptic units are reduced (LTD). 2) Multiple layers, as in EC to DG to CA3, allow the compounding of pattern separation, but not pattern completion. 3) The variance of the input signal carried by the mossy fibers is important for separation, not the raw strength, which may explain why the mossy fiber inputs are few and relatively strong, rather than many and relatively weak like the other hippocampal pathways. 4) The EC projects to CA3 both directly and indirectly via the DG, which suggests that the two-stage pathway may dominate during pattern separation and the one-stage pathway may dominate during completion; methods the hippocampus may use to enhance this effect are discussed.
[PDF] O'Reilly, R.C. & Johnson, M.H. (1994). Object Recognition and Sensitive Periods: A Computational Analysis of Visual Imprinting. Neural Computation, 6, 357-389.
Abstract:
Evidence from a variety of methods suggests that a localized portion of the domestic chick brain, the Intermediate and Medial Hyperstriatum Ventrale (IMHV), is critical for filial imprinting. Data further suggest that IMHV is performing the object recognition component of imprinting, as chicks with IMHV lesions are impaired on other tasks requiring object recognition. We present a neural network model of translation invariant object recognition developed from computational and neurobiological considerations that incorporates some features of the known local circuitry of IMHV. In particular, we propose that the recurrent excitatory and lateral inhibitory circuitry in the model, and observed in IMHV, produces hysteresis on the activation state of the units in the model and the principal excitatory neurons in IMHV. Hysteresis, when combined with a simple Hebbian covariance learning mechanism, has been shown in earlier work to produce translation invariant visual representations. To test the idea that IMHV might be implementing this type of object recognition algorithm, we have used a simple neural network model to simulate a variety of different empirical phenomena associated with the imprinting process. These phenomena include reversibility, sensitive periods, generalization, and temporal contiguity effects observed in behavioral studies of chicks. In addition to supporting the notion that these phenomena, and imprinting itself, result from the IMHV properties captured in the simplified model, the simulations also generate several predictions and clarify apparent contradictions in the behavioral data.
[PDF] O'Reilly, R.C. (1994). Temporally Local Unsupervised Learning: The MaxIn Algorithm for Maximizing Input Information. Proceedings of the 1993 Connectionist Models Summer School,M.C. Mozer, P. Smolensky & A.S. Weigend (Eds), Hillsdale, NJ; Lawrence Erlbaum Associates
Abstract:
There are many appealing aspects of self-organizing learning rules, among them the notion that they are more ``biologically plausible'' than supervised learning algorithms. This plausibility usually derives from the ability to compute the algorithm with variables available locally to the unit or ``neuron'', typically using some variant of a Hebbian learning rule. Ironically, however, the locality in time of the variables that determine the learning is often ignored. This temporal non-locality presents a problem both from a biological and a psychological standpoint. In this paper, I present an alternative objective function for self-organizing algorithms that is local in both space and time, and results in a simple learning rule that can be implemented with properties of neuronal synaptic modification.
[PDF] Farah, M.J., O'Reilly, R.C. & Vecera, S.P. (1993). Dissociated Overt and Covert Recognition as an Emergent Property of a Lesioned Neural Network. Psychological Review, 100, 571-588.
Abstract:
Covert recognition of faces in prosopagnosia, in which patients cannot overtly recognize faces but nevertheless mainfest recognition when tested in certain indirect ways, has been interpreted as the functioning of an intact visual face recognition system deprived of access to other brain systems necessary for consciousness. The authors proposa an alternative hypothesis: that the visual face recognition system is damaged but not obliterated in these patients and that damaged neural networks will mainfest their residual knowledge in just the kinds of tasks used to measure overt recognition. To test this, a simple model of face recognition is lesioned in the parts of the model corresponding to visual processing. The model demonstrates covert recognition in 3 qualitatively different tasks. Implications for the nature of prosopagnosia, and for other types of dissociations between conscious and unconscious perception, are discussed.
[PDF] O'Reilly, R.C. & McClelland, J.L. (1992). The Self-Organization of Spatially Invariant Representations. Technical Report {PDP.CNS.92.5} Carnegie Mellon University, Department of Psychology.
Abstract:
The problem of computing object-based visual representations can be construed as the development of invariancies to visual dimensions irrelevant for object identity. This view, when implemented in a neural network, suggests a different set of algorithms for computing object-based visual representations than the ``traditional'' approach pioneered by Marr (1980). A biologically plausible self-organizing neural network model that develops spatially invariant representations is presented. There are four features of the self-organizing algorithm that contribute to the development of spatially invariant representations: temporal continuity of environmental stimuli, hysteresis of the activation state (via recurrent activation loops and lateral inhibition in an interactive network), Hebbian learning, and a split pathway between ``what'' and ``where'' representations. These constraints are tested with a backprop network, which allows for the evaluation of the individual contributions of each constraint on the development of spatially invariant representations. Subsequently, a complete model embodying a modified Hebbian learning rule and interactive connectivity is developed from biological and computational considerations. The activational stability and weight function maximization properties of this interactive network are analyzed using a Lyapunov function approach. The model is tested first on the same simple stimuli used in the backprop simulation, and then with a more complex environment consisting of right and left diagonal lines. The results indicate that the hypothesized constraints, implemented in a Hebbian network, were capable of producing spatially invariant representations. Further, evidence for the gradual integration of both featural complexity and spatial invariance over increasing layers in the network, thought to be important for real-world applications, was obtained. As the approach is generalizable to other dimensions such as orientation and size, it could provide the basis of a more complete biologically plausible object recognition system. Indeed, this work forms the basis of a recent model of object recognition in the domestic chick (O'Reilly & Johnson, 1994).
[PDF] O'Reilly, R.C., Kosslyn, S.M., Marsolek, C.J. & Chabris, C.F. (1990). Receptive Field Characteristics that Allow Parietal Lobe Neurons to Encode Spatial Properties of Visual Input: A computational investigation. Journal of Cognitive Neuroscience, 2, 141-155.

Back to my home page.