[pdp-discuss] Leabra: learning continous values over time?

[Randall C. O'Reilly]

Frank -- both the learning and activation equations in leabra are optimized 
for binary-ish output states -- it really is good at learning whether some 
thing is present or not, and therefore not good at learning continuous 
values.  The way to solve this problem in leabra is to use a coarse-coded 
distributed representation of continuous values, which perhaps not 
coincidentally is how the brain does it as well.  The ScalarValLayerSpec is 
designed to do exactly this, and makes it easy to do so.  Read the info about 
it (maybe only avail in the comments on the object itself -- not sure) and 
hopefully you can get it configured ok.  Basically it sets up a number of 
neurons with gaussian tuning curves for preferred values within an overall 
range of continuous values to be represented.

Also, use the Wizard for creating the context layer -- it sets up everything 
correctly.  Best of luck,

- Randy

On Wednesday 24 January 2007 14:46, Frank Leoné wrote:
> Dear all,
>
> I am trying to implement a network to learn to map a retinal map to a
> saccadic direction. So the input is a retinal map of 11 (horizontal
> location) x 11 (disparity) = 121 neurons, where one spot is stimulated,
> resulting in Gaussian like activity pattern over several neurons. On the
> output side, only two output neurons exist, one representing the version
> component, the other vergence, both normalized between 0 and 1. In between
> is a hidden layer.
>
> All this I would like do using Leabra, but uptil now without much success.
> The error oscillates a lot, often from zero (though the actual error is
> quite large) to values of 1 and far higher. Also, if I compare the output
> to the targets on individual trials, it becomes clear the network always
> produces the same output, regardless of the input. A standard
> backpropagation network does learn the problem though.
>
> I hope someone of you can help me. I'm about to order the book, as it will
> probably also be of great help and sounds really interesting, but it is
> kinda important that I find a solution to my problem earlier than the
> arrival date of the book.
>
> More concrete, my questions are:
>
> - How do I make Leabra learn data with continuous values, as I just
> sketched. I read in another post here that Leabra is made for
> categorization, but can also be used to fit continuous data. I did change
> the dwa parameter and made activation and weight functions linear, but
> without any luck. So, starting with a 'default' Leabra network, what do I
> need to change to make it learn in my situation?
>
> - In addition, I want the network to learn this problem over time. I want
> to be able to only present the retinal stimulation at the beginning, next
> some trials without any retinal input, and then it needs to give the
> output. How do I do this? I added a context layer (and also two, as used in
> the grammar example in O'Reilly's thesis) and the sequenceEpoch and
> process, but it didn't work. But it might well be that it didn't work
> because it didn't work without the delay in the first place :) So the first
> question is the most important.
>
> Thank a lot in advance!
>
> With kind regards,
>
> Frank
>
> _________________________________________________________________
> Live Search, for accurate results! http://www.live.nl
>
> _______________________________________________
> PDP-Discuss mailing list
> PDP-Discuss at psych.Colorado.EDU
> http://psych.colorado.edu/mailman/listinfo/pdp-discuss