[pdp-discuss] Leabra: learning continous values over time?

[Frank Leoné]

Dear all,

I am trying to implement a network to learn to map a retinal map to a 
saccadic direction. So the input is a retinal map of 11 (horizontal 
location) x 11 (disparity) = 121 neurons, where one spot is stimulated, 
resulting in Gaussian like activity pattern over several neurons. On the 
output side, only two output neurons exist, one representing the version 
component, the other vergence, both normalized between 0 and 1. In between 
is a hidden layer.

All this I would like do using Leabra, but uptil now without much success. 
The error oscillates a lot, often from zero (though the actual error is 
quite large) to values of 1 and far higher. Also, if I compare the output to 
the targets on individual trials, it becomes clear the network always 
produces the same output, regardless of the input. A standard 
backpropagation network does learn the problem though.

I hope someone of you can help me. I'm about to order the book, as it will 
probably also be of great help and sounds really interesting, but it is 
kinda important that I find a solution to my problem earlier than the 
arrival date of the book.

More concrete, my questions are:

- How do I make Leabra learn data with continuous values, as I just 
sketched. I read in another post here that Leabra is made for 
categorization, but can also be used to fit continuous data. I did change 
the dwa parameter and made activation and weight functions linear, but 
without any luck. So, starting with a 'default' Leabra network, what do I 
need to change to make it learn in my situation?

- In addition, I want the network to learn this problem over time. I want to 
be able to only present the retinal stimulation at the beginning, next some 
trials without any retinal input, and then it needs to give the output. How 
do I do this? I added a context layer (and also two, as used in the grammar 
example in O'Reilly's thesis) and the sequenceEpoch and process, but it 
didn't work. But it might well be that it didn't work because it didn't work 
without the delay in the first place :) So the first question is the most 
important.

Thank a lot in advance!

With kind regards,

Frank

_________________________________________________________________
Live Search, for accurate results! http://www.live.nl