[pdp-discuss] Turning off kwta in a leabra layer?

[Randall C. O'Reilly]

the pat_k uses the external inputs to determine the k on a trial-by-trial 
basis -- k = # of units with external input > pat_q.

are you computing your error relative to the scalar output value from the 
first unit in the scalar val layer, or over the whole activation pattern?  
the former is preferred, as it reflects what the network is actually trying 
to learn -- there are many ways to to encode the same scalar val, and you 
probably don't care about that..  In the new version, it automatically 
computes SSE based on the scalar val, but in 3.2, you need to manually do 
this yourself using a script..

input layer kwta params DO matter for 1 thing (only): they determine the 
expected level of activation over the layer, which then influences netinput 
scaling and also hebbian learning..  So, set it to reflect the expected 
activation level.

- Randy

On Tuesday 05 June 2007 07:04, Frank Leoné wrote:
> Hi,
>
> Another question, related to kwta: I am using the scalarval layers and its
> 2D counterpart and am trying to learn the values perfectly. The type of
> inhibition and the value are really important to make sure this succeeds.
>
> Now I use pat_k with normal inhibition, 0.25, on the outputlayers. But what
> does pat_k exactly do? It does state that it takes the number of units with
> an activity bigger than pat_q. But what is this pat_q? It does seem to work
> alright though but I'm having the feeling that he can't go down anymore
> because of the kwta values. I get an minimal avarage error of 5 (so
> everything avg instead of sum), but it should be possible to get it down
> further.
>
> A second question is: what settings should I use for the inputlayer? At
> first I though it wouldn't matter, but it certainly seems to matter. Now I
> use pat_k, avg inhibition, 0.1, to make sure all the activities are put in
> and it seems to work, but isn't there a better way to do this?
>
> In addition I'm glad to hear any tips related to the kwta-values, they
> influence the results a lot.
>
> Thanks in advance!
>
>        Frank
>
> _________________________________________________________________
> Live Search, for accurate results! http://www.live.nl
>
> _______________________________________________
> PDP-Discuss mailing list
> PDP-Discuss at psych.Colorado.EDU
> http://psych.colorado.edu/mailman/listinfo/pdp-discuss