From ftmleone at hotmail.com Fri Jun 1 03:19:27 2007 From: ftmleone at hotmail.com (Frank Leoné) Date: Fri Jun 1 03:19:34 2007 Subject: [PDP-discuss] Using input layer for both input and target? In-Reply-To: <200702052041.53042.Randy.OReilly@colorado.edu> Message-ID: Dear all, The proposals given for how to use the input layer both as input and target layer work great, thanks! But I need something a little different, which should be easy to solve: I want the first event in a group to have an event spec where layer A is the inputlayer, while all other events in that group have a different event spec, where A is a targetlayer. This can be done by hand and works great. But, as you might expect, this is not something you want to do by hand for up to 10000 groups. First of all my question is whether there is a function in the program which can do easily. If not, I think it would be good functionality to add in the upcoming version. So an function way to specify different eventspec within a group (and possible between groups). For now I think I'm just going to write a script which loops through all the events and makes the appropriate changes. This will probably not be too difficult, I'll just have to right variables and structures. Thanks in advance, with kind regards, Frank _________________________________________________________________ Talk with your online friends with Messenger http://www.join.msn.com/messenger/overview From ftmleone at hotmail.com Fri Jun 1 04:29:29 2007 From: ftmleone at hotmail.com (Frank Leoné) Date: Fri Jun 1 04:29:32 2007 Subject: [PDP-discuss] Using input layer for both input and target? In-Reply-To: Message-ID: I made a script alright and it works. Possibly it is also useful for other, so, this is the code: void setEventSpecs (int numEnv, int numSpecOne, int numSpecOther) { int e = 0; int t = 0; for (e = 0; e < .projects.environments[numEnv].events.gp.size; e++) for (t = 0; t < .projects.environments[numEnv].events.gp[e].size; t++) { if (t == 0) .projects[0].environments[numEnv].events.gp[e][t].spec.spec = .projects[0].environments[numEnv].event_specs[numSpecOne]; else .projects[0].environments[numEnv].events.gp[e][t].spec.spec = .projects[0].environments[numEnv].event_specs[numSpecOther]; } } It receives the number of the environment, the number of the event spec for the first event and the number of the event spec for the other events. Next it just applies them. It should be easy enough to change it to a more general case, e.g. by using an array of targetspecs with the same length as one group of events and applying them. void setEventSpecs (int numEnv, int targetSpecs[]) { int e = 0; int t = 0; for (e = 0; e < .projects.environments[numEnv].events.gp.size; e++) for (t = 0; t < .projects.environments[numEnv].events.gp[e].size; t++) { .projects[0].environments[numEnv].events.gp[e][t].spec.spec = targetSpecs[t]; } } Didn't test this one, but should work. Hopefully its useful for someone and something like this can be included in the upcoming version. kind regards, Frank _________________________________________________________________ Hotmail en Messenger on the move http://www.msn.nl/services From ftmleone at hotmail.com Tue Jun 5 07:04:38 2007 From: ftmleone at hotmail.com (Frank Leoné) Date: Tue Jun 5 07:04:52 2007 Subject: [pdp-discuss] Turning off kwta in a leabra layer? In-Reply-To: <200703081901.35599.Randy.OReilly@colorado.edu> Message-ID: Hi, Another question, related to kwta: I am using the scalarval layers and its 2D counterpart and am trying to learn the values perfectly. The type of inhibition and the value are really important to make sure this succeeds. Now I use pat_k with normal inhibition, 0.25, on the outputlayers. But what does pat_k exactly do? It does state that it takes the number of units with an activity bigger than pat_q. But what is this pat_q? It does seem to work alright though but I'm having the feeling that he can't go down anymore because of the kwta values. I get an minimal avarage error of 5 (so everything avg instead of sum), but it should be possible to get it down further. A second question is: what settings should I use for the inputlayer? At first I though it wouldn't matter, but it certainly seems to matter. Now I use pat_k, avg inhibition, 0.1, to make sure all the activities are put in and it seems to work, but isn't there a better way to do this? In addition I'm glad to hear any tips related to the kwta-values, they influence the results a lot. Thanks in advance! Frank _________________________________________________________________ Live Search, for accurate results! http://www.live.nl From Randy.OReilly at colorado.edu Thu Jun 7 01:33:16 2007 From: Randy.OReilly at colorado.edu (Randall C. O'Reilly) Date: Thu Jun 7 01:33:21 2007 Subject: [pdp-discuss] Turning off kwta in a leabra layer? In-Reply-To: References: Message-ID: <200706070133.16463.Randy.OReilly@colorado.edu> the pat_k uses the external inputs to determine the k on a trial-by-trial basis -- k = # of units with external input > pat_q. are you computing your error relative to the scalar output value from the first unit in the scalar val layer, or over the whole activation pattern? the former is preferred, as it reflects what the network is actually trying to learn -- there are many ways to to encode the same scalar val, and you probably don't care about that.. In the new version, it automatically computes SSE based on the scalar val, but in 3.2, you need to manually do this yourself using a script.. input layer kwta params DO matter for 1 thing (only): they determine the expected level of activation over the layer, which then influences netinput scaling and also hebbian learning.. So, set it to reflect the expected activation level. - Randy On Tuesday 05 June 2007 07:04, Frank Leon? wrote: > Hi, > > Another question, related to kwta: I am using the scalarval layers and its > 2D counterpart and am trying to learn the values perfectly. The type of > inhibition and the value are really important to make sure this succeeds. > > Now I use pat_k with normal inhibition, 0.25, on the outputlayers. But what > does pat_k exactly do? It does state that it takes the number of units with > an activity bigger than pat_q. But what is this pat_q? It does seem to work > alright though but I'm having the feeling that he can't go down anymore > because of the kwta values. I get an minimal avarage error of 5 (so > everything avg instead of sum), but it should be possible to get it down > further. > > A second question is: what settings should I use for the inputlayer? At > first I though it wouldn't matter, but it certainly seems to matter. Now I > use pat_k, avg inhibition, 0.1, to make sure all the activities are put in > and it seems to work, but isn't there a better way to do this? > > In addition I'm glad to hear any tips related to the kwta-values, they > influence the results a lot. > > Thanks in advance! > > Frank > > _________________________________________________________________ > Live Search, for accurate results! http://www.live.nl > > _______________________________________________ > PDP-Discuss mailing list > PDP-Discuss@psych.Colorado.EDU > http://psych.colorado.edu/mailman/listinfo/pdp-discuss From ftmleone at hotmail.com Thu Jun 7 13:43:47 2007 From: ftmleone at hotmail.com (Frank Leoné) Date: Thu Jun 7 13:43:53 2007 Subject: [pdp-discuss] Turning off kwta in a leabra layer? In-Reply-To: <200706070133.16463.Randy.OReilly@colorado.edu> Message-ID: Thanks for the reply! Now I get the effect of the input inhibition settings and the way of calculating the error. I don't get what pat_q stands for though and I can't find a definition anywhere. Could you explain it to me? And do you have a script to calculate the error the right way? Would really be of great help if you had. Otherwise I'll just figure out how to do it myself. Thanks again, kind regards, Frank _________________________________________________________________ Live Search, for accurate results! http://www.live.nl From Randy.OReilly at colorado.edu Thu Jun 7 19:52:18 2007 From: Randy.OReilly at colorado.edu (Randall C. O'Reilly) Date: Thu Jun 7 19:52:22 2007 Subject: [pdp-discuss] Turning off kwta in a leabra layer? In-Reply-To: References: Message-ID: <200706071952.18889.Randy.OReilly@colorado.edu> pat_q is just a threshold on the unit input value for computing what k should be -- in pseudocode: if(unit->ext >= pat_q) pat_k++; I don't have a script handy, but you just do something like this: Layer* lay = network.layers.OutputLayerName; // insert your name here.. float trg_val = lay.units[0].targ; float act_val = lay.units[0].act_m; float err = trg_val - act_val; float sq_err = err * err; you can then stick the sq_err value (and/or err) in the stat vals of a ScriptStat -- you might find an example of this somewhere in the demos/leabra directory or in the projects from the textbook: zcat projfile.proj.gz | grep ScriptStat to find files with script stats. - Randy On Thursday 07 June 2007 13:43, Frank Leon? wrote: > Thanks for the reply! Now I get the effect of the input inhibition settings > and the way of calculating the error. I don't get what pat_q stands for > though and I can't find a definition anywhere. Could you explain it to me? > > And do you have a script to calculate the error the right way? Would really > be of great help if you had. Otherwise I'll just figure out how to do it > myself. > > Thanks again, > > kind regards, > > Frank > > _________________________________________________________________ > Live Search, for accurate results! http://www.live.nl > > _______________________________________________ > PDP-Discuss mailing list > PDP-Discuss@psych.Colorado.EDU > http://psych.colorado.edu/mailman/listinfo/pdp-discuss From ftmleone at hotmail.com Mon Jun 18 14:46:00 2007 From: ftmleone at hotmail.com (Frank Leoné) Date: Mon Jun 18 14:46:07 2007 Subject: [PDP-discuss] Using input layer for both input and target? In-Reply-To: Message-ID: Hi all, I can teach the network to use an input layer as target during the same trial with the settings previously suggested (two scripts to set the hard clamp and min, plus + nothing). But, if I then add two other input layers, which are not meant to be also used as targets, the error rises far slower. Even if the layers are not connected to the hidden layer and other input layers. If I add a recurrent connection between them and the hidden layer, it seems to work better, so it seems it also tries to learn these two layers. How can I stop that? I want these input layers not to function different from the normal situation. And if want to use the same layer as target later on in the same trial or another layer only as target at the same time, can I then still use the same settings? It seems so, but just to be sure. Thanks in advance, kind regards, Frank _________________________________________________________________ Live Search, for accurate results! http://www.live.nl From ftmleone at hotmail.com Sun Jun 24 13:27:20 2007 From: ftmleone at hotmail.com (Frank Leoné) Date: Sun Jun 24 13:27:25 2007 Subject: [PDP-discuss] Higher error in test than train? In-Reply-To: Message-ID: Hi all, I'm experiencing a strange problem. After I have trained my network, it reaches an error of about 2, which is good enough. But if I then run a test-run (so without changing the weights) with the same data, the error is about the ten-fold. How can this be if it is exactly the same data? Earlier I got an even higher error and when I then continued training the error rose really high, without going down anymore. This was because I had the send delta on in the train trial and not in the test-trial, which resulted in the properties of the neurons being changed (I think). kind regards, Frank _________________________________________________________________ Live Search, for accurate results! http://www.live.nl From Randy.OReilly at colorado.edu Sun Jun 24 13:35:07 2007 From: Randy.OReilly at colorado.edu (Randall C. O'Reilly) Date: Sun Jun 24 13:35:12 2007 Subject: [PDP-discuss] Higher error in test than train? In-Reply-To: References: Message-ID: <200706241335.07269.Randy.OReilly@colorado.edu> this shouldn't happen, and so very likely reflects some kind of misconfiguration -- you are using different processes for train and test and there are various parameters in the trial and settle proc that you should double-check -- did you implement that different error stat? copy it to both places? - Randy On Sunday 24 June 2007 13:27, Frank Leon? wrote: > Hi all, > > I'm experiencing a strange problem. After I have trained my network, it > reaches an error of about 2, which is good enough. But if I then run a > test-run (so without changing the weights) with the same data, the error is > about the ten-fold. How can this be if it is exactly the same data? > > Earlier I got an even higher error and when I then continued training the > error rose really high, without going down anymore. This was because I had > the send delta on in the train trial and not in the test-trial, which > resulted in the properties of the neurons being changed (I think). > > kind regards, > > Frank > > _________________________________________________________________ > Live Search, for accurate results! http://www.live.nl > > _______________________________________________ > PDP-Discuss mailing list > PDP-Discuss@psych.Colorado.EDU > http://psych.colorado.edu/mailman/listinfo/pdp-discuss From Randy.OReilly at colorado.edu Mon Jun 25 23:59:51 2007 From: Randy.OReilly at colorado.edu (Randall C. O'Reilly) Date: Mon Jun 25 23:59:52 2007 Subject: [PDP-discuss] Higher error in test than train? In-Reply-To: References: Message-ID: <200706252359.51193.Randy.OReilly@colorado.edu> Very strange.. I wonder if the network might be learning some sequential dependency in the input -- are you using permuted order or sequential order of events? You might want to look at the trial-by-trial behavior just after switching off learning -- does it deteriorate immediately or more slowly over trials? Also try setting the lrate = 0 in the conspec instead of using test -- if that gives the same bad results, then you DEFINITELY have a case of the network learning in some way that significantly improves performance.. These networks can be mighty crafty sometimes.. ;) - Randy On Monday 25 June 2007 04:28, Frank Leon? wrote: > Hi mister O'Reilly, > > Thanks for the fast reply. I checked all the variables, but didn't find any > differences. On the contrary: if I use the same processes for training and > testing and in the sequence epoch just switch "online" to "test", the same > high error arrises. If I then switch it back to online, the error is low > again. The nasty thing is, not only the error is higher, but also the > actual output it worse: so it is not that the number is incorrect, the > network actually gives a different output. Both the SE-stat as the > errorscript give far worse values, consistent with worse output. > > The network is made up of scalarvallayers, with one hidden layer with a low > decay and sequences of trials. Nothing really special. If I train the > network without multiple events in the groups, so without any need of > memory, the problem stays. > > I'll try to figure out what the problem is, hopefully you got some helpful > hints. > > Thanks in advance, > > kind regards, > > Frank > > _________________________________________________________________ > Talk with your online friends with Messenger > http://www.join.msn.com/messenger/overview From allison.kaufman at email.ucr.edu Tue Jun 26 15:02:16 2007 From: allison.kaufman at email.ucr.edu (Allison B. Kaufman) Date: Tue Jun 26 15:03:02 2007 Subject: [pdp-discuss] Monitor values for projections (in tutorial) References: <200706252359.51193.Randy.OReilly@colorado.edu> Message-ID: <007701c7b835$5e252aa0$bc22b68b@Cetacea> Greetings - I am new to PDP++ and am working my way through the manual's tutorial. I have reached the section on setting up a GridLog to display the weights in the network at the end of each test process, and am trying to set up a monitor stat for each projection. When I do, I get the following error message - "monitorstat projection does not have layer's set or selected variable does not apply to connections" If I click "ok" and continue to set up the GridLog, I think I'm getting more columns in the header than I should. There's one for the sum-squared error, one for the epoch, one that reads "1st_act_Fm_H" , and then 8 small boxes that I can't pull apart which have "Outp[0][0][0].wt" written across the first 5. I have not had any problems or discrepancies from the manuel up until now. Is there someone who might be able to help me with this error? Thank you so much! Best, Allison Allison B. Kaufman University of California, Riverside Department of Neuroscience 1208 Spieth Hall Riverside, CA 92521 (951) 827-2102 allison.kaufman@email.ucr.edu http://semantics.ucr.edu/~allison/