From Randy.OReilly at colorado.edu Mon Jan 1 22:23:22 2007 From: Randy.OReilly at colorado.edu (Randall C. O'Reilly) Date: Mon Jan 1 22:23:26 2007 Subject: [pdp-discuss] writing out delta weight changes to file In-Reply-To: References: Message-ID: <200701012223.22964.Randy.OReilly@colorado.edu> r.pdw is teh variable you want. those values don't live in the monitor stat itself. the monitor stat just creates them in a textlog. so, you'd need to access from the log: .logs.TextLogName.data.etc. it might just be easier to access from the network itself: e.g., .networks.mynet.layers.mylayer.unit[0].recv.gp[0][0].pdw. - Randy On Saturday 30 December 2006 11:27, rama0042@umn.edu wrote: > Dear List, > > I am having trouble writing out delta weight changes to a data file using > the script language. > > I want to make a list of all of the connection weight changes from one > trial of learning to the next, for both the input-hidden and the > hidden-output connections. > > The first question I have is: are r.pdw and s.pdw the appropriate items if > I want to access the *change* in weights from one trial to the next? I > think the answer is yes, but I want to doublecheck. > > More importantly, here is the main problem that I have: > > I have no problem creating a textlog: I can create a log of r.pdw or s.pdw > values, which it reports under a column titled, for example, > "Hidd[0][0][0].pdw". However, I can't figure out how to access these > values, either individually (eg: Hidd[0][0][1]), or in an iterative fashion > (iterating over the indices) and write to an output file using the script > language. > > I've tried all kinds of variations on a line like this: > > foutweightdeltas<<.processes[2].loop_stats[3].Hidd[i][j][k].pdw.val; > > The error message says that there is no such monitorstat. So, I've also > tried variations on this kind of thing: > > foutweightdeltas<<.processes[2].loop_stats[3].sum_r.pdw_Hidd.val<<; > > Again, it doesn't recognize this (calling it a syntax error, and choking on > the "r."). > > Any ideas? > > Thanks, > Pradeep From rama0042 at umn.edu Tue Jan 2 09:01:26 2007 From: rama0042 at umn.edu (rama0042@umn.edu) Date: Tue Jan 2 09:01:29 2007 Subject: [pdp-discuss] writing out delta weight changes to file In-Reply-To: <200701012223.22964.Randy.OReilly@colorado.edu> References: <200701012223.22964.Randy.OReilly@colorado.edu> Message-ID: Hi Dr. O'Reilly, Thanks for the tips - we'll give it a try! Pradeep On Jan 1 2007, Randall C. O'Reilly wrote: > r.pdw is teh variable you want. those values don't live in the monitor > stat itself. the monitor stat just creates them in a textlog. so, you'd > need to access from the log: .logs.TextLogName.data.etc. it might just be > easier to access from the network itself: e.g., > .networks.mynet.layers.mylayer.unit[0].recv.gp[0][0].pdw. > >- Randy > >On Saturday 30 December 2006 11:27, rama0042@umn.edu wrote: >> Dear List, >> >> I am having trouble writing out delta weight changes to a data file using >> the script language. >> >> I want to make a list of all of the connection weight changes from one >> trial of learning to the next, for both the input-hidden and the >> hidden-output connections. >> >> The first question I have is: are r.pdw and s.pdw the appropriate items >> if I want to access the *change* in weights from one trial to the next? >> I think the answer is yes, but I want to doublecheck. >> >> More importantly, here is the main problem that I have: >> >> I have no problem creating a textlog: I can create a log of r.pdw or >> s.pdw values, which it reports under a column titled, for example, >> "Hidd[0][0][0].pdw". However, I can't figure out how to access these >> values, either individually (eg: Hidd[0][0][1]), or in an iterative >> fashion (iterating over the indices) and write to an output file using >> the script language. >> >> I've tried all kinds of variations on a line like this: >> >> foutweightdeltas<<.processes[2].loop_stats[3].Hidd[i][j][k].pdw.val; >> >> The error message says that there is no such monitorstat. So, I've also >> tried variations on this kind of thing: >> >> foutweightdeltas<<.processes[2].loop_stats[3].sum_r.pdw_Hidd.val<<; >> >> Again, it doesn't recognize this (calling it a syntax error, and >> choking on the "r."). >> >> Any ideas? >> >> Thanks, >> Pradeep > -- Pradeep Ramanathan M.S., M.A., CCC-SLP Speech-Language Pathologist/Doctoral Candidate University of Minnesota From ftmleone at hotmail.com Wed Jan 24 12:46:41 2007 From: ftmleone at hotmail.com (Frank Leoné) Date: Wed Jan 24 12:46:47 2007 Subject: [pdp-discuss] Leabra: learning continous values over time? Message-ID: Dear all, I am trying to implement a network to learn to map a retinal map to a saccadic direction. So the input is a retinal map of 11 (horizontal location) x 11 (disparity) = 121 neurons, where one spot is stimulated, resulting in Gaussian like activity pattern over several neurons. On the output side, only two output neurons exist, one representing the version component, the other vergence, both normalized between 0 and 1. In between is a hidden layer. All this I would like do using Leabra, but uptil now without much success. The error oscillates a lot, often from zero (though the actual error is quite large) to values of 1 and far higher. Also, if I compare the output to the targets on individual trials, it becomes clear the network always produces the same output, regardless of the input. A standard backpropagation network does learn the problem though. I hope someone of you can help me. I'm about to order the book, as it will probably also be of great help and sounds really interesting, but it is kinda important that I find a solution to my problem earlier than the arrival date of the book. More concrete, my questions are: - How do I make Leabra learn data with continuous values, as I just sketched. I read in another post here that Leabra is made for categorization, but can also be used to fit continuous data. I did change the dwa parameter and made activation and weight functions linear, but without any luck. So, starting with a 'default' Leabra network, what do I need to change to make it learn in my situation? - In addition, I want the network to learn this problem over time. I want to be able to only present the retinal stimulation at the beginning, next some trials without any retinal input, and then it needs to give the output. How do I do this? I added a context layer (and also two, as used in the grammar example in O'Reilly's thesis) and the sequenceEpoch and process, but it didn't work. But it might well be that it didn't work because it didn't work without the delay in the first place :) So the first question is the most important. Thank a lot in advance! With kind regards, Frank _________________________________________________________________ Live Search, for accurate results! http://www.live.nl From Randy.OReilly at colorado.edu Wed Jan 24 21:04:25 2007 From: Randy.OReilly at colorado.edu (Randall C. O'Reilly) Date: Wed Jan 24 21:04:25 2007 Subject: [pdp-discuss] Leabra: learning continous values over time? In-Reply-To: References: Message-ID: <200701242304.25572.Randy.OReilly@colorado.edu> Frank -- both the learning and activation equations in leabra are optimized for binary-ish output states -- it really is good at learning whether some thing is present or not, and therefore not good at learning continuous values. The way to solve this problem in leabra is to use a coarse-coded distributed representation of continuous values, which perhaps not coincidentally is how the brain does it as well. The ScalarValLayerSpec is designed to do exactly this, and makes it easy to do so. Read the info about it (maybe only avail in the comments on the object itself -- not sure) and hopefully you can get it configured ok. Basically it sets up a number of neurons with gaussian tuning curves for preferred values within an overall range of continuous values to be represented. Also, use the Wizard for creating the context layer -- it sets up everything correctly. Best of luck, - Randy On Wednesday 24 January 2007 14:46, Frank Leon? wrote: > Dear all, > > I am trying to implement a network to learn to map a retinal map to a > saccadic direction. So the input is a retinal map of 11 (horizontal > location) x 11 (disparity) = 121 neurons, where one spot is stimulated, > resulting in Gaussian like activity pattern over several neurons. On the > output side, only two output neurons exist, one representing the version > component, the other vergence, both normalized between 0 and 1. In between > is a hidden layer. > > All this I would like do using Leabra, but uptil now without much success. > The error oscillates a lot, often from zero (though the actual error is > quite large) to values of 1 and far higher. Also, if I compare the output > to the targets on individual trials, it becomes clear the network always > produces the same output, regardless of the input. A standard > backpropagation network does learn the problem though. > > I hope someone of you can help me. I'm about to order the book, as it will > probably also be of great help and sounds really interesting, but it is > kinda important that I find a solution to my problem earlier than the > arrival date of the book. > > More concrete, my questions are: > > - How do I make Leabra learn data with continuous values, as I just > sketched. I read in another post here that Leabra is made for > categorization, but can also be used to fit continuous data. I did change > the dwa parameter and made activation and weight functions linear, but > without any luck. So, starting with a 'default' Leabra network, what do I > need to change to make it learn in my situation? > > - In addition, I want the network to learn this problem over time. I want > to be able to only present the retinal stimulation at the beginning, next > some trials without any retinal input, and then it needs to give the > output. How do I do this? I added a context layer (and also two, as used in > the grammar example in O'Reilly's thesis) and the sequenceEpoch and > process, but it didn't work. But it might well be that it didn't work > because it didn't work without the delay in the first place :) So the first > question is the most important. > > Thank a lot in advance! > > With kind regards, > > Frank > > _________________________________________________________________ > Live Search, for accurate results! http://www.live.nl > > _______________________________________________ > PDP-Discuss mailing list > PDP-Discuss@psych.Colorado.EDU > http://psych.colorado.edu/mailman/listinfo/pdp-discuss From ftmleone at hotmail.com Thu Jan 25 01:29:29 2007 From: ftmleone at hotmail.com (Frank Leoné) Date: Thu Jan 25 01:29:34 2007 Subject: [pdp-discuss] Leabra: learning continous values over time? In-Reply-To: <200701242304.25572.Randy.OReilly@colorado.edu> Message-ID: An HTML attachment was scrubbed... URL: http://psych.colorado.edu/pipermail/pdp-discuss/attachments/20070125/d9b48dd5/attachment.html From allan.randall at ntt.ca Tue Jan 30 13:28:34 2007 From: allan.randall at ntt.ca (allan.randall@ntt.ca) Date: Tue Jan 30 13:28:47 2007 Subject: [pdp-discuss] oldlba, 3.1 and libraries Message-ID: <200701302028.l0UKSZFH032513@mail29.atl.registeredsite.com> Hi, I have been running 3.2a07 (leabra++) and doing pretty well with it. However, occasionally, I have to run one of the demos from the "Explorations" book, so I use oldlba++, which is the version with the out-dated stuff to make the demos not break. The only problem is that I have found that the demos still break or have problems occasionally, so oldlba++ is not foolproof. So I decided that it would be nice to have a working 3.1 around to run leabra++ when I want to show demos, without fear of compatibility problems. The demos ran fine under 3.1 (installed binaries) on my old Fedora 2 Linux machine, but I have since upgraded to a new Fedora 5 system, and I get missing library errors. Installing from RPM rpm -Uvh pdp++-binext-3.1-0.i386.rpm gets me: error: Failed dependencies: libreadline.so.4 is needed by pdp++-binext-3.1-0.i386 libstdc++-libc6.2-2.so.3 is needed by pdp++-binext-3.1-0.i386 libstdc++-libc6.1-1.so.2 is needed by pdp++-binext-3.1-0.i386 If I try to install the binaries from the tar file (no rpm), I get missing file errors for the same libraries. Perhaps the 3.1 binaries are just incompatible with Fedora 5, and I need to stick with oldlba? I don't think I need to build 3.1 from source, but maybe that is what I need to do? Anyone know the answer? And/or a way to decrease risk of failure when running demos in oldlba++? Cheers, Allan Randall, Allan.Randall@ntt.ca From Randy.OReilly at colorado.edu Tue Jan 30 17:39:36 2007 From: Randy.OReilly at colorado.edu (Randall C. O'Reilly) Date: Tue Jan 30 17:39:40 2007 Subject: [pdp-discuss] oldlba, 3.1 and libraries In-Reply-To: <200701302028.l0UKSZFH032513@mail29.atl.registeredsite.com> References: <200701302028.l0UKSZFH032513@mail29.atl.registeredsite.com> Message-ID: <200701301939.37038.Randy.OReilly@colorado.edu> I think you'd probably have to recompile. Various changes would have to be made to get it to work with more recent g++ and libs.. What exactly doesn't work with oldlba++ for you?? I have used that to teach with and not encountered any problems (that I can remember at least.. :) - Randy On Tuesday 30 January 2007 15:28, allan.randall@ntt.ca wrote: > Hi, > > I have been running 3.2a07 (leabra++) and doing pretty well with it. > However, occasionally, I have to run one of the demos from the > "Explorations" book, so I use oldlba++, which is the version with the > out-dated stuff to make the demos not break. The only problem is that I > have found that the demos still break or have problems occasionally, so > oldlba++ is not foolproof. > > So I decided that it would be nice to have a working 3.1 around to run > leabra++ when I want to show demos, without fear of compatibility problems. > The demos ran fine under 3.1 (installed binaries) on my old Fedora 2 Linux > machine, but I have since upgraded to a new Fedora 5 system, and I get > missing library errors. > > Installing from RPM > rpm -Uvh pdp++-binext-3.1-0.i386.rpm > > gets me: > > error: Failed dependencies: > libreadline.so.4 is needed by pdp++-binext-3.1-0.i386 > libstdc++-libc6.2-2.so.3 is needed by pdp++-binext-3.1-0.i386 > libstdc++-libc6.1-1.so.2 is needed by pdp++-binext-3.1-0.i386 > > If I try to install the binaries from the tar file (no rpm), I get missing > file errors for the same libraries. > > Perhaps the 3.1 binaries are just incompatible with Fedora 5, and I need to > stick with oldlba? I don't think I need to build 3.1 from source, but maybe > that is what I need to do? > > Anyone know the answer? And/or a way to decrease risk of failure when > running demos in oldlba++? > > Cheers, > > Allan Randall, Allan.Randall@ntt.ca > > > > > _______________________________________________ > PDP-Discuss mailing list > PDP-Discuss@psych.Colorado.EDU > http://psych.colorado.edu/mailman/listinfo/pdp-discuss From ftmleone at hotmail.com Wed Jan 31 09:52:40 2007 From: ftmleone at hotmail.com (Frank Leoné) Date: Wed Jan 31 09:52:49 2007 Subject: [pdp-discuss] Leabra: learning continous values over time? In-Reply-To: Message-ID: Hi all, Again I want to expres my gratitude: it now works! Especially this, for me previous undiscovered, layertype really opens new interesting possibilities, I like it! It does not work flawlessly though: the error first goes down quite fast, but after some time rises even faster, to really high levels, just to go down again (not as far down as it went previously) and to rise later on, etcetera. Needles to say, it is quite hard to reach low error levels with this kind of behaviour. Does anyone got an idea what the reason might be? I'm using a leabra network with a context layer (made with the wizard). Also I have one other minor problem: for the learning over time I want the retinal input to switch off after the first trial. So, the first trial the input is a value between a minimum and a maximum, converted in a distributed code. But in the rest of the trials no such input should be given: the input layer should give a zero input. But if I enter zero as input, it is converted to the center of my distributed code, which is ofcourse normally the correct behavior. So I thought: lets take a number far outside the borders, but that point is just moved to the border itself. So: how can I input nothing into a scalarvallayer? I now fixed it by just clamping the entire pattern, but it would be nice to use the scalarvallayer again. Thanks in advance! Frank PS. I can't seem to reach the latest archives. Is there something wrong or is it just me? :) _________________________________________________________________ Hotmail en Messenger on the move http://www.msn.nl/services From ftmleone at hotmail.com Wed Jan 31 12:37:28 2007 From: ftmleone at hotmail.com (Frank Leoné) Date: Wed Jan 31 12:37:33 2007 Subject: [pdp-discuss] Leabra: learning continous values over time? In-Reply-To: Message-ID: I think I solved my first problem, the fluctuating error. While reading one of the articles, I came across the sigmoidal weight functions to improve learning in binary settings. Turning that off (wt sig gain and off set to 1) stopped the network from oscillating. Quite understandable, I just had to notice the option. Hopefully this post can at least help someone else with the same kind of problems. Now I just have to get the error somewhat lower than it is now.... The other minor problem still exists though. _________________________________________________________________ FREE pop-up blocking with the new Windows Live Toolbar - get it now! http://toolbar.msn.click-url.com/go/onm00200415ave/direct/01/