Script for averaging epoch error for large training sessions

[flatmax@cse.unsw.edu.au (Matthew Flax)]

Please find an octave script attatched which displays graphlog data in an 
averaged fashion for training sessions which last for several hundreds of 
thousands of epochs.

-- 
Matt

For electronic musicians ...
Vector Bass          : http://mffmvectorbass.sourceforge.net/
For developers ...
TimeScale Audio Mod  : http://mffmtimescale.sourceforge.net/
Multimedia Time Code : http://mffmtimecode.sourceforge.net/
3D Audio Library     : http://mffm3daudiolib.sourceforge.net/
function error=logAvgDisplay
  %# You will need octave to use this script : http://www.octave.org
  %# You will be training neural networks in PDP++
  %# This script slides an overlapping window across error data saved by a graph log.
  %# The purpose of this is to display averaged and smoothed errors.
  %# When one is training a large network for several hundres of
  %# thousands of epochs, this script will come in handy

  %# How to use ?
  %#     a] Save the graph log data from PDP++ to a file
  %#     b] Change the variable "name" here to represent your log file
  %#     c] For the first hundred thousand epochs try a windowSize of 500
  %#        For larger numbers of epochs try a windowSize of 1000
  

  name='generatePhase.error.log';
  fp = fopen(name);
  %# get rid of header
  fscanf(fp, '%s',1 );  fscanf(fp, '%s',1 );  fscanf(fp, '%s',1 );
  %# scan data
  [m,count]=fscanf(fp, '%s %d %f',[5,189569] );

  %# load vectors
  indexes=m(4,:);
  error=m(5,:);

  clear m; %# get rid of redundant

  fclose(fp); %# close file

  count=length(indexes)

  %#plot(indexes,error); return

	%# set up window/overlap variables
  windowSize=1000;
  overlapFactor=0.5;
  overlap=windowSize*overlapFactor
  windowCount=count/(windowSize*overlapFactor)-1

	%# average the error
  for j=1:windowCount
    startSample=indexes(1)+(j-1)*windowSize*overlapFactor;
    sampleSequence=1+(startSample:startSample+windowSize);
    summedError(j)=sum(error(sampleSequence));
  end

	%# smooth the error
  filterLength=5;
  filter=ones(1, filterLength); filter=filter/length(filter);

  smoothedError=conv(summedError,filter);

	%# plot
  xaxis=1:windowCount;
  xlabel(['Averaged over ' num2str(windowSize) ' epochs, with ' \
	  num2str(overlapFactor) ' factor overlap'])
  title('Smoothed averaged epoch error')
  p=0;
  if p==1
    plot(summedError);
    hold on
  else
    plot(smoothedError((filterLength):length(smoothedError)-filterLength+1))
  end
  if p==1
    hold off
  end

return