C# Class SharpNeat.Phenomes.NeuralNets.FastCyclicNetwork

A neural network class that represents a network with recurrent (cyclic) connections. This is a much faster implementation of CyclicNetwork. The speedup is approximately 5x depending on hardware and CLR platform, see http://sharpneat.sourceforge.net/network_optimization.html for detailed info. The speedup is achieved by compactly storing all required data in arrays and in a way that maximizes in-order memory accesses; This allows us to maximize use of CPU caches. In contrast the CyclicNetwork class represents the network directly, that is, as a network of neuron/node objects; This has additional overhead such as the standard data associated with each object in dotNet which results in less efficient packing of the true neural net data in memory, which in turns results in less efficient use of CPU memory caches. Finally, representing the network directly as a graph of connected nodes is not conducive to writing code with in-order memory accesses. Algorithm Overview. 1) Loop connections. Each connection gets its input signal from its source neuron, applies its weight and stores its output value./ Connections are ordered by source neuron index, thus all memory accesses here are sequential/in-order. 2) Loop connections (again). Each connection adds its output value to its target neuron, thus each neuron accumulates or 'collects' its input signal in its pre-activation variable. Because connections are sorted by source neuron index and not target index, this loop generates out-of order memory accesses, but is the only loop to do so. 3) Loop neurons. Pass each neuron's pre-activation signal through the activation function and set its post-activation signal value. The activation loop is now complete and we can go back to (1) or stop.