C# Class Encog.Neural.Networks.Training.Lma.LevenbergMarquardtTraining

Trains a neural network using a Levenberg Marquardt algorithm (LMA). This training technique is based on the mathematical technique of the same name. The LMA interpolates between the Gauss-Newton algorithm (GNA) and the method of gradient descent (similar to what is used by backpropagation. The lambda parameter determines the degree to which GNA and Gradient Descent are used. A lower lambda results in heavier use of GNA, whereas a higher lambda results in a heavier use of gradient descent. Each iteration starts with a low lambda that builds if the improvement to the neural network is not desirable. At some point the lambda is high enough that the training method reverts totally to gradient descent. This allows the neural network to be trained effectively in cases where GNA provides the optimal training time, but has the ability to fall back to the more primitive gradient descent method LMA finds only a local minimum, not a global minimum. References: C. R. Souza. (2009). Neural Network Learning by the Levenberg-Marquardt Algorithm with Bayesian Regularization. Website, available from: http://crsouza.blogspot.com/2009/11/neural-network-learning-by-levenberg_18.html http://www.heatonresearch.com/wiki/LMA http://en.wikipedia.org/wiki/Levenberg%E2%80%93Marquardt_algorithm http://en.wikipedia.org/wiki/Finite_difference_method http://mathworld.wolfram.com/FiniteDifference.html http://www-alg.ist.hokudai.ac.jp/~jan/alpha.pdf - http://www.inference.phy.cam.ac.uk/mackay/Bayes_FAQ.html
Inheritance: Encog.ML.Train.BasicTraining, IMultiThreadable
Mostra file Open project: encog/encog-silverlight-core Class Usage Examples

Public Methods

Method Description
CalculateHessian ( double jacobian, double errors ) : void

Calculate the Hessian matrix.

Iteration ( ) : void

Perform one iteration.

LevenbergMarquardtTraining ( BasicNetwork network, IMLDataSet training ) : System

Construct the LMA object.

Pause ( ) : TrainingContinuation
Resume ( TrainingContinuation state ) : void
Trace ( double m ) : double

Return the sum of the diagonal.

UpdateWeights ( ) : double

Update the weights.

Private Methods

Method Description
CalculateSumOfSquaredWeights ( ) : double

Calculate the sum squared of the weights.

Method Details

CalculateHessian() public method

Calculate the Hessian matrix.
public CalculateHessian ( double jacobian, double errors ) : void
jacobian double The Jacobian matrix.
errors double The errors.
return void

Iteration() public method

Perform one iteration.
public Iteration ( ) : void
return void

LevenbergMarquardtTraining() public method

Construct the LMA object.
public LevenbergMarquardtTraining ( BasicNetwork network, IMLDataSet training ) : System
network BasicNetwork The network to train. Must have a single output neuron.
training IMLDataSet The training data to use. Must be indexable.
return System

Pause() public method

public Pause ( ) : TrainingContinuation
return Encog.Neural.Networks.Training.Propagation.TrainingContinuation

Resume() public method

public Resume ( TrainingContinuation state ) : void
state Encog.Neural.Networks.Training.Propagation.TrainingContinuation
return void

Trace() public static method

Return the sum of the diagonal.
public static Trace ( double m ) : double
m double The matrix to sum.
return double

UpdateWeights() public method

Update the weights.
public UpdateWeights ( ) : double
return double