C# Class Accord.MachineLearning.Sarsa

Sarsa learning algorithm.
The class provides implementation of Sarsa algorithm, known as on-policy Temporal Difference control.
显示文件 Open project: accord-net/framework Class Usage Examples

Public Methods

Method Description
GetAction ( int state ) : int

Get next action from the specified state.

The method returns an action according to current exploration policy.

Sarsa ( int states, int actions, IExplorationPolicy explorationPolicy ) : System

Initializes a new instance of the Sarsa class.

Action estimates are randomized in the case of this constructor is used.

Sarsa ( int states, int actions, IExplorationPolicy explorationPolicy, bool randomize ) : System

Initializes a new instance of the Sarsa class.

The randomize parameter specifies if initial action estimates should be randomized with small values or not. Randomization of action values may be useful, when greedy exploration policies are used. In this case randomization ensures that actions of the same type are not chosen always.

UpdateState ( int previousState, int previousAction, double reward ) : void

Update Q-function's value for the previous state-action pair.

Updates Q-function's value for the previous state-action pair in the case if the next state is terminal.

UpdateState ( int previousState, int previousAction, double reward, int nextState, int nextAction ) : void

Update Q-function's value for the previous state-action pair.

Updates Q-function's value for the previous state-action pair in the case if the next state is non terminal.

Method Details

GetAction() public method

Get next action from the specified state.
The method returns an action according to current exploration policy.
public GetAction ( int state ) : int
state int Current state to get an action for.
return int

Sarsa() public method

Initializes a new instance of the Sarsa class.
Action estimates are randomized in the case of this constructor is used.
public Sarsa ( int states, int actions, IExplorationPolicy explorationPolicy ) : System
states int Amount of possible states.
actions int Amount of possible actions.
explorationPolicy IExplorationPolicy Exploration policy.
return System

Sarsa() public method

Initializes a new instance of the Sarsa class.
The randomize parameter specifies if initial action estimates should be randomized with small values or not. Randomization of action values may be useful, when greedy exploration policies are used. In this case randomization ensures that actions of the same type are not chosen always.
public Sarsa ( int states, int actions, IExplorationPolicy explorationPolicy, bool randomize ) : System
states int Amount of possible states.
actions int Amount of possible actions.
explorationPolicy IExplorationPolicy Exploration policy.
randomize bool Randomize action estimates or not.
return System

UpdateState() public method

Update Q-function's value for the previous state-action pair.
Updates Q-function's value for the previous state-action pair in the case if the next state is terminal.
public UpdateState ( int previousState, int previousAction, double reward ) : void
previousState int Curren state.
previousAction int Action, which lead from previous to the next state.
reward double Reward value, received by taking specified action from previous state.
return void

UpdateState() public method

Update Q-function's value for the previous state-action pair.
Updates Q-function's value for the previous state-action pair in the case if the next state is non terminal.
public UpdateState ( int previousState, int previousAction, double reward, int nextState, int nextAction ) : void
previousState int Curren state.
previousAction int Action, which lead from previous to the next state.
reward double Reward value, received by taking specified action from previous state.
nextState int Next state.
nextAction int Next action.
return void