SGD<N extends num, E, T extends Signal<N, E, T>, S extends Scale<N>, P extends Sample<N, E, T, S>> class

Stochastic Gradient Descent with optional (classic or Nesterov) momentum.

Add-convention (the accumulated gradient g already points in the error-reducing direction): v = μ·v + g Δw = lr·v (classic momentum) Δw = lr·(μ·v + g) (Nesterov)

With momentum = 0 this is plain SGD (Δw = lr·g). Pair with a small batchSize for mini-batch/online SGD.

Inheritance

Constructors

SGD(ANN<N, E, T, S> ann, SamplesSet<P> samplesSet, {double learningRate = 0.1, double momentum = 0.0, bool nesterov = false, int batchSize = 0, LearningRateScheduleBuilder<N, E, T>? lrSchedule, String? subject})

Properties

algorithmName → String
The training algorithm name.
finalinherited
ann → ANN<N, E, T, S>
The ANN to train.
finalinherited
baseLearningRate → double
The base (initial) learning rate.
finalinherited
batchSize → int
Mini-batch size. 0 (default) = full-batch (one weight update per epoch); 1 = online/pure SGD; otherwise samples are shuffled each epoch and the weights are updated once per mini-batch.
finalinherited
bestTrainingError → double
The lowest training error seen by checkBestTrainingError since the last resetBestTraining.
no setterinherited
elapsedTime → Duration?
no setterinherited
enableSelectInitialANN ↔ bool
If true will select the initial ANN calling selectInitialANN.
getter/setter pairinherited
endTime → DateTime?
The end time of the last training session or null if not finished yet.
no setterinherited
globalError → double
Returns the current training global error (set by train).
no setterinherited
globalLearnError → double
The global error while updating weights.
no setterinherited
gradientClip → double
Per-value gradient clipping bound; 0 disables it. Each gradient entry is clamped to [-gradientClip, gradientClip] before the update.
finalinherited
hashCode → int
The hash code for this object.
no setterinherited
initialAnnEpochs ↔ int
Number of epochs to perform in the ANNs in the selection pool.
getter/setter pairinherited
initialAnnPoolSize ↔ int
The initial ANN pool size.
getter/setter pairinherited
lastGlobalError → double
no setterinherited
lastGlobalLearnError → double
The previous global error while updating weights.
no setterinherited
learningRate → double
Returns the current learning rate of the Backpropagation.
no setterinherited
learningRateEntry → E
no setterinherited
logEnabled ↔ bool
If true logging will be enabled.
getter/setter pairinherited
logger → TrainingLogger
finalinherited
logProgressEnabled ↔ bool
If true logging of progress will be enabled.
getter/setter pairinherited
lrSchedule → LearningRateScheduleBuilder<N, E, T>?
Optional learning-rate schedule (e.g. StepDecayStrategy, CosineAnnealingStrategy); when null a static base learning rate is used.
finalinherited
momentum → double
Returns the current momentum rate of the Backpropagation.
no setterinherited
momentumEntry → E
no setterinherited
momentumFactor → double
Momentum coefficient (named momentumFactor to avoid clashing with Propagation.momentum, which is the momentum-strategy value).
final
nesterov → bool
final
noImprovementLimit → int
Limit of epochs without improvements. Used to trigger strategies.
no setterinherited
noImprovementRatio ↔ double
Minimal improvement ratio.
getter/setter pairinherited
optimizerStep ↔ int
The number of weight-update steps applied so far (one per full-batch epoch, or one per mini-batch). Used e.g. for Adam bias correction; reset by reset and settable when resuming from a checkpoint.
getter/setter pairinherited
parameters → String
no setterinherited
random → Random
no setterinherited
runtimeType → Type
A representation of the runtime type of the object.
no setterinherited
samples → List<P>
Returns the samples of samplesSet
no setterinherited
samplesSet → SamplesSet<P>
The samples set for training.
finalinherited
samplesSubject → String
Returns the subject of samplesSet
no setterinherited
signalInstance → T
no setterinherited
startTime → DateTime?
The start time of the last training session or null if reset.
no setterinherited
subject → String
The training subject. Defaults to samplesSet.subject.
finalinherited
totalFailedEpochs → int
no setterinherited
totalTrainedEpochs → int
Returns the total number of epochs of all the training session. A call to reset won't reset this value.
no setterinherited
totalTrainingActivations → int
Returns the total number of activations of all the training session. A call to reset won't reset this value.
no setterinherited
trainedEpochs → int
Returns the number of epochs of the last training session.
no setterinherited
trainingActivations → int
Returns the number of activations of the last training session.
no setterinherited
trainingSamplesSize → int
no setterinherited
weightDecay → double
Decoupled L2 weight decay coefficient (AdamW-style); 0 disables it. Applied uniformly after the per-optimizer update: Δw −= lr·wd·w.
finalinherited

Methods

backPropagateLastLayerError(Layer<N, E, T, S> layer, int layerIndex, T expected) → void
inherited
backPropagateMiddleLayerError(Layer<N, E, T, S> layer, int layerIndex) → void
inherited
checkBestTrainingError(double trainingError) → void
inherited
computeEntryWeightUpdate(E weight, E weightLastUpdate, E gradient, E previousGradient, T previousUpdateDeltas, T noImprovementCounter, int weightsEntryIndex, E neuronOutput) → E
inherited
computeEntryWeightUpdateSIMD(E weight, E weightLastUpdate, E gradient, E previousGradient, T previousUpdateDeltas, T noImprovementCounter, int weightsEntryIndex, E neuronOutput) → E
Implementation of the weight update for an entry (SIMD).
inherited
computeGlobalError(List<P> samples) → double
inherited
computeWeightUpdate(N weight, N weightLastUpdate, num gradient, num previousGradient, List<num> previousUpdateDeltas, List<num> noImprovementCounter, int weightIndex, N neuronOutput) → double
Not used by GradientOptimizer subclasses (they override the richer updateWeightEntry seam); retained to satisfy the abstract Propagation.computeWeightUpdate contract.
inherited
createLearningRateStrategy() → ParameterStrategy<N, E, T>
inherited
createMomentumStrategy() → ParameterStrategy<N, E, T>
inherited
createWeightStateBuffers({double fill = 0}) → List<List<T>>
Allocates a per-weight state buffer group with the same shape as the network weights (per layer -> per source neuron -> Signal over the target neurons), initialized to fill. The group is registered so reset reinitializes it. Optimizer subclasses use this for their persistent state (e.g. Adam's first/second moments).
inherited
generateRandomValue(double range) → double
inherited
generateRandomValuePositive(double range) → double
inherited
generateRandomWeightUpdate(double range, double min, double max, double multiplier) → double
inherited
generateRandomWeightUpdateByFactor(double weight, double factor, {double zeroPoint = 0.01, double multiplier = 1.0}) → double
inherited
initializeParameters() → void
Initialize training parameters.
inherited
initializeTraining() → void
inherited
learn(List<P> samples, double targetGlobalError) → bool
Mini-batch epoch: shuffles the samples and updates the weights once per mini-batch (reusing the protected backprop helpers). Falls back to the full-batch Propagation.learn when batchSize is 0 or covers all samples.
inherited
loadGradients(List state) → void
Restores gradients previously produced by saveGradients.
inherited
loadOptimizerState(List state) → void
Restores optimizer state previously produced by saveOptimizerState.
inherited
logError(String message, [dynamic error, StackTrace? stackTrace]) → void
inherited
logInfo(String message) → void
inherited
logProgress(String message) → void
inherited
logWarn(String message) → void
inherited
noSuchMethod(Invocation invocation) → dynamic
Invoked when a nonexistent method or property is accessed.
inherited
recordTrainingBlock(double globalError, int epochs) → void
Records the outcome of an externally-driven training block (e.g. the async WebGPU trainers, which run epochs off the synchronous _trainImpl loop) so the public bookkeeping getters (globalError, trainedEpochs, totalTrainedEpochs, trainingActivations) reflect it. Also updates the best-training snapshot via checkBestTrainingError.
inherited
reset() → void
Reset this instance for a future training sessions.
inherited
resetBestTraining() → void
Discards the best weights/error tracked by checkBestTrainingError.
inherited
restoreGlobalLearnErrors(double last, double current) → void
Restores the epoch error-tracking used by iRProp+ backtracking (and by the learning-rate/momentum strategies). Used when resuming from a checkpoint.
inherited
saveGradients() → List<List<List<double>>>
Serializes the current accumulated gradients (layer.gradients) — the value that becomes previousGradient on the next epoch's resetGradients. Needed for an exact checkpoint resume of optimizers that read the previous gradient (Quickprop, iRProp+).
inherited
saveOptimizerState() → List<List<List<double>>>
Serializes the registered optimizer state buffers (for checkpointing). Each buffer group is flattened to a list of per-source-neuron value lists (layer-major, matching the allocation order).
inherited
selectInitialANN(List<P> samples, double targetGlobalError, [Random? random]) → void
Selects the initial ANN.
inherited
setLearningRate(double learningRate) → void
inherited
setMomentum(double momentum) → void
inherited
toString() → String
A string representation of this object.
inherited
train(int epochs, double targetGlobalError) → double
Train the samples for n epochs and returns the last global error.
inherited
trainUntilGlobalError({double? targetGlobalError, int epochsBlock = 50, int maxEpochs = 1000000, double maxEpochsLimitRatio = 3, int maxRetries = 5, double retryIncreaseMaxEpochsRatio = 1.50, Random? random}) → bool
Train the ann until targetGlobalError, with maxEpochs per training session and a maxRetries when a training session can't reach the target global error.
inherited
updateGlobalLearnError(double globalLearnError) → void
Publishes the epoch's learn error, rolling the previous value.
inherited
updateLayerWeights(Layer<N, E, T, S> layer, int layerIndex) → void
inherited
updateParameters() → void
Update training parameters.
inherited
updateWeightEntry({required int layerIndex, required int neuronIndex, required int entryIndex, required E weight, required E gradient, required E previousGradient, required E neuronOutput}) → E
The per-weight-entry update rule (SIMD). Returns the delta to ADD to the weight entry. Optimizers index their own state buffers with (layerIndex, neuronIndex, entryIndex).
override

Operators

operator ==(Object other) → bool
The equality operator.
inherited