AMSGrad class

Implements the AMSGrad optimizer.

AMSGrad is a variant of the Adam optimizer that aims to fix a potential convergence issue. It achieves this by using the maximum of past squared gradients (v_hat) for normalization, rather than just the exponential average (v).

This ensures that the adaptive learning rate is non-increasing, which can improve stability and guarantee convergence in some scenarios where Adam might fail.

Example

var optimizer = AMSGrad(model.parameters, learningRate: 0.001);

Inheritance

• Object
• Optimizer
• AMSGrad

Constructors

AMSGrad(List<Tensor> parameters, {required double learningRate, double beta1 = 0.9, double beta2 = 0.999, double epsilon = 1e-8})

Properties

beta1 → double

final

beta2 → double

final

epsilon → double

final

hashCode → int

The hash code for this object.

no setter, inherited

learningRate → double

The step size for the gradient updates.

final, inherited

parameters → List<Tensor>

The list of model parameters (weights and biases) that this optimizer will update.

final, inherited

runtimeType → Type

A representation of the runtime type of the object.

no setter, inherited

Methods

noSuchMethod(Invocation invocation) → dynamic

Invoked when a nonexistent method or property is accessed.

inherited

step() → void

Performs a single optimization step according to the AMSGrad update rule.

override

toString() → String

A string representation of this object.

inherited

zeroGrad() → void

Resets the gradients of all parameters to zero.

inherited

Operators

operator ==(Object other) → bool

The equality operator.

inherited