AdamGPU class

Applies the Adam optimizer to every element of its parameter tensors. Adam combines the advantages of AdaGrad and RMSProp by computing adaptive learning rates for each parameter using the first and second moments of the gradients.

Inheritance

Constructors

AdamGPU(List<GPUTensor> parameters, double learningRate, {double beta1 = 0.9, double beta2 = 0.999, double epsilon = 1e-7, double weightDecay = 0.0})
Initializes the Adam optimizer and allocates VRAM for the moment caches. Requires the list of trainable parameters and the learningRate. Optional parameters beta1, beta2, epsilon, and weightDecay allow fine-tuning of the algorithm.

Properties

beta1 ↔ double
The exponential decay rate for the first moment estimates (momentum).
getter/setter pair
beta2 ↔ double
The exponential decay rate for the second moment estimates (variance).
getter/setter pair
currentStep ↔ int
Tracks the current optimization iteration for bias correction. Exposed publicly so training can be perfectly paused, saved, and resumed.
getter/setter pair
epsilon ↔ double
A very small value added to the denominator to prevent division by zero.
getter/setter pair
hashCode → int
The hash code for this object.
no setterinherited
learningRate ↔ double
getter/setter pairinherited
mCaches ↔ List<GPUTensor>
Persistent VRAM caches storing the first moment (moving average of the gradient) for each parameter.
getter/setter pair
parameters ↔ List<GPUTensor>
List of parameters the gradient is responsible for updating.
getter/setter pairinherited
runtimeType → Type
A representation of the runtime type of the object.
no setterinherited
vCaches ↔ List<GPUTensor>
Persistent VRAM caches storing the second moment (moving average of the squared gradient) for each parameter.
getter/setter pair
weightDecay ↔ double
The L2 penalty factor applied to the weights to prevent overfitting.
getter/setter pair

Methods

free() → void
Frees VRAM for all persistently allocated moment caches.
noSuchMethod(Invocation invocation) → dynamic
Invoked when a nonexistent method or property is accessed.
inherited
refreshStep(CommandBuffer tape) → void
Patches the compiled tape with the current step and advances it. Call this once per training step, right before SeqModel replays the optimize tape.
override
step(CommandBuffer tape) → void
Appends the Adam weight update operations to the provided tape. Increments the currentStep automatically after processing all parameters.
override
toString() → String
A string representation of this object.
inherited
zeroGrad(CommandBuffer tape) → void
Appends the zero-gradient operations to the tape to clear the gradients for the next pass.
override

Operators

operator ==(Object other) → bool
The equality operator.
inherited