gpu_version/optimizer/optimizer library

Holds the optimizers for the GPU.

Classes

AdamGPU

Applies the Adam optimizer to every element of its parameter tensors. Adam combines the advantages of AdaGrad and RMSProp by computing adaptive learning rates for each parameter using the first and second moments of the gradients.

OptimizerGPU

Optimizers are used to update a parameters value to reduce the error of the system. For that the gradient of the requested value is used. Optimizers CAN be static so their execution tapes can be reused of they may require recompilation on every training step.

SGDGPU

Applies the SGD optimizer to every element of its parameter tensors. SGD is a very simply descent method and my get stuck in local minima. This function does not require to be reallocated on each training step and can be reused indefinitely.