Persistent VRAM caches storing the second moment (moving average of the squared gradient) for each parameter.
List<GPUTensor> vCaches = <GPUTensor>[];