dropoutGPU<T> function

GPUTensor<T> dropoutGPU<T>(GPUTensor<T> input, double rate, CommandBuffer tape, { GPUTensor<T>? outTensor, GPUTensor<T>? maskTensor, })

Randomly zeroes out elements of the input tensor with a probability equal to the specified rate. To preserve the expected mathematical sum of the tensor during inference, it scales the remaining active elements by 1 / (1 - rate). Outputs the modified tensor alongside the generated dropout mask. The operations are appended to tape for execution.

Implementation

GPUTensor<T> dropoutGPU<T>(GPUTensor<T> input, double rate, CommandBuffer tape,
    {GPUTensor<T>? outTensor, GPUTensor<T>? maskTensor}) {
  int seed = Random().nextInt(1000000);

  GPUTensor<T> out = outTensor ?? GPUTensor<T>.empty(input.shape);
  GPUTensor<T> mask = maskTensor ?? GPUTensor<T>.empty(input.shape);
  out.subMap["mask"] = mask;

  tape.putInt(OP_DROPOUT_FORWARD);
  tape.putString(input.id);
  tape.putString(out.id);
  tape.putString(mask.id);
  tape.putFloat(rate);
  tape.putInt(seed);

  int elementCount = 1;
  for (int i = 0; i < input.shape.length; i = i + 1) {
    elementCount = elementCount * input.shape[i];
  }

  out.creator = GPUNode(
    <GPUTensor>[input],
    (CommandBuffer bTape) {
      bTape.putInt(OP_DROPOUT_BACKWARD);
      bTape.putString(out.gradId);
      bTape.putString(mask.id);
      bTape.putString(input.gradId);
    },
    opName: 'dropout_gpu',
    cost: elementCount * 2,
  );

  return out;
}