addBiasToMatMulOutGPU function

GPUTensor<Matrix> addBiasToMatMulOutGPU(
  1. GPUTensor<Matrix> m,
  2. GPUTensor<Vector> b,
  3. CommandBuffer tape, {
  4. GPUTensor<Matrix>? outTensor,
})

Appends commands for addition of a 1D bias vector b to a 2D matrix m via broadcasting over the matrix to the tape.

Implementation

GPUTensor<Matrix> addBiasToMatMulOutGPU(
    GPUTensor<Matrix> m, GPUTensor<Vector> b, CommandBuffer tape,
    {GPUTensor<Matrix>? outTensor}) {
  int rows = m.shape[0];
  int cols = m.shape[1];

  GPUTensor<Matrix> out =
      outTensor ?? GPUTensor<Matrix>.empty(<int>[rows, cols]);

  tape.putInt(OP_BROADCAST_ADD);
  tape.putString(m.id);
  tape.putString(b.id);
  tape.putString(out.id);

  out.creator = GPUNode(
    <GPUTensor>[m, b],
    (CommandBuffer bTape) {
      bTape.putInt(OP_ADD_INTO);
      bTape.putString(out.gradId);
      bTape.putString(m.gradId);

      // FIXED: Sum across columns to isolate the row biases!
      bTape.putInt(OP_SUM_REDUCE_ROWS);
      bTape.putString(out.gradId);
      bTape.putString(b.gradId);
    },
    opName: 'addBiasToMatMulOutGPU',
    cost: rows * cols,
  );

  return out;
}