matrixELU function

Tensor<Matrix> matrixELU(Tensor<Matrix> m, { double alpha = 1.0, })

Mathematical operation for the ELU function on a matrix.

Implementation

Tensor<Matrix> matrixELU(Tensor<Matrix> m, {double alpha = 1.0}) {
  int numRows = m.value.length;
  int numCols = m.value[0].length;
  Matrix outValue = [];
  for (int i = 0; i < numRows; i++) {
    Vector row = [];
    for (int j = 0; j < numCols; j++) {
      row.add(m.value[i][j] > 0 ? m.value[i][j] : alpha * (exp(m.value[i][j]) - 1));
    }
    outValue.add(row);
  }

  Tensor<Matrix> out = Tensor<Matrix>(outValue);
  out.creator = Node([m], () {
    for (int i = 0; i < numRows; i++) {
      for (int j = 0; j < numCols; j++) {
        m.grad[i][j] += out.grad[i][j] * (m.value[i][j] > 0 ? 1.0 : out.value[i][j] + alpha);
      }
    }
  }, opName: 'elu_matrix', cost: numRows * numCols);
  return out;
}