vectorLeakyReLU function

Tensor<Vector> vectorLeakyReLU(
  1. Tensor<Vector> v, {
  2. double alpha = 0.01,
})

Mathematical operation for element-wise Leaky ReLU on a vector.

Implementation

Tensor<Vector> vectorLeakyReLU(Tensor<Vector> v, {double alpha = 0.01}) {
  int N = v.value.length;
  Vector outValue = [];
  for (int i = 0; i < N; i++) {
    outValue.add(v.value[i] > 0 ? v.value[i] : alpha * v.value[i]);
  }
  Tensor<Vector> out = Tensor<Vector>(outValue);

  out.creator = Node([v], () {
    for (int i = 0; i < N; i++) {
      v.grad[i] += out.grad[i] * (v.value[i] > 0 ? 1.0 : alpha);
    }
  }, opName: 'leaky_relu', cost: N);
  return out;
}