swish function

double swish(
  1. double x
)

Similar to ReLU and Softplus; negative results do occur, but they approach 0 up until x ≈ -10. Delivers comparable or superior results to ReLU.

Implementation

double swish(double x) => x * sigmoid(x);