cpu_version/activationFunctions/activation_function library

Export the activation funcitons of this engine

Classes

ActivationFunction<T>

The abstract base class (or interface) for all activation functions.

ELUMatrix

Applies the Exponential Linear Unit (ELU) activation function over a given input of type Tensor<Matrix>. In contrast to the standard ReLU function, this function has a small negative value for negative inputs, which can help prevent the dying ReLU problem and speed up learning by pushing the mean activation closer to zero.

ELUVector

Applies the Exponential Linear Unit (ELU) activation function over a given input of type Tensor<Vector>. In contrast to the standard ReLU function, this function has a small negative value for negative inputs, which can help prevent the dying ReLU problem and speed up learning by pushing the mean activation closer to zero.

LeakyReLUMatrix

Applies leaky ReLu over a given input of type Tensor<Matrix>. In contrast to the standard ReLU function, this function multiplies negative values with a small negative number alpha to prevent the dying gradient problem.

LeakyReLUVector

Applies leaky ReLu over a given input of type Tensor<Vector>. In contrast to the standard ReLU function, this function multiplies negative values with a small negative number alpha to prevent the dying gradient problem.

MishMatrix

Applies the mish activation function (similar to swish) to each element inside of a given input of type Tensor<Matrix>. Mish mirrors Relu with a continuous curve. The calculation is handled via the mishMatrix math operation.

MishVector

Applies the mish activation function (similar to swish) to each element inside of a given input of type Tensor<Vector>. Mish mirrors Relu with a continuous curve. The calculation is handled via the mishVector math operation.

ReLU

An activation function that applies the Rectified Linear Unit (ReLU) to a Vector.

ReLUMatrix

An activation function that applies the Rectified Linear Unit (ReLU) to a Matrix.

Sigmoid

The Sigmoid activation function.

SigmoidMatrix

An activation function that applies the Sigmoid function to a Matrix.

SoftmaxMatrix

Applies the softmax function over a given input of type Tensor<Matrix>. Creates a probability distribution over all values of each row summing up to 1. The calculation is handled via the softmaxMatrix math operation.

SoftmaxVector

Applies the softmax function over a given input of type Tensor<Vector>. Creates a probability distribution over all values summing up to 1. The calculation is handled via the softmaxVector math operation.

SwishMatrix

Applies the Sigmoid-weighted Linear Unit (SiLU / Swish) activation function to each element inside of a given input of type Tensor<Matrix>. Defined mathematically as f(x) = x * sigmoid(x), this modern activation function often outperforms standard ReLU in deeper hidden layers.

SwishVector

Applies the Sigmoid-weighted Linear Unit (SiLU / Swish) activation function to each element inside of a given input of type Tensor<Vector>. Defined mathematically as f(x) = x * sigmoid(x), this modern activation function often outperforms standard ReLU in deeper hidden layers.