LayerNormalization class

Data normalization Layer. Normalize each record in batch independently.

Can be seen as column-wise normalization.

For now supports two different normalization types:

Layer acts similar in fit, evaluate and predict modes

Example:

final data = Matrix.column([-2, -1, 0, 1, 2]);
final minmax = LayerNormalization(type: NormalizationType.minMax);
final zScore = LayerNormalization(type: NormalizationType.zScore);
minmax.init(5);
zScore.init(5);

// min-max normilized
print(minmax.act(data).flattenRow()); // transpose result
// Output: matrix 1⨯5 [[0.0, 0.25, 0.5, 0.75, 1.0]]

// z-score normilized
print(zScore.act(data).flattenRow()); // transpose result
//Output: matrix 1⨯5 [[-1.4142135, -0.707106781, 0.0, 0.707106781, 1.4142135]]

For batchSize > 1 in NeuralNetwork these Layer applies normalization logic for each column of bacth matrix

Example:

final batch = Matrix.fromLists(
    [[-4, -1, 0, 2, 3], // first data record
     [-4, -2, 1, 2, 3]] // second data record
  ).T; // transpose so records are represented as columns

  print(batch);
  // Output:
  // matrix 5⨯2
  // [[-4.0, -4.0]
  // [-1.0, -2.0]
  // [0.0, 1.0]
  // [2.0, 2.0]
  // [3.0, 3.0]]


  final minmax = LayerNormalization(type: NormalizationType.minMax);
  minmax.init(5);

  var result = minmax.act(batch);
  print(result);
  // Output:
  // matrix 5⨯2
  // [[0.0, 0.0]
  // [0.42857142857142855, 0.2857142857142857]
  // [0.5714285714285714, 0.7142857142857142]
  // [0.8571428571428571, 0.8571428571428571]
  // [1.0, 1.0]]
Inheritance

Constructors

LayerNormalization({NormalizationType type = NormalizationType.minMax, String? name})

Properties

activatedDerivativeBuffer ↔ List<Matrix>?
Derivatives of the activation function used in the learning process
getter/setter pairinherited
b ↔ Matrix?
Matrix.column of the biases of the Layer
getter/setter pairinherited
hashCode → int
The hash code for this object.
no setterinherited
inputDataBuffer ↔ Matrix?
Input data buffer used in the learning proccess
getter/setter pairinherited
name ↔ String?
The name of the layer
getter/setter pairinherited
runtimeType → Type
A representation of the runtime type of the object.
no setterinherited
trainable ↔ bool
Identify if this is trainable
getter/setter pairinherited
type ↔ NormalizationType
getter/setter pair
units ↔ int
Number of rows in output Matrix.column after activation (or applying) of the Layer over input data
getter/setter pairinherited
useBiases → bool
Defines if train biases or keep them as zero vector
finalinherited
w ↔ Matrix?
Matrix of the weights of the Layer
getter/setter pairinherited
wasInitialized ↔ bool
Identify if initialization was called
getter/setter pairinherited

Methods

act(dynamic inputs, {bool train = false}) → Matrix
Apply Layer's logic to the inputs
override
clear() → void
After-training method, typically clear buffered data from the training process
inherited
init([dynamic parametr]) → void
Initialization of Layer's parametrs should be called before calling (act()) the Layer
override
noSuchMethod(Invocation invocation) → dynamic
Invoked when a nonexistent method or property is accessed.
inherited
toString() → String
A string representation of this object.
override

Operators

operator ==(Object other) → bool
The equality operator.
inherited