forward method

ValueVector forward(
  1. ValueVector x
)

Forward pass for Layer Normalization.

Takes a vector x and returns the normalized vector.

Implementation

ValueVector forward(ValueVector x) {
  final mean = x.mean();

  // CORRECTED PART: Manually perform element-wise subtraction
  // This creates a new vector where each element is (x_i - mean).
  final x_minus_mean = ValueVector(x.values.map((v) => v - mean).toList());

  final variance = x_minus_mean.squared().mean();

  // Normalize x to have mean 0 and variance 1. Use the new vector here.
  final xHat = x_minus_mean / (variance + epsilon).sqrt();

  // Scale and shift with learnable parameters
  return (xHat * gamma) + beta;
}