forward method

@override Tensor<Matrix> forward(Tensor input)

The core logic of the layer's transformation.

Subclasses must implement this method to define how they process input tensors and return an output tensor.

Implementation

@override
Tensor<Matrix> forward(Tensor<dynamic> input) {
  Vector wordIndices = (input as Tensor<Vector>).value;
  Matrix outputSequence = [];

  for (double indexDouble in wordIndices) {
    int index = indexDouble.toInt();
    outputSequence.add(embeddings.value[index]);
  }

  Tensor<Matrix> out = Tensor<Matrix>(outputSequence);

  out.creator = Node([embeddings], () {
    for (int i = 0; i < wordIndices.length; i++) {
      int index = wordIndices[i].toInt();
      for (int j = 0; j < embeddingDimension; j++) {
        embeddings.grad[index][j] += out.grad[i][j];
      }
    }
  }, opName: 'embedding_lookup', cost: 0);

  return out;
}