Embedding constructor

Embedding(
  1. int numEmbeddings,
  2. int embeddingDim, {
  3. Device device = Device.CPU,
  4. int seed = 0,
})

Constructs a table of shape [numEmbeddings, embeddingDim] initialized with a small-scale normal (std = 1/sqrt(dim)) — this keeps output activations roughly unit-scaled before the first forward pass through the rest of the model.

Implementation

Embedding(
  this.numEmbeddings,
  this.embeddingDim, {
  Device device = Device.CPU,
  int seed = 0,
}) : weight = _initWeight(numEmbeddings, embeddingDim, device, seed);