embeddingLookupBatchGPU function

GPUTensor<Tensor3D> embeddingLookupBatchGPU(
  1. GPUTensor<Matrix> batchIndices,
  2. GPUTensor<Matrix> weights,
  3. CommandBuffer tape, {
  4. GPUTensor<Tensor3D>? outTensor,
})

Acts as a high performance lookup of rows in a given matrix weights via their row indices indices. In contrast to embeddingLookupGPU this method performs a batch process to speed up lookup. Can be used for fast odering of elements. The operation is appended to tape for execution.

Implementation

GPUTensor<Tensor3D> embeddingLookupBatchGPU(GPUTensor<Matrix> batchIndices,
    GPUTensor<Matrix> weights, CommandBuffer tape,
    {GPUTensor<Tensor3D>? outTensor}) {
  int batchSize = batchIndices.shape[0];
  int sequenceLength = batchIndices.shape[1];
  int embeddingDim = weights.shape[1];

  GPUTensor<Tensor3D> out = outTensor ??
      GPUTensor<Tensor3D>.empty([batchSize, sequenceLength, embeddingDim]);

  tape.putInt(OP_EMBEDDING_FORWARD);
  tape.putString(batchIndices.id);
  tape.putString(weights.id);
  tape.putString(out.id);

  out.creator = GPUNode(
    [weights],
    (CommandBuffer bTape) {
      bTape.putInt(OP_EMBEDDING_BACKWARD);
      bTape.putString(out.gradId);
      bTape.putString(batchIndices.id);
      bTape.putString(weights.gradId);
    },
    opName: 'embedding_lookup_batch_gpu',
    cost: batchSize * sequenceLength * embeddingDim,
  );

  return out;
}