sampleWindow function

(Tensor, Tensor) sampleWindow(
  1. List<double> ids,
  2. int blockSize,
  3. Random rng, {
  4. required Device device,
})

Sample one [blockSize] window at a random start position and return the (x, y = next-token) pair as rank-1 tensors on device.

Implementation

(Tensor, Tensor) sampleWindow(
  List<double> ids,
  int blockSize,
  math.Random rng, {
  required Device device,
}) {
  final maxStart = ids.length - blockSize - 1;
  if (maxStart <= 0) {
    throw ArgumentError(
      'sampleWindow: corpus (${ids.length}) too short for blockSize $blockSize',
    );
  }
  final start = rng.nextInt(maxStart);
  final x = List<double>.generate(blockSize, (i) => ids[start + i]);
  final y = List<double>.generate(blockSize, (i) => ids[start + i + 1]);
  return (
    Tensor.fromList([blockSize], x, device: device),
    Tensor.fromList([blockSize], y, device: device),
  );
}