predict method

GPUTensor<OutputType> predict(
  1. GPUTensor<InputType> newInferInput
)

Compiles a lightweight static inference tape tailored to the shape of newInferInput. This safely resizes internal layer VRAM caches (e.g., matching a batch size of 1) and returns the output tensor reference.

Implementation

GPUTensor<OutputType> predict(GPUTensor<InputType> newInferInput) {
  inferInput = newInferInput;
  inferTape = CommandBuffer();
  GPUTensor<dynamic> current = inferInput!;

  for (int i = 0; i < layers.length; i = i + 1) {
    current = layers[i].forward(current, inferTape!, <GPUTensor>[]);
  }

  if (current is! GPUTensor<OutputType>) {
    throw StateError('[SeqModel Inference Error] Architecture Mismatch:\n'
        'The Sequential Model is defined to return a GPUTensor<$OutputType>, '
        'but the final layer produced a ${current.runtimeType}.');
  }

  inferResult = current;
  return inferResult!;
}