GPUEngine class

The Cuda Engine in responsible for communication with the native runtimes as well as managing the communication between dart and its native FFI-bindings. In order to use the GPU acceleration, at the very beginning of the program it is required to call initialize with the Target provided to load the relevant binary. This class is to be used globally as a sole instance of communication and data transfer. Its handle acts as the id of the native executor instance. If there is currently no instance running this value will be 0. Currently supported are WebGPU with android_arm64, android_x86_64 and cudafrom version 1.12.1 (2026-09). Use dispose to delete the currently running instance of GPUEngine and free all memory again.

Constructors

GPUEngine()

Properties

hashCode → int

The hash code for this object.

no setter, inherited

runtimeType → Type

A representation of the runtime type of the object.

no setter, inherited

Methods

noSuchMethod(Invocation invocation) → dynamic

Invoked when a nonexistent method or property is accessed.

inherited

toString() → String

A string representation of this object.

inherited

Operators

operator ==(Object other) → bool

The equality operator.

inherited

Static Properties

handle ↔ int

Id of the currently running GPUEngine. If its value is 0, no engine is running.

getter/setter pair

Static Methods

addPointers(Pointer<Float> dest, Pointer<Float> src, int length) → void

Allows a direct memory copy from src to dest for faster transfer.

dispose() → void

Deletes the current instance of the GPUEngine and frees all allocated memory.

free(String name) → void

Frees a given tensors allocation on the device(GPU).

getTensorNames() → List<String>

Retrieves all tensor names currently allocated on the GPU. Allocates memory to store the String list natively and deletes it before returning its value,

initialize({bool debug = false, Target target = Target.cuda}) → Future<void>

Initialises the GPUEngine. If no target platform is provided initialising with cuda is attempted. debug enables full printout of every interaction for in debugging. Initialize has to be called at least once before using the GPUEngine.

initRandomUniform(String name, double scale, int seed) → void

Fills the tensor name with uniformly distributed numbers scaled with scale.

load(String name, Pointer<Float> data, List<int> shape) → void

Allocates and transfers tensor data from host to device (GPU). Requires the tensors name, data and shape.

retrieve(String name, Pointer<Float> destination) → void

Transfers a tensors data from device(GPU) to host. Requires the tensors name and transfer destination.

run(Uint8List tapeBytes) → void

Passes a compiled CommandBuffer tape to the native engine and executes it. Bytes are copied into memory before dispatch.