gpu_version/ffi/ffi library

Holds the exports for the ffi interaction with the C++ library

Classes

CommandBuffer
CommandBuffer in this library is used as a tape to record actions for the native backend to execute. Commands are stored as a continuous stream of raw bytes. Each command constists of two parts:
GPUEngine
The Cuda Engine in responsible for communication with the native runtimes as well as managing the communication between dart and its native FFI-bindings. In order to use the GPU acceleration, at the very beginning of the program it is required to call initialize with the Target provided to load the relevant binary. This class is to be used globally as a sole instance of communication and data transfer. Its handle acts as the id of the native executor instance. If there is currently no instance running this value will be 0. Currently supported are WebGPU with android_arm64, android_x86_64 and cudafrom version 1.12.1 (2026-09). Use dispose to delete the currently running instance of GPUEngine and free all memory again.
TapeDecoder
The TapeDecoder class offers a quality of life tool to allow decoding and visual printing of the contents of a CommandBuffer. To decode a CommandBuffer the following command can be used:

Enums

Target
Targeted device architecture

Constants

OP_ABS_BACKWARD → const int
OP_ABS_ELEMENTWISE → const int
OP_ADAM_UPDATE → const int
OP_ADD → const int
OP_ADD_INTO → const int
OP_ADD_SCALAR → const int
OP_ARGMAX_FORWARD → const int
OP_AVG_POOL_2D_BACKWARD → const int
OP_AVG_POOL_2D_FORWARD → const int
OP_BATCH_NORM_1D_BACKWARD → const int
OP_BATCH_NORM_1D_FORWARD → const int
OP_BATCH_NORM_2D_BACKWARD → const int
OP_BATCH_NORM_2D_FORWARD → const int
OP_BCE_LOSS_BACKWARD → const int
OP_BCE_LOSS_FORWARD → const int
OP_BROADCAST_ADD → const int
OP_CAUSAL_MASK_BACKWARD → const int
OP_CAUSAL_MASK_FORWARD → const int
OP_CLAMP_BACKWARD → const int
OP_CLAMP_ELEMENTWISE → const int
OP_CLIP_GRAD_VALUE → const int
OP_COL2IM → const int
OP_CONCATENATE → const int
OP_CONCATENATE_BACKWARD → const int
OP_CONV2D_BACKWARD_INPUT → const int
OP_CONV2D_BACKWARD_KERNEL → const int
OP_CONV2D_FORWARD → const int
OP_CONV2D_MULTI_BACKWARD_INPUT → const int
OP_CONV2D_MULTI_BACKWARD_WEIGHT → const int
OP_CONV2D_MULTI_FORWARD → const int
OP_COPY → const int
OP_CROSS_ENTROPY_BACKWARD → const int
OP_CROSS_ENTROPY_FORWARD → const int
OP_DIVIDE → const int
OP_DIVIDE_BACKWARD → const int
OP_DROPOUT_BACKWARD → const int
OP_DROPOUT_FORWARD → const int
OP_EMBEDDING_BACKWARD → const int
OP_EMBEDDING_FORWARD → const int
OP_EXP_BACKWARD → const int
OP_EXP_ELEMENTWISE → const int
OP_FILL → const int
OP_GELU_BACKWARD → const int
OP_GELU_FORWARD → const int
OP_GLOBAL_AVG_POOL_BACKWARD → const int
OP_GLOBAL_AVG_POOL_FORWARD → const int
OP_IM2COL → const int
OP_LAYER_NORM_BACKWARD → const int
OP_LAYER_NORM_FORWARD → const int
OP_LOAD_SAMPLE → const int
OP_LOG_BACKWARD → const int
OP_LOG_ELEMENTWISE → const int
OP_MARKOV_COUNT → const int
OP_MARKOV_NORMALIZE → const int
OP_MARKOV_PREDICT → const int
OP_MATMUL → const int
OP_MATMUL_BIAS_RELU_FORWARD → const int
OP_MAX_POOL_1D_BACKWARD → const int
OP_MAX_POOL_1D_FORWARD → const int
OP_MAX_POOL_2D_BACKWARD → const int
OP_MAX_POOL_2D_FORWARD → const int
OP_MSE_LOSS_BACKWARD → const int
OP_MSE_LOSS_FORWARD → const int
OP_MULTIPLY → const int
OP_MULTIPLY_BACKWARD → const int
OP_PAD2D → const int
OP_PAD2D_BACKWARD → const int
OP_POW_BACKWARD → const int
OP_POW_ELEMENTWISE → const int
OP_RELU → const int
OP_RELU_BACKWARD → const int
OP_RMS_NORM_BACKWARD → const int
OP_RMS_NORM_FORWARD → const int
OP_ROPE_BACKWARD → const int
OP_ROPE_FORWARD → const int
OP_SCALE_MATRIX → const int
OP_SCALE_MATRIX_BACKWARD → const int
OP_SGD_UPDATE → const int
OP_SIGMOID → const int
OP_SIGMOID_BACKWARD → const int
OP_SLICE_COLUMN → const int
OP_SLICE_COLUMN_BACKWARD → const int
OP_SLICE_ROW → const int
OP_SLICE_ROW_BACKWARD → const int
OP_SOFTMAX_BACKWARD → const int
OP_SOFTMAX_FORWARD → const int
OP_SQRT_BACKWARD → const int
OP_SQRT_ELEMENTWISE → const int
OP_STACK_ROWS → const int
OP_STACK_ROWS_BACKWARD → const int
OP_STORE_SAMPLE → const int
OP_SUBTRACT → const int
OP_SUBTRACT_INTO → const int
OP_SUM_REDUCE → const int
OP_SUM_REDUCE_BACKWARD → const int
OP_SUM_REDUCE_COLUMNS → const int
OP_SUM_REDUCE_ROWS → const int
OP_TANH → const int
OP_TANH_BACKWARD → const int
OP_TRANSPOSE → const int
OP_ZERO_GRAD → const int

Typedefs

DartAddPointers = void Function(Pointer<Float>, Pointer<Float>, int)
DartCreate = int Function(bool)
DartFree = void Function(int)
DartFreeStr = void Function(Pointer<Utf8>)
DartFreeT = void Function(int, Pointer<Utf8>)
DartGetNames = Pointer<Utf8> Function(int)
DartInitRandom = void Function(int, Pointer<Utf8>, double, int)
DartLoad = void Function(int, Pointer<Utf8>, Pointer<Float>, int, Pointer<Int32>)
DartRetrieve = void Function(int, Pointer<Utf8>, Pointer<Float>)
DartRun = void Function(int, Pointer<Uint8>, int)
NativeAddPointers = Void Function(Pointer<Float>, Pointer<Float>, Int32)
FFI- Advanced operation to allow direct pointer access. Adds the content of a source content into a destination pointer with a given length
NativeCreate = Int64 Function(Bool)
FFI- Binding for creating a tensor on the GPU
NativeFree = Void Function(Int64)
FFI- Binding for freeing/deleting the GPU backend
NativeFreeStr = Void Function(Pointer<Utf8>)
FFI- Binding for freeing the made list of tensor names
NativeFreeT = Void Function(Int64, Pointer<Utf8>)
FFI- Binding for freeing a tensor on the GPU
NativeGetNames = Pointer<Utf8> Function(Int64)
FFI- Binding for retrieving the ids of all tensors currently on the GPU. This creates a List of Strings that is stored in the backend and has to be deleted again
NativeInitRandom = Void Function(Int64, Pointer<Utf8>, Float, Int32)
FFI- Initialise a tensor with a random initialization
NativeLoad = Void Function(Int64, Pointer<Utf8>, Pointer<Float>, Int32, Pointer<Int32>)
FFI- Binding for pushing a tensor to the GPU
NativeRetrieve = Void Function(Int64, Pointer<Utf8>, Pointer<Float>)
FFI- Binding for pulling a tensor from the GPU
NativeRun = Void Function(Int64, Pointer<Uint8>, Int32)
FFI- Binding for running a given byteList of commands (usually via a CommandBuffer)