gpu_version/gpu_version library

Holds and exports the subclasses for GPU processing

Classes

AdamGPU
Applies the Adam optimizer to every element of its parameter tensors. Adam combines the advantages of AdaGrad and RMSProp by computing adaptive learning rates for each parameter using the first and second moments of the gradients.
AveragePooling2DTL
Slides a two-dimensional window over a given matrix. Each window returns the average over all its values.
BatchNorm1DTL
Applies Batch Normalization over a 1D input vector to enforce a mean of 0 and a variance of 1.
BatchNorm2DTL
Applies Batch Normalization over a 3D input Tensor3D to enforce a mean of 0 and a variance of 1. It standardizes the spatial dimensions for each sub-element in the input Tensor3D (which are matrices) independently. Usually, the different matrices building the Tensor3D can be viewed as different channels.
CommandBuffer
CommandBuffer in this library is used as a tape to record actions for the native backend to execute. Commands are stored as a continuous stream of raw bytes. Each command constists of two parts:
Conv2DTL
Applies a 2D convolution over a multi-channel input using learnable spatial filters. Works for Matrix and Tensor3D inputs. On Matrix input it is assumed to contain 1 channel.
ConvLSTMTL
Advanced Layer to process sequences of 2D spatial data. In this case it is represented as Tensor3Ds with format 'Time, Height, Width'. It uses convolutional gates instead of standard matrix multiplications. Therefore this architecture is able to capture both time and spatial information.
DenseReluTL
Represents a standard fully connected layer followed by a ReLU activation function. Utilizes a fused GPU kernel to perform matrix multiplication, bias broadcasting, and ReLU activation in a single optimized pass.
DenseTL
Represents a standard fully connected layer of a neural network. It performs a matrix multiplication and adds a bias vector to the result.
DropoutTL
A dropout layer that randomly mutes some elements of the provided GPUTensor. Works on Scalar, Vector, Matrix and Vector3D. It scales the remaining active elements by 1 / (1 - rate).
DualLSTMTL
Advanced Layer to process sequences of data. It is designed to capture short and long term dependencies of data entries. It is given a Matrix in the shape of (Time, Features) and returns a prediction for the next set of features. A Low tier resolution LSTM updates every step to capture short-term dependencies while a long-term high tier resolution layer provides trend prediction.
EmbeddingMatrixTL
Embedding layer to act as a discrete lookup table. Processes a batch of sequences of indices (Matrix) finding its embedding and returning a batch of sequences of embeddings as a Tensor3D.
EmbeddingTL
Embedding layer to act as a discrete lookup table. Processes a single sequence of indices (Vector) finding its embedding and returning a sequence of embeddings as a matrix.
FlattenTL
Flattens a Tensor3D into a Matrix.
GeluLayerMatrixTL
Applies the gelu function over a Matrix
GeluLayerTL
Applies the gelu function over a Vector.
GlobalAveragePooling1DTL
Collapses a matrix input to a vector containing the average values of its columns.
GlobalAveragePoolingTL
Collapses a matrix input into a vector of the average values of its feature columns. Does not store intermediate values.
GPUEngine
The Cuda Engine in responsible for communication with the native runtimes as well as managing the communication between dart and its native FFI-bindings. In order to use the GPU acceleration, at the very beginning of the program it is required to call initialize with the Target provided to load the relevant binary. This class is to be used globally as a sole instance of communication and data transfer. Its handle acts as the id of the native executor instance. If there is currently no instance running this value will be 0. Currently supported are WebGPU with android_arm64, android_x86_64 and cudafrom version 1.12.1 (2026-09). Use dispose to delete the currently running instance of GPUEngine and free all memory again.
LayerNormalizationTL
Applies Layer Normalization over a batch of inputs. Standardizes the features for each sample to have a mean of 0 and a variance of 1.
LSTMTL
Standard Long Short-Term Memory (LSTM) Layer. Processes a sequence of data to capture long-term dependencies.
MaxPooling1DTL
Slides a one-dimensional window over a given vector. Each window returns the maximum of all its values.
MaxPooling2DTL
Slides a two-dimensional window over a given matrix. Each window returns the maximum of all its values.
MultiHeadAttentionTL
Applies Multi-Head Attention to a sequence. Projects the input into Query, Key, and Value matrices, splits them into multiple heads, calculates scaled dot-product attention, and projects the concatenated results back to the original dimension.
OptimizerGPU
Optimizers are used to update a parameters value to reduce the error of the system. For that the gradient of the requested value is used. Optimizers CAN be static so their execution tapes can be reused of they may require recompilation on every training step.
PositionalEncodingTL
Injects spatial/sequential context into tokens. Because self-attention is permutation-invariant, this layer adds fixed sine and cosine waves of different frequencies to the embeddings, allowing the model to perceive the absolute and relative positions of words in a sequence.
ReLULayerMatrixTapeLayer
Applies the ReLu-function to each element in a matrix.
ReLULayerTL
Applies the ReLu-function to each element in a vector.
RNNTL
A standard Recurrent Neural Network (RNN) Layer. Processes a sequence of data sequentially, maintaining a hidden state that is updated at each time step based on the current input and the previous hidden state.
SeqModel<InputType, OutputType>
Sequential aid for constructing models to run. The Sequential Model wraps a list of TapeLayers to manage static graph compilation via command tapes. It handles VRAM lifecycles, training loops, and high-speed inference.
SGDGPU
Applies the SGD optimizer to every element of its parameter tensors. SGD is a very simply descent method and my get stuck in local minima. This function does not require to be reallocated on each training step and can be reused indefinitely.
SigmoidMatrixTL
Applies the Sigmoid-function to each element in a matrix.
SingleHeadAttentionTL
Applies Scaled Dot-Product Attention over a sequence. Projects the input into Query, Key, and Value matrices to determine contextual relationships between tokens before re-combining them based on the calculated attention scores.
TapeDecoder
The TapeDecoder class offers a quality of life tool to allow decoding and visual printing of the contents of a CommandBuffer. To decode a CommandBuffer the following command can be used:
TapeLayer<InputType, OutputType>
TapeLayer is built to be an additional abstraction that allows for complex mathematical bundling. Operations like Dense Neural Networks are supported as a single TapeLayer and can be used to build models reliably. Each TapeLayer has the following individually assigned values:
TransformerEncoderBlockTapeLayer
A full Transformer Encoder Block. Combines Multi-Head Self-Attention, Residual Connections, Layer Normalization, and a position-wise Feed-Forward Network (FFN) using GELU activation.

Enums

Target
Targeted device architecture

Constants

OP_ABS_BACKWARD → const int
OP_ABS_ELEMENTWISE → const int
OP_ADAM_UPDATE → const int
OP_ADD → const int
OP_ADD_INTO → const int
OP_ADD_SCALAR → const int
OP_ARGMAX_FORWARD → const int
OP_AVG_POOL_2D_BACKWARD → const int
OP_AVG_POOL_2D_FORWARD → const int
OP_BATCH_NORM_1D_BACKWARD → const int
OP_BATCH_NORM_1D_FORWARD → const int
OP_BATCH_NORM_2D_BACKWARD → const int
OP_BATCH_NORM_2D_FORWARD → const int
OP_BCE_LOSS_BACKWARD → const int
OP_BCE_LOSS_FORWARD → const int
OP_BROADCAST_ADD → const int
OP_CAUSAL_MASK_BACKWARD → const int
OP_CAUSAL_MASK_FORWARD → const int
OP_CLAMP_BACKWARD → const int
OP_CLAMP_ELEMENTWISE → const int
OP_CLIP_GRAD_VALUE → const int
OP_COL2IM → const int
OP_CONCATENATE → const int
OP_CONCATENATE_BACKWARD → const int
OP_CONV2D_BACKWARD_INPUT → const int
OP_CONV2D_BACKWARD_KERNEL → const int
OP_CONV2D_FORWARD → const int
OP_CONV2D_MULTI_BACKWARD_INPUT → const int
OP_CONV2D_MULTI_BACKWARD_WEIGHT → const int
OP_CONV2D_MULTI_FORWARD → const int
OP_COPY → const int
OP_CROSS_ENTROPY_BACKWARD → const int
OP_CROSS_ENTROPY_FORWARD → const int
OP_DIVIDE → const int
OP_DIVIDE_BACKWARD → const int
OP_DROPOUT_BACKWARD → const int
OP_DROPOUT_FORWARD → const int
OP_EMBEDDING_BACKWARD → const int
OP_EMBEDDING_FORWARD → const int
OP_EXP_BACKWARD → const int
OP_EXP_ELEMENTWISE → const int
OP_FILL → const int
OP_GELU_BACKWARD → const int
OP_GELU_FORWARD → const int
OP_GLOBAL_AVG_POOL_BACKWARD → const int
OP_GLOBAL_AVG_POOL_FORWARD → const int
OP_IM2COL → const int
OP_LAYER_NORM_BACKWARD → const int
OP_LAYER_NORM_FORWARD → const int
OP_LOAD_SAMPLE → const int
OP_LOG_BACKWARD → const int
OP_LOG_ELEMENTWISE → const int
OP_MARKOV_COUNT → const int
OP_MARKOV_NORMALIZE → const int
OP_MARKOV_PREDICT → const int
OP_MATMUL → const int
OP_MATMUL_BIAS_RELU_FORWARD → const int
OP_MAX_POOL_1D_BACKWARD → const int
OP_MAX_POOL_1D_FORWARD → const int
OP_MAX_POOL_2D_BACKWARD → const int
OP_MAX_POOL_2D_FORWARD → const int
OP_MSE_LOSS_BACKWARD → const int
OP_MSE_LOSS_FORWARD → const int
OP_MULTIPLY → const int
OP_MULTIPLY_BACKWARD → const int
OP_PAD2D → const int
OP_PAD2D_BACKWARD → const int
OP_POW_BACKWARD → const int
OP_POW_ELEMENTWISE → const int
OP_RELU → const int
OP_RELU_BACKWARD → const int
OP_RMS_NORM_BACKWARD → const int
OP_RMS_NORM_FORWARD → const int
OP_ROPE_BACKWARD → const int
OP_ROPE_FORWARD → const int
OP_SCALE_MATRIX → const int
OP_SCALE_MATRIX_BACKWARD → const int
OP_SGD_UPDATE → const int
OP_SIGMOID → const int
OP_SIGMOID_BACKWARD → const int
OP_SLICE_COLUMN → const int
OP_SLICE_COLUMN_BACKWARD → const int
OP_SLICE_ROW → const int
OP_SLICE_ROW_BACKWARD → const int
OP_SOFTMAX_BACKWARD → const int
OP_SOFTMAX_FORWARD → const int
OP_SQRT_BACKWARD → const int
OP_SQRT_ELEMENTWISE → const int
OP_STACK_ROWS → const int
OP_STACK_ROWS_BACKWARD → const int
OP_STORE_SAMPLE → const int
OP_SUBTRACT → const int
OP_SUBTRACT_INTO → const int
OP_SUM_REDUCE → const int
OP_SUM_REDUCE_BACKWARD → const int
OP_SUM_REDUCE_COLUMNS → const int
OP_SUM_REDUCE_ROWS → const int
OP_TANH → const int
OP_TANH_BACKWARD → const int
OP_TRANSPOSE → const int
OP_ZERO_GRAD → const int

Typedefs

DartAddPointers = void Function(Pointer<Float>, Pointer<Float>, int)
DartCreate = int Function(bool)
DartFree = void Function(int)
DartFreeStr = void Function(Pointer<Utf8>)
DartFreeT = void Function(int, Pointer<Utf8>)
DartGetNames = Pointer<Utf8> Function(int)
DartInitRandom = void Function(int, Pointer<Utf8>, double, int)
DartLoad = void Function(int, Pointer<Utf8>, Pointer<Float>, int, Pointer<Int32>)
DartRetrieve = void Function(int, Pointer<Utf8>, Pointer<Float>)
DartRun = void Function(int, Pointer<Uint8>, int)
NativeAddPointers = Void Function(Pointer<Float>, Pointer<Float>, Int32)
FFI- Advanced operation to allow direct pointer access. Adds the content of a source content into a destination pointer with a given length
NativeCreate = Int64 Function(Bool)
FFI- Binding for creating a tensor on the GPU
NativeFree = Void Function(Int64)
FFI- Binding for freeing/deleting the GPU backend
NativeFreeStr = Void Function(Pointer<Utf8>)
FFI- Binding for freeing the made list of tensor names
NativeFreeT = Void Function(Int64, Pointer<Utf8>)
FFI- Binding for freeing a tensor on the GPU
NativeGetNames = Pointer<Utf8> Function(Int64)
FFI- Binding for retrieving the ids of all tensors currently on the GPU. This creates a List of Strings that is stored in the backend and has to be deleted again
NativeInitRandom = Void Function(Int64, Pointer<Utf8>, Float, Int32)
FFI- Initialise a tensor with a random initialization
NativeLoad = Void Function(Int64, Pointer<Utf8>, Pointer<Float>, Int32, Pointer<Int32>)
FFI- Binding for pushing a tensor to the GPU
NativeRetrieve = Void Function(Int64, Pointer<Utf8>, Pointer<Float>)
FFI- Binding for pulling a tensor from the GPU
NativeRun = Void Function(Int64, Pointer<Uint8>, Int32)
FFI- Binding for running a given byteList of commands (usually via a CommandBuffer)