full_library library

A comprehensive machine learning library for Flutter.

Classes

ActivationFunction<T>

The abstract base class (or interface) for all activation functions.

Adagrad

Implements the Adagrad optimization algorithm. Adapts the learning rate individually for each parameter based on the accumulated history of squared gradients.

Adam

Implements the Adam (Adaptive Moment Estimation) optimization algorithm. Computes individual adaptive learning rates from exponentially decaying averages of past gradients (first moment) and squared gradients (second moment), incorporating bias correction terms.

AdamGPU

Applies the Adam optimizer to every element of its parameter tensors. Adam combines the advantages of AdaGrad and RMSProp by computing adaptive learning rates for each parameter using the first and second moments of the gradients.

AdamW

Implements the AdamW optimization algorithm with decoupled weight decay. Modifies the standard Adam update by applying weight decay directly to the parameters rather than incorporating it into the gradient moments.

AMSGrad

Implements the AMSGrad optimization algorithm, a variant of Adam. Maintains the maximum of past squared gradient averages in _vHat to prevent learning rate scaling from exploding, ensuring non-increasing effective step sizes for improved convergence.

AveragePooling2DLayer

Applies a 2D average pooling operation over an input Matrix tensor. Downsamples the input representation by taking the average value over pooling windows defined by poolSize and stride.

AveragePooling2DTL

Slides a two-dimensional window over a given matrix. Each window returns the average over all its values.

BatchNorm1D

Applies Batch Normalization over a 1D Vector tensor. Normalizes the input features across the batch to have zero mean and unit variance, followed by a learnable affine transformation using gamma and beta. Tracks running statistics runningMean and runningVariance for evaluation during inference.

BatchNorm1DTL

Applies Batch Normalization over a 1D input vector to enforce a mean of 0 and a variance of 1.

BatchNorm2D

Applies Batch Normalization over a 3D Tensor3D tensor across channels. Normalizes the channel dimensions across spatial elements to have zero mean and unit variance, followed by a learnable affine transformation using gamma and beta. Tracks running statistics runningMean and runningVariance for evaluation during inference.

BatchNorm2DTL

Applies Batch Normalization over a 3D input Tensor3D to enforce a mean of 0 and a variance of 1. It standardizes the spatial dimensions for each sub-element in the input Tensor3D (which are matrices) independently. Usually, the different matrices building the Tensor3D can be viewed as different channels.

CommandBuffer

CommandBuffer in this library is used as a tape to record actions for the native backend to execute. Commands are stored as a continuous stream of raw bytes. Each command constists of two parts:

Conv2DLayer

Applies a 2D convolution over an input Matrix tensor. Convolves the input with a set of learnable 2D filters (kernels) to produce a 3D feature map (Tensor3D), adds channel-wise biases, and optionally applies an activation function.

Conv2DTL

Applies a 2D convolution over a multi-channel input using learnable spatial filters. Works for Matrix and Tensor3D inputs. On Matrix input it is assumed to contain 1 channel.

ConvLSTMLayer

Applies a Convolutional Long Short-Term Memory (ConvLSTM) recurrent step over a sequence of 2D matrices formatted as a Tensor3D. Replaces standard matrix multiplications in an LSTM with 2D convolutions (conv2d) using 'same' padding to preserve spatial dimensions across recurrent state transitions. Computes forget, input, cell, and output gates sequentially over the sequence length and returns the final hidden state Matrix.

ConvLSTMTL

Advanced Layer to process sequences of 2D spatial data. In this case it is represented as Tensor3Ds with format 'Time, Height, Width'. It uses convolutional gates instead of standard matrix multiplications. Therefore this architecture is able to capture both time and spatial information.

DenseLayer

Represents a standard fully connected layer operating on a 1D Vector tensor. Performs a matrix-vector multiplication with weights, adds a biases vector, and optionally applies an activation function to the resulting vector.

DenseLayerMatrix

Represents a standard fully connected layer operating on a 2D Matrix tensor for batched inputs. Performs matrix multiplication of input with weights, broadcasts the biases vector across rows, and optionally applies an activation function to the output matrix.

DenseReluTL

Represents a standard fully connected layer followed by a ReLU activation function. Utilizes a fused GPU kernel to perform matrix multiplication, bias broadcasting, and ReLU activation in a single optimized pass.

DenseTL

Represents a standard fully connected layer of a neural network. It performs a matrix multiplication and adds a bias vector to the result.

DropoutLayer

Applies Dropout regularization over an input Vector tensor. During training, randomly zeroes out elements of the input tensor with probability rate and scales the remaining elements by 1.0 / (1.0 - rate) to maintain expected activation magnitude. During evaluation (isTraining = false), the input tensor is passed through unchanged.

DropoutLayerMatrix

Applies Dropout regularization over a 2D Matrix tensor for batched or spatial representations. During training, randomly zeroes out elements of the input matrix with probability rate and scales the remaining elements by 1.0 / (1.0 - rate) to maintain expected activation magnitude. During evaluation (isTraining = false), the input tensor is passed through unchanged.

DropoutTL

A dropout layer that randomly mutes some elements of the provided GPUTensor. Works on Scalar, Vector, Matrix and Vector3D. It scales the remaining active elements by 1 / (1 - rate).

DualLSTMLayer

A Multi-Timeline Long Short-Term Memory (MT-LSTM) layer operating over a 2D Matrix sequence. Maintains two separate tiers of LSTM cells running at different temporal granularities:

DualLSTMTL

Advanced Layer to process sequences of data. It is designed to capture short and long term dependencies of data entries. It is given a Matrix in the shape of (Time, Features) and returns a prediction for the next set of features. A Low tier resolution LSTM updates every step to capture short-term dependencies while a long-term high tier resolution layer provides trend prediction.

ELUMatrix

Applies the Exponential Linear Unit (ELU) activation function over a given input of type Tensor<Matrix>. In contrast to the standard ReLU function, this function has a small negative value for negative inputs, which can help prevent the dying ReLU problem and speed up learning by pushing the mean activation closer to zero.

ELUVector

Applies the Exponential Linear Unit (ELU) activation function over a given input of type Tensor<Vector>. In contrast to the standard ReLU function, this function has a small negative value for negative inputs, which can help prevent the dying ReLU problem and speed up learning by pushing the mean activation closer to zero.

EmbeddingLayer

Maps a 1D Vector tensor of integer indices (token IDs) to continuous embedding vectors, outputting a 2D Matrix tensor. Operates as an embedding lookup table of shape [vocabularySize, embeddingDimension] on the CPU with dedicated autograd backpropagation.

EmbeddingLayerMatrix

Maps a batched 2D Matrix tensor of integer indices [batchSize, sequenceLength] to dense continuous embeddings, outputting a 3D Tensor3D tensor. Operates as a batched embedding lookup table of shape [vocabularySize, embeddingDimension] on the CPU with dedicated autograd backpropagation.

EmbeddingMatrixTL

Embedding layer to act as a discrete lookup table. Processes a batch of sequences of indices (Matrix) finding its embedding and returning a batch of sequences of embeddings as a Tensor3D.

EmbeddingTL

Embedding layer to act as a discrete lookup table. Processes a single sequence of indices (Vector) finding its embedding and returning a sequence of embeddings as a matrix.

FlattenLayer

Flattens a 2D Matrix tensor into a contiguous 1D Vector tensor. Preserves the underlying flat data elements while resetting dimensionality and attaching an autograd Node to propagate gradients backward.

FlattenTL

Flattens a Tensor3D into a Matrix.

GeluLayerMatrixTL

Applies the gelu function over a Matrix

GeluLayerTL

Applies the gelu function over a Vector.

GlobalAveragePooling1D

Applies a 1D global average pooling operation over an input Matrix sequence of shape [sequenceLength, numFeatures]. Computes the mean across all sequence timesteps for each feature dimension, reducing the 2D Matrix to a 1D Vector of size [numFeatures]. Registers an autograd Node to evenly distribute incoming gradients back across all sequence steps.

GlobalAveragePooling1DTL

Collapses a matrix input to a vector containing the average values of its columns.

GlobalAveragePoolingLayer

Applies a global average pooling operation over an input Matrix tensor. Computes the mean along the spatial dimensions to reduce a 2D Matrix to a 1D Vector.

GlobalAveragePoolingTL

Collapses a matrix input into a vector of the average values of its feature columns. Does not store intermediate values.

GPUEngine

The Cuda Engine in responsible for communication with the native runtimes as well as managing the communication between dart and its native FFI-bindings. In order to use the GPU acceleration, at the very beginning of the program it is required to call initialize with the Target provided to load the relevant binary. This class is to be used globally as a sole instance of communication and data transfer. Its handle acts as the id of the native executor instance. If there is currently no instance running this value will be 0. Currently supported are WebGPU with android_arm64, android_x86_64 and cudafrom version 1.12.1 (2026-09). Use dispose to delete the currently running instance of GPUEngine and free all memory again.

GPUNode

GPUTensor<T>

A GPUTensor in this library is used as the fundamental unit to hold structured collections of float values on the GPU. For CPU computation please refer to Tensor. These values are stored on the GPU (usually inside the VRAM) with a unique identifier id. Globally a counter called idCounter ensures uniqueness of Tensors up to the integer limit of instances.

Layer<I, O>

Layer is built to be an additional abstraction that allows for complex mathematical bundling on the CPU. Operations like Dense Neural Networks are supported as a single Layer and can be used to build models reliably. Each Layer has the following individually assigned values:

LayerNormalization

Applies Layer Normalization over a 2D Matrix tensor across features. Normalizes the activations of each sample (row) across the feature dimension (columns) to have zero mean and unit variance, followed by a learnable affine transformation using gamma and beta. Constructs an autograd Node to backpropagate gradients to inputs and parameters.

LayerNormalizationTL

Applies Layer Normalization over a batch of inputs. Standardizes the features for each sample to have a mean of 0 and a variance of 1.

LeakyReLUMatrix

Applies leaky ReLu over a given input of type Tensor<Matrix>. In contrast to the standard ReLU function, this function multiplies negative values with a small negative number alpha to prevent the dying gradient problem.

LeakyReLUVector

Applies leaky ReLu over a given input of type Tensor<Vector>. In contrast to the standard ReLU function, this function multiplies negative values with a small negative number alpha to prevent the dying gradient problem.

Logger

LSTMLayer

Applies a Long Short-Term Memory (LSTM) recurrent neural network pass over a sequential 2D Matrix tensor. Processes input sequences of shape [sequenceLength, featureSize] timestep-by-timestep using standard gating mechanisms (forget gate, input gate, candidate cell state, and output gate) to maintain long-term dependencies. Returns the final hidden state Vector after processing all sequence steps.

LSTMTL

Standard Long Short-Term Memory (LSTM) Layer. Processes a sequence of data to capture long-term dependencies.

MaxPooling1DLayer

Applies a 1D max pooling operation over an input Vector tensor. Downsamples the 1D representation by selecting the maximum value within sliding windows defined by poolSize and stride. Tracks input indices of maximum values to route backpropagating gradients directly to those elements via an autograd Node.

MaxPooling1DTL

Slides a one-dimensional window over a given vector. Each window returns the maximum of all its values.

MaxPooling2DLayer

Applies a 2D max pooling operation over an input Matrix tensor. Downsamples the spatial representation by extracting the maximum value within pooling windows defined by poolSize and stride. Tracks the flat 1D input indices of maximum values to route backpropagating gradients directly to those elements via an autograd Node.

MaxPooling2DTL

Slides a two-dimensional window over a given matrix. Each window returns the maximum of all its values.

MishMatrix

Applies the mish activation function (similar to swish) to each element inside of a given input of type Tensor<Matrix>. Mish mirrors Relu with a continuous curve. The calculation is handled via the mishMatrix math operation.

MishVector

Applies the mish activation function (similar to swish) to each element inside of a given input of type Tensor<Vector>. Mish mirrors Relu with a continuous curve. The calculation is handled via the mishVector math operation.

Momentum

Implements the Stochastic Gradient Descent with Momentum optimization algorithm. Accelerates gradient vectors in the relevant direction by accumulating past velocity updates, dampening oscillations across optimization steps.

MultiHeadAttention

Applies Multi-Head Attention over an input Matrix tensor. Projects the input sequence into multiple subspaces by running numHeads parallel SingleHeadAttention modules, concatenates the head outputs along the column dimension, and linearly projects the result back to dModel using wo.

MultiHeadAttentionTL

Applies Multi-Head Attention to a sequence. Projects the input into Query, Key, and Value matrices, splits them into multiple heads, calculates scaled dot-product attention, and projects the concatenated results back to the original dimension.

MultiTierLSTMLayer

Applies a Multi-Tier Hierarchical Long Short-Term Memory recurrent pass over a 2D Matrix sequence. Extends multi-timeline recurrence to an arbitrary number of stacked tiers defined by tierClockCycles.

NAG

Implements Nesterov Accelerated Gradient (NAG) optimization. Incorporates momentum by computing the parameter update step using a look-ahead velocity term.

Node

Optimizer

The abstract base class for all optimization algorithms. Tracks a collection of trainable parameters and manages optimization steps and gradient resets.

OptimizerGPU

Optimizers are used to update a parameters value to reduce the error of the system. For that the gradient of the requested value is used. Optimizers CAN be static so their execution tapes can be reused of they may require recompilation on every training step.

PositionalEncoding

Injects sinusoidal positional encodings into an input sequence Matrix tensor. Generates fixed sine and cosine frequency encodings up to maxLength across dModel feature dimensions to encode token positions, adding them directly to the input tensor.

PositionalEncodingTL

Injects spatial/sequential context into tokens. Because self-attention is permutation-invariant, this layer adds fixed sine and cosine waves of different frequencies to the embeddings, allowing the model to perceive the absolute and relative positions of words in a sequence.

ReLU

An activation function that applies the Rectified Linear Unit (ReLU) to a Vector.

ReLULayer

Applies the Rectified Linear Unit (ReLU) activation function element-wise over an input Vector tensor. Computes max(0, x) on each element of the input vector.

ReLULayerMatrix

Applies the Rectified Linear Unit (ReLU) activation function element-wise over a 2D Matrix tensor. Computes max(0, x) on each element across all rows and columns of the matrix.

ReLULayerMatrixTapeLayer

Applies the ReLu-function to each element in a matrix.

ReLULayerTL

Applies the ReLu-function to each element in a vector.

ReLUMatrix

An activation function that applies the Rectified Linear Unit (ReLU) to a Matrix.

RMSprop

Implements the RMSprop optimization algorithm. Maintains a moving average of squared gradients to normalize gradient magnitudes, dividing the learning rate by the square root of recent gradient variances.

RNN

Applies a standard Elman Recurrent Neural Network (RNN) pass over a sequential 2D Matrix tensor. Processes input sequences of shape [sequenceLength, featureSize] sequentially across timesteps, updating the recurrent hidden state vector h via input-to-hidden projections (wXh), hidden-to-hidden transitions (wHh), and a bias vector (bh), followed by an activation function. Returns the final hidden state Vector after processing all sequence steps.

RNNTL

A standard Recurrent Neural Network (RNN) Layer. Processes a sequence of data sequentially, maintaining a hidden state that is updated at each time step based on the current input and the previous hidden state.

SeqModel<InputType, OutputType>

Sequential aid for constructing models to run. The Sequential Model wraps a list of TapeLayers to manage static graph compilation via command tapes. It handles VRAM lifecycles, training loops, and high-speed inference.

SGD

Implements the Stochastic Gradient Descent (SGD) optimization algorithm. Updates parameters in the opposite direction of the gradient scaled directly by learningRate.

SGDGPU

Applies the SGD optimizer to every element of its parameter tensors. SGD is a very simply descent method and my get stuck in local minima. This function does not require to be reallocated on each training step and can be reused indefinitely.

Sigmoid

The Sigmoid activation function.

SigmoidMatrix

An activation function that applies the Sigmoid function to a Matrix.

SigmoidMatrixTL

Applies the Sigmoid-function to each element in a matrix.

SingleHeadAttention

Applies single-head scaled dot-product attention over an input sequence Matrix tensor. Projects input token embeddings into Query (wq), Key (wk), and Value (wv) representations, computes attention scores scaled by 1.0 / sqrt(dK), applies softmax normalization to extract attention weights, and computes the context matrix by weighting wv.

SingleHeadAttentionTL

Applies Scaled Dot-Product Attention over a sequence. Projects the input into Query, Key, and Value matrices to determine contextual relationships between tokens before re-combining them based on the calculated attention scores.

SNetwork

A sequential neural network architecture that linearly stacks and executes Layer instances. Manages sequential forward propagation, parameter aggregation across constituent layers, training loops via fit, weight serialization to JSON via save and load, and autograd graph visualization.

SoftmaxMatrix

Applies the softmax function over a given input of type Tensor<Matrix>. Creates a probability distribution over all values of each row summing up to 1. The calculation is handled via the softmaxMatrix math operation.

SoftmaxVector

Applies the softmax function over a given input of type Tensor<Vector>. Creates a probability distribution over all values summing up to 1. The calculation is handled via the softmaxVector math operation.

SwishMatrix

Applies the Sigmoid-weighted Linear Unit (SiLU / Swish) activation function to each element inside of a given input of type Tensor<Matrix>. Defined mathematically as f(x) = x * sigmoid(x), this modern activation function often outperforms standard ReLU in deeper hidden layers.

SwishVector

Applies the Sigmoid-weighted Linear Unit (SiLU / Swish) activation function to each element inside of a given input of type Tensor<Vector>. Defined mathematically as f(x) = x * sigmoid(x), this modern activation function often outperforms standard ReLU in deeper hidden layers.

TapeDecoder

The TapeDecoder class offers a quality of life tool to allow decoding and visual printing of the contents of a CommandBuffer. To decode a CommandBuffer the following command can be used:

TapeLayer<InputType, OutputType>

TapeLayer is built to be an additional abstraction that allows for complex mathematical bundling. Operations like Dense Neural Networks are supported as a single TapeLayer and can be used to build models reliably. Each TapeLayer has the following individually assigned values:

Tensor<T>

A Tensor in this library is used as the fundamental unit to hold a structured collection of float values on the CPU. These values are stored for performance enhancement inside of Float32List. Globally a counter called idCounter ensures uniqueness of Tensors up to the integer limit of instances.

TransformerEncoderBlock

Applies a standard Transformer Encoder Block over an input sequence Matrix tensor. Combines a multi-head self-attention module (mha) with a position-wise feed-forward network (ffn). Each sub-layer incorporates residual connections (addMatrix) followed by layer normalization (LayerNormalization).

TransformerEncoderBlockTapeLayer

A full Transformer Encoder Block. Combines Multi-Head Self-Attention, Residual Connections, Layer Normalization, and a position-wise Feed-Forward Network (FFN) using GELU activation.

Enums

Target

Targeted device architecture

Constants

OP_ABS_BACKWARD → const int

OP_ABS_ELEMENTWISE → const int

OP_ADAM_UPDATE → const int

OP_ADD → const int

OP_ADD_INTO → const int

OP_ADD_SCALAR → const int

OP_ARGMAX_FORWARD → const int

OP_AVG_POOL_2D_BACKWARD → const int

OP_AVG_POOL_2D_FORWARD → const int

OP_BATCH_NORM_1D_BACKWARD → const int

OP_BATCH_NORM_1D_FORWARD → const int

OP_BATCH_NORM_2D_BACKWARD → const int

OP_BATCH_NORM_2D_FORWARD → const int

OP_BCE_LOSS_BACKWARD → const int

OP_BCE_LOSS_FORWARD → const int

OP_BROADCAST_ADD → const int

OP_CAUSAL_MASK_BACKWARD → const int

OP_CAUSAL_MASK_FORWARD → const int

OP_CLAMP_BACKWARD → const int

OP_CLAMP_ELEMENTWISE → const int

OP_CLIP_GRAD_VALUE → const int

OP_COL2IM → const int

OP_CONCATENATE → const int

OP_CONCATENATE_BACKWARD → const int

OP_CONV2D_BACKWARD_INPUT → const int

OP_CONV2D_BACKWARD_KERNEL → const int

OP_CONV2D_FORWARD → const int

OP_CONV2D_MULTI_BACKWARD_INPUT → const int

OP_CONV2D_MULTI_BACKWARD_WEIGHT → const int

OP_CONV2D_MULTI_FORWARD → const int

OP_COPY → const int

OP_CROSS_ENTROPY_BACKWARD → const int

OP_CROSS_ENTROPY_FORWARD → const int

OP_DIVIDE → const int

OP_DIVIDE_BACKWARD → const int

OP_DROPOUT_BACKWARD → const int

OP_DROPOUT_FORWARD → const int

OP_EMBEDDING_BACKWARD → const int

OP_EMBEDDING_FORWARD → const int

OP_EXP_BACKWARD → const int

OP_EXP_ELEMENTWISE → const int

OP_FILL → const int

OP_GELU_BACKWARD → const int

OP_GELU_FORWARD → const int

OP_GLOBAL_AVG_POOL_BACKWARD → const int

OP_GLOBAL_AVG_POOL_FORWARD → const int

OP_IM2COL → const int

OP_LAYER_NORM_BACKWARD → const int

OP_LAYER_NORM_FORWARD → const int

OP_LOAD_SAMPLE → const int

OP_LOG_BACKWARD → const int

OP_LOG_ELEMENTWISE → const int

OP_MARKOV_COUNT → const int

OP_MARKOV_NORMALIZE → const int

OP_MARKOV_PREDICT → const int

OP_MATMUL → const int

OP_MATMUL_BIAS_RELU_FORWARD → const int

OP_MAX_POOL_1D_BACKWARD → const int

OP_MAX_POOL_1D_FORWARD → const int

OP_MAX_POOL_2D_BACKWARD → const int

OP_MAX_POOL_2D_FORWARD → const int

OP_MSE_LOSS_BACKWARD → const int

OP_MSE_LOSS_FORWARD → const int

OP_MULTIPLY → const int

OP_MULTIPLY_BACKWARD → const int

OP_PAD2D → const int

OP_PAD2D_BACKWARD → const int

OP_POW_BACKWARD → const int

OP_POW_ELEMENTWISE → const int

OP_RELU → const int

OP_RELU_BACKWARD → const int

OP_RMS_NORM_BACKWARD → const int

OP_RMS_NORM_FORWARD → const int

OP_ROPE_BACKWARD → const int

OP_ROPE_FORWARD → const int

OP_SCALE_MATRIX → const int

OP_SCALE_MATRIX_BACKWARD → const int

OP_SGD_UPDATE → const int

OP_SIGMOID → const int

OP_SIGMOID_BACKWARD → const int

OP_SLICE_COLUMN → const int

OP_SLICE_COLUMN_BACKWARD → const int

OP_SLICE_ROW → const int

OP_SLICE_ROW_BACKWARD → const int

OP_SOFTMAX_BACKWARD → const int

OP_SOFTMAX_FORWARD → const int

OP_SQRT_BACKWARD → const int

OP_SQRT_ELEMENTWISE → const int

OP_STACK_ROWS → const int

OP_STACK_ROWS_BACKWARD → const int

OP_STORE_SAMPLE → const int

OP_SUBTRACT → const int

OP_SUBTRACT_INTO → const int

OP_SUM_REDUCE → const int

OP_SUM_REDUCE_BACKWARD → const int

OP_SUM_REDUCE_COLUMNS → const int

OP_SUM_REDUCE_ROWS → const int

OP_TANH → const int

OP_TANH_BACKWARD → const int

OP_TRANSPOSE → const int

OP_ZERO_GRAD → const int

Functions

absGPU<T>(GPUTensor<T> a, CommandBuffer tape, {GPUTensor<T>? outTensor}) → GPUTensor<T>

Appends the commands for applying the abs(v) exponential function on every element of a to tape.

adamUpdateGPU(GPUTensor data, GPUTensor m, GPUTensor v, double lr, double beta1, double beta2, double eps, int step, double weightDecay, CommandBuffer tape) → void

Performs an in place update of the provided data values via the adam function and its assigned gradient. The first moment m and second moment v as well as beta1, beta2, eps,weightDecay, and step need to be provided. The operation is appended to tape for execution.

add(Tensor<Scalar> a, Tensor<Scalar> b) → Tensor<Scalar>

Simple addition of tensor a and b.

add3D(Tensor<Tensor3D> a, Tensor<Tensor3D> b) → Tensor<Tensor3D>

Adds the elements of a to the elements of b. a and b are required to have the same size.

add3DGPU(GPUTensor<Tensor3D> a, GPUTensor<Tensor3D> b, CommandBuffer tape, {GPUTensor<Tensor3D>? outTensor}) → GPUTensor<Tensor3D>

Adds the command for addition of two Tensor3Ds a and b to the tape. Creates and allocates a Tensor to store the results.

addBiasToFeatureMapGPU(GPUTensor<Matrix> m, GPUTensor<Matrix> b, CommandBuffer tape, {GPUTensor<Matrix>? outTensor}) → GPUTensor<Matrix>

Appends commands for broadcast additions of a matrix wrapped scalar b to a full matrix m.

addBiasToMatMulOutGPU(GPUTensor<Matrix> m, GPUTensor<Vector> b, CommandBuffer tape, {GPUTensor<Matrix>? outTensor}) → GPUTensor<Matrix>

Appends commands for addition of a 1D bias vector b to a 2D matrix m via broadcasting over the matrix to the tape.

addGPU<T>(GPUTensor<T> a, GPUTensor<T> b, CommandBuffer tape, {GPUTensor<T>? outTensor}) → GPUTensor<T>

Adds the command for addition of two GPUTensors a and b of type T to the tape. Creates and allocates a Tensor to store the results.

addMatrix(Tensor<Matrix> a, Tensor<Matrix> b) → Tensor<Matrix>

Adds a matrix a to a matrix b.

addMatrixAndVector(Tensor<Matrix> m, Tensor<Vector> v) → Tensor<Matrix>

Adds a vector v to every row of matrix m.

addMatrixAndVectorGPU(GPUTensor<Matrix> m, GPUTensor<Vector> v, CommandBuffer tape, {GPUTensor<Matrix>? outTensor}) → GPUTensor<Matrix>

Appends a command for broadcast add a vector v to tape.

addMatrixGPU(GPUTensor<Matrix> a, GPUTensor<Matrix> b, CommandBuffer tape, {GPUTensor<Matrix>? outTensor}) → GPUTensor<Matrix>

Adds the command for addition of two matrices a and b to the tape. Creates and allocates a Tensor to store the results.

addScalar(Tensor<Vector> v, double s) → Tensor<Vector>

Adds a constant value to each element of a vector.

addScalar3DGPU(GPUTensor<Tensor3D> t, double scalar, CommandBuffer tape, {GPUTensor<Tensor3D>? outTensor}) → GPUTensor<Tensor3D>

Appends a command for adding a scalar scalar to every element of a Tensor3D v to tape.

addScalarMatrixGPU(GPUTensor<Matrix> m, GPUTensor<Scalar> b, CommandBuffer tape, {GPUTensor<Matrix>? outTensor}) → GPUTensor<Matrix>

Appends a command for adding a scalar b to every element of a matrix m to tape.

addScalarToMatrix(Tensor<Matrix> m, Tensor<Scalar> s) → Tensor<Matrix>

Adds a scalar s to every element of a matrix m.

addScalarVectorGPU(GPUTensor<Vector> v, double scalar, CommandBuffer tape, {GPUTensor<Vector>? outTensor}) → GPUTensor<Vector>

Appends a command for adding a scalar b to every element of a vector v to tape.

addVector(Tensor<Vector> a, Tensor<Vector> b) → Tensor<Vector>

Adds two vectors together element.

addVectorGPU(GPUTensor<Vector> a, GPUTensor<Vector> b, CommandBuffer tape, {GPUTensor<Vector>? outTensor}) → GPUTensor<Vector>

Adds the command for addition of two vectors a and b to the tape. Creates and allocates a Tensor to store the results.

applyRopeGPU(GPUTensor<Tensor3D> input, GPUTensor<Matrix> cosTable, GPUTensor<Matrix> sinTable, int batchSize, int seqLen, int numHeads, int headDim, CommandBuffer tape) → GPUTensor<Tensor3D>

Applies Rotary Position Embeddings (RoPE) to a multi-head attention state tensor. Encodes relative positional information by pairing and rotating feature dimensions using precomputed values supplied in cosTable and sinTable. The backward pass computes the inverse rotation gradients by intrinsically transposing the rotation matrix (negating the sine terms).

argmaxGPU(GPUTensor<Matrix> logits, CommandBuffer tape) → GPUTensor<Vector>

Identifies the predicted class token for each sequence in a batch. Examines the vocabulary dimension of the provided logits and returns a vector containing the index of the maximum value (argmax) for each batch row. This operator is used exclusively during inference and has no backward pass.

avgPool1d(Tensor<Vector> input, int poolSize, int stride) → Tensor<Vector>

Slides a window of size poolSize over the vector with the step-size stride. Returns a vector containing the averages of the observed windows.

avgPool2d(Tensor<Matrix> input, int poolSize, int stride) → Tensor<Matrix>

Slides a 2 dimensional kernel over a Matrix averaging the window together. The sliding window size is defined by poolSizexpoolSize with stride as the step size in both dimensions.

avgPool2dGPU(GPUTensor<Matrix> input, int poolSize, int stride, CommandBuffer tape, {GPUTensor<Matrix>? outTensor}) → GPUTensor<Matrix>

Slides a pool of size poolSizexpoolSize over matrix input. The resulting matrix consists of the average of the elements in each window. The value of stride tells how big the step size of the sliding window is. The operations are appended to tape for execution.

batchNorm1dGPU(GPUTensor<Vector> input, GPUTensor<Vector> gamma, GPUTensor<Vector> beta, GPUTensor<Vector> runningMean, GPUTensor<Vector> runningVariance, double momentum, double epsilon, bool isTraining, CommandBuffer tape, {GPUTensor<Vector>? outTensor, GPUTensor<Vector>? savedMeanTensor, GPUTensor<Vector>? savedInvVarTensor}) → GPUTensor<Vector>

Normalizes a 1D feature vector using per-feature running mean and variance. Scales and shifts the normalized data using learnable gamma and beta parameter vectors. During inference, it computes the output strictly using the pre-calculated running statistics rather than the current batch's statistics. The operations are appended to tape for execution.

batchNorm1dMath(Tensor<Vector> x, Tensor<Vector> gamma, Tensor<Vector> beta, Vector runningMean, Vector runningVariance, int numFeatures, bool isTraining, double momentum, double epsilon) → Tensor<Vector>

Applies Batch Normalization over a 1D vector.

batchNorm2dGPU(GPUTensor<Tensor3D> input, GPUTensor<Vector> gamma, GPUTensor<Vector> beta, GPUTensor<Vector> runningMean, GPUTensor<Vector> runningVariance, double momentum, double epsilon, bool isTraining, CommandBuffer tape, {GPUTensor<Tensor3D>? outTensor, GPUTensor<Vector>? savedMeanTensor, GPUTensor<Vector>? savedInvVarTensor}) → GPUTensor<Tensor3D>

Normalizes a 3D spatial tensor independently for each channel. Uses the mean and variance evaluated across the combined spatial dimensions (height and width) of a given channel. Returns a 3D tensor of identical shape scaled by gamma and shifted by beta. The operations are appended to tape for execution.

batchNorm2dMath(Tensor<Tensor3D> x, Tensor<Vector> gamma, Tensor<Vector> beta, Vector runningMean, Vector runningVariance, int numChannels, bool isTraining, double momentum, double epsilon) → Tensor<Tensor3D>

Applies Batch Normalization over a 3D tensor (channels, height, width).

binaryCrossEntropy(Tensor<Scalar> prediction, Tensor<Scalar> target) → Tensor<Scalar>

Calculates the binary cross entropy by comparing the prediction of the network to the target required.

binaryCrossEntropyGPU<T>(GPUTensor<T> prediction, GPUTensor<T> target, CommandBuffer tape, {GPUTensor<Scalar>? outTensor}) → GPUTensor<Scalar>

///////////////////////////////// Loss Functions (400-499) /// ///////////////////////////////// Appends commands for calculating the binary cross entropy over the elements of m of type T to tape.

broadcastAddVectorToMatrixGPU(GPUTensor<Matrix> m, GPUTensor<Vector> v, CommandBuffer tape, {GPUTensor<Matrix>? outTensor}) → GPUTensor<Matrix>

Appends commands for addition of a 1D bias vector b to a 2D matrix m via broadcasting over the matrix to the tape. This command is inference only! No gradient path is calculated.

buildMarkovTableGPU(GPUTensor<Vector> sequence, int order, int numStates, CommandBuffer tape, {GPUTensor<Matrix>? outTensor, GPUTensor<Matrix>? probTableTensor}) → GPUTensor<Matrix>

Constructs a Markov chain probability transition table from a given state sequence. Calculates the frequencies of state transitions based on the specified Markov order and numStates, and normalizes these counts to output a probability matrix of shape [numStates^order, numStates]. This Table does not have a gradient calculation. The operations are appended to tape for execution.

causalMaskGPU(GPUTensor<Tensor3D> input, int batchSize, int numHeads, int seqLen, CommandBuffer tape) → GPUTensor<Tensor3D>

Applies a causal attention mask to prevent sequences from attending to future tokens. Modifies the [seqLen, seqLen] attention matrices within the 3D tensor by overriding elements above the main diagonal with -10000.0, effectively zeroing out their probability post-softmax. Gradients are blocked from flowing back through the masked positions.

clampGPU<T>(GPUTensor<T> a, double minVal, double maxVal, CommandBuffer tape, {GPUTensor<T>? outTensor}) → GPUTensor<T>

Appends the commands for clamping every element of v between minVal - maxVal to tape.

clipGradValueGPU(GPUTensor tensor, double clipValue, CommandBuffer tape) → void

Performs an in place update of the provided data values by clipping them in between the intervall (-clipValue)-(clipValue). The operation is appended to tape for execution.

concatenate(Tensor<Vector> a, Tensor<Vector> b) → Tensor<Vector>

Appends two vectors after each oder with a and b becoming ab.

concatenate3D(Tensor<Tensor3D> a, Tensor<Tensor3D> b) → Tensor<Tensor3D>

Appends two Tensor3D objects together on their 0th axis.

concatenate3DGPU(GPUTensor<Tensor3D> a, GPUTensor<Tensor3D> b, CommandBuffer tape, {GPUTensor<Tensor3D>? outTensor}) → GPUTensor<Tensor3D>

Appends two Tensor3Ds a and b together in order a-b. The operation is appended to tape for execution.

concatenateGPU(GPUTensor<Vector> a, GPUTensor<Vector> b, CommandBuffer tape, {GPUTensor<Vector>? outTensor}) → GPUTensor<Vector>

Appends two vectors a and b together in order a-b. The operation is appended to tape for execution.

concatenateMatricesByColumn(List<Tensor<Matrix>> matrices) → Tensor<Matrix>

Concatenates two matrices via their columns

concatenateMatricesByColumnGPU(List<GPUTensor<Matrix>> matrices, CommandBuffer tape, {GPUTensor<Matrix>? outTensor}) → GPUTensor<Matrix>

Appends a list of matrices matrices together in column order. The operation is appended to tape for execution.

conv2d(Tensor<Matrix> input, Tensor<Matrix> kernel, {String padding = 'valid'}) → Tensor<Matrix>

Slides a kernel kernel over the matrix and calculates the elementwise multiplication at each position. Allows for optional zero padding with padding = 'same' where input size equals output size.

conv2dMultiChannelGPU(GPUTensor input, GPUTensor<Tensor3D> weight, GPUTensor<Vector> bias, int kH, int kW, CommandBuffer tape, {String padding = 'valid', int strideH = 1, int strideW = 1, GPUTensor<Tensor3D>? outTensor}) → GPUTensor<Tensor3D>

Performs an advanced 2D convolution on multiple channels with input and weight, alongside a bias vector.

conv2dSimpleGPU(GPUTensor<Matrix> input, GPUTensor<Matrix> kernel, CommandBuffer tape, {GPUTensor<Matrix>? outTensor}) → GPUTensor<Matrix>

Computes a basic, single-channel 2D convolution between an input matrix and a kernel matrix.

cosineSimilarityGPU(GPUTensor<Vector> a, GPUTensor<Vector> b, CommandBuffer tape) → GPUTensor<Scalar>

Computes the cosine similarity between two vectors, measuring their directional alignment. Calculates the dot product of a and b and divides it by the product of their individual L2 norms. Returns a scalar between -1.0 (opposite) and 1.0 (identical direction). The operations are appended to tape for execution.

crossEntropyLossGPU(GPUTensor<Matrix> logits, GPUTensor<Vector> targets, CommandBuffer tape) → GPUTensor<Vector>

Computes the categorical Cross-Entropy loss between predicted logits and target class indices. Leverages a fused, numerically stable softmax internally to calculate the log-probabilities. Outputs an unreduced vector containing one scalar loss value per batch sequence.

divide3DGPU(GPUTensor<Tensor3D> a, GPUTensor<Tensor3D> b, CommandBuffer tape, {GPUTensor<Tensor3D>? outTensor}) → GPUTensor<Tensor3D>

Appends the commands for element wise division of two Tensor3Ds a through b to tape. Creates and allocates a Tensor to store the results.

divideGPU<T>(GPUTensor<T> a, GPUTensor<T> b, CommandBuffer tape, {GPUTensor<T>? outTensor}) → GPUTensor<T>

Appends the commands for element wise division of two GPUTensors a through b of type T to tape. Creates and allocates a Tensor to store the results.

divideMatrixGPU(GPUTensor<Matrix> a, GPUTensor<Matrix> b, CommandBuffer tape, {GPUTensor<Matrix>? outTensor}) → GPUTensor<Matrix>

Appends the commands for element wise division of two matrices a through b to tape. Creates and allocates a Tensor to store the results.

divideVectorGPU(GPUTensor<Vector> a, GPUTensor<Vector> b, CommandBuffer tape, {GPUTensor<Vector>? outTensor}) → GPUTensor<Vector>

dot(Tensor<Vector> a, Tensor<Vector> b) → Tensor<Scalar>

Calculates the eucledian dot product of a and b.

dotProductGPU(GPUTensor<Vector> a, GPUTensor<Vector> b, CommandBuffer tape, {GPUTensor<Vector>? multipliedTensor, GPUTensor<Scalar>? outTensor}) → GPUTensor<Scalar>

Calculates the dot product between vector a and b. The operations are appended to tape for execution. The operations are appended to tape for execution.

dropoutGPU<T>(GPUTensor<T> input, double rate, CommandBuffer tape, {GPUTensor<T>? outTensor, GPUTensor<T>? maskTensor}) → GPUTensor<T>

Randomly zeroes out elements of the input tensor with a probability equal to the specified rate. To preserve the expected mathematical sum of the tensor during inference, it scales the remaining active elements by 1 / (1 - rate). Outputs the modified tensor alongside the generated dropout mask. The operations are appended to tape for execution.

dropoutMatrixMath(Tensor<Matrix> input, double rate, bool isTraining) → Tensor<Matrix>

If isTraining is set to true this function sets values with the probability rate to zero. The remaining values get scaled up tp conserve the total sum of the values.

dropoutVectorMath(Tensor<Vector> input, double rate, bool isTraining) → Tensor<Vector>

If isTraining is set to true this function sets values with the probability rate to zero. The remaining values get scaled up tp conserve the total sum of the values.

elementWiseMultiply(Tensor<Vector> a, Tensor<Vector> b) → Tensor<Vector>

Multiplies each element of a with each element of b.

elementWiseMultiply3D(Tensor<Tensor3D> a, Tensor<Tensor3D> b) → Tensor<Tensor3D>

Multiplies the elements of a with the elements of b. a and b are required to have the same size.

elementWiseMultiply3DGPU(GPUTensor<Tensor3D> a, GPUTensor<Tensor3D> b, CommandBuffer tape, {GPUTensor<Tensor3D>? outTensor}) → GPUTensor<Tensor3D>

Appends the commands for element wise multiplication of two Tensor3Ds a and b to tape. Creates and allocates a Tensor to store the results.

elementWiseMultiplyGPU(GPUTensor<Vector> a, GPUTensor<Vector> b, CommandBuffer tape, {GPUTensor<Vector>? outTensor}) → GPUTensor<Vector>

Appends the commands for element wise multiplication of two vectors a and b to tape. Creates and allocates a Tensor to store the results.

elementWiseMultiplyMatrix(Tensor<Matrix> a, Tensor<Matrix> b) → Tensor<Matrix>

Multiplies the elements of matrix a with the elements of matrix b. This is NOT matrix multiplication.

elementWiseMultiplyMatrixGPU(GPUTensor<Matrix> a, GPUTensor<Matrix> b, CommandBuffer tape, {GPUTensor<Matrix>? outTensor}) → GPUTensor<Matrix>

Appends the commands for element wise multiplication of two matrices a and b to tape. Creates and allocates a Tensor to store the results.

eluMatrix(Tensor<Matrix> m, double alpha) → Tensor<Matrix>

Applies the elu activation function to each element inside of m with multiplying negative values of m with a small negative number alpha to prevent negatives.

eluVector(Tensor<Vector> v, double alpha) → Tensor<Vector>

Applies the elu activation function to each element inside of v with multiplying negative values of v with a small negative number alpha to prevent negatives.

embeddingLookupBatchGPU(GPUTensor<Matrix> batchIndices, GPUTensor<Matrix> weights, CommandBuffer tape, {GPUTensor<Tensor3D>? outTensor}) → GPUTensor<Tensor3D>

Acts as a high performance lookup of rows in a given matrix weights via their row indices indices. In contrast to embeddingLookupGPU this method performs a batch process to speed up lookup. Can be used for fast odering of elements. The operation is appended to tape for execution.

embeddingLookupGPU(GPUTensor<Vector> indices, GPUTensor<Matrix> weights, CommandBuffer tape, {GPUTensor<Matrix>? outTensor}) → GPUTensor<Matrix>

Acts as a high performance lookup of rows in a given matrix weights via their row indices indices. Can be used for fast odering of elements. The operation is appended to tape for execution.

euclideanDistanceGPU(GPUTensor<Vector> a, GPUTensor<Vector> b, CommandBuffer tape) → GPUTensor<Scalar>

Calculates the straight-line Euclidean distance between two vectors. Computes the element-wise difference between vector a and b, squares the differences, sums them, and extracts the square root. The operations are appended to tape for execution.

flatMse(Tensor pred, Tensor target) → Tensor<Scalar>

Computes the Mean Squared Error (MSE) loss between flattened elements of pred and target tensors. Builds an autograd Node attached to the returned Scalar Tensor to propagate gradients back to pred.

flatten3DToMatrixGPU(GPUTensor<Tensor3D> t, CommandBuffer tape, {GPUTensor<Matrix>? outTensor}) → GPUTensor<Matrix>

geluGPU(GPUTensor<Vector> v, CommandBuffer tape, {GPUTensor<Vector>? outTensor}) → GPUTensor<Vector>

Appends commands for applying the gelu activation function on every element of vector v to tape.

geluMatrixGPU(GPUTensor<Matrix> m, CommandBuffer tape, {GPUTensor<Matrix>? outTensor}) → GPUTensor<Matrix>

Appends commands for applying the gelu activation function on every element of matrix m to tape.

globalAveragePooling(Tensor<Matrix> input) → Tensor<Vector>

Pools the average over each column in input.

globalAveragePoolingGPU(GPUTensor<Matrix> input, CommandBuffer tape, {GPUTensor<Vector>? outTensor}) → GPUTensor<Vector>

Collapses a matrix input to a vector of their average values of their feature columns. The operations are appended to tape for execution.

im2colGPU(GPUTensor input, int kH, int kW, CommandBuffer tape, {GPUTensor<Matrix>? outTensor}) → GPUTensor<Matrix>

Extracts sliding local pathces fromm an image Tensor and flattens them into columns of a matrix. Kernel dimensions kH and kW need to be set. The operations are appended to tape for execution.

l2NormGPU(GPUTensor<Vector> v, CommandBuffer tape) → GPUTensor<Scalar>

Computes the L2 norm (Euclidean length) of a vector. Squares each element of the vector, sums them together, and takes the square root of the result. The operations are appended to tape for execution. The operations are appended to tape for execution.

layerNormMatrixGPU(GPUTensor<Matrix> m, GPUTensor<Vector> gamma, GPUTensor<Vector> beta, GPUTensor<Vector> meanCache, GPUTensor<Vector> rstdCache, double epsilon, CommandBuffer tape, {GPUTensor<Matrix>? outTensor}) → GPUTensor<Matrix>

Normalizes a 2D matrix independently for each row. Calculates the mean and variance across the columns of a single row, normalizes the row's elements, and applies the gamma and beta transformations. Outputs a normalized matrix of the original dimensions. The operations are appended to tape for execution.

leakyReluMatrix(Tensor<Matrix> m, double alpha) → Tensor<Matrix>

Applies leaky ReLu over a matrix m with multiplying negative values with a small negative number alpha to prevent the dying gradient problem.

leakyReluVector(Tensor<Vector> v, double alpha) → Tensor<Vector>

Applies leaky ReLu over a matrix v with multiplying negative values with a small negative number alpha to prevent the dying gradient problem.

loadSampleGPU(GPUTensor<Matrix> dataset, int sampleIndex, CommandBuffer tape, {GPUTensor<Vector>? outTensor}) → GPUTensor<Vector>

logGPU<T>(GPUTensor<T> a, CommandBuffer tape, {GPUTensor<T>? outTensor}) → GPUTensor<T>

Appends the commands for applying the log(v) exponential function on every element of v to tape.

maeLossGPU(GPUTensor<Vector> preds, GPUTensor<Vector> targets, CommandBuffer tape) → GPUTensor<Scalar>

Calculates the Mean Absolute Error (MAE) loss between predictions and targets. Computes the absolute differences between the preds and targets vectors, sums the errors, and divides by the total number of elements to yield a single average error scalar. The operations are appended to tape for execution.

main() → Future<void>

Demonstrates training, saving, loading, and evaluating an SNetwork model on the XOR problem.

markovPredictGPU(GPUTensor<Matrix> historyBatch, GPUTensor<Matrix> probTable, int numStates, CommandBuffer tape, {GPUTensor<Matrix>? outTensor}) → GPUTensor<Matrix>

Predicts the probability distribution of the next states for a batch of histories. Maps each sequence in the historyBatch to its corresponding row in the provided probTable and outputs a matrix of shape [batchSize, numStates] containing the next-state probabilities. The operations are appended to tape for execution.

matMul(Tensor<Matrix> a, Tensor<Matrix> b) → Tensor<Matrix>

Calculates the matrix multiplication of two matrices a and b.

matMulBiasReluGPU(GPUTensor<Matrix> x, GPUTensor<Matrix> w, GPUTensor<Vector> b, CommandBuffer tape, List<GPUTensor> intermediates, {GPUTensor<Matrix>? reluOutTensor, GPUTensor<Matrix>? preReluOutTensor}) → GPUTensor<Matrix>

///////////////////////////////// Fused Kernels (1000+) /// ///////////////////////////////// Executes a fused matrix multiplication, bias addition, and ReLU activation sequence. Multiplies matrix x by weight matrix w, broadcasts and adds the bias vector b, and applies a ReLU activation in a single optimized pass. Returns the activated output matrix and pushes the pre-activation tensor to the intermediates list to manage VRAM lifecycles. The operations are appended to tape for execution.

matMulGPU(GPUTensor<Matrix> a, GPUTensor<Matrix> b, CommandBuffer tape, {GPUTensor<Matrix>? outTensor}) → GPUTensor<Matrix>

///////////////////////////////// Matrix Operations (200-299) /// ///////////////////////////////// Appends the commands for matrix multiplication of a with b to tape.

matVecMul(Tensor<Matrix> M, Tensor<Vector> v) → Tensor<Vector>

Multiplies a vector v with a matrix M, resulting in a new vector.

matVecMulGPU(GPUTensor<Matrix> mMat, GPUTensor<Vector> v, CommandBuffer tape, {GPUTensor<Vector>? outTensor}) → GPUTensor<Vector>

Appends the commands for multiplication of matrixa with vectorb to tape.

maxPool1d(Tensor<Vector> input, int poolSize, int stride) → Tensor<Vector>

Slides a window of size poolSize over the vector with the step-size stride. Returns a vector containing the maximum in the observed windows.

maxPool1dGPU(GPUTensor<Vector> input, int poolSize, int stride, CommandBuffer tape, {GPUTensor<Vector>? outTensor, GPUTensor<Vector>? indicesTensor}) → GPUTensor<Vector>

Slides a pool of size poolSize over vector input. The resulting vector consists of the maximum elements in each window. The value of stride tells how big the step size of the sliding window is. The operations are appended to tape for execution.

maxPool2d(Tensor<Matrix> input, int poolSize, int stride) → Tensor<Matrix>

Slides a window of size poolSize over the matrix with the step-size stride. Returns a matrix containing the maximum in the observed windows.

maxPool2dGPU(GPUTensor<Matrix> input, int poolSize, int stride, CommandBuffer tape, {GPUTensor<Matrix>? outTensor, GPUTensor<Matrix>? indicesTensor}) → GPUTensor<Matrix>

Slides a pool of size poolSizexpoolSize over matrix input. The resulting matrix consists of the maximum elements in each window. The value of stride tells how big the step size of the sliding window is in x and y direction. The operations are appended to tape for execution.

mishMatrix(Tensor<Matrix> m) → Tensor<Matrix>

Applies the mish activation function (similar to swish) to each element inside of m. Mish mirrors Relu with a continuous curve.

mishVector(Tensor<Vector> v) → Tensor<Vector>

Applies the mish activation function (similar to swish) to each element inside of v. Mish mirrors Relu with a continuous curve.

mse(Tensor<Vector> predictions, Tensor<Vector> targets) → Tensor<Scalar>

Calculates the mean square error/difference between the elements of a and b. Both Tensors need to have the same size.

mseGPU(GPUTensor<Vector> predictions, GPUTensor<Vector> targets, CommandBuffer tape, {GPUTensor<Scalar>? outTensor}) → GPUTensor<Scalar>

Appends commands for calculating the mean square error over between the vectors predictions and targets to tape.

mseMatrix(Tensor<Matrix> predictions, Tensor<Matrix> targets) → Tensor<Scalar>

Calculates the mean squared error between two matrices a and b. Error here means the difference between each element of the matrices. The provided matrices are required to have the same shape.

mseMatrixGPU(GPUTensor<Matrix> predictions, GPUTensor<Matrix> targets, CommandBuffer tape, {GPUTensor<Scalar>? outTensor}) → GPUTensor<Scalar>

Appends commands for calculating the mean square error over between the matrices predictions and targets to tape.

multiply(Tensor<Scalar> a, Tensor<Scalar> b) → Tensor<Scalar>

Simple multiplication of tensor a and b.

multiplyGPU<T>(GPUTensor<T> a, GPUTensor<T> b, CommandBuffer tape, {GPUTensor<T>? outTensor}) → GPUTensor<T>

Appends the commands for element wise multiplication of two GPUTensors a and b of type T to tape. Creates and allocates a Tensor to store the results.

multiplyScalarGPU(GPUTensor<Scalar> a, GPUTensor<Scalar> b, CommandBuffer tape, {GPUTensor<Scalar>? outTensor}) → GPUTensor<Scalar>

padMatrix(Matrix input, int padding) → Matrix

Pads the matrix on all sides with the given padding.

padMatrixGPU(GPUTensor<Matrix> input, int padSize, CommandBuffer tape, {GPUTensor<Matrix>? outTensor}) → GPUTensor<Matrix>

Expands a matrix input symmetrically to all sides with 0.0 padding with length of padSize. The operations are appended to tape for execution.

powGPU<T>(GPUTensor<T> a, double exponent, CommandBuffer tape, {GPUTensor<T>? outTensor}) → GPUTensor<T>

Appends the commands for applying pow(v,exponent) exponential function on every element of v to tape.

relu(Tensor<Vector> v) → Tensor<Vector>

Applies the ReLU function to each value of v. When the values is bigger then 0 it is the output, otherwise 0.

reluGPU(GPUTensor<Vector> v, CommandBuffer tape, {GPUTensor<Vector>? outTensor}) → GPUTensor<Vector>

///////////////////////////////// Activations (300-399) /// ///////////////////////////////// Appends commands for applying Relu on every element of vector v to tape.

reluMatrix(Tensor<Matrix> m) → Tensor<Matrix>

Applies the ReLU function to every element of a matrix. Every negative value is set to 0.

reluMatrixGPU(GPUTensor<Matrix> m, CommandBuffer tape, {GPUTensor<Matrix>? outTensor}) → GPUTensor<Matrix>

Appends commands for applying Relu on every element of matrix m to tape.

reshape3DToMatrixGPU(GPUTensor<Tensor3D> t, int rows, int cols, CommandBuffer tape, {GPUTensor<Matrix>? outTensor}) → GPUTensor<Matrix>

reshapeMatrixTo3DGPU(GPUTensor<Matrix> m, int c, int h, int w, CommandBuffer tape, {GPUTensor<Tensor3D>? outTensor}) → GPUTensor<Tensor3D>

reshapeVectorToMatrix(Tensor<Vector> v, int numRows, int numCols) → Tensor<Matrix>

Reformats a vector to a matrix with given dimensions numRowsxnumCols.

reshapeVectorToMatrixGPU(GPUTensor<Vector> v, int numRows, int numCols, CommandBuffer tape, {GPUTensor<Matrix>? outTensor}) → GPUTensor<Matrix>

///////////////////////////////// Data management (0-99) /// /////////////////////////////////

rmsNormMatrixGPU(GPUTensor<Matrix> input, GPUTensor<Vector>? weight, double epsilon, bool scalePlusOne, CommandBuffer tape) → GPUTensor<Matrix>

///////////////////////////////// Transformer & LLM Ops (2000+) /// ///////////////////////////////// Applies Root Mean Square (RMS) Normalization across the feature dimension of an input matrix. Optionally scales the normalized features using a learnable weight vector. If scalePlusOne is true (as required by Gemma models), the scaling factor applied is (1.0 + weight). The reciprocal standard deviation is cached in rstdCache to accelerate backpropagation.

scaleMatrix(Tensor<Matrix> m, double s) → Tensor<Matrix>

Multiplies each element of a matrix with a constant s.

scaleMatrixGPU(GPUTensor<Matrix> m, double s, CommandBuffer tape, {GPUTensor<Matrix>? outTensor}) → GPUTensor<Matrix>

Appends commands for additions of a scalar constant s to every element of matrix m to tape.

scatterHeadsGPU(List<GPUTensor<Matrix>> heads, int dModel, CommandBuffer tape, {GPUTensor<Matrix>? outTensor}) → GPUTensor<Matrix>

Constructs a matrix from smaller matrices packing them into a bigger one in order. Primarily used for attention-heads. The operations are appended to tape for execution.

selectMatrixFrom3DGPU(GPUTensor<Tensor3D> t, int index, CommandBuffer tape, {GPUTensor<Matrix>? outTensor}) → GPUTensor<Matrix>

Extracts a single matrix out of a Tensor3D t given by index. The operation is appended to tape for execution.

selectRow(Tensor<Matrix> m, int rowIndex) → Tensor<Vector>

Returns the vector of the selected row of rowIndex inside of m.

selectRowGPU(GPUTensor<Matrix> m, int rowIndex, CommandBuffer tape, {GPUTensor<Vector>? outTensor}) → GPUTensor<Vector>

Extracts a single row out of a matrix m given by rowIndex. The operation is appended to tape for execution.

sgdUpdateGPU(GPUTensor data, double lr, CommandBuffer tape) → void

///////////////////////////////// Optimizers (500-599) /// ///////////////////////////////// Performs an in place update of the provided data values via standard gradient descent, learning rate lr and its assigned gradient. The operation is appended to tape for execution.

sigmoid(Tensor<Vector> v) → Tensor<Vector>

Applies the Sigmoid activation function to each value of a vector.

sigmoid3DGPU(GPUTensor<Tensor3D> t, CommandBuffer tape, {GPUTensor<Tensor3D>? outTensor}) → GPUTensor<Tensor3D>

Appends commands for applying the sigmoid function on every element of Tensor3D t to tape.

sigmoidGPU(GPUTensor<Vector> v, CommandBuffer tape, {GPUTensor<Vector>? outTensor}) → GPUTensor<Vector>

Appends commands for applying the sigmoid function on every element of vector v to tape.

sigmoidMatrix(Tensor<Matrix> m) → Tensor<Matrix>

Applies the sigmoid function to every value of matrix m.

sigmoidMatrixGPU(GPUTensor<Matrix> m, CommandBuffer tape, {GPUTensor<Matrix>? outTensor}) → GPUTensor<Matrix>

Appends commands for applying the sigmoid function on every element of matrix m to tape.

sigmoidScalar(Tensor<Scalar> s) → Tensor<Scalar>

Applies the sigmoid activation function to a scalar tensor.

sigmoidScalarGPU<T>(GPUTensor<T> s, CommandBuffer tape, {GPUTensor<T>? outTensor}) → GPUTensor<T>

Appends commands for applying the sigmoid function on every element of m (type t) to tape.

sliceColumnGPU(GPUTensor<Matrix> input, int startCol, int endCol, CommandBuffer tape, {GPUTensor<Matrix>? outTensor}) → GPUTensor<Matrix>

///////////////////////////////// Tensor Manipulations(700-799) /// ///////////////////////////////// Extracts a continous subset of columns out of a matrix m given by range startCol-endCol. The operation is appended to tape for execution.

softmaxMatrix(Tensor<Matrix> m) → Tensor<Matrix>

Applies the softmax function to every element. Creates probability distribution over all values summing up to 1.

softmaxMatrixGPU(GPUTensor<Matrix> m, CommandBuffer tape, {GPUTensor<Matrix>? outTensor}) → GPUTensor<Matrix>

Appends commands for applying the softmax function on every element of matrix m to tape.

softmaxVector(Tensor<Vector> v) → Tensor<Vector>

Applies the softmax function over a vector, converting them into a probability value.

softplus(Tensor<Vector> v) → Tensor<Vector>

Differentiable approximation of the Relu function.

sqrtGPU<T>(GPUTensor<T> a, CommandBuffer tape, {GPUTensor<T>? outTensor}) → GPUTensor<T>

Appends the commands for applying the sqrt(v) exponential function on every element of v to tape.

stackMatricesGPU(List<GPUTensor<Matrix>> matrices, CommandBuffer tape, {GPUTensor<Tensor3D>? outTensor}) → GPUTensor<Tensor3D>

Appends a list of matrices matrices together together to form a Tensor3D. The operation is appended to tape for execution.

stackMatricesTo3D(List<Tensor<Matrix>> matrices) → Tensor<Tensor3D>

Uses a stack of matrices to construct a Tensor3D by concatenating them. Gradient propagation is conserved.

subtract3DGPU(GPUTensor<Tensor3D> a, GPUTensor<Tensor3D> b, CommandBuffer tape, {GPUTensor<Tensor3D>? outTensor}) → GPUTensor<Tensor3D>

Appends the commands for subtraction of two Tensor3Ds a and b of type T to tape. Creates and allocates a Tensor to store the results.

subtractGPU<T>(GPUTensor<T> a, GPUTensor<T> b, CommandBuffer tape, {GPUTensor<T>? outTensor}) → GPUTensor<T>

Appends the commands for subtraction of two GPUTensors a and b of type T to the tape. Creates and allocates a Tensor to store the results.

subtractMatrixGPU(GPUTensor<Matrix> a, GPUTensor<Matrix> b, CommandBuffer tape, {GPUTensor<Matrix>? outTensor}) → GPUTensor<Matrix>

Appends the commands for subtraction of two matrices a and b of type T to tape. Creates and allocates a Tensor to store the results.

subtractVectorGPU(GPUTensor<Vector> a, GPUTensor<Vector> b, CommandBuffer tape, {GPUTensor<Vector>? outTensor}) → GPUTensor<Vector>

sum(Tensor<Vector> v) → Tensor<Scalar>

Sums up all elements of the input vector tensor.

sumGPU(GPUTensor<Vector> v, CommandBuffer tape, {GPUTensor<Scalar>? outTensor}) → GPUTensor<Scalar>

///////////////////////////////// Reductions (600-699) /// ///////////////////////////////// Appends commands for summing all elements of vector v to tape.

sumMatrix(Tensor<Matrix> m) → Tensor<Scalar>

Sums all elements of matrix m together.

sumMatrixGPU(GPUTensor<Matrix> m, CommandBuffer tape, {GPUTensor<Scalar>? outTensor}) → GPUTensor<Scalar>

Appends commands for summing all elements of matrix m to tape.

sumReduceColumnsGPU(GPUTensor<Matrix> m, CommandBuffer tape, {GPUTensor<Vector>? outTensor}) → GPUTensor<Vector>

Sums all matrix elements of m via their column order returning the sum of each column as a vector. The operation is appended to tape for execution.

sumReduceRowsGPU(GPUTensor<Matrix> m, CommandBuffer tape, {GPUTensor<Vector>? outTensor}) → GPUTensor<Vector>

Sums all matrix elements of m via their row order returning the sum of each row as a vector. The operation is appended to tape for execution.

swishMatrix(Tensor<Matrix> m) → Tensor<Matrix>

Applies the swish activation function to each element inside of m. Swish mirrors Relu with a continuous curve.

swishVector(Tensor<Vector> v) → Tensor<Vector>

Applies the swish activation function to each element inside of v. Swish mirrors Relu with a continuous curve.

tanh3DGPU(GPUTensor<Tensor3D> t, CommandBuffer tape, {GPUTensor<Tensor3D>? outTensor}) → GPUTensor<Tensor3D>

Appends commands for applying the tanh function on every element of Tensor3D t to tape.

tanhMatrix(Tensor<Matrix> m) → Tensor<Matrix>

Applies the tanh fucntion to every values of matrix m.

tanhMatrixGPU(GPUTensor<Matrix> m, CommandBuffer tape, {GPUTensor<Matrix>? outTensor}) → GPUTensor<Matrix>

Appends commands for applying the tanh function on every element of matrix m to tape.

transpose(Tensor<Matrix> a) → Tensor<Matrix>

Switches rows and columns.

transposeGPU(GPUTensor<Matrix> a, CommandBuffer tape, {GPUTensor<Matrix>? outTensor}) → GPUTensor<Matrix>

Appends a command for switching rows and columns to tape.

vectorExp(Tensor<Vector> v) → Tensor<Vector>

Calculates the e^x exponent for each element x in v.

vectorExpGPU(GPUTensor<Vector> v, CommandBuffer tape, {GPUTensor<Vector>? outTensor}) → GPUTensor<Vector>

Appends the commands for applying the e^v exponential function on vector v to tape.

vectorLog(Tensor<Vector> v) → Tensor<Vector>

Calculates the log(x) exponent for each element x in v.

vectorTanh(Tensor<Vector> v) → Tensor<Vector>

Applies the tanh activation function to every element of a vector.

vectorTanhGPU(GPUTensor<Vector> v, CommandBuffer tape, {GPUTensor<Vector>? outTensor}) → GPUTensor<Vector>

Appends commands for applying the tanh function on every element of vector t to tape.

Typedefs

DartAddPointers = void Function(Pointer<Float>, Pointer<Float>, int)

DartCreate = int Function(bool)

DartFree = void Function(int)

DartFreeStr = void Function(Pointer<Utf8>)

DartFreeT = void Function(int, Pointer<Utf8>)

DartGetNames = Pointer<Utf8> Function(int)

DartInitRandom = void Function(int, Pointer<Utf8>, double, int)

DartLoad = void Function(int, Pointer<Utf8>, Pointer<Float>, int, Pointer<Int32>)

DartRetrieve = void Function(int, Pointer<Utf8>, Pointer<Float>)

DartRun = void Function(int, Pointer<Uint8>, int)

Mat = Matrix

Matrix = List<List<double>>

NativeAddPointers = Void Function(Pointer<Float>, Pointer<Float>, Int32)

FFI- Advanced operation to allow direct pointer access. Adds the content of a source content into a destination pointer with a given length

NativeCreate = Int64 Function(Bool)

FFI- Binding for creating a tensor on the GPU

NativeFree = Void Function(Int64)

FFI- Binding for freeing/deleting the GPU backend

NativeFreeStr = Void Function(Pointer<Utf8>)

FFI- Binding for freeing the made list of tensor names

NativeFreeT = Void Function(Int64, Pointer<Utf8>)

FFI- Binding for freeing a tensor on the GPU

NativeGetNames = Pointer<Utf8> Function(Int64)

FFI- Binding for retrieving the ids of all tensors currently on the GPU. This creates a List of Strings that is stored in the backend and has to be deleted again

NativeInitRandom = Void Function(Int64, Pointer<Utf8>, Float, Int32)

FFI- Initialise a tensor with a random initialization

NativeLoad = Void Function(Int64, Pointer<Utf8>, Pointer<Float>, Int32, Pointer<Int32>)

FFI- Binding for pushing a tensor to the GPU

NativeRetrieve = Void Function(Int64, Pointer<Utf8>, Pointer<Float>)

FFI- Binding for pulling a tensor from the GPU

NativeRun = Void Function(Int64, Pointer<Uint8>, Int32)

FFI- Binding for running a given byteList of commands (usually via a CommandBuffer)

Sc = Scalar

Scalar = double

T3D = Tensor3D

Tensor3D = List<List<List<double>>>

Vec = Vector

Vector = List<double>