tensor/tensor_math_gpu
library
Functions
-
absGPU<T>(GPUTensor<T> a, CommandBuffer tape, {GPUTensor<T>? outTensor})
→ GPUTensor<T>
-
Appends the commands for applying the abs(v) exponential function on every element of
a to tape.
-
adamUpdateGPU(GPUTensor data, GPUTensor m, GPUTensor v, double lr, double beta1, double beta2, double eps, int step, double weightDecay, CommandBuffer tape)
→ void
-
Performs an in place update of the provided
data values via the adam function and its assigned gradient.
The first moment m and second moment v as well as beta1, beta2, eps,weightDecay, and step need to be provided.
The operation is appended to tape for execution.
-
add3DGPU(GPUTensor<Tensor3D> a, GPUTensor<Tensor3D> b, CommandBuffer tape, {GPUTensor<Tensor3D>? outTensor})
→ GPUTensor<Tensor3D>
-
Adds the command for addition of two Tensor3Ds
a and b to the tape.
Creates and allocates a Tensor to store the results.
-
addBiasToFeatureMapGPU(GPUTensor<Matrix> m, GPUTensor<Matrix> b, CommandBuffer tape, {GPUTensor<Matrix>? outTensor})
→ GPUTensor<Matrix>
-
Appends commands for broadcast additions of a matrix wrapped scalar
b to a full matrix m.
-
addBiasToMatMulOutGPU(GPUTensor<Matrix> m, GPUTensor<Vector> b, CommandBuffer tape, {GPUTensor<Matrix>? outTensor})
→ GPUTensor<Matrix>
-
Appends commands for addition of a 1D bias vector
b to a 2D matrix m via broadcasting over the matrix to the tape.
-
addGPU<T>(GPUTensor<T> a, GPUTensor<T> b, CommandBuffer tape, {GPUTensor<T>? outTensor})
→ GPUTensor<T>
-
Adds the command for addition of two GPUTensors
a and b of type T to the tape.
Creates and allocates a Tensor to store the results.
-
addMatrixAndVectorGPU(GPUTensor<Matrix> m, GPUTensor<Vector> v, CommandBuffer tape, {GPUTensor<Matrix>? outTensor})
→ GPUTensor<Matrix>
-
Appends a command for broadcast add a vector
v to tape.
-
addMatrixGPU(GPUTensor<Matrix> a, GPUTensor<Matrix> b, CommandBuffer tape, {GPUTensor<Matrix>? outTensor})
→ GPUTensor<Matrix>
-
Adds the command for addition of two matrices
a and b to the tape.
Creates and allocates a Tensor to store the results.
-
addScalar3DGPU(GPUTensor<Tensor3D> t, double scalar, CommandBuffer tape, {GPUTensor<Tensor3D>? outTensor})
→ GPUTensor<Tensor3D>
-
Appends a command for adding a scalar
scalar to every element of a Tensor3D v to tape.
-
addScalarMatrixGPU(GPUTensor<Matrix> m, GPUTensor<Scalar> b, CommandBuffer tape, {GPUTensor<Matrix>? outTensor})
→ GPUTensor<Matrix>
-
Appends a command for adding a scalar
b to every element of a matrix m to tape.
-
addScalarVectorGPU(GPUTensor<Vector> v, double scalar, CommandBuffer tape, {GPUTensor<Vector>? outTensor})
→ GPUTensor<Vector>
-
Appends a command for adding a scalar
b to every element of a vector v to tape.
-
addVectorGPU(GPUTensor<Vector> a, GPUTensor<Vector> b, CommandBuffer tape, {GPUTensor<Vector>? outTensor})
→ GPUTensor<Vector>
-
Adds the command for addition of two vectors
a and b to the tape.
Creates and allocates a Tensor to store the results.
-
applyRopeGPU(GPUTensor<Tensor3D> input, GPUTensor<Matrix> cosTable, GPUTensor<Matrix> sinTable, int batchSize, int seqLen, int numHeads, int headDim, CommandBuffer tape)
→ GPUTensor<Tensor3D>
-
Applies Rotary Position Embeddings (RoPE) to a multi-head attention state tensor.
Encodes relative positional information by pairing and rotating feature dimensions using precomputed values supplied in
cosTable and sinTable. The backward pass computes the
inverse rotation gradients by intrinsically transposing the rotation matrix (negating the sine terms).
-
argmaxGPU(GPUTensor<Matrix> logits, CommandBuffer tape)
→ GPUTensor<Vector>
-
Identifies the predicted class token for each sequence in a batch.
Examines the vocabulary dimension of the provided
logits and returns a vector containing
the index of the maximum value (argmax) for each batch row. This operator is used exclusively
during inference and has no backward pass.
-
avgPool2dGPU(GPUTensor<Matrix> input, int poolSize, int stride, CommandBuffer tape, {GPUTensor<Matrix>? outTensor})
→ GPUTensor<Matrix>
-
Slides a pool of size
poolSizexpoolSize over matrix input. The resulting matrix consists of the average of the elements in each window.
The value of stride tells how big the step size of the sliding window is.
The operations are appended to tape for execution.
-
batchNorm1dGPU(GPUTensor<Vector> input, GPUTensor<Vector> gamma, GPUTensor<Vector> beta, GPUTensor<Vector> runningMean, GPUTensor<Vector> runningVariance, double momentum, double epsilon, bool isTraining, CommandBuffer tape, {GPUTensor<Vector>? outTensor, GPUTensor<Vector>? savedMeanTensor, GPUTensor<Vector>? savedInvVarTensor})
→ GPUTensor<Vector>
-
Normalizes a 1D feature vector using per-feature running mean and variance.
Scales and shifts the normalized data using learnable
gamma and beta parameter vectors.
During inference, it computes the output strictly using the pre-calculated running statistics rather than the current batch's statistics.
The operations are appended to tape for execution.
-
batchNorm2dGPU(GPUTensor<Tensor3D> input, GPUTensor<Vector> gamma, GPUTensor<Vector> beta, GPUTensor<Vector> runningMean, GPUTensor<Vector> runningVariance, double momentum, double epsilon, bool isTraining, CommandBuffer tape, {GPUTensor<Tensor3D>? outTensor, GPUTensor<Vector>? savedMeanTensor, GPUTensor<Vector>? savedInvVarTensor})
→ GPUTensor<Tensor3D>
-
Normalizes a 3D spatial tensor independently for each channel.
Uses the mean and variance evaluated across the combined spatial dimensions (height and width) of a given channel.
Returns a 3D tensor of identical shape scaled by
gamma and shifted by beta.
The operations are appended to tape for execution.
-
binaryCrossEntropyGPU<T>(GPUTensor<T> prediction, GPUTensor<T> target, CommandBuffer tape, {GPUTensor<Scalar>? outTensor})
→ GPUTensor<Scalar>
-
/////////////////////////////////
Loss Functions (400-499) ///
/////////////////////////////////
Appends commands for calculating the binary cross entropy over the elements of
m of type T to tape.
-
broadcastAddVectorToMatrixGPU(GPUTensor<Matrix> m, GPUTensor<Vector> v, CommandBuffer tape, {GPUTensor<Matrix>? outTensor})
→ GPUTensor<Matrix>
-
Appends commands for addition of a 1D bias vector
b to a 2D matrix m via broadcasting over the matrix to the tape.
This command is inference only! No gradient path is calculated.
-
buildMarkovTableGPU(GPUTensor<Vector> sequence, int order, int numStates, CommandBuffer tape, {GPUTensor<Matrix>? outTensor, GPUTensor<Matrix>? probTableTensor})
→ GPUTensor<Matrix>
-
Constructs a Markov chain probability transition table from a given state
sequence.
Calculates the frequencies of state transitions based on the specified Markov order and numStates,
and normalizes these counts to output a probability matrix of shape [numStates^order, numStates]. This Table does not have a gradient calculation.
The operations are appended to tape for execution.
-
causalMaskGPU(GPUTensor<Tensor3D> input, int batchSize, int numHeads, int seqLen, CommandBuffer tape)
→ GPUTensor<Tensor3D>
-
Applies a causal attention mask to prevent sequences from attending to future tokens.
Modifies the
[seqLen, seqLen] attention matrices within the 3D tensor by overriding elements
above the main diagonal with -10000.0, effectively zeroing out their probability post-softmax.
Gradients are blocked from flowing back through the masked positions.
-
clampGPU<T>(GPUTensor<T> a, double minVal, double maxVal, CommandBuffer tape, {GPUTensor<T>? outTensor})
→ GPUTensor<T>
-
Appends the commands for clamping every element of
v between minVal - maxVal to tape.
-
clipGradValueGPU(GPUTensor tensor, double clipValue, CommandBuffer tape)
→ void
-
Performs an in place update of the provided
data values by clipping them in between the intervall (-clipValue)-(clipValue).
The operation is appended to tape for execution.
-
concatenate3DGPU(GPUTensor<Tensor3D> a, GPUTensor<Tensor3D> b, CommandBuffer tape, {GPUTensor<Tensor3D>? outTensor})
→ GPUTensor<Tensor3D>
-
Appends two Tensor3Ds
a and b together in order a-b.
The operation is appended to tape for execution.
-
concatenateGPU(GPUTensor<Vector> a, GPUTensor<Vector> b, CommandBuffer tape, {GPUTensor<Vector>? outTensor})
→ GPUTensor<Vector>
-
Appends two vectors
a and b together in order a-b.
The operation is appended to tape for execution.
-
concatenateMatricesByColumnGPU(List<GPUTensor<Matrix>> matrices, CommandBuffer tape, {GPUTensor<Matrix>? outTensor})
→ GPUTensor<Matrix>
-
Appends a list of matrices
matrices together in column order.
The operation is appended to tape for execution.
-
conv2dMultiChannelGPU(GPUTensor input, GPUTensor<Tensor3D> weight, GPUTensor<Vector> bias, int kH, int kW, CommandBuffer tape, {String padding = 'valid', int strideH = 1, int strideW = 1, GPUTensor<Tensor3D>? outTensor})
→ GPUTensor<Tensor3D>
-
Performs an advanced 2D convolution on multiple channels with
input and weight, alongside a bias vector.
-
conv2dSimpleGPU(GPUTensor<Matrix> input, GPUTensor<Matrix> kernel, CommandBuffer tape, {GPUTensor<Matrix>? outTensor})
→ GPUTensor<Matrix>
-
Computes a basic, single-channel 2D convolution between an
input matrix and a kernel matrix.
-
cosineSimilarityGPU(GPUTensor<Vector> a, GPUTensor<Vector> b, CommandBuffer tape)
→ GPUTensor<Scalar>
-
Computes the cosine similarity between two vectors, measuring their directional alignment.
Calculates the dot product of
a and b and divides it by the product of their individual L2 norms.
Returns a scalar between -1.0 (opposite) and 1.0 (identical direction).
The operations are appended to tape for execution.
-
crossEntropyLossGPU(GPUTensor<Matrix> logits, GPUTensor<Vector> targets, CommandBuffer tape)
→ GPUTensor<Vector>
-
Computes the categorical Cross-Entropy loss between predicted logits and target class indices.
Leverages a fused, numerically stable softmax internally to calculate the log-probabilities.
Outputs an unreduced vector containing one scalar loss value per batch sequence.
-
divide3DGPU(GPUTensor<Tensor3D> a, GPUTensor<Tensor3D> b, CommandBuffer tape, {GPUTensor<Tensor3D>? outTensor})
→ GPUTensor<Tensor3D>
-
Appends the commands for element wise division of two Tensor3Ds
a through b to tape.
Creates and allocates a Tensor to store the results.
-
divideGPU<T>(GPUTensor<T> a, GPUTensor<T> b, CommandBuffer tape, {GPUTensor<T>? outTensor})
→ GPUTensor<T>
-
Appends the commands for element wise division of two GPUTensors
a through b of type T to tape.
Creates and allocates a Tensor to store the results.
-
divideMatrixGPU(GPUTensor<Matrix> a, GPUTensor<Matrix> b, CommandBuffer tape, {GPUTensor<Matrix>? outTensor})
→ GPUTensor<Matrix>
-
Appends the commands for element wise division of two matrices
a through b to tape.
Creates and allocates a Tensor to store the results.
-
divideVectorGPU(GPUTensor<Vector> a, GPUTensor<Vector> b, CommandBuffer tape, {GPUTensor<Vector>? outTensor})
→ GPUTensor<Vector>
-
-
dotProductGPU(GPUTensor<Vector> a, GPUTensor<Vector> b, CommandBuffer tape, {GPUTensor<Vector>? multipliedTensor, GPUTensor<Scalar>? outTensor})
→ GPUTensor<Scalar>
-
Calculates the dot product between vector
a and b.
The operations are appended to tape for execution.
The operations are appended to tape for execution.
-
dropoutGPU<T>(GPUTensor<T> input, double rate, CommandBuffer tape, {GPUTensor<T>? outTensor, GPUTensor<T>? maskTensor})
→ GPUTensor<T>
-
Randomly zeroes out elements of the input tensor with a probability equal to the specified
rate.
To preserve the expected mathematical sum of the tensor during inference, it scales the remaining active elements by 1 / (1 - rate).
Outputs the modified tensor alongside the generated dropout mask.
The operations are appended to tape for execution.
-
elementWiseMultiply3DGPU(GPUTensor<Tensor3D> a, GPUTensor<Tensor3D> b, CommandBuffer tape, {GPUTensor<Tensor3D>? outTensor})
→ GPUTensor<Tensor3D>
-
Appends the commands for element wise multiplication of two Tensor3Ds
a and b to tape.
Creates and allocates a Tensor to store the results.
-
elementWiseMultiplyGPU(GPUTensor<Vector> a, GPUTensor<Vector> b, CommandBuffer tape, {GPUTensor<Vector>? outTensor})
→ GPUTensor<Vector>
-
Appends the commands for element wise multiplication of two vectors
a and b to tape.
Creates and allocates a Tensor to store the results.
-
elementWiseMultiplyMatrixGPU(GPUTensor<Matrix> a, GPUTensor<Matrix> b, CommandBuffer tape, {GPUTensor<Matrix>? outTensor})
→ GPUTensor<Matrix>
-
Appends the commands for element wise multiplication of two matrices
a and b to tape.
Creates and allocates a Tensor to store the results.
-
embeddingLookupBatchGPU(GPUTensor<Matrix> batchIndices, GPUTensor<Matrix> weights, CommandBuffer tape, {GPUTensor<Tensor3D>? outTensor})
→ GPUTensor<Tensor3D>
-
Acts as a high performance lookup of rows in a given matrix
weights via their row indices indices.
In contrast to embeddingLookupGPU this method performs a batch process to speed up lookup.
Can be used for fast odering of elements.
The operation is appended to tape for execution.
-
embeddingLookupGPU(GPUTensor<Vector> indices, GPUTensor<Matrix> weights, CommandBuffer tape, {GPUTensor<Matrix>? outTensor})
→ GPUTensor<Matrix>
-
Acts as a high performance lookup of rows in a given matrix
weights via their row indices indices.
Can be used for fast odering of elements.
The operation is appended to tape for execution.
-
euclideanDistanceGPU(GPUTensor<Vector> a, GPUTensor<Vector> b, CommandBuffer tape)
→ GPUTensor<Scalar>
-
Calculates the straight-line Euclidean distance between two vectors.
Computes the element-wise difference between vector
a and b, squares the differences, sums them, and extracts the square root.
The operations are appended to tape for execution.
-
flatten3DToMatrixGPU(GPUTensor<Tensor3D> t, CommandBuffer tape, {GPUTensor<Matrix>? outTensor})
→ GPUTensor<Matrix>
-
-
geluGPU(GPUTensor<Vector> v, CommandBuffer tape, {GPUTensor<Vector>? outTensor})
→ GPUTensor<Vector>
-
Appends commands for applying the gelu activation function on every element of vector
v to tape.
-
geluMatrixGPU(GPUTensor<Matrix> m, CommandBuffer tape, {GPUTensor<Matrix>? outTensor})
→ GPUTensor<Matrix>
-
Appends commands for applying the gelu activation function on every element of matrix
m to tape.
-
globalAveragePoolingGPU(GPUTensor<Matrix> input, CommandBuffer tape, {GPUTensor<Vector>? outTensor})
→ GPUTensor<Vector>
-
Collapses a matrix
input to a vector of their average values of their feature columns.
The operations are appended to tape for execution.
-
im2colGPU(GPUTensor input, int kH, int kW, CommandBuffer tape, {GPUTensor<Matrix>? outTensor})
→ GPUTensor<Matrix>
-
Extracts sliding local pathces fromm an image Tensor and flattens them into columns of a matrix.
Kernel dimensions
kH and kW need to be set.
The operations are appended to tape for execution.
-
l2NormGPU(GPUTensor<Vector> v, CommandBuffer tape)
→ GPUTensor<Scalar>
-
Computes the L2 norm (Euclidean length) of a vector. Squares each element of the vector, sums them together, and takes the square root of the result.
The operations are appended to
tape for execution.
The operations are appended to tape for execution.
-
layerNormMatrixGPU(GPUTensor<Matrix> m, GPUTensor<Vector> gamma, GPUTensor<Vector> beta, GPUTensor<Vector> meanCache, GPUTensor<Vector> rstdCache, double epsilon, CommandBuffer tape, {GPUTensor<Matrix>? outTensor})
→ GPUTensor<Matrix>
-
Normalizes a 2D matrix independently for each row.
Calculates the mean and variance across the columns of a single row, normalizes the row's elements, and applies the
gamma and beta transformations.
Outputs a normalized matrix of the original dimensions.
The operations are appended to tape for execution.
-
loadSampleGPU(GPUTensor<Matrix> dataset, int sampleIndex, CommandBuffer tape, {GPUTensor<Vector>? outTensor})
→ GPUTensor<Vector>
-
-
logGPU<T>(GPUTensor<T> a, CommandBuffer tape, {GPUTensor<T>? outTensor})
→ GPUTensor<T>
-
Appends the commands for applying the log(v) exponential function on every element of
v to tape.
-
maeLossGPU(GPUTensor<Vector> preds, GPUTensor<Vector> targets, CommandBuffer tape)
→ GPUTensor<Scalar>
-
Calculates the Mean Absolute Error (MAE) loss between predictions and targets.
Computes the absolute differences between the
preds and targets vectors, sums the errors, and divides by the total number of elements to yield a single average error scalar.
The operations are appended to tape for execution.
-
markovPredictGPU(GPUTensor<Matrix> historyBatch, GPUTensor<Matrix> probTable, int numStates, CommandBuffer tape, {GPUTensor<Matrix>? outTensor})
→ GPUTensor<Matrix>
-
Predicts the probability distribution of the next states for a batch of histories.
Maps each sequence in the
historyBatch to its corresponding row in the provided probTable
and outputs a matrix of shape [batchSize, numStates] containing the next-state probabilities.
The operations are appended to tape for execution.
-
matMulBiasReluGPU(GPUTensor<Matrix> x, GPUTensor<Matrix> w, GPUTensor<Vector> b, CommandBuffer tape, List<GPUTensor> intermediates, {GPUTensor<Matrix>? reluOutTensor, GPUTensor<Matrix>? preReluOutTensor})
→ GPUTensor<Matrix>
-
/////////////////////////////////
Fused Kernels (1000+) ///
/////////////////////////////////
Executes a fused matrix multiplication, bias addition, and ReLU activation sequence.
Multiplies matrix
x by weight matrix w, broadcasts and adds the bias vector b,
and applies a ReLU activation in a single optimized pass. Returns the activated output matrix
and pushes the pre-activation tensor to the intermediates list to manage VRAM lifecycles.
The operations are appended to tape for execution.
-
matMulGPU(GPUTensor<Matrix> a, GPUTensor<Matrix> b, CommandBuffer tape, {GPUTensor<Matrix>? outTensor})
→ GPUTensor<Matrix>
-
/////////////////////////////////
Matrix Operations (200-299) ///
/////////////////////////////////
Appends the commands for matrix multiplication of
a with b to tape.
-
matVecMulGPU(GPUTensor<Matrix> mMat, GPUTensor<Vector> v, CommandBuffer tape, {GPUTensor<Vector>? outTensor})
→ GPUTensor<Vector>
-
Appends the commands for multiplication of matrix
a with vectorb to tape.
-
maxPool1dGPU(GPUTensor<Vector> input, int poolSize, int stride, CommandBuffer tape, {GPUTensor<Vector>? outTensor, GPUTensor<Vector>? indicesTensor})
→ GPUTensor<Vector>
-
Slides a pool of size
poolSize over vector input. The resulting vector consists of the maximum elements in each window.
The value of stride tells how big the step size of the sliding window is.
The operations are appended to tape for execution.
-
maxPool2dGPU(GPUTensor<Matrix> input, int poolSize, int stride, CommandBuffer tape, {GPUTensor<Matrix>? outTensor, GPUTensor<Matrix>? indicesTensor})
→ GPUTensor<Matrix>
-
Slides a pool of size
poolSizexpoolSize over matrix input. The resulting matrix consists of the maximum elements in each window.
The value of stride tells how big the step size of the sliding window is in x and y direction.
The operations are appended to tape for execution.
-
mseGPU(GPUTensor<Vector> predictions, GPUTensor<Vector> targets, CommandBuffer tape, {GPUTensor<Scalar>? outTensor})
→ GPUTensor<Scalar>
-
Appends commands for calculating the mean square error over between the vectors
predictions and targets to tape.
-
mseMatrixGPU(GPUTensor<Matrix> predictions, GPUTensor<Matrix> targets, CommandBuffer tape, {GPUTensor<Scalar>? outTensor})
→ GPUTensor<Scalar>
-
Appends commands for calculating the mean square error over between the matrices
predictions and targets to tape.
-
multiplyGPU<T>(GPUTensor<T> a, GPUTensor<T> b, CommandBuffer tape, {GPUTensor<T>? outTensor})
→ GPUTensor<T>
-
Appends the commands for element wise multiplication of two GPUTensors
a and b of type T to tape.
Creates and allocates a Tensor to store the results.
-
multiplyScalarGPU(GPUTensor<Scalar> a, GPUTensor<Scalar> b, CommandBuffer tape, {GPUTensor<Scalar>? outTensor})
→ GPUTensor<Scalar>
-
-
padMatrixGPU(GPUTensor<Matrix> input, int padSize, CommandBuffer tape, {GPUTensor<Matrix>? outTensor})
→ GPUTensor<Matrix>
-
Expands a matrix
input symmetrically to all sides with 0.0 padding with length of padSize.
The operations are appended to tape for execution.
-
powGPU<T>(GPUTensor<T> a, double exponent, CommandBuffer tape, {GPUTensor<T>? outTensor})
→ GPUTensor<T>
-
Appends the commands for applying pow(
v,exponent) exponential function on every element of v to tape.
-
reluGPU(GPUTensor<Vector> v, CommandBuffer tape, {GPUTensor<Vector>? outTensor})
→ GPUTensor<Vector>
-
/////////////////////////////////
Activations (300-399) ///
/////////////////////////////////
Appends commands for applying Relu on every element of vector
v to tape.
-
reluMatrixGPU(GPUTensor<Matrix> m, CommandBuffer tape, {GPUTensor<Matrix>? outTensor})
→ GPUTensor<Matrix>
-
Appends commands for applying Relu on every element of matrix
m to tape.
-
reshape3DToMatrixGPU(GPUTensor<Tensor3D> t, int rows, int cols, CommandBuffer tape, {GPUTensor<Matrix>? outTensor})
→ GPUTensor<Matrix>
-
-
reshapeMatrixTo3DGPU(GPUTensor<Matrix> m, int c, int h, int w, CommandBuffer tape, {GPUTensor<Tensor3D>? outTensor})
→ GPUTensor<Tensor3D>
-
-
reshapeVectorToMatrixGPU(GPUTensor<Vector> v, int numRows, int numCols, CommandBuffer tape, {GPUTensor<Matrix>? outTensor})
→ GPUTensor<Matrix>
-
/////////////////////////////////
Data management (0-99) ///
/////////////////////////////////
-
rmsNormMatrixGPU(GPUTensor<Matrix> input, GPUTensor<Vector>? weight, double epsilon, bool scalePlusOne, CommandBuffer tape)
→ GPUTensor<Matrix>
-
/////////////////////////////////
Transformer & LLM Ops (2000+) ///
/////////////////////////////////
Applies Root Mean Square (RMS) Normalization across the feature dimension of an input matrix.
Optionally scales the normalized features using a learnable
weight vector. If scalePlusOne is true (as required by Gemma models), the scaling factor applied is (1.0 + weight). The reciprocal standard deviation is cached in rstdCache to accelerate backpropagation.
-
scaleMatrixGPU(GPUTensor<Matrix> m, double s, CommandBuffer tape, {GPUTensor<Matrix>? outTensor})
→ GPUTensor<Matrix>
-
Appends commands for additions of a scalar constant
s to every element of matrix m to tape.
-
scatterHeadsGPU(List<GPUTensor<Matrix>> heads, int dModel, CommandBuffer tape, {GPUTensor<Matrix>? outTensor})
→ GPUTensor<Matrix>
-
Constructs a matrix from smaller matrices packing them into a bigger one in order. Primarily used for attention-heads.
The operations are appended to
tape for execution.
-
selectMatrixFrom3DGPU(GPUTensor<Tensor3D> t, int index, CommandBuffer tape, {GPUTensor<Matrix>? outTensor})
→ GPUTensor<Matrix>
-
Extracts a single matrix out of a Tensor3D
t given by index.
The operation is appended to tape for execution.
-
selectRowGPU(GPUTensor<Matrix> m, int rowIndex, CommandBuffer tape, {GPUTensor<Vector>? outTensor})
→ GPUTensor<Vector>
-
Extracts a single row out of a matrix
m given by rowIndex.
The operation is appended to tape for execution.
-
sgdUpdateGPU(GPUTensor data, double lr, CommandBuffer tape)
→ void
-
/////////////////////////////////
Optimizers (500-599) ///
/////////////////////////////////
Performs an in place update of the provided
data values via standard gradient descent, learning rate lr and its assigned gradient.
The operation is appended to tape for execution.
-
sigmoid3DGPU(GPUTensor<Tensor3D> t, CommandBuffer tape, {GPUTensor<Tensor3D>? outTensor})
→ GPUTensor<Tensor3D>
-
Appends commands for applying the sigmoid function on every element of Tensor3D
t to tape.
-
sigmoidGPU(GPUTensor<Vector> v, CommandBuffer tape, {GPUTensor<Vector>? outTensor})
→ GPUTensor<Vector>
-
Appends commands for applying the sigmoid function on every element of vector
v to tape.
-
sigmoidMatrixGPU(GPUTensor<Matrix> m, CommandBuffer tape, {GPUTensor<Matrix>? outTensor})
→ GPUTensor<Matrix>
-
Appends commands for applying the sigmoid function on every element of matrix
m to tape.
-
sigmoidScalarGPU<T>(GPUTensor<T> s, CommandBuffer tape, {GPUTensor<T>? outTensor})
→ GPUTensor<T>
-
Appends commands for applying the sigmoid function on every element of
m (type t) to tape.
-
sliceColumnGPU(GPUTensor<Matrix> input, int startCol, int endCol, CommandBuffer tape, {GPUTensor<Matrix>? outTensor})
→ GPUTensor<Matrix>
-
/////////////////////////////////
Tensor Manipulations(700-799) ///
/////////////////////////////////
Extracts a continous subset of columns out of a matrix
m given by range startCol-endCol.
The operation is appended to tape for execution.
-
softmaxMatrixGPU(GPUTensor<Matrix> m, CommandBuffer tape, {GPUTensor<Matrix>? outTensor})
→ GPUTensor<Matrix>
-
Appends commands for applying the softmax function on every element of matrix
m to tape.
-
sqrtGPU<T>(GPUTensor<T> a, CommandBuffer tape, {GPUTensor<T>? outTensor})
→ GPUTensor<T>
-
Appends the commands for applying the sqrt(v) exponential function on every element of
v to tape.
-
stackMatricesGPU(List<GPUTensor<Matrix>> matrices, CommandBuffer tape, {GPUTensor<Tensor3D>? outTensor})
→ GPUTensor<Tensor3D>
-
Appends a list of matrices
matrices together together to form a Tensor3D.
The operation is appended to tape for execution.
-
subtract3DGPU(GPUTensor<Tensor3D> a, GPUTensor<Tensor3D> b, CommandBuffer tape, {GPUTensor<Tensor3D>? outTensor})
→ GPUTensor<Tensor3D>
-
Appends the commands for subtraction of two Tensor3Ds
a and b of type T to tape.
Creates and allocates a Tensor to store the results.
-
subtractGPU<T>(GPUTensor<T> a, GPUTensor<T> b, CommandBuffer tape, {GPUTensor<T>? outTensor})
→ GPUTensor<T>
-
Appends the commands for subtraction of two GPUTensors
a and b of type T to the tape.
Creates and allocates a Tensor to store the results.
-
subtractMatrixGPU(GPUTensor<Matrix> a, GPUTensor<Matrix> b, CommandBuffer tape, {GPUTensor<Matrix>? outTensor})
→ GPUTensor<Matrix>
-
Appends the commands for subtraction of two matrices
a and b of type T to tape.
Creates and allocates a Tensor to store the results.
-
subtractVectorGPU(GPUTensor<Vector> a, GPUTensor<Vector> b, CommandBuffer tape, {GPUTensor<Vector>? outTensor})
→ GPUTensor<Vector>
-
-
sumGPU(GPUTensor<Vector> v, CommandBuffer tape, {GPUTensor<Scalar>? outTensor})
→ GPUTensor<Scalar>
-
/////////////////////////////////
Reductions (600-699) ///
/////////////////////////////////
Appends commands for summing all elements of vector
v to tape.
-
sumMatrixGPU(GPUTensor<Matrix> m, CommandBuffer tape, {GPUTensor<Scalar>? outTensor})
→ GPUTensor<Scalar>
-
Appends commands for summing all elements of matrix
m to tape.
-
sumReduceColumnsGPU(GPUTensor<Matrix> m, CommandBuffer tape, {GPUTensor<Vector>? outTensor})
→ GPUTensor<Vector>
-
Sums all matrix elements of
m via their column order returning the sum of each column as a vector.
The operation is appended to tape for execution.
-
sumReduceRowsGPU(GPUTensor<Matrix> m, CommandBuffer tape, {GPUTensor<Vector>? outTensor})
→ GPUTensor<Vector>
-
Sums all matrix elements of
m via their row order returning the sum of each row as a vector.
The operation is appended to tape for execution.
-
tanh3DGPU(GPUTensor<Tensor3D> t, CommandBuffer tape, {GPUTensor<Tensor3D>? outTensor})
→ GPUTensor<Tensor3D>
-
Appends commands for applying the tanh function on every element of Tensor3D
t to tape.
-
tanhMatrixGPU(GPUTensor<Matrix> m, CommandBuffer tape, {GPUTensor<Matrix>? outTensor})
→ GPUTensor<Matrix>
-
Appends commands for applying the tanh function on every element of matrix
m to tape.
-
transposeGPU(GPUTensor<Matrix> a, CommandBuffer tape, {GPUTensor<Matrix>? outTensor})
→ GPUTensor<Matrix>
-
Appends a command for switching rows and columns to
tape.
-
vectorExpGPU(GPUTensor<Vector> v, CommandBuffer tape, {GPUTensor<Vector>? outTensor})
→ GPUTensor<Vector>
-
Appends the commands for applying the e^v exponential function on vector
v to tape.
-
vectorTanhGPU(GPUTensor<Vector> v, CommandBuffer tape, {GPUTensor<Vector>? outTensor})
→ GPUTensor<Vector>
-
Appends commands for applying the tanh function on every element of vector
t to tape.
