TransformerBlock class
TransformerBlock implements a single transformer block with pre-layer normalization.
Constructors
- TransformerBlock(int dModel, int nHeads, {int ffnMult = 4, bool rope = false})
- Creates a TransformerBlock.
Properties
- ffn → FeedForward
-
Feed-forward network layer.
final
- hashCode → int
-
The hash code for this object.
no setterinherited
- ln1 → LayerNorm
-
First layer normalization.
final
- ln2 → LayerNorm
-
Second layer normalization.
final
- mha → MultiHeadAttention
-
Multi-head attention layer (optionally with RoPE positional encoding).
final
- runtimeType → Type
-
A representation of the runtime type of the object.
no setterinherited
Methods
-
forward(
Tensor2D x, {KVCache? cache}) → Tensor2D -
Runs the transformer block forward pass.
xis the input tensor.cacheis an optional key-value cache for attention. Returns the output tensor after attention and feed-forward layers. -
noSuchMethod(
Invocation invocation) → dynamic -
Invoked when a nonexistent method or property is accessed.
inherited
-
toString(
) → String -
A string representation of this object.
inherited
Operators
-
operator ==(
Object other) → bool -
The equality operator.
inherited