TransformerBlock class
Constructors
- TransformerBlock(int embedDim, int numHeads, {int? ffnDim, double dropoutP = 0.0, bool attnBias = false, Activation activation = Activation.relu, Device device = Device.CPU, int seed = 0})
Properties
- activation → Activation
-
final
- dropout → Dropout
-
final
- embedDim → int
-
final
- ffn1 → Linear
-
final
- ffn2 → Linear
-
final
- ffnDim → int
-
final
- hashCode → int
-
The hash code for this object.
no setterinherited
- ln1 → LayerNorm
-
final
- ln2 → LayerNorm
-
final
- mha → MultiHeadAttention
-
final
- numHeads → int
-
final
- runtimeType → Type
-
A representation of the runtime type of the object.
no setterinherited
- training ↔ bool
-
Whether this module is in training mode. Layers that behave
differently between training and inference (e.g.
Dropout) read this flag in theircallmethod. Defaults to training mode.getter/setter pairinherited
Methods
-
call(
Tensor x, {Tensor? mask, MHACache? cache}) → Tensor -
eval(
) → void -
Put this module (and any registered submodules) into evaluation mode.
inherited
-
noSuchMethod(
Invocation invocation) → dynamic -
Invoked when a nonexistent method or property is accessed.
inherited
-
parameters(
) → List< Tensor> -
Trainable tensors owned by this module (and its submodules).
override
-
submodules(
) → List< Module> -
Submodules owned by this module. Subclasses that compose other
modules should override this so
train()/eval()propagate. Default: empty.override -
toString(
) → String -
A string representation of this object.
inherited
-
train(
) → void -
Put this module (and any registered submodules) into training mode.
inherited
-
zeroGrad(
) → void -
Zero every parameter's gradient. Safe to call before each
backward.inherited
Operators
-
operator ==(
Object other) → bool -
The equality operator.
inherited