TransformerEncoder class

A complete Transformer Encoder model.

This model takes a sequence of token indices and produces contextualized embeddings for each token. It does not have a language model head for next-token prediction, unlike the decoder-only Transformer.

Inheritance

• Object
• Module
• TransformerEncoder

Constructors

TransformerEncoder({int vocabSize = 100, int embedSize = 64, int blockSize = 128, int numLayers = 6, int numHeads = 8})

Properties

blocks → List<TransformerEncoderBlock>

final

blockSize → int

final

embedSize → int

final

finalLayerNorm → LayerNorm

final

hashCode → int

The hash code for this object.

no setter, inherited

numHeads → int

final

numLayers → int

final

positionEmbeddings → List<ValueVector>

final

runtimeType → Type

A representation of the runtime type of the object.

no setter, inherited

tokenEmbeddings → List<ValueVector>

final

vocabSize → int

final

Methods

forward(List<int> idx) → List<ValueVector>

Standard forward pass for token IDs (used in NLP tasks).

forwardEmbeddings(List<ValueVector> embeddedInputs) → List<ValueVector>

NEW: Forward pass for already pre-computed embeddings (used in ViT).

noSuchMethod(Invocation invocation) → dynamic

Invoked when a nonexistent method or property is accessed.

inherited

parameters() → List<Value>

override

toString() → String

A string representation of this object.

inherited

zeroGrad() → void

inherited

Operators

operator ==(Object other) → bool

The equality operator.

inherited