MHACache class

Constructors

MHACache.empty(int numKvHeads)

Properties

hashCode → int

The hash code for this object.

no setter, inherited

k → List<Tensor?>

Per-KV-head running K, shape [T_seen, headDim]. null until the first append.

final

numKvHeads → int

Number of KV-head slots stored. For standard MHA this equals the number of Q heads; for GQA (Llama / Mistral / Qwen) it is smaller than the Q-head count and each cached slot is shared by several Q heads via head-grouping.

final

runtimeType → Type

A representation of the runtime type of the object.

no setter, inherited

seqLen → int

Length of the cached sequence so far (0 if empty). All KV heads share the same T, so we read from KV head 0.

no setter

v → List<Tensor?>

Per-KV-head running V, shape [T_seen, headDim]. null until the first append.

final

Methods

appendK(int kvHead, Tensor newK) → Tensor

Append [N_new, headDim] K and V for a single KV head. Returns the updated concatenated tensor (also stored in the cache).

appendV(int kvHead, Tensor newV) → Tensor

noSuchMethod(Invocation invocation) → dynamic

Invoked when a nonexistent method or property is accessed.

inherited

toString() → String

A string representation of this object.

inherited

Operators

operator ==(Object other) → bool

The equality operator.

inherited