KVCache class

Key-value cache for auto-regressive decoding.

Stores previously computed key/value tensors for each layer to avoid recomputation during token-by-token generation.

Constructors

KVCache(int numLayers)

Properties

crossAttnCache → List<(Tensor, Tensor)?>

final

hashCode → int

The hash code for this object.

no setter, inherited

runtimeType → Type

A representation of the runtime type of the object.

no setter, inherited

selfAttnCache → List<(Tensor, Tensor)?>

final

Methods

noSuchMethod(Invocation invocation) → dynamic

Invoked when a nonexistent method or property is accessed.

inherited

toString() → String

A string representation of this object.

inherited

Operators

operator ==(Object other) → bool

The equality operator.

inherited