llama_kv_cache_view class final
An updateable view of the KV cache.
- Inheritance
- Implemented types
- Available extensions
Constructors
Properties
-
cells
↔ Pointer<
llama_kv_cache_view_cell> -
Information for an individual cell.
getter/setter pair
-
cells_sequences
↔ Pointer<
llama_seq_id> -
The sequences for each cell. There will be n_max_seq items per cell.
getter/setter pair
- hashCode → int
-
The hash code for this object.
no setterinherited
- max_contiguous ↔ int
-
Maximum contiguous empty slots in the cache.
getter/setter pair
- max_contiguous_idx ↔ int
-
Index to the start of the max_contiguous slot range. Can be negative
when cache is full.
getter/setter pair
- n_cells ↔ int
-
Number of KV cache cells. This will be the same as the context size.
getter/setter pair
- n_max_seq ↔ int
-
Maximum number of sequences that can exist in a cell. It's not an error
if there are more sequences in a cell than this value, however they will
not be visible in the view cells_sequences.
getter/setter pair
- runtimeType → Type
-
A representation of the runtime type of the object.
no setterinherited
- token_count ↔ int
-
Number of tokens in the cache. For example, if there are two populated
cells, the first with 1 sequence id in it and the second with 2 sequence
ids then you'll have 3 tokens.
getter/setter pair
- used_cells ↔ int
-
Number of populated cache cells.
getter/setter pair
Methods
-
noSuchMethod(
Invocation invocation) → dynamic -
Invoked when a nonexistent method or property is accessed.
inherited
-
toString(
) → String -
A string representation of this object.
inherited
Operators
-
operator ==(
Object other) → bool -
The equality operator.
inherited