LlamaSession class final

In-RAM conversation state on top of a LlamaContext.

Owns the token history for one sequence id and the corresponding KV cache cursor. Disk tiering and serialization come in M6.

Constructors

LlamaSession(LlamaContext context, {int seqId = 0})

Properties

context → LlamaContext
final
hashCode → int
The hash code for this object.
no setterinherited
hasPendingPrompt → bool
True when there are tokens in tokens that have not been decoded yet.
no setter
kvHead → int
Position one past the last token committed to the KV cache. Tokens at index >= kvHead need prefill before sampling.
no setter
runtimeType → Type
A representation of the runtime type of the object.
no setterinherited
seqId → int
final
tokenCount → int
Number of tokens in the conversation history.
no setter
tokenizer → Tokenizer
final
tokens → List<int>
Full token history (prompt + generated).
no setter

Methods

appendText(String text, {bool addSpecial = false, bool parseSpecial = true}) → void
Encode text and append the resulting tokens to the conversation. They become the next prefill input.
appendTokens(List<int> ids) → void
Append already-tokenized ids to the conversation.
captureRawState() → Uint8List
Snapshot the raw KV state for this sequence. The bytes are opaque to the wrapper — pass them to restoreRawState later (after model load) to resume.
captureRawStateExt({required StateSeqFlags flags}) → Uint8List
Like captureRawState, but uses the _ext form so the caller can request partial-only state (StateSeqFlags.partialOnly) or keep the snapshot on-device (StateSeqFlags.onDevice).
clear() → void
Drop all tokens and reset the KV cache for this session's sequence.
generate({SamplerParams sampler = const SamplerParams(), int maxTokens = 256, ContextShiftPolicy shiftPolicy = ContextShiftPolicy.off, ContextShift shift = ContextShift.defaults}) → Stream<GenerationEvent>
Generate tokens until stop. Performs prefill of any pending tokens.
noSuchMethod(Invocation invocation) → dynamic
Invoked when a nonexistent method or property is accessed.
inherited
restoreRawState(Uint8List bytes, List<int> tokens, int kvHead) → void
Apply a previously captured KV blob to this session's sequence and replace the in-memory token history with tokens. The caller is responsible for verifying that bytes came from a compatible model.
restoreRawStateExt(Uint8List bytes, List<int> tokens, int kvHead, {required StateSeqFlags flags}) → void
_ext counterpart to restoreRawState. Passes flags through to llama_state_seq_set_data_ext. bytes must have been produced by captureRawStateExt with a compatible flag set.
shiftContext({required int nKeep, int? nDiscard}) → int
Drop the oldest non-keep window from the KV cache and slide the remainder left, matching llama-server's --context-shift behaviour.
toString() → String
A string representation of this object.
inherited

Operators

operator ==(Object other) → bool
The equality operator.
inherited