A loaded llama-family model session.
Implementations must yield text with stop sequences removed and terminate at the first stop sequence.
Concurrency contract
A session runs at most one generation at a time, regardless of LlamaSessionCapabilities.maxSequences — sequences are independent KV caches sharing one loaded model, not parallel decode lanes. Calling generate while a run is in flight supersedes it: the engine cancels the current run and starts the newest call once cancellation completes (an intermediate superseded call's stream closes without emitting text). Callers that care about the superseded run's output should cancel and drain its stream before starting the next run.
- Available extensions
Properties
- capabilities → LlamaSessionCapabilities
-
What this session can do beyond generating (state persistence, state
size reporting).
no setter
- hashCode → int
-
The hash code for this object.
no setterinherited
- runtimeType → Type
-
A representation of the runtime type of the object.
no setterinherited
Methods
-
cancel(
) → Future< void> - Requests cancellation of the in-flight generation, if any.
-
clearSequence(
int sequenceId) → Future< void> - Erases one sequence's KV cache and prompt ledger. A no-op on engines without sequence support.
-
dispose(
) → Future< void> - Releases resources held by this session.
-
dropStashedState(
String key) → Future< int> -
Drops the stash entry under
key; returns the bytes freed (0when no entry existed or stashing is unsupported). -
generate(
String prompt, {int maxTokens = 256, double temperature = 0.8, int? topK, double? topP, int? seed, List< String> stopSequences = const <String>[], List<Uint8List> ? media, List<LlamaChatTurn> ? turns, int sequenceId = 0, LlamaStatsCallback? onStats}) → Stream<String> -
Generates text for
prompt, yielding decoded pieces as they arrive. -
generateEvents(
String prompt, {int maxTokens = 256, double temperature = 0.8, int? topK, double? topP, int? seed, List< String> stopSequences = const <String>[], List<Uint8List> ? media, List<LlamaChatTurn> ? turns, int sequenceId = 0}) → Stream<LlamaGenerationEvent> -
Available on LlamaSession, provided by the LlamaSessionEvents extension
Generates text forpromptas a stream of typed events instead of raw strings. -
loadState(
String path, {int sequenceId = 0}) → Future< int> -
Restores KV-cache state previously written by saveState into
sequenceId, replacing that sequence's current cache. -
noSuchMethod(
Invocation invocation) → dynamic -
Invoked when a nonexistent method or property is accessed.
inherited
-
restoreStashedState(
String key, {int sequenceId = 0}) → Future< int> -
Restores stashed state saved under
keyintosequenceId, replacing that sequence's cache. The entry is kept until dropStashedState. -
saveState(
String path, {int sequenceId = 0}) → Future< int> -
Saves one sequence's KV-cache state to
path. -
setImageTokenBudget(
int? imageTokenBudget) → Future< void> - Changes the vision encoder's per-image token budget for subsequent image turns; null restores the model's metadata default.
-
stashState(
String key, {int sequenceId = 0}) → Future< LlamaStashResult> -
Copies one sequence's KV state into an engine-side, in-memory stash
under
key— the fast swap path (no disk, no serialization). -
stateSizeBytes(
{int sequenceId = 0}) → Future< int> - Measures the byte size of one sequence's current state (KV cache plus bookkeeping) without writing it anywhere.
-
toString(
) → String -
A string representation of this object.
inherited
Operators
-
operator ==(
Object other) → bool -
The equality operator.
inherited