BackendStatePersistence class abstract

Optional backend capability for persisting the KV cache to disk and restoring it later, mirroring llama_state_save_file / llama_state_load_file in llama.cpp. Saving captures the native runtime state of contextHandle together with the token sequence that produced it. Loading restores the native KV cache and returns the saved token sequence; subsequent inference can skip prompt evaluation when callers re-issue a prompt with the restored token prefix and prompt-prefix reuse enabled.

Constructors

BackendStatePersistence()

Properties

hashCode → int

The hash code for this object.

no setter, inherited

runtimeType → Type

A representation of the runtime type of the object.

no setter, inherited

Methods

noSuchMethod(Invocation invocation) → dynamic

Invoked when a nonexistent method or property is accessed.

inherited

stateLoadFile(int contextHandle, String path, int tokenCapacity) → Future<StateLoadResult>

Restores the KV cache of contextHandle from a file previously written by stateSaveFile. tokenCapacity caps how many tokens the caller is willing to receive — typically the context size of the loaded model. Throws if the file is corrupt or was produced by a different llama.cpp build.

stateSaveFile(int contextHandle, String path, List<int> tokens) → Future<bool>

Writes the KV cache state of contextHandle together with the token sequence in tokens to path. The file format is the one llama.cpp emits — opaque, version-tied, and not portable across llama.cpp builds.

toString() → String

A string representation of this object.

inherited

Operators

operator ==(Object other) → bool

The equality operator.

inherited