LlamaChatClient class

Bridges the M.E.AI chat abstractions to a model running through LlamaCppFlutter.

This is the inner client: wrap it with FunctionInvokingChatClient so tool calls it surfaces (as FunctionCallContent) are executed and fed back. All model-family specifics — prompt rendering and the prose/reasoning/ tool-call stream split — live in the injected ChatFormat. The model is loaded by a hosted service and supplied through sessionProvider; this client does not own its lifecycle.

Constructors

LlamaChatClient({required SessionProvider sessionProvider, required ChatFormat format, required int contextSize, ChatFormat? formatResolver()?, SamplingDefaults sampling = const SamplingDefaults(), ImageTiling? imageTiling, PromptInspector? inspector, bool isThinkingEnabled()?, LoggerFactory? loggerFactory})

Properties

contextSize → int

The model's context window in tokens, recorded on each PromptSnapshot so the UI can gauge how full the context is.

final

format → ChatFormat

The model family's prompt rendering and output decoding, used when formatResolver is absent or returns null.

final

formatResolver → ChatFormat? Function()?

Optional late-bound format, read per request after the session resolves. This lets a host defer the choice until the model file is actually on disk — e.g. picking the format from the GGUF's embedded chat template during load — instead of guessing from the file name at construction time. Returning null falls back to format.

final

hashCode → int

The hash code for this object.

no setter, inherited

imageTiling → ImageTiling?

When set, each attached image is split into overlapping crops before prompt rendering (see ImageTiling), boosting vision fidelity for OCR and dense documents at the cost of one prefill per crop. Null passes images through whole.

final

inspector → PromptInspector?

Optional sink that records each rendered prompt and its resolved sampling config so the UI can show exactly what was sent to the model.

final

isThinkingEnabled → bool Function()?

Reads whether to request the family's reasoning channel, evaluated per request so a runtime toggle takes effect on the next turn. Null means thinking is always off. The result is still gated on ChatFormat.supportsThinking.

final

runtimeType → Type

A representation of the runtime type of the object.

no setter, inherited

sampling → SamplingDefaults

Generation defaults used when the per-request ChatOptions doesn't override them.

final

sessionProvider → SessionProvider

Resolves the ready session; the caller owns its lifecycle.

final

Methods

dispose() → void

Performs application-defined tasks associated with freeing, releasing, or resetting unmanaged resources.

getResponse({required Iterable<ChatMessage> messages, ChatOptions? options, CancellationToken? cancellationToken}) → Future<ChatResponse>

Sends a chat request and returns the complete response.

getService<T>({Object? key}) → T?

Gets a service of the specified type.

inherited

getStreamingResponse({required Iterable<ChatMessage> messages, ChatOptions? options, CancellationToken? cancellationToken}) → Stream<ChatResponseUpdate>

Sends a chat request and returns a stream of response updates.

noSuchMethod(Invocation invocation) → dynamic

Invoked when a nonexistent method or property is accessed.

inherited

toString() → String

A string representation of this object.

inherited

Operators

operator ==(Object other) → bool

The equality operator.

inherited