InferenceChat class - chat library

Constructors

InferenceChat({required Future<InferenceModelSession> sessionCreator()?, required int maxTokens, int tokenBuffer = 2000, bool supportImage = false, bool supportAudio = false, bool supportsFunctionCalls = false, int maxFunctionBufferLength = defaultMaxFunctionBufferLength, List<Tool> tools = const [], ModelType modelType = ModelType.gemmaIt, bool isThinking = false, ModelFileType fileType = ModelFileType.task, ToolChoice toolChoice = ToolChoice.auto, String? systemInstruction})

Properties

currentTokens → int: no setter
fileType → ModelFileType: final
fullHistory → List<Message>: no setter
hashCode → int: The hash code for this object.
no setterinherited
imageMessageCount → int: no setter
isThinking → bool: final
maxFunctionBufferLength → int: final
maxTokens → int: final
modelType → ModelType: final
runtimeType → Type: A representation of the runtime type of the object.
no setterinherited
session ↔ InferenceModelSession: getter/setter pair
sessionCreator → Future<InferenceModelSession> Function()?: final
supportAudio → bool: final
supportImage → bool: final
supportsFunctionCalls → bool: final
supportsImages → bool: no setter
tokenBuffer → int: final
toolChoice → ToolChoice: final
tools → List<Tool>: Tools available to this chat, fixed for the chat's lifetime. A cached unmodifiable VIEW over _tools (allocated once, reflects the backing list) — tools.isNotEmpty is read once per generated token on the streaming path, so a per-read List.unmodifiable(_tools) copy would be an O(tools) allocation on the hottest generation loop.
latefinal

Methods

addQuery(Message message) → Future<void>
addQueryChunk(Message message, [bool noTool = false, bool prefix = false]) → Future<void>
clearHistory({List<Message>? replayHistory}) → Future<void>
close() → Future<void>
createToolsPrompt() → String: Creates tools prompt based on model type and tool choice. Made package-private for testing.
generateChatResponse() → Future<ModelResponse>
generateChatResponseAsync() → Stream<ModelResponse>: Streams this turn's response token-by-token.
generateChatResponseWithTools({required FutureOr<Map<String, dynamic>> onToolCall(FunctionCallResponse call), int maxToolTurns = 8, bool isCancelled()?, void onMaxToolTurns()?}) → Stream<ModelResponse>: Drive flutter_gemma's function-calling loop to completion. Stream this turn's text/thinking tokens; whenever the model calls a tool, run onToolCall and feed its result back as a tool-response message, then continue — until a turn has no calls (the model's final answer) or maxToolTurns / isCancelled stops it.
generateContent(List<ChatMessage> prompt) → Future<ChatMessage>: Available on InferenceChat, provided by the GenAiChat extension
STATEFUL batch: stage the whole list into THIS chat, then generate once.
generateContentStream(List<ChatMessage> prompt) → Stream<ChatMessage>: Available on InferenceChat, provided by the GenAiChat extension
initSession() → Future<void>
noSuchMethod(Invocation invocation) → dynamic: Invoked when a nonexistent method or property is accessed.
inherited
sendMessage(ChatMessage message) → Future<ChatMessage>: Available on InferenceChat, provided by the GenAiChat extension
Send one turn; returns the model turn as a role:model ChatMessage (text + tool calls + thinking as parts).
sendMessageStream(ChatMessage message) → Stream<ChatMessage>: Available on InferenceChat, provided by the GenAiChat extension
Streaming variant — partial role:model ChatMessages, one per delta.
stopGeneration() → Future<void>
toString() → String: A string representation of this object.
inherited

Operators

operator ==(Object other) → bool: The equality operator.
inherited