InferenceModel class abstract

Represents an LLM model instance.

Implementers

Constructors

InferenceModel()

Properties

activeBackend → PreferredBackend?
Backend that the runtime initialized for this model, when known.
no setter
chat ↔ InferenceChat?
getter/setter pair
fileType → ModelFileType
no setter
hashCode → int
The hash code for this object.
no setterinherited
maxTokens → int
no setter
runtimeType → Type
A representation of the runtime type of the object.
no setterinherited
session → InferenceModelSession?
The single session created via createSession. Singleton lane — each createSession call overwrites this field with a new session and closes the previous one.
no setter
sessions → List<InferenceModelSession>
Live sessions owned by this model — union of the legacy session (if any) and every active openSession result. Returns an unmodifiable view; mutate via openSession, session.close(), or close.
no setter

Methods

close() → Future<void>
createChat({double temperature = .8, int randomSeed = 1, int topK = 1, double? topP, int tokenBuffer = 256, String? loraPath, bool? supportImage, bool? supportAudio, List<Tool> tools = const [], bool? supportsFunctionCalls, bool isThinking = false, ModelType? modelType, ToolChoice toolChoice = ToolChoice.auto, int? maxFunctionBufferLength, String? systemInstruction}) → Future<InferenceChat>
createSession({double temperature = .8, int randomSeed = 1, int topK = 1, double? topP, String? loraPath, bool? enableVisionModality, bool? enableAudioModality, String? systemInstruction, bool enableThinking = false, List<Tool> tools = const []}) → Future<InferenceModelSession>
Creates a new InferenceModelSession for generation.
noSuchMethod(Invocation invocation) → dynamic
Invoked when a nonexistent method or property is accessed.
inherited
openChat({double temperature = .8, int randomSeed = 1, int topK = 1, double? topP, int tokenBuffer = 256, String? loraPath, bool? supportImage, bool? supportAudio, List<Tool> tools = const [], bool? supportsFunctionCalls, bool isThinking = false, ModelType? modelType, ToolChoice toolChoice = ToolChoice.auto, int? maxFunctionBufferLength, String? systemInstruction}) → Future<InferenceChat>
Same as createChat, but uses openSession internally so the resulting chat owns an independent session that does not touch the legacy session field or other open chats. Use this when you need concurrent chats on a single loaded model.
openSession({double temperature = .8, int randomSeed = 1, int topK = 1, double? topP, String? loraPath, bool? enableVisionModality, bool? enableAudioModality, String? systemInstruction, bool enableThinking = false, List<Tool> tools = const []}) → Future<InferenceModelSession>
Opens a new session detached from session. Each call returns a fresh independent session sharing the loaded model weights but with isolated context (history / KV cache). Use this for concurrent dialogues on a single loaded model.
toString() → String
A string representation of this object.
inherited

Operators

operator ==(Object other) → bool
The equality operator.
inherited