FfiInferenceModel class
- Inheritance
-
- Object
- InferenceModel
- FfiInferenceModel
Constructors
- FfiInferenceModel({required LiteRtLmFfiClient ffiClient, required int maxTokens, required ModelType modelType, ModelFileType fileType = ModelFileType.litertlm, PreferredBackend? activeBackend, bool supportImage = false, bool supportAudio = false, int? maxConcurrentSessions, required VoidCallback onClose})
Properties
- activeBackend → PreferredBackend?
-
Backend that the runtime initialized for this model, when known.
no setteroverride
- chat ↔ InferenceChat?
-
getter/setter pairinherited
- fileType → ModelFileType
-
no setteroverride
- hashCode → int
-
The hash code for this object.
no setterinherited
- maxTokens → int
-
no setteroverride
- runtimeType → Type
-
A representation of the runtime type of the object.
no setterinherited
- session → InferenceModelSession?
-
The single session created via createSession. Singleton lane —
each createSession call overwrites this field with a new session
and closes the previous one.
no setteroverride
-
sessions
→ List<
InferenceModelSession> -
Live sessions owned by this model — union of the legacy session
(if any) and every active openSession result. Returns an
unmodifiable view; mutate via openSession,
session.close(), or close.no setterinherited
Methods
-
close(
) → Future< void> -
override
-
createChat(
{double temperature = .8, int randomSeed = 1, int topK = 1, double? topP, int tokenBuffer = 256, String? loraPath, bool? supportImage, bool? supportAudio, List< Tool> tools = const [], bool? supportsFunctionCalls, bool isThinking = false, ModelType? modelType, ToolChoice toolChoice = ToolChoice.auto, int? maxFunctionBufferLength, String? systemInstruction}) → Future<InferenceChat> -
inherited
-
createSession(
{double temperature = .8, int randomSeed = 1, int topK = 1, double? topP, String? loraPath, bool? enableVisionModality, bool? enableAudioModality, String? systemInstruction, bool enableThinking = false, List< Tool> tools = const []}) → Future<InferenceModelSession> -
Creates a new InferenceModelSession for generation.
override
-
noSuchMethod(
Invocation invocation) → dynamic -
Invoked when a nonexistent method or property is accessed.
inherited
-
openChat(
{double temperature = .8, int randomSeed = 1, int topK = 1, double? topP, int tokenBuffer = 256, String? loraPath, bool? supportImage, bool? supportAudio, List< Tool> tools = const [], bool? supportsFunctionCalls, bool isThinking = false, ModelType? modelType, ToolChoice toolChoice = ToolChoice.auto, int? maxFunctionBufferLength, String? systemInstruction}) → Future<InferenceChat> -
Same as createChat, but uses openSession internally so the
resulting chat owns an independent session that does not touch the
legacy session field or other open chats. Use this when you need
concurrent chats on a single loaded model.
inherited
-
openSession(
{double temperature = .8, int randomSeed = 1, int topK = 1, double? topP, String? loraPath, bool? enableVisionModality, bool? enableAudioModality, String? systemInstruction, bool enableThinking = false, List< Tool> tools = const []}) → Future<InferenceModelSession> -
Opens a new session detached from session. Each call returns a
fresh independent session sharing the loaded model weights but with
isolated context (history / KV cache). Use this for concurrent
dialogues on a single loaded model.
inherited
-
toString(
) → String -
A string representation of this object.
inherited
Operators
-
operator ==(
Object other) → bool -
The equality operator.
inherited