EmbeddedLlmDriver class abstract
Abstract contract for on-device GGUF / local embedded LLM engines.
Implemented by companion packages like flutter_ui_mcp_embedded
using native FFI bindings (llamadart / llama.cpp / WebGPU).
Constructors
Properties
- hashCode → int
-
The hash code for this object.
no setterinherited
- isLoaded → bool
-
Whether an embedded model is currently initialized and ready for inference.
no setter
- loadedModelPath → String?
-
The file path or URL of the currently loaded model, if any.
no setter
- runtimeType → Type
-
A representation of the runtime type of the object.
no setterinherited
Methods
-
dispose(
) → Future< void> - Release resources and unload the model.
-
generateResponse(
{required List< ChatMessage> messages, List<MCPTool> ? availableTools, double? temperature, int? maxTokens, int? topK, double? topP, double? penalty, void onStreamChunk(String chunk)?}) → Future<LLMResponse> - Generate a completion or streaming response using the loaded model.
-
getVramInfo(
) → Future< ({int free, int total})> - Total and free VRAM in bytes if GPU is supported.
-
initialize(
String modelPath, {int gpuLayers = 0, int contextSize = 4096, void onProgress(double progress)?}) → Future< void> - Initialize and load a GGUF model into memory.
-
isGpuSupported(
) → Future< bool> - Whether GPU acceleration is supported on the current host.
-
noSuchMethod(
Invocation invocation) → dynamic -
Invoked when a nonexistent method or property is accessed.
inherited
-
toString(
) → String -
A string representation of this object.
inherited
Operators
-
operator ==(
Object other) → bool -
The equality operator.
inherited