LocalLlamaProvider class

Local in-process GGUF LLM inference provider for Neomage.

Implements SAIA's saia.LocalAiService interface. Integrates with the HardwareProfiler from neom_ollama to dynamically profile device resources at runtime and optimize context size/thread settings.

Constructors

LocalLlamaProvider()

Properties

hashCode → int

The hash code for this object.

no setter, inherited

isReady → bool

Whether a local model is loaded and ready for inference.

no setter, override

modelInfo → Map<String, dynamic>

Returns metadata about the model (name, size, quantization).

no setter, override

runtimeType → Type

A representation of the runtime type of the object.

no setter, inherited

Methods

chat(List<Map<String, String>> messages, {int maxTokens = 256}) → Future<String>

Generates a response with conversation history context.

override

dispose() → Future<void>

Releases the local model from memory to free up resources.

override

generate(String prompt, {int maxTokens = 256}) → Future<String>

Generates a response from a prompt (single turn, no conversation history).

override

initialize({String? modelPath}) → Future<bool>

Loads the local model.

override

noSuchMethod(Invocation invocation) → dynamic

Invoked when a nonexistent method or property is accessed.

inherited

toString() → String

A string representation of this object.

inherited

Operators

operator ==(Object other) → bool

The equality operator.

inherited