local_ai_llama_cpp library

llama.cpp adapter for LocalAI Kit.

Runs any GGUF model through llama.cpp behind the core LocalLlm and LocalEmbedding interfaces. All llama_cpp_dart types stay inside this package (architecture §2 rule 2) and every native call happens in a worker isolate.

final ai = await LocalAI.initialize(
  const LocalAIConfig(llm: LlmConfig(modelId: 'qwen-2.5-0.5b-instruct-gguf')),
  plugins: const [LlamaCppAdapterPlugin()],
);

Classes

BackendPlan: llama.cpp load parameters derived from a RuntimePreference.
BackendSelection: Chooses llama.cpp load parameters for a RuntimePreference.
ChatTemplate: Renders chat histories into a prompt string for a LlamaChatFormat.
ContextWindow: Sliding-window truncation over a chat history.
ContextWindowResult: Result of applying the context window to a request.
EmbeddingVectors: Vector helpers used by LlamaCppEmbeddingAdapter.
GgufLocator: Locates .gguf weights under models/<type>/<modelId>/.
JsonSchemaToGbnf: Translates the JsonSchema subset the kit supports into GBNF.
LlamaCppAdapterPlugin: Registers the llama.cpp LLM and embedding factories under the llama-cpp provider key.
LlamaCppEmbeddingAdapter: Embeds text with an embedding-mode GGUF model (nomic-embed, bge, e5, …).
LlamaCppLlmAdapter: LocalLlm implementation backed by llama.cpp, for any GGUF model.
LlamaCppRuntime: Global llama.cpp runtime configuration.
PromptPlan: What the worker should feed llama.cpp for the next request.
PromptPlanner: Decides between continuing and resetting the cached context.
SamplerSpec: Sampling configuration for one request.
StopScanResult: Text that can be emitted now, plus whether generation should stop.
StopSequenceScanner: Incremental scanner for a fixed set of stop sequences.

Enums

LlamaChatFormat: Turn-marker dialects this adapter knows how to render.
LlamaStopReason: Why a generation ended, mirroring LlmFinishReason without importing it into the isolate protocol.

local_ai_llama_cpp library

Classes

Enums

local_ai_llama_cpp package

local_ai_llama_cpp library