modelport_llamacpp library

llama.cpp adapter for ModelPort.

await ModelPortFlutter.init(adapters: [LlamaCppAdapter()]);
final llm = await TextGenerator.load('hf://org/qwen2.5-0.5b-instruct');
await for (final piece in llm.chat([ChatMessage.user('Hi!')])) {
  stdout.write(piece);
}

On Android, native libraries must be extracted from the APK so llama.cpp can find its CPU backends. Add this to android/app/build.gradle.kts:

android {
    packaging { jniLibs { useLegacyPackaging = true } }
}

Classes

LlamaCppAdapter

Runs llamacpp variants (GGUF files) through llm_llamacpp.

LlamaCppSession

A GGUF model loaded with llama.cpp.

Functions

describeLoadError(Object error, String variantId) → ModelPortException

Turns llama.cpp load failures into errors that say how to fix them.

toGenerationOptions(GenerationConfig config) → GenerationOptions

Converts ModelPort sampling settings to llm_llamacpp's options.

toLlmMessage(ChatMessage message) → LLMMessage

Converts a ModelPort chat message to llm_llamacpp's type.