chat library
Chat-format internals: per-family prompt templates, stream decoders, tool-call parsing, the LlamaChatClient implementation, prompt diagnostics, and the token-stream transformers (StopSequenceFilter, TokenSmoother).
Most apps only need resolveChatFormat/detectChatFormatNameForGguf
from the main package:llama_cpp_flutter/llama_cpp_flutter.dart
entrypoint; import this one to construct or extend formats directly —
e.g. registering a custom family with registerChatFormat, tuning a
built-in template, or reusing the decoders.
Classes
- ChatFormat
- Pairs prompt rendering with output decoding for one model family.
- ChatmlChatFormat
-
ChatML's ChatFormat: ChatmlChatTemplate rendering paired with a
MarkedToolCallDecoder that buffers on
<tool_call>. - ChatmlChatTemplate
- Renders ChatML prompts from M.E.AI chat messages and tool declarations.
- GemmaChatFormat
- Gemma 4's ChatFormat: GemmaChatTemplate rendering paired with GemmaStreamDecoder output splitting.
- GemmaChatTemplate
- Renders Gemma 4 prompts from M.E.AI chat messages and tool declarations.
- GemmaPrompt
- A rendered Gemma prompt plus the stop sequences generation should halt on.
- GemmaStreamDecoder
-
Turns the
Stream<String>fromLlamaCppFlutter.generateinto a stream ofChatResponseUpdates for an M.E.AI chat client. - GemmaTurn
- The parsed result of one generated model turn.
- Lfm2ChatFormat
- LFM's ChatFormat: Lfm2ChatTemplate rendering paired with Lfm2StreamDecoder output splitting.
- Lfm2ChatTemplate
- Renders Liquid LFM prompts from M.E.AI chat messages and tool declarations.
- Lfm2Prompt
- A rendered LFM prompt plus the stop sequences generation should halt on.
- Lfm2StreamDecoder
-
Turns the
Stream<String>fromLlamaCppFlutter.generateinto a stream ofChatResponseUpdates for an M.E.AI chat client. - Lfm2Turn
- The parsed result of one generated model turn.
- Llama3ChatFormat
-
Llama 3's ChatFormat: Llama3ChatTemplate rendering paired with a
MarkedToolCallDecoder that buffers on
<|python_tag|>. - Llama3ChatTemplate
- Renders Llama 3 prompts from M.E.AI chat messages and tool declarations.
- LlamaChatClient
-
Bridges the M.E.AI chat abstractions to a model running through
LlamaCppFlutter. - MarkedToolCallDecoder
- Splits a raw token stream into Microsoft.Extensions.AI updates for any family without a reasoning channel.
- MistralChatFormat
-
Mistral's ChatFormat: MistralChatTemplate rendering paired with a
MarkedToolCallDecoder that buffers on
[TOOL_CALLS]. - MistralChatTemplate
- Renders Mistral prompts from M.E.AI chat messages and tool declarations.
- ParsedTurn
- Prose plus any tool calls extracted from a single generated model turn.
- PromptInspector
- Holds the most recently rendered PromptSnapshot and notifies listeners when it changes.
- PromptSnapshot
- An immutable record of one rendered prompt and the sampling configuration it was generated with.
- QwenChatFormat
-
Qwen's ChatFormat: QwenChatTemplate rendering paired with
QwenStreamDecoder, which surfaces the
<think>reasoning channel. - QwenChatTemplate
- Renders Qwen prompts from M.E.AI chat messages and tool declarations.
- QwenStreamDecoder
-
Turns the
Stream<String>from the runtime intoChatResponseUpdates, separating three kinds of content: - RenderedPrompt
- A rendered prompt ready for a text-completion inference engine.
- StopSequenceFilter
- Filters a text stream so generation stops at the first stop sequence.
- TokenSmoother
- Re-paces a token stream into a steady, typewriter-style character stream.
Enums
- LfmToolCallSyntax
- How assistant tool-call bodies are rendered.
- LfmToolTagStyle
- How Liquid-family prompts wrap tool declarations and tool responses.
Extensions
-
SmoothedStream
on Stream<
String> - Convenience methods for smoothing text streams.
Constants
- defaultChatFormatName → const String
-
The name used when
llama.formatis unset, for backwards compatibility.
Properties
-
supportedChatFormatNames
→ Set<
String> -
The set of accepted
llama.formatnames (an empty string is also accepted and resolves to defaultChatFormatName).no setter
Functions
-
chatTurnsFromMessages(
Iterable< ChatMessage> messages) → List<LlamaChatTurn> -
Builds the engine-neutral structured view of
messagespassed to LlamaSession.generate alongside the rendered prompt for image turns. -
detectChatFormatName(
{String? architecture, String? chatTemplate}) → String? - Maps GGUF metadata values to a registered chat-format name, or null when neither signal identifies a known family.
-
detectChatFormatNameForGguf(
Uint8List headerPrefix) → String? -
Detects the chat-format name for the GGUF whose first bytes are
headerPrefix, or null when the header is unreadable or matches no known family. -
messagesWithInstructions(
Iterable< ChatMessage> messages, String? instructions) → List<ChatMessage> -
Returns
messageswithinstructionsmaterialized as the leading system message. -
registerChatFormat(
String name, ChatFormat format) → void -
Registers
formatundernameso resolveChatFormat can produce it. -
resolveChatFormat(
String? name) → ChatFormat? -
Resolves
nameto a ChatFormat, ornullwhennameis unknown.
Typedefs
- AtomicChunkPredicate = bool Function(String chunk)
-
Whether
chunkshould be released whole rather than paced grapheme by grapheme. The default treats every chunk as ordinary text. -
SessionProvider
= Future<
LlamaSession> Function() - Resolves the loaded LlamaSession this client generates against.