ai library
AI toolkit for PhoenixDB: embeddings, LLM clients, retrieval-augmented generation, semantic caching and conversation memory, all stored in on-device PhoenixCollections.
import 'package:phoenixdb/ai.dart';
import 'package:phoenixdb/phoenixdb.dart';
final kb = await AsyncPhoenixCollection.open('kb', dimensions: 1024);
final rag = RagPipeline(
store: kb,
embedder: OpenAICompatibleEmbedder.voyage(
apiKey: voyageKey, model: 'voyage-3.5', dimensions: 1024),
chat: AnthropicChatModel(apiKey: anthropicKey),
);
await rag.ingest(RagDocument('faq', faqText, metadata: {'lang': 'en'}));
final answer = await rag.ask('How do I reset my password?');
print(answer.text); // cites sources as [1], [2]…
print(answer.citations.map((s) => s.documentId));
- Chat models — AnthropicChatModel (Claude, via the Messages API) and OpenAICompatibleChatModel (OpenAI, Ollama, Gemini, vLLM, LM Studio, Mistral, Groq, OpenRouter, …), behind one ChatModel interface with streaming.
- Embedders — OpenAICompatibleEmbedder (OpenAI, Voyage AI, Ollama, …), the offline HashingEmbedder, and CachedEmbedder, which persists vectors in a PhoenixDatabase so no text is embedded twice.
- Retrieval — TextChunker and RagPipeline: hybrid vector + BM25 retrieval with MMR, grounded answers with numbered citations.
- Tools and agents — Tool lets a model call your code, and Agent runs the loop: ask, execute, feed the results back, repeat. Pair it with knowledgeBaseTool and the model can search on-device data itself.
- Caching and memory — SemanticCache reuses answers to prompts that mean the same thing; ConversationMemory keeps recent turns and recalls relevant older ones.
Network clients use dart:io, so this library runs on the Dart VM and
Flutter's native platforms (not the web — neither does PhoenixDB).
Classes
- Agent
- Runs a model in a loop, executing the tools it asks for.
- AgentRun
- The outcome of Agent.run.
- AgentStep
- One tool the agent ran.
- AnthropicChatModel
- A ChatModel backed by Claude.
- CachedChatModel
- A ChatModel that consults a SemanticCache before calling the model.
- CachedEmbedder
- Caches another embedder's vectors, in memory and optionally in a PhoenixDatabase, so a text is embedded (and paid for) once.
- ChatMessage
- One conversation turn.
- ChatModel
- A chat-completion model.
- ChatResponse
- A complete model response.
- ChatUsage
- Token accounting for one response.
- ConversationMemory
- Persists a conversation and assembles its context for the next turn.
- Embedder
- Turns texts into fixed-size vectors.
- HashingEmbedder
- A deterministic, offline embedder based on feature hashing of words, word bigrams and character trigrams.
- OpenAICompatibleChatModel
- A ChatModel for any OpenAI-compatible chat-completions server.
- OpenAICompatibleEmbedder
-
An Embedder for any OpenAI-compatible
/embeddingsendpoint. - RagAnswer
- A generated answer with its evidence.
- RagDocument
- A document to ingest.
- RagPipeline
- Ingests documents into a DocumentStore and answers questions from them.
- RagSource
- A retrieved chunk, numbered for citation.
- RagStream
- An answer being streamed.
- SemanticCache
- Caches model responses by prompt similarity.
- TextChunk
- A slice of a document.
- TextChunker
- Splits text into chunks of at most chunkSize characters, preferring natural boundaries: paragraphs, then lines, then sentences, then words.
- Tool
- Something the model can ask to have run.
- ToolCall
- A model's request to run a tool.
- ToolCallingModel
- A model that can be given Tools.
- ToolResult
- What a tool returned, on its way back to the model.
Enums
- ChatRole
- Who wrote a message.
- ClaudeEffort
-
Thinking effort for Claude (
output_config.effort). - EmbedPurpose
- Whether a text is being stored or used to search. Asymmetric embedding models (Voyage AI, E5, …) embed the two differently.
Functions
-
citedSources(
String text, List< RagSource> sources) → List<RagSource> -
The members of
sourcesthattextcites as[n]or[n, m], in order of first citation. Numbers without a matching source are ignored. -
decodeToolInput(
Object? raw) → Map< String, Object?> - Decodes a provider's tool-call arguments, which arrive as a JSON string in the OpenAI shape and as an object in the Anthropic one.
-
knowledgeBaseTool(
RagPipeline rag, {String name = 'search_knowledge_base', String description = 'Search the local knowledge base for passages relevant to a query. ' 'Use it whenever the answer might be in the stored documents. ' 'Returns numbered passages with their document ids.', int maxResults = 5}) → Tool - A tool that searches a RagPipeline's collection, so a model can look things up in on-device data instead of guessing.
Exceptions / Errors
- LlmException
- Base class of every model-client failure.
- LlmHttpException
- The provider answered with an HTTP error.
- LlmRefusalException
- The model (or a safety classifier) declined the request.