contextengine 0.2.0

Provider-agnostic AI orchestration, memory, and context-optimization layer for Flutter apps.

contextengine

Provider-agnostic AI orchestration, memory, and context-optimization layer for Flutter/Dart apps.

Drop it between your app and any LLM provider. It adds the thing none of the provider SDKs give you for free: automatic, token-efficient context assembly backed by structured memory and indexing.

"I stopped writing prompt-building, memory, and history-trimming code by hand."

Platforms

Mobile and desktop only (iOS, Android, macOS, Windows, Linux). OpenAIProvider and FileStorage use dart:io, which does not compile on Flutter Web. A web-compatible provider adapter (using package:http instead of dart:io) and a web-compatible storage adapter (IndexedDB) are not yet built.

What's real vs. what's reference

Real (production-usable):

• Context pipeline: sliding window, relevance gate, budget allocator, summarizer with fallback
• GraphifyCompressor: input token optimization — symbol compression, structural shorthand, whitespace normalization (30-60% savings)
• CavemanMode: output token optimization — brevity directives, maxOutputTokens capping (20-70% savings)
• Memory engine: fact storage, conflict resolution, versioning, importance scoring
• Graph adapter: entity indexing, keyword-scored retrieval
• Sync manager: offline queue, idempotent writes, partial failure handling
• Observability: 10 callback hooks at every pipeline stage
• Privacy: regex-based field redaction (email, phone, SSN)
• FileStorage: file-backed persistent storage (data survives restarts)
• OpenAIProvider: real LLM adapter using dart:io HttpClient (talks to OpenAI API)
• Tool calling: schema translation for OpenAI/Anthropic/Gemini
• JSON output: repair-retry for malformed JSON
• Cost estimation: per-provider pricing tables
• Multimodal: content blocks with capability gating
• Concurrency guard: prevents overlapping sendMessage calls

Reference / stub (not production-ready):

• InsecureXorStorage — XOR "encryption" for demonstrating the hook. NOT secure. Use platform secure storage with AES-GCM in production.
• InMemoryVectorStore — bag-of-words hash embedding. Lexical, not semantic. Wire to real embedding API (OpenAI/Gemini/Voyage) for semantic recall.
• FactExtractor regex mode — catches 4 fixed sentence shapes. Enable LLM extraction mode (enableLlmExtraction: true) for real conversational accuracy.
• Token counting — chars/4 estimation. Real adapters need tiktoken or provider-specific counting endpoints.

Not yet built:

• Gemini/Claude/Ollama provider adapters
• Isar/Hive/Drift storage adapters
• Real AES-GCM encryption adapter
• Isolate usage for heavy compute (summarizer, embeddings, bulk import)

Quick start

import 'package:contextengine/contextengine.dart';

// Real usage with OpenAI and file persistence
final engine = ContextEngine(
  provider: OpenAIProvider(apiKey: 'sk-...'),
  storage: FileStorage(dbPath: '/path/to/db.json'),
  memory: MemoryEngine(
    graphAdapter: LocalGraphAdapter(),
    sessionId: 'pet_bruno',
  ),
  config: ContextEngineConfig(
    tokenBudget: TokenBudget(maxInputTokens: 6000),
    systemPrompt: 'You are a pet health assistant.',
  ),
);

await engine.initialize(sessionId: 'pet_bruno');

// Direct fact ingestion — no LLM cost
await engine.remember(Fact(
  id: 'f1', subject: 'Bruno', relation: 'HAS_ALLERGY', object: 'Chicken',
));

// Conversational message — full pipeline
final response = await engine.sendMessage('His ear is bothering him again');

// Streaming
await for (final chunk in engine.sendMessageStream('Tell me more')) {
  print(chunk.text);
}

// Observability
engine.lastPromptDebug();  // exact prompt sent
engine.lastTokenUsage();   // per-section token breakdown

Token optimization

Save tokens on both sides of the LLM call — opt-in, zero overhead when disabled:

final engine = ContextEngine(
  provider: OpenAIProvider(apiKey: 'sk-...'),
  storage: FileStorage(dbPath: '/path/to/db.json'),
  config: ContextEngineConfig(
    tokenBudget: TokenBudget(maxInputTokens: 6000),
    systemPrompt: 'You are a pet health assistant.',

    // Input optimization: compress context before sending (30-60% savings)
    graphifyCompressor: GraphifyCompressor(),

    // Output optimization: instruct LLM to minimize output (20-70% savings)
    cavemanMode: CavemanMode(level: CavemanLevel.moderate),
  ),
);

GraphifyCompressor (input tokens):

• Known facts:\n → [F], Summary of → [S], You are a → [R]
• Fact(Bruno HAS_ALLERGY Chicken, conf=0.8, extraction) → Bruno→HAS_ALLERGY→Chicken
• Strips filler: please , could you , I would like to
• Custom symbols: GraphifyCompressor(customSymbols: {'MyPhrase': '[MP]'})

CavemanMode (output tokens):

• mild — "Be concise." (~20% savings, no output cap)
• moderate — "No filler. No hedging. Direct answers only." (~40% savings, 256 token cap)
• extreme — "Respond like a telegram. Single words only." (~70% savings, 64 token cap)

Testing

dart test       # 336 tests pass
dart analyze    # zero issues

Architecture

                         Flutter App
                              |
                     +------------------+
                     |  ContextEngine   |   <- single public entry point
                     +--------+---------+
              +---------------+---------------+
              v               v               v
        ContextPipeline   MemoryEngine    ProviderAdapter
     (sliding window,   (facts, graph,  (OpenAI/Mock/
      budgeter,          vectors,       custom — via
      relevance gate,    importance)    dart:io HttpClient)
      graphify,          |
      caveman mode)      |
              |               v
              v          SyncManager
         LocalStorage   (offline queue)
       (File/InMemory)

Design principles

1. Wrap, don't reimplement — no custom LLM client, no custom vector math.
2. Token efficiency is the core metric — every decision justified in tokens saved.
3. Local-first — works instantly with zero network; sync is background.
4. Inspectable, not magic — lastPromptDebug() shows exactly what was sent.
5. Modular — zero required dependency on any provider/storage/memory backend.
6. Fail soft — summarization fails, fall back to truncation; current message never dropped.

Two lanes

• remember() — direct fact ingestion, no LLM round trip.
• sendMessage() — conversational input through the full token-optimization pipeline.

Performance

• Parallel token counting (batchCountTokens)
• Token count cache — unchanged messages skip recounting
• Parallel memory retrieval — graph + vector run simultaneously
• Entity index fast path — graph query avoids full scan
• GraphifyCompressor — 30-60% input token reduction
• CavemanMode — 20-70% output token reduction

License

MIT — see LICENSE.