contextengine 0.2.0

Provider-agnostic AI orchestration, memory, and context-optimization layer for Flutter apps.

Changelog

0.2.0

Token optimization release — minimize both input and output token costs.

New features

• GraphifyCompressor — input token optimization (30-60% savings):

  • Symbol compression: replaces common phrases with short tokens (Known facts:\n → [F], Summary of → [S], You are a → [R])
  • Structural shorthand: converts Fact(S R O, conf=X, src) → S→R→O
  • Whitespace normalization: collapses excessive whitespace/newlines
  • Filler removal: strips please , could you , I would like to
  • Custom symbol maps via customSymbols parameter
  • estimateSavings() for pre-compression token savings estimation

• CavemanMode — output token optimization (20-70% savings):

  • Three intensity levels: mild, moderate, extreme
  • Injects brevity directives into system prompt
  • Automatic maxOutputTokens capping per level (moderate=256, extreme=64)
  • Works with or without an existing system prompt

• Both features are opt-in via ContextEngineConfig:

  ContextEngineConfig(
    graphifyCompressor: GraphifyCompressor(),
    cavemanMode: CavemanMode(level: CavemanLevel.moderate),
  )
  

Pipeline changes

• Steps 9-10 added to _buildRequest():
  • Step 9: GraphifyCompressor runs after budget allocation
  • Step 10: CavemanMode modifies system prompt and sets maxOutputTokens
• Pipeline log includes graphify compression: ~N tokens saved and caveman mode: level (maxOut=N) entries
• Zero overhead when features are disabled (null by default)

Test coverage

• 336 tests (up from 178), all passing
• New test files:
  • graphify_compressor_test.dart — 28 tests (symbol compression, Unicode, idempotency, custom symbols, large input, disabled mode)
  • caveman_mode_test.dart — 21 tests (all levels, prompt modification, maxOutputTokens, pipeline integration)
  • platform_compatibility_test.dart — 30 tests (all pure-Dart modules verified independent of dart:io)
  • production_readiness_test.dart — 30 tests (concurrency, 100K char input, Unicode/emoji/CJK/RTL, budget extremes, fact conflict cascades, privacy+compression combo, sync edge cases, storage quota)
  • edge_cases_comprehensive_test.dart — 49 tests (lifecycle errors, memory filtering, RelevanceGate boundaries, JSON repair, token budget extremes, sliding window cache, budget allocator trimming, tool schemas, Fact model)

No breaking changes

• All existing APIs unchanged
• GraphifyCompressor and CavemanMode are null by default in config
• Existing 178 tests continue to pass

0.1.2

Add repository field to pubspec.yaml.

0.1.1

Bug fixes and hardening before first public publish:

• Fix FileStorage not writing to disk when enableBufferedWrites is false (flush was skipped by an inverted guard clause)
• Fix InsecureXorStorage.deleteMessages silently passing DateTime.now() instead of null, changing "delete all" semantics to "delete up to now"
• Fix CostEstimator._findClosestPricing returning wrong pricing for model variants (e.g. gpt-4o-mini-2024 matched gpt-4o instead of gpt-4o-mini); now prefers the longest prefix match
• Fix FileStorage._DbState.fromJson crashing on corrupted role fields; now falls back to Role.user via orElse
• Fix LocalStorage.deleteMessages interface declaring non-nullable DateTime before while implementations accept nullable; interface now declares DateTime? before
• Remove unused enableBufferedWrites field from FileStorage
• Remove KNOW.md from package
• Fix 6 analyzer warnings in test files (unused imports, unused variables, single-quote consistency)

0.1.0

Initial pre-release. Core pipeline, memory, and orchestration layer is functional. Not yet recommended for production use without reviewing the reference/stub status of individual modules.

Real (production-usable)

• Context pipeline: sliding window (token-counted), relevance gate, budget allocator (priority-ordered trimming), summarizer with truncation fallback
• Memory engine: fact storage, conflict resolution (supersede, confidence-weighted, callback), versioning, importance scoring
• Graph adapter: entity indexing, keyword-scored retrieval
• Sync manager: offline queue, idempotent writes, partial failure handling, conflict resolution
• Observability: 10 callback hooks at every pipeline stage
• Privacy: regex-based field redaction (email, phone, SSN)
• FileStorage: file-backed persistent storage (survives app restarts)
• OpenAIProvider: real LLM adapter using dart:io HttpClient
• Tool calling: schema translation for OpenAI/Anthropic/Gemini formats
• JSON output: repair-retry for malformed JSON
• Cost estimation: per-provider pricing tables
• Multimodal: content blocks with capability gating
• Concurrency guard: prevents overlapping sendMessage calls
• Performance: parallel token counting, token count cache, parallel memory retrieval, entity index fast path

Reference / stub (not production-ready)

• InsecureXorStorage: XOR "encryption" — NOT secure, demonstrates the hook only. Use platform secure storage with AES-GCM in production.
• InMemoryVectorStore: bag-of-words hash embedding. Lexical, not semantic. Wire to a real embedding API for semantic recall.
• FactExtractor regex mode: catches 4 fixed sentence shapes. Enable LLM mode (enableLlmExtraction: true) for real conversational accuracy.
• Token counting: chars/4 estimation. Real adapters need tiktoken or provider-specific counting endpoints.

Not yet built

• Gemini, Claude, Ollama provider adapters
• Isar, Hive, Drift storage adapters
• Real AES-GCM encryption adapter
• Isolate usage for heavy compute
• CI pipeline