genkit_flutter_edge_ai library

Genkit Dart plugin for flutter_edge_ai — local on-device AI inference.

Wraps flutter_edge_ai as a Genkit model provider, enabling on-device inference with Google Gemma, DeepSeek, Qwen, Llama, and other supported architectures.

Quick Start

import 'package:genkit/genkit.dart';
import 'package:genkit_flutter_edge_ai/genkit_flutter_edge_ai.dart';
import 'package:flutter_edge_ai/flutter_edge_ai.dart';

// 1. Initialize flutter_edge_ai and install a model (host app responsibility).
await FlutterEdgeAi.initialize();
await FlutterEdgeAi.installModel(modelType: ModelType.gemmaIt)
  .fromNetwork('https://...')
  .install();

// 2. Create Genkit with the plugin.
final ai = Genkit(plugins: [
  GenkitFlutterEdgeAiPlugin(
    models: [
      FlutterEdgeAiModelConfig(
        name: 'gemma-3-nano',
        modelType: ModelType.gemmaIt,
      ),
    ],
    embedders: [
      FlutterEdgeAiEmbedderConfig(name: 'embedding-gemma-300m'),
    ],
  ),
]);

// 3. Generate.
final response = await ai.generate(
  model: flutterEdgeAi.model('gemma-3-nano'),
  prompt: 'Tell me a joke',
);
print(response.text);

Classes

ContextWindowMiddleware

Middleware that trims the oldest conversation turns so a request fits the on-device model's context window (KV-cache budget).

DefaultFlutterEdgeAiRuntime

Default runtime that delegates to gemma.FlutterEdgeAi static methods.

FlutterEdgeAiEmbedConfig

Configuration options for flutter_edge_ai embedding generation.

FlutterEdgeAiEmbedderConfig

Configuration for an embedder exposed by GenkitFlutterEdgeAiPlugin.

FlutterEdgeAiModelConfig

Configuration for a single model exposed by GenkitFlutterEdgeAiPlugin.

FlutterEdgeAiModelOptions

Configuration options for flutter_edge_ai model inference.

FlutterEdgeAiPluginHandle

Convenience handle for referencing flutter-edge-ai models and embedders.

FlutterEdgeAiRuntime

Abstraction over flutter_edge_ai static API for testability.

GenkitFlutterEdgeAiPlugin

Genkit plugin that bridges flutter_edge_ai for on-device AI inference and embedding generation.

Constants

flutterEdgeAi → const FlutterEdgeAiPluginHandle

Global convenience instance for referencing flutter-edge-ai models and embedders.

kContextWindowMiddlewareName → const String

Registry name of the context-window trimmer. Shared by the plugin's GenkitPlugin.middleware registration and the trimContext ref so the two can never drift apart.

Functions

contextWindowMiddlewareDef() → GenerateMiddlewareDef<Map<String, dynamic>>

The middleware definition to register from the plugin's GenkitPlugin.middleware hook.

trimContext({int? maxInputTokens, int? reserveOutputTokens}) → GenerateMiddlewareRef<Map<String, dynamic>>

Reference the context-window trimmer in a generate call: