engine library

Engine SPI of flutter_edge_ai: everything an engine, backend, tokenizer, resolver or skill-executor package needs, in one web-safe import. Re-exports the app library (flutter_edge_ai.dart).

Native-only helpers (dart:io / dart:ffi) are in engine_io.dart, web-only ones (dart:js_interop) in engine_web.dart. Nothing under lib/src is public: import these libraries, never a src/ path.

The namespace is pinned by the symbol golden in test/public_surface/.

Classes

AbstainReason
Why a decider abstained.
AbstainSignalKind
What DeciderProvenance.abstainSignal is.
AssetSource
Asset source - copies from Flutter assets
BundledSource
Bundled source - native platform resources
CanceledError
Download canceled
CancelToken
Token for cancelling model downloads
CatalogModel
A model in a catalog: what to show and how to group it. What to install is on its variants.
CatalogVariant
One installable form of a CatalogModel: its files and where they run. One subclass per installer, so a variant reaches only the installer that can take it.
ChatRequest
InferenceModel.openChat's arguments as the caller passed them (nulls unset), for InferenceModel.resolveChatRequest.
ChatTurnHooks
What an engine (or a test) changes about one chat's turns: an engine returns one from InferenceModel.createChatTurnHooks, once per chat, and a test passes one to InferenceChat's constructor.
CorruptionDetectionResult
Result of corruption detection
Decider
Picks one of its options for an input — or abstains when none fits.
DeciderAnswer
A decider's answer.
DeciderCapabilities
What a decider can do.
DeciderInput
What a decider is asked about: the input and, for deciders that read one, the conversation so far.
DeciderMethod
How a decider scores a request against its options.
DeciderOption
One option a decider can choose: an intent, an action, a screen, a tool, a label, a route.
DeciderProvenance
Where an answer came from.
DeciderQuestionType
The kind of question an answer answers, in Jev's terms: what DeciderAnswer.toJevJson writes.
DeciderRequest
A Decider.choose call as core checked it, handed to Decider.decide.
DiagnosticReport
Diagnostic information for troubleshooting
DownloadError
Download error types using sealed classes for exhaustive pattern matching
DownloadProgress
Progress information for model downloads
EmbedderVariant
A model FlutterEdgeAi.installEmbedder installs.
EmbeddingBackendProvider
A pluggable embedding backend (LiteRT C API + Gecko/EmbeddingGemma, ONNX, or a third-party backend). Same probe-chain shape as InferenceEngineProvider: selected by probing the embedding model spec, highest-priority first match.
EmbeddingDeciderCalibration
How an embedding decider decides to abstain ("none of these"), and whether its probabilities are calibrated.
EmbeddingInput
One thing to embed, shaped like Message: optional text plus media.
EmbeddingInstallation
Result of embedding model installation
EmbeddingInstallationBuilder
Fluent builder for embedding model installation
EmbeddingModel
An embedding model, as returned by FlutterEdgeAi.getActiveEmbedder().
EmbeddingModelSpec
Specification for an embedding model: a model file plus its tokenizer (.tflite, .onnx, …), or a single .litertlm or .gguf file that carries its own tokenizer (isSelfContained).
EmbeddingPass
What an engine runs: in core's worker isolate on native, inline on the page on web.
EmbeddingPassDescriptor<C extends Object>
Everything core needs to build an engine's EmbeddingPass, with the engine's own configuration C opaque to core (D2): a new engine knob never needs a core release.
EmbeddingPassInfo
What a loaded pass can do. Core checks the model's declared media against it at load, and every request against it before the request is sent (D4).
EmbeddingTokenizer
Turns raw text (with a TaskType prefix already applied) into a TokenizedInput. One instance per loaded tokenizer file; built once by TokenEmbeddingPass.load and reused for every encode call.
EmbeddingTokenizerProvider
A pluggable source of EmbeddingTokenizerFactory — the "text -> token ids" half of an embedding.
EmbeddingTokenizerRegistry
Holds embedding tokenizer providers registered via FlutterEdgeAi.initialize. Same probe-chain selection as EmbeddingRegistry and EngineRegistry.
EncodedMediaFormat
An encoded media container (a file format, not a decoded image or waveform).
EngineTurn
One turn, as the engine produces it.
ErrorHandlingResult
Result of error handling
EstimatedDimensions
Estimated image dimensions
FileSource
File source - external files (mobile only)
FileSystemService
Abstraction for file system operations Allows different implementations for different platforms (mobile/web) and easy mocking in tests
FlutterEdgeAi
Modern API facade for Flutter Edge AI
FlutterEdgeAiDesktop
Desktop implementation of FlutterEdgeAi plugin
FlutterEdgeAiPlugin
Low-level platform-interface tier (the federated-plugin SPI) for flutter_edge_ai. It hosts the abstract runtime types (InferenceModel, InferenceModelSession, InferenceChat, EmbeddingModel, SpeechRecognizer, SpeechSynthesizer, …) and the platform-dispatch singleton instance.
ForbiddenError
HTTP 403 - Access forbidden
ForwardResult
A graph's raw output for one input: a flat, row-major buffer plus its tensor shape.
FunctionCallParser
Facade for backward compatibility. Delegates to model-specific FunctionCallFormat implementations.
FunctionCallResponse
Gemma3Specs
Gemma 3 SigLIP vision encoder specifications
GeneralVisionSpecs
General vision encoder specifications
HuggingFaceResolver
A pluggable resolver that turns a Hugging Face repo id into a ResolvedHfModel by reading that repo's deployment metadata.
HuggingFaceResolverSource
Opt-in capability interface: a provider — an InferenceEngineProvider, EmbeddingBackendProvider, SttBackendProvider or TtsBackendProvider — that also ships a Hugging Face HuggingFaceResolver for its own models.
ImageErrorHandler
Comprehensive error handling and debugging utilities for AI image processing to prevent corruption that causes repeating text patterns in model responses.
ImageProcessor
Comprehensive image processing utilities to prevent AI image corruption and ensure proper vision encoder compatibility.
ImageTokenizer
Handles proper image tokenization for multimodal AI models to prevent "Prompt contained 0 image tokens but received 1 images" errors and corruption that causes repeating text patterns.
InferenceChat
InferenceDeciderCalibration
How an inference decider (InferenceModel.createDecider) turns its option scores into probabilities, and when it abstains.
InferenceEngineProvider
A pluggable inference engine (MediaPipe .task, LiteRT-LM .litertlm, or a third-party .onnx/.gguf engine).
InferenceInstallation
Result of inference model installation
InferenceInstallationBuilder
Fluent builder for inference model installation
InferenceModel
Represents an LLM model instance.
InferenceModelSession
Session managing response generation from the model.
InferenceModelSpec
Specification for inference models (main model + optional LoRA)
InferenceVariant
A model FlutterEdgeAi.installModel installs: an LLM.
JsonSchemaResponseFormat
A ResponseFormat of JSON values valid against schema. Made by ResponseFormat.jsonSchema and ResponseFormat.jsonObject.
Message
One message of a conversation: text, media, or a tool call or result.
MessageMedia
A media buffer MessageTranscriptCodec.encode hands to its MessageTranscriptCodec.writeMedia.
MessageMediaReference
A media reference MessageTranscriptCodec.decode hands to its MessageTranscriptCodec.readMedia.
MessageTranscriptCodec
Writes a list of Messages as versioned JSON and reads it back (#824).
ModelArtifact
One file a catalog model installs: a Hugging Face file pinned to a commit.
ModelCatalog
A list of models, as one package or FlutterEdgeAi.catalog offers them.
ModelCatalogSource
A provider that ships a catalog of the models it runs. An engine or backend passed to FlutterEdgeAi.initialize that implements this adds its catalog to FlutterEdgeAi.catalog.
ModelFamily
A model family: what a model is, beyond its file — the language-model families of ModelType, the speech-to-text families of SttModelType and the text-to-speech families of TtsModelType.
ModelFile
Represents a single file that belongs to a model
ModelFileManager
ModelResponse
Base interface for model responses from InferenceChat Can be either TextResponse, FunctionCallResponse, or ThinkingResponse
ModelRuntimeDefaults
Runtime defaults resolved from a Hugging Face repo's deployment metadata (e.g. a litertlm_manifest.json) or listed by a catalog variant, in flutter_edge_ai's own runtime vocabulary.
ModelSource
Sealed class representing all possible model sources Provides type-safe alternative to string URLs
ModelSpec
Base specification for any model (inference or embedding)
ModelThinkingFilter
MultimodalImageHandler
Main integration class for handling multimodal image processing in Flutter Edge AI to prevent AI image corruption and repeating text pattern issues.
MultimodalImageResult
Result of multimodal image processing
MultimodalOrder
The order in which an engine sends a user message's images, audio and text to the model.
MultiModelEngine
Opt-in capability interface: an InferenceEngineProvider whose models can be open side by side, which FlutterEdgeAi.openModel requires.
NetworkError
Network connectivity error
NetworkSource
Network source - downloads from HTTPS/HTTP URLs
NotFoundError
HTTP 404 - Resource not found
NpuTarget
The NPU a model file was compiled for. Engines extend this with the details their files carry.
NpuVendor
An NPU vendor. A class with constants rather than an enum, so a vendor added in a later version does not break a switch written against this one.
OptionScorer
An engine's calls for scoring options, on one loaded model: what its InferenceModel.openOptionScorer returns. Core makes one call at a time on a scorer (a decider runs its decisions one after another); every call runs on the model's own engine, ordered with its other native work.
OptionScoringRun
What one OptionScorer.score call returned.
OrphanedFileInfo
Information about a potentially orphaned file
ParallelFunctionCallResponse
Multiple function calls in a single model response.
ProcessedImage
Represents a processed image ready for AI model consumption
RankedOption
One candidate of an answer.
RateLimitedError
HTTP 429 - Rate limit exceeded
ResolvedChatRequest
A ChatRequest with every value set, as the engine resolved it in InferenceModel.resolveChatRequest. Every field but runtimeInjectsToolDeclarations is required, so a value an engine forgets to carry over is a compile error, not a silent default.
ResolvedHfFile
One file of a model resolved from a Hugging Face repo that comes as more than one file: a member of a DIRECTORY model (an ORT-GenAI model is genai_config.json + model.onnx+`model.onnx_data` + tokenizer files), or an embedder's model file or its tokenizer file (see ResolvedHfModel.files).
ResolvedHfModel
A model resolved from a Hugging Face repo's deployment metadata into flutter_edge_ai's install + runtime vocabulary.
ResolvedSampling
Sampling parameters with every field set: what an engine sends.
ResponseFormat
What every reply of a session must be: fixed for the session's lifetime.
ResponseValidationResult
Result of response validation
RuntimeConfig
Runtime config for building a model — the per-call params getActiveModel collects, kept as a small holder so the provider contract stays stable as params evolve.
SamplingParams
Sampling parameters for a session, as the caller asked for them.
ScoreKind
What a score in DeciderAnswer.ranked is.
SdkResponseParser
Parser for LiteRT-LM SDK Chat Completions JSON responses.
SdkTextExtractor
Pure-Dart extractor for LiteRT-LM SDK JSON response chunks.
ServerError
HTTP 5xx - Server error
SessionMetrics
Session metrics containing token usage and performance statistics.
SessionRequest
Everything an engine needs to open a session (InferenceModel.openEngineSession). A later option is a new field, not a new parameter on every engine.
SkillExecutorProvider
A pluggable skill executor — the seam between core and the opt-in flutter_edge_ai_agent package's agentic "skills" runtime.
SkillExecutorRegistry
Holds skill executors registered via FlutterEdgeAi.initialize (skillExecutors:). Same probe-chain selection as EngineRegistry / EmbeddingRegistry: the registered executor with the highest SkillExecutorProvider.priority whose SkillExecutorProvider.canExecute returns true for a skill type wins (first-registered breaks ties). There is no central type map — the opt-in flutter_edge_ai_agent executors self-select.
SpeechRecognizer
Represents a speech-to-text model instance.
SpeechSynthesizer
Represents a text-to-speech model instance.
StorageStats
Storage statistics
StructuredOutputSupport
What a loaded model can enforce: engine, platform and the backend it was loaded on. Read it on InferenceModel.structuredOutputSupport.
SttBackendProvider
A pluggable STT backend (LiteRT C API, or a third-party backend). Same probe-chain shape as EmbeddingBackendProvider/InferenceEngineProvider: selected by probing the STT model spec, highest-priority first match.
SttInstallation
Result of STT model installation
SttInstallationBuilder
Fluent builder for STT (speech-to-text) model installation
SttModelSpec
Specification for STT models (model.tflite + tokenizer.json).
SttVariant
A model FlutterEdgeAi.installStt installs.
TextResponse
Text token during streaming
ThinkingResponse
Thinking process content from the model
TokenEmbeddingPass
An EmbeddingPass for an engine that tokenizes in Dart. Text only.
TokenizedInput
One text's tokenized form, ready for TokenEmbeddingPass.runTokens.
TokenPassGraph
What TokenEmbeddingPass.openGraph reports about the opened graph.
Tool
TtsBackendProvider
A pluggable TTS backend (LiteRT C API, or a third-party backend). Same probe-chain shape as EmbeddingBackendProvider/InferenceEngineProvider: selected by probing the TTS model spec, highest-priority first match.
TtsInstallation
Result of TTS model installation.
TtsInstallationBuilder
Fluent builder for TTS (text-to-speech) model installation.
TtsModelSpec
Specification for a TTS model — a SELECTABLE bundle. ttsModelType carries the model family so one generic backend dispatches to the right runtime profile; sources is one entry per file in that type's manifest.
TtsVariant
A model FlutterEdgeAi.installTts installs: a bundle of files under one base URL. Its files are exactly what core fetches for type under baseUrl (TtsModelType.manifest, each at its fetch location): the install checks their sizes, and refuses with ArgumentError a variant whose files name another URL.
UnauthorizedError
HTTP 401 - Authentication required
UnknownError
Unknown error
ValidationResult
Result of image validation
VisionEncoderValidator
Validates images for compatibility with AI vision encoders to prevent corruption that causes models to interpret images as repeating text patterns.
VisionSpecs
Base class for vision encoder specifications

Enums

ActivationDataType
Numeric type of the model's activations, for native .litertlm models (Android, iOS, desktop).
CorruptionAction
Actions to take when corruption is detected
EdgeAiLogLevel
Verbosity for flutter_edge_ai's internal logs.
EmbeddingModality
A kind of input an embedding model can turn into a vector.
EmbeddingOutputContract
How to turn a ForwardResult into the final embedding vector.
ErrorType
Types of errors that can occur in image processing
MediaPosition
Where a message's media go relative to its text.
MessageMediaType
Which media field a buffer belongs to: Message.imageBytes and Message.images are image, Message.audioBytes is audio.
MessageType
ModelFileType
ModelManagementType
Base enumeration for different model management types
ModelReplacePolicy
Policy for what happens to a previously-installed model when a new one is set active. Lives with the spec value types (it's a per-spec install policy); re-exported from model_file_manager_interface.dart for backward compat.
ModelType
The language-model families.
PreferredBackend
Hardware backend for model inference.
ResponseAction
Actions to take for corrupted responses
StructuredDelivery
How a structured reply reaches the consumer.
SttModelType
Speech-to-text model families supported by the pluggable STT backends. Only moonshine has a shipped SttModelProfile/pipeline (flutter_edge_ai_speech); the others are follow-ons that need a log-mel frontend.
TaskType
What an embedding is for. Every engine prepends prefix to the TEXT part of an input, native and web alike; media parts never get a prefix.
ToolChoice
Controls whether the model should call tools.
TtsModelType
Text-to-speech model families supported by the pluggable TTS backends. matcha, qwen3 and inflect have shipped TtsModelProfile/pipelines (flutter_edge_ai_speech); kokoro/supertonic are documented follow-ons (fail-loud until wired).
TurnEnd
How one generation turn ended.
VisionEncoderType
Vision encoder types with their specifications
WebStorageMode
Storage mode for web platform models

Mixins

CloseNotifier
Lets the lifecycle OWNER (core) react when a model built by a separate engine package closes. The package constructs the model; core registers a reset callback via addCloseListener; the concrete model fires the listeners from its own close()'s finally. Best practice: the engine createModel(spec, config) contract stays a pure factory — no lifecycle callback leaks into the contract that third-party engines implement.
RawSdkResponseSession
Mixin for sessions that surface the SDK's structured raw JSON response (LiteRT-LM Gemma 4 path with tool_calls). Allows InferenceChat to read the structured tool calls without a hard dependency on a concrete session type, and lets non-FFI sessions opt out by simply not implementing this mixin.

Extensions

DownloadErrorMessage on DownloadError
Extension to convert DownloadError to user-friendly messages
MessageExtension on Message
TtsModelTypeManifest on TtsModelType
The filenames a given TTS model needs, installed together as a bundle from one source. Fail-loud for unwired families.

Constants

defaultMaxFunctionBufferLength → const int
Default maximum length for function call buffer before flushing as text. Must accommodate verbose formats (DeepSeek tags, parallel calls).
supportedLoraRanks → const List<int>

Properties

edgeAiLogLevel ↔ EdgeAiLogLevel
Active log level. Top-level (Dart-canonical for process-global logger state) so edgeAiLog reads it without importing the public facade.
getter/setter pair
isDesktop → bool
Check if current platform is desktop
no setter

Functions

detectImageMimeType(Uint8List bytes) → String
sniffEncodedMediaFormat's MIME type for an IMAGE, else 'image/png' — the fallback the LLM web arms have always used for bytes they could not identify. Audio bytes get the fallback too: this answers "what kind of image is this", and has never named an audio type.
edgeAiLog(String message, {EdgeAiLogLevel level = EdgeAiLogLevel.info}) → void
The plugin's debug log. Silent in release builds and under EdgeAiLogLevel.none; in debug builds it prints when level <= edgeAiLogLevel.
finalizeEmbedding(EmbeddingOutputContract contract, ForwardResult result, {List<int>? attentionMask}) → List<double>
Turns a raw ForwardResult into the final vector per contract.
isLiteRtLmFilename(String filename) → bool
Whether filename ends in .litertlm, compared ASCII case-insensitively (model.LiteRTLM matches; only A–Z fold, so a non-ASCII look-alike never does).
meanPoolAndNormalize(ForwardResult result, {List<int>? attentionMask}) → List<double>
Mean-pools a token-level ForwardResult over the sequence axis and L2-normalizes it into a single embedding vector of length dim.
sanitizeHfDirName(String repo, {String? variant}) → String
A filesystem-safe, single-segment directory name for a directory model (ORT-GenAI) installed from the Hugging Face repo repo, optionally from its execution-provider subfolder variant.
sniffEncodedMediaFormat(Uint8List bytes) → EncodedMediaFormat?
The container bytes start with, from their magic bytes, or null when they match none of EncodedMediaFormat.known.
toolCallsOf(Message message) → List<FunctionCallResponse>
The calls a MessageType.toolCall message carries, for an engine that replays a history.

Typedefs

EmbeddingTokenizerFactory = Future<EmbeddingTokenizer> Function(String tokenizerPath)
Factory that builds an EmbeddingTokenizer for a given on-disk tokenizer path.
MessageMediaReader = Future<Uint8List> Function(MessageMediaReference reference)
Returns the bytes a MessageMediaWriter stored under MessageMediaReference.reference, for MessageTranscriptCodec.decode.
MessageMediaWriter = Future<String> Function(MessageMedia media)
Stores MessageMedia.bytes and returns a reference the app can resolve later (a file name, a key), for MessageTranscriptCodec.encode.

Exceptions / Errors

ContextLimitException
The error an engine puts on a turn's text when it refused an input too long for what the context window had left, together with end TurnEnd.contextLimit. The engine's context is as it was before the turn, and InferenceChat takes the turn's messages back out of its history.
DeciderCancelledException
A createDecider or a choose was cancelled through its CancelToken, or the decider was closed while a choose ran. A cancelled createDecider returns no decider.
DeciderException
Common base of every decider failure. Not sealed, so a later kind of failure is an additive change; catch this to handle them all.
DeciderSpaceMismatchException
A decider was fitted in another embedding space than the embedder asked to load it, or than the one core now serves in place of the decider's own: another model file, dimension, backend or task prefix.
DeciderUnsupportedException
The decider, or the model asked for one, cannot do what was asked — for example an InferenceModel that does not score options, or a decider that takes no conversation context given some.
DownloadCancelledException
Exception thrown when a download is cancelled
DownloadException
Custom exception for download failures
EmbedderClosedException
The model was closed, or closed before this request started — so the request never ran.
EmbedderCloseTimeoutException
The previous embedder's close did not finish within the limit (60 s), measured from the moment that close STARTED, so FlutterEdgeAi.getActiveEmbedder built nothing: two native models are never resident at once.
EmbeddingCancelledException
embedAll was cancelled through its CancelToken. No partial results are returned.
EmbeddingEngineException
The engine failed while embedding.
EmbeddingException
Common base of every embedding failure. Not sealed, so a later kind of failure is an additive change; catch this to handle them all.
EmbeddingFormatNotSupportedException
A media part's container is not one the engine accepts (e.g. HEIC where the engine decodes JPEG/PNG/GIF/BMP only).
EmbeddingInputTooLongException
The input is longer than maxInputTokens — task prefix, BOS and EOS included. Nothing was truncated; split the text and embed the pieces.
EmbeddingLoadException
The engine could not load the model, or refused the requested configuration (e.g. a modality it cannot serve).
EmbeddingModalityNotLoadedException
The input carries modality, which this model was not loaded with (loaded is what it was loaded with). Declare media at load: getActiveEmbedder(supportImage: true).
ImageProcessingException
Exception thrown when image processing fails
ImageTokenizationException
Exception thrown when image tokenization fails
IncompleteStructuredOutputException
The turn ended before the value was complete: end is TurnEnd.outputLimit, TurnEnd.contextLimit, TurnEnd.stopped or TurnEnd.closed. partialText is what was generated; in an InferenceChat it is also in the chat's history, except for a turn stopped before any text, which the chat takes back, its user message included. A stop is incomplete even when partialText parses.
ModelStorageException
Exception thrown when model storage operations fail
ModelValidationException
An installed model file is not what it should be: smaller than a model can be, or not the size the install asked for (expectedSize on the builders, and the size of every fromCatalog install; not checked on the web). A file that fails the size check is not made active, and is deleted with its record so the next install downloads it again — unless an active model reads it, or the delete fails, which the message says.
NoActiveModelError
Thrown when a model is asked for and none is installed and active: the app has to install one first (FlutterEdgeAi.installModel() or installEmbedder()).
NonConformingStructuredOutputException
The turn ended normally but text is not a value of the format (not JSON, trailing text, or a dialect keyword violated at schemaPath): an engine accepted a format it did not enforce. An engine bug.
ResponseFormatUnsupportedException
The format is refused by this engine, platform or backend, or a schema keyword at schemaPath cannot be enforced. Returned (not thrown) by InferenceModel.refusalFor; openSession and openChat throw that refusal before opening an engine session. An engine whose native constraint compiler can reject a schema only at first send may instead put this exception on that turn's text; core passes it through and takes the turn back.
StructuredOutputException
Base of the typed-response failures. Sealed: apps catch these, engines never subclass them, and a fourth kind is a deliberate break.
UnsupportedEmbeddingCombinationException
The input carries several parts — several modalities, or several images (requested names its modalities) — and the engine embeds one part per vector.
VisionEncoderValidationException
Exception thrown when validation fails