engine library
Engine SPI of flutter_edge_ai: everything an engine, backend, tokenizer,
resolver or skill-executor package needs, in one web-safe import. Re-exports
the app library (flutter_edge_ai.dart).
Native-only helpers (dart:io / dart:ffi) are in engine_io.dart,
web-only ones (dart:js_interop) in engine_web.dart. Nothing under
lib/src is public: import these libraries, never a src/ path.
The namespace is pinned by the symbol golden in test/public_surface/.
Classes
- AbstainReason
- Why a decider abstained.
- AbstainSignalKind
- What DeciderProvenance.abstainSignal is.
- AssetSource
- Asset source - copies from Flutter assets
- BundledSource
- Bundled source - native platform resources
- CanceledError
- Download canceled
- CancelToken
- Token for cancelling model downloads
- CatalogModel
- A model in a catalog: what to show and how to group it. What to install is on its variants.
- CatalogVariant
- One installable form of a CatalogModel: its files and where they run. One subclass per installer, so a variant reaches only the installer that can take it.
- ChatRequest
- InferenceModel.openChat's arguments as the caller passed them (nulls unset), for InferenceModel.resolveChatRequest.
- ChatTurnHooks
-
What an engine (or a test) changes about one chat's turns: an engine
returns one from
InferenceModel.createChatTurnHooks, once per chat, and a test passes one to InferenceChat's constructor. - CorruptionDetectionResult
- Result of corruption detection
- Decider
- Picks one of its options for an input — or abstains when none fits.
- DeciderAnswer
- A decider's answer.
- DeciderCapabilities
- What a decider can do.
- DeciderInput
- What a decider is asked about: the input and, for deciders that read one, the conversation so far.
- DeciderMethod
- How a decider scores a request against its options.
- DeciderOption
- One option a decider can choose: an intent, an action, a screen, a tool, a label, a route.
- DeciderProvenance
- Where an answer came from.
- DeciderQuestionType
- The kind of question an answer answers, in Jev's terms: what DeciderAnswer.toJevJson writes.
- DeciderRequest
- A Decider.choose call as core checked it, handed to Decider.decide.
- DiagnosticReport
- Diagnostic information for troubleshooting
- DownloadError
- Download error types using sealed classes for exhaustive pattern matching
- DownloadProgress
- Progress information for model downloads
- EmbedderVariant
-
A model
FlutterEdgeAi.installEmbedderinstalls. - EmbeddingBackendProvider
-
A pluggable embedding backend (LiteRT C API + Gecko/EmbeddingGemma, ONNX,
or a third-party backend). Same probe-chain shape as
InferenceEngineProvider: selected by probing the embedding model spec, highest-priority first match. - EmbeddingDeciderCalibration
- How an embedding decider decides to abstain ("none of these"), and whether its probabilities are calibrated.
- EmbeddingInput
-
One thing to embed, shaped like
Message: optional text plus media. - EmbeddingInstallation
- Result of embedding model installation
- EmbeddingInstallationBuilder
- Fluent builder for embedding model installation
- EmbeddingModel
-
An embedding model, as returned by
FlutterEdgeAi.getActiveEmbedder(). - EmbeddingModelSpec
-
Specification for an embedding model: a model file plus its tokenizer
(
.tflite,.onnx, …), or a single.litertlmor.gguffile that carries its own tokenizer (isSelfContained). - EmbeddingPass
- What an engine runs: in core's worker isolate on native, inline on the page on web.
-
EmbeddingPassDescriptor<
C extends Object> -
Everything core needs to build an engine's EmbeddingPass, with the
engine's own configuration
Copaque to core (D2): a new engine knob never needs a core release. - EmbeddingPassInfo
- What a loaded pass can do. Core checks the model's declared media against it at load, and every request against it before the request is sent (D4).
- EmbeddingTokenizer
-
Turns raw text (with a TaskType prefix already applied) into a
TokenizedInput. One instance per loaded tokenizer file; built once by
TokenEmbeddingPass.loadand reused for everyencodecall. - EmbeddingTokenizerProvider
- A pluggable source of EmbeddingTokenizerFactory — the "text -> token ids" half of an embedding.
- EmbeddingTokenizerRegistry
-
Holds embedding tokenizer providers registered via
FlutterEdgeAi.initialize. Same probe-chain selection asEmbeddingRegistryandEngineRegistry. - EncodedMediaFormat
- An encoded media container (a file format, not a decoded image or waveform).
- EngineTurn
- One turn, as the engine produces it.
- ErrorHandlingResult
- Result of error handling
- EstimatedDimensions
- Estimated image dimensions
- FileSource
- File source - external files (mobile only)
- FileSystemService
- Abstraction for file system operations Allows different implementations for different platforms (mobile/web) and easy mocking in tests
- FlutterEdgeAi
- Modern API facade for Flutter Edge AI
- FlutterEdgeAiDesktop
- Desktop implementation of FlutterEdgeAi plugin
- FlutterEdgeAiPlugin
-
Low-level platform-interface tier (the federated-plugin SPI) for
flutter_edge_ai. It hosts the abstract runtime types (
InferenceModel,InferenceModelSession,InferenceChat,EmbeddingModel,SpeechRecognizer,SpeechSynthesizer, …) and the platform-dispatch singleton instance. - ForbiddenError
- HTTP 403 - Access forbidden
- ForwardResult
- A graph's raw output for one input: a flat, row-major buffer plus its tensor shape.
- FunctionCallParser
-
Facade for backward compatibility.
Delegates to model-specific
FunctionCallFormatimplementations. - FunctionCallResponse
- Gemma3Specs
- Gemma 3 SigLIP vision encoder specifications
- GeneralVisionSpecs
- General vision encoder specifications
- HuggingFaceResolver
- A pluggable resolver that turns a Hugging Face repo id into a ResolvedHfModel by reading that repo's deployment metadata.
- HuggingFaceResolverSource
-
Opt-in capability interface: a provider — an
InferenceEngineProvider,EmbeddingBackendProvider,SttBackendProviderorTtsBackendProvider— that also ships a Hugging Face HuggingFaceResolver for its own models. - ImageErrorHandler
- Comprehensive error handling and debugging utilities for AI image processing to prevent corruption that causes repeating text patterns in model responses.
- ImageProcessor
- Comprehensive image processing utilities to prevent AI image corruption and ensure proper vision encoder compatibility.
- ImageTokenizer
- Handles proper image tokenization for multimodal AI models to prevent "Prompt contained 0 image tokens but received 1 images" errors and corruption that causes repeating text patterns.
- InferenceChat
- InferenceDeciderCalibration
-
How an inference decider (
InferenceModel.createDecider) turns its option scores into probabilities, and when it abstains. - InferenceEngineProvider
-
A pluggable inference engine (MediaPipe
.task, LiteRT-LM.litertlm, or a third-party.onnx/.ggufengine). - InferenceInstallation
- Result of inference model installation
- InferenceInstallationBuilder
- Fluent builder for inference model installation
- InferenceModel
- Represents an LLM model instance.
- InferenceModelSession
- Session managing response generation from the model.
- InferenceModelSpec
- Specification for inference models (main model + optional LoRA)
- InferenceVariant
-
A model
FlutterEdgeAi.installModelinstalls: an LLM. - JsonSchemaResponseFormat
- A ResponseFormat of JSON values valid against schema. Made by ResponseFormat.jsonSchema and ResponseFormat.jsonObject.
- Message
- One message of a conversation: text, media, or a tool call or result.
- MessageMedia
- A media buffer MessageTranscriptCodec.encode hands to its MessageTranscriptCodec.writeMedia.
- MessageMediaReference
- A media reference MessageTranscriptCodec.decode hands to its MessageTranscriptCodec.readMedia.
- MessageTranscriptCodec
- Writes a list of Messages as versioned JSON and reads it back (#824).
- ModelArtifact
- One file a catalog model installs: a Hugging Face file pinned to a commit.
- ModelCatalog
-
A list of models, as one package or
FlutterEdgeAi.catalogoffers them. - ModelCatalogSource
-
A provider that ships a catalog of the models it runs. An engine or
backend passed to
FlutterEdgeAi.initializethat implements this adds its catalog toFlutterEdgeAi.catalog. - ModelFamily
-
A model family: what a model is, beyond its file — the language-model
families of ModelType, the speech-to-text families of
SttModelTypeand the text-to-speech families ofTtsModelType. - ModelFile
- Represents a single file that belongs to a model
- ModelFileManager
- ModelResponse
- Base interface for model responses from InferenceChat Can be either TextResponse, FunctionCallResponse, or ThinkingResponse
- ModelRuntimeDefaults
-
Runtime defaults resolved from a Hugging Face repo's deployment metadata
(e.g. a
litertlm_manifest.json) or listed by a catalog variant, in flutter_edge_ai's own runtime vocabulary. - ModelSource
- Sealed class representing all possible model sources Provides type-safe alternative to string URLs
- ModelSpec
- Base specification for any model (inference or embedding)
- ModelThinkingFilter
- MultimodalImageHandler
- Main integration class for handling multimodal image processing in Flutter Edge AI to prevent AI image corruption and repeating text pattern issues.
- MultimodalImageResult
- Result of multimodal image processing
- MultimodalOrder
- The order in which an engine sends a user message's images, audio and text to the model.
- MultiModelEngine
-
Opt-in capability interface: an
InferenceEngineProviderwhose models can be open side by side, whichFlutterEdgeAi.openModelrequires. - NetworkError
- Network connectivity error
- NetworkSource
- Network source - downloads from HTTPS/HTTP URLs
- NotFoundError
- HTTP 404 - Resource not found
- NpuTarget
- The NPU a model file was compiled for. Engines extend this with the details their files carry.
- NpuVendor
-
An NPU vendor. A class with constants rather than an enum, so a vendor
added in a later version does not break a
switchwritten against this one. - OptionScorer
-
An engine's calls for scoring options, on one loaded model: what its
InferenceModel.openOptionScorerreturns. Core makes one call at a time on a scorer (a decider runs its decisions one after another); every call runs on the model's own engine, ordered with its other native work. - OptionScoringRun
- What one OptionScorer.score call returned.
- OrphanedFileInfo
- Information about a potentially orphaned file
- ParallelFunctionCallResponse
- Multiple function calls in a single model response.
- ProcessedImage
- Represents a processed image ready for AI model consumption
- RankedOption
- One candidate of an answer.
- RateLimitedError
- HTTP 429 - Rate limit exceeded
- ResolvedChatRequest
- A ChatRequest with every value set, as the engine resolved it in InferenceModel.resolveChatRequest. Every field but runtimeInjectsToolDeclarations is required, so a value an engine forgets to carry over is a compile error, not a silent default.
- ResolvedHfFile
-
One file of a model resolved from a Hugging Face repo that comes as more
than one file: a member of a DIRECTORY model (an ORT-GenAI model is
genai_config.json+model.onnx+`model.onnx_data`+ tokenizer files), or an embedder's model file or its tokenizer file (see ResolvedHfModel.files). - ResolvedHfModel
- A model resolved from a Hugging Face repo's deployment metadata into flutter_edge_ai's install + runtime vocabulary.
- ResolvedSampling
- Sampling parameters with every field set: what an engine sends.
- ResponseFormat
- What every reply of a session must be: fixed for the session's lifetime.
- ResponseValidationResult
- Result of response validation
- RuntimeConfig
-
Runtime config for building a model — the per-call params
getActiveModelcollects, kept as a small holder so the provider contract stays stable as params evolve. - SamplingParams
- Sampling parameters for a session, as the caller asked for them.
- ScoreKind
- What a score in DeciderAnswer.ranked is.
- SdkResponseParser
- Parser for LiteRT-LM SDK Chat Completions JSON responses.
- SdkTextExtractor
- Pure-Dart extractor for LiteRT-LM SDK JSON response chunks.
- ServerError
- HTTP 5xx - Server error
- SessionMetrics
- Session metrics containing token usage and performance statistics.
- SessionRequest
- Everything an engine needs to open a session (InferenceModel.openEngineSession). A later option is a new field, not a new parameter on every engine.
- SkillExecutorProvider
-
A pluggable skill executor — the seam between core and the opt-in
flutter_edge_ai_agentpackage's agentic "skills" runtime. - SkillExecutorRegistry
-
Holds skill executors registered via
FlutterEdgeAi.initialize(skillExecutors:). Same probe-chain selection asEngineRegistry/EmbeddingRegistry: the registered executor with the highest SkillExecutorProvider.priority whose SkillExecutorProvider.canExecute returns true for a skill type wins (first-registered breaks ties). There is no central type map — the opt-influtter_edge_ai_agentexecutors self-select. - SpeechRecognizer
- Represents a speech-to-text model instance.
- SpeechSynthesizer
- Represents a text-to-speech model instance.
- StorageStats
- Storage statistics
- StructuredOutputSupport
-
What a loaded model can enforce: engine, platform and the backend it was
loaded on. Read it on
InferenceModel.structuredOutputSupport. - SttBackendProvider
- A pluggable STT backend (LiteRT C API, or a third-party backend). Same probe-chain shape as EmbeddingBackendProvider/InferenceEngineProvider: selected by probing the STT model spec, highest-priority first match.
- SttInstallation
- Result of STT model installation
- SttInstallationBuilder
- Fluent builder for STT (speech-to-text) model installation
- SttModelSpec
- Specification for STT models (model.tflite + tokenizer.json).
- SttVariant
-
A model
FlutterEdgeAi.installSttinstalls. - TextResponse
- Text token during streaming
- ThinkingResponse
- Thinking process content from the model
- TokenEmbeddingPass
- An EmbeddingPass for an engine that tokenizes in Dart. Text only.
- TokenizedInput
-
One text's tokenized form, ready for
TokenEmbeddingPass.runTokens. - TokenPassGraph
- What TokenEmbeddingPass.openGraph reports about the opened graph.
- Tool
- TtsBackendProvider
- A pluggable TTS backend (LiteRT C API, or a third-party backend). Same probe-chain shape as EmbeddingBackendProvider/InferenceEngineProvider: selected by probing the TTS model spec, highest-priority first match.
- TtsInstallation
- Result of TTS model installation.
- TtsInstallationBuilder
- Fluent builder for TTS (text-to-speech) model installation.
- TtsModelSpec
- Specification for a TTS model — a SELECTABLE bundle. ttsModelType carries the model family so one generic backend dispatches to the right runtime profile; sources is one entry per file in that type's manifest.
- TtsVariant
-
A model
FlutterEdgeAi.installTtsinstalls: a bundle of files under one base URL. Its files are exactly what core fetches for type under baseUrl (TtsModelType.manifest, each at its fetch location): the install checks their sizes, and refuses with ArgumentError a variant whose files name another URL. - HTTP 401 - Authentication required
- UnknownError
- Unknown error
- ValidationResult
- Result of image validation
- VisionEncoderValidator
- Validates images for compatibility with AI vision encoders to prevent corruption that causes models to interpret images as repeating text patterns.
- VisionSpecs
- Base class for vision encoder specifications
Enums
- ActivationDataType
-
Numeric type of the model's activations, for native
.litertlmmodels (Android, iOS, desktop). - CorruptionAction
- Actions to take when corruption is detected
- EdgeAiLogLevel
- Verbosity for flutter_edge_ai's internal logs.
- EmbeddingModality
- A kind of input an embedding model can turn into a vector.
- EmbeddingOutputContract
- How to turn a ForwardResult into the final embedding vector.
- ErrorType
- Types of errors that can occur in image processing
- MediaPosition
- Where a message's media go relative to its text.
- MessageMediaType
- Which media field a buffer belongs to: Message.imageBytes and Message.images are image, Message.audioBytes is audio.
- MessageType
- ModelFileType
- ModelManagementType
- Base enumeration for different model management types
- ModelReplacePolicy
-
Policy for what happens to a previously-installed model when a new one is
set active. Lives with the spec value types (it's a per-spec install policy);
re-exported from
model_file_manager_interface.dartfor backward compat. - ModelType
- The language-model families.
- PreferredBackend
- Hardware backend for model inference.
- ResponseAction
- Actions to take for corrupted responses
- StructuredDelivery
- How a structured reply reaches the consumer.
- SttModelType
-
Speech-to-text model families supported by the pluggable STT backends.
Only moonshine has a shipped
SttModelProfile/pipeline (flutter_edge_ai_speech); the others are follow-ons that need a log-mel frontend. - TaskType
- What an embedding is for. Every engine prepends prefix to the TEXT part of an input, native and web alike; media parts never get a prefix.
- ToolChoice
- Controls whether the model should call tools.
- TtsModelType
-
Text-to-speech model families supported by the pluggable TTS backends.
matcha, qwen3 and inflect have shipped
TtsModelProfile/pipelines (flutter_edge_ai_speech); kokoro/supertonic are documented follow-ons (fail-loud until wired). - TurnEnd
- How one generation turn ended.
- VisionEncoderType
- Vision encoder types with their specifications
- WebStorageMode
- Storage mode for web platform models
Mixins
- CloseNotifier
-
Lets the lifecycle OWNER (core) react when a model built by a separate
engine package closes. The package constructs the model; core registers a
reset callback via addCloseListener; the concrete model fires the
listeners from its own
close()'sfinally. Best practice: the enginecreateModel(spec, config)contract stays a pure factory — no lifecycle callback leaks into the contract that third-party engines implement. - RawSdkResponseSession
-
Mixin for sessions that surface the SDK's structured raw JSON response
(LiteRT-LM Gemma 4 path with
tool_calls). Allows InferenceChat to read the structured tool calls without a hard dependency on a concrete session type, and lets non-FFI sessions opt out by simply not implementing this mixin.
Extensions
- DownloadErrorMessage on DownloadError
- Extension to convert DownloadError to user-friendly messages
- MessageExtension on Message
- TtsModelTypeManifest on TtsModelType
- The filenames a given TTS model needs, installed together as a bundle from one source. Fail-loud for unwired families.
Constants
- defaultMaxFunctionBufferLength → const int
- Default maximum length for function call buffer before flushing as text. Must accommodate verbose formats (DeepSeek tags, parallel calls).
-
supportedLoraRanks
→ const List<
int>
Properties
- edgeAiLogLevel ↔ EdgeAiLogLevel
-
Active log level. Top-level (Dart-canonical for process-global logger
state) so edgeAiLog reads it without importing the public facade.
getter/setter pair
- isDesktop → bool
-
Check if current platform is desktop
no setter
Functions
-
detectImageMimeType(
Uint8List bytes) → String -
sniffEncodedMediaFormat's MIME type for an IMAGE, else
'image/png'— the fallback the LLM web arms have always used for bytes they could not identify. Audio bytes get the fallback too: this answers "what kind of image is this", and has never named an audio type. -
edgeAiLog(
String message, {EdgeAiLogLevel level = EdgeAiLogLevel.info}) → void -
The plugin's debug log. Silent in release builds and under
EdgeAiLogLevel.none; in debug builds it prints when
level<= edgeAiLogLevel. -
finalizeEmbedding(
EmbeddingOutputContract contract, ForwardResult result, {List< int> ? attentionMask}) → List<double> -
Turns a raw ForwardResult into the final vector per
contract. -
isLiteRtLmFilename(
String filename) → bool -
Whether
filenameends in.litertlm, compared ASCII case-insensitively (model.LiteRTLMmatches; onlyA–Zfold, so a non-ASCII look-alike never does). -
meanPoolAndNormalize(
ForwardResult result, {List< int> ? attentionMask}) → List<double> -
Mean-pools a token-level ForwardResult over the sequence axis and
L2-normalizes it into a single embedding vector of length
dim. -
sanitizeHfDirName(
String repo, {String? variant}) → String -
A filesystem-safe, single-segment directory name for a directory model
(ORT-GenAI) installed from the Hugging Face repo
repo, optionally from its execution-provider subfoldervariant. -
sniffEncodedMediaFormat(
Uint8List bytes) → EncodedMediaFormat? -
The container
bytesstart with, from their magic bytes, or null when they match none of EncodedMediaFormat.known. -
toolCallsOf(
Message message) → List< FunctionCallResponse> - The calls a MessageType.toolCall message carries, for an engine that replays a history.
Typedefs
-
EmbeddingTokenizerFactory
= Future<
EmbeddingTokenizer> Function(String tokenizerPath) - Factory that builds an EmbeddingTokenizer for a given on-disk tokenizer path.
-
MessageMediaReader
= Future<
Uint8List> Function(MessageMediaReference reference) - Returns the bytes a MessageMediaWriter stored under MessageMediaReference.reference, for MessageTranscriptCodec.decode.
-
MessageMediaWriter
= Future<
String> Function(MessageMedia media) - Stores MessageMedia.bytes and returns a reference the app can resolve later (a file name, a key), for MessageTranscriptCodec.encode.
Exceptions / Errors
- ContextLimitException
-
The error an engine puts on a turn's text when it refused an input too
long for what the context window had left, together with
endTurnEnd.contextLimit. The engine's context is as it was before the turn, and InferenceChat takes the turn's messages back out of its history. - DeciderCancelledException
-
A
createDecideror achoosewas cancelled through itsCancelToken, or the decider was closed while achooseran. A cancelledcreateDeciderreturns no decider. - DeciderException
- Common base of every decider failure. Not sealed, so a later kind of failure is an additive change; catch this to handle them all.
- DeciderSpaceMismatchException
- A decider was fitted in another embedding space than the embedder asked to load it, or than the one core now serves in place of the decider's own: another model file, dimension, backend or task prefix.
- DeciderUnsupportedException
-
The decider, or the model asked for one, cannot do what was asked — for
example an
InferenceModelthat does not score options, or a decider that takes no conversation context given some. - DownloadCancelledException
- Exception thrown when a download is cancelled
- DownloadException
- Custom exception for download failures
- EmbedderClosedException
- The model was closed, or closed before this request started — so the request never ran.
- EmbedderCloseTimeoutException
-
The previous embedder's close did not finish within the limit (60 s),
measured from the moment that close STARTED, so
FlutterEdgeAi.getActiveEmbedderbuilt nothing: two native models are never resident at once. - EmbeddingCancelledException
-
embedAllwas cancelled through itsCancelToken. No partial results are returned. - EmbeddingEngineException
- The engine failed while embedding.
- EmbeddingException
- Common base of every embedding failure. Not sealed, so a later kind of failure is an additive change; catch this to handle them all.
- EmbeddingFormatNotSupportedException
- A media part's container is not one the engine accepts (e.g. HEIC where the engine decodes JPEG/PNG/GIF/BMP only).
- EmbeddingInputTooLongException
- The input is longer than maxInputTokens — task prefix, BOS and EOS included. Nothing was truncated; split the text and embed the pieces.
- EmbeddingLoadException
- The engine could not load the model, or refused the requested configuration (e.g. a modality it cannot serve).
- EmbeddingModalityNotLoadedException
-
The input carries modality, which this model was not loaded with
(loaded is what it was loaded with). Declare media at load:
getActiveEmbedder(supportImage: true). - ImageProcessingException
- Exception thrown when image processing fails
- ImageTokenizationException
- Exception thrown when image tokenization fails
- IncompleteStructuredOutputException
-
The turn ended before the value was complete: end is
TurnEnd.outputLimit, TurnEnd.contextLimit, TurnEnd.stopped or
TurnEnd.closed. partialText is what was generated; in an
InferenceChatit is also in the chat's history, except for a turn stopped before any text, which the chat takes back, its user message included. A stop is incomplete even when partialText parses. - ModelStorageException
- Exception thrown when model storage operations fail
- ModelValidationException
-
An installed model file is not what it should be: smaller than a model
can be, or not the size the install asked for (
expectedSizeon the builders, and the size of everyfromCataloginstall; not checked on the web). A file that fails the size check is not made active, and is deleted with its record so the next install downloads it again — unless an active model reads it, or the delete fails, which the message says. - NoActiveModelError
-
Thrown when a model is asked for and none is installed and active: the
app has to install one first (
FlutterEdgeAi.installModel()orinstallEmbedder()). - NonConformingStructuredOutputException
- The turn ended normally but text is not a value of the format (not JSON, trailing text, or a dialect keyword violated at schemaPath): an engine accepted a format it did not enforce. An engine bug.
- ResponseFormatUnsupportedException
-
The format is refused by this engine, platform or backend, or a schema
keyword at schemaPath cannot be enforced. Returned (not thrown) by
InferenceModel.refusalFor;openSessionandopenChatthrow that refusal before opening an engine session. An engine whose native constraint compiler can reject a schema only at first send may instead put this exception on that turn's text; core passes it through and takes the turn back. - StructuredOutputException
- Base of the typed-response failures. Sealed: apps catch these, engines never subclass them, and a fourth kind is a deliberate break.
- UnsupportedEmbeddingCombinationException
- The input carries several parts — several modalities, or several images (requested names its modalities) — and the engine embeds one part per vector.
- VisionEncoderValidationException
- Exception thrown when validation fails