internal_io library

Classes

BenchmarkInfo
What LiteRT-LM's benchmark reports about a conversation: its tokens over every turn so far, the last decode's speed, the first token's latency and the engine's load time. LiteRT-LM's Kotlin/Swift BenchmarkInfo, summed over the conversation rather than its last turn.
BundleInfo
What a .litertlm file says about itself, read before any engine exists: its container version, its system metadata, and its sections with their declared constraints and the NPU each compiled section targets.
BundleSampler
The sampler a .litertlm file declares: each value null when the file does not set it.
BundleSamplerRead
The outcome of reading a bundle's sampler: sampler, empty when the bundle ships none, or the error that stopped the read.
BundleSection
One section of a .litertlm container.
CancelSignal
A cancel the runtime can check and wait for. The adapter passes core's CancelToken through it.
Constraint
The LLGuidance constraint of a conversation: a JSON Schema or a regex.
ConversationExtras
A conversation's extras beyond what every conversation is created with.
ConversationHandle
Per-conversation operations a session needs from the FFI layer.
EmbedderInfo
What a loaded embedder reports.
EmbedderInput
One embedding request: text, images and audio, embedded as one vector.
EngineBundle
A model file as readEngineBundle read it.
EngineCreateRequest
Everything litert_lm_engine_create is made from, decided on the calling isolate. Plain values only, so it crosses to the isolate that builds the engine by copy: the settings are created, used and freed there (createEngineOffMainIsolate), and no settings pointer is ever valid on two isolates.
EngineHost
What Engine.create reaches outside Dart, behind one object so a VM test can drive a whole load without a native library or a device.
EngineSamplerLatch
LiteRT-LM builds an engine's sampler at its first generation and keeps it for every later conversation (google-ai-edge/LiteRT-LM#2080), so a later session that asks for another sampler does not get it. This records the first and says what a later generation should report.
ExecutorConversation<C extends Object>
Which conversation's turn LiteRT-LM's executor ran last, as far as the client knows. A send to any other (or when it is not known) switches KV caches inside the native send call, which then runs off the calling isolate.
LibraryLoadFailure
An error thrown while libLiteRtLm, or a library it needs, loads or is checked: what Engine.create reports as a LiteRtLmLibraryException. Each keeps the type the load threw (the Flutter adapter's backend fallback tells an Exception from an Error), so this marks it instead of replacing it.
LiteRtLmConversationHandle
One conversation owned by a LiteRtLmFfiClient. Each call to LiteRtLmFfiClient.createConversationHandle returns a fresh handle: the engine pointer is shared across handles, the conversation pointer is private to this handle. The handle owns that pointer's lifetime and routes every per-conversation native call through the client's private _…On(conv, …) methods.
LiteRtLmEmbedder
The .litertlm embedder: text, images on a bundle with a vision encoder, audio on a bundle with an audio encoder, and any combination of them in one vector; CPU, or the NPU for a per-SoC bundle.
LiteRtLmEmbeddingConfig
How to create a LiteRtLmEmbedder. All of it is sendable to another isolate (flutter_edge_ai_litertlm's describe() hands it to core's worker).
LiteRtLmFfiClient
High-level Dart wrapper around the LiteRT-LM C API.
LitertlmFlatBuffer
Tables, vectors of tables, strings, union discriminators and scalar fields, little-endian, over the header bytes. Out-of-range offsets throw RangeError — checked here, before every read, rather than left to ByteData: on the web the bytes are a JS-backed view, and an out-of-range read there is the DataView's own JS RangeError, which dart2js delivers as an ArgumentError and dart2wasm as a JS exception no on clause matches (measured in the browser suite on Chrome 155).
LiteRtLmNpuHost
What choosing an NPU vendor asks of the host. The real one is LiteRtLmPlatformNpuHost; tests pass a fake, because every answer here is a property of a device a test host is not.
LiteRtLmPlatformNpuHost
MediaOrder
Where a message's images and audio go relative to its text.
NativeTurn
One native operation on a conversation — a streamed send or a rewind — from its start until native has posted its end.
NativeTurns<C extends Object>
The native operation running on each conversation. Nothing deletes a conversation, and no later operation starts on it, before the one that runs has ended: a cancel only asks native to stop, and the task — and its callbacks — finish later.
NativeTurnStream<C extends Object>
The stream of one native send, fed by the native callbacks.
NpuChoice
Which vendor's NPU a load runs on (NpuChosen), or why none (NpuRefused).
NpuChosen
The load runs on vendor's NPU.
NpuPlatform
What the NPU code needs from the app's platform layer that plain Dart cannot answer: the build's NativeAssetsManifest.json, and on Android a dispatch directory the app's native side prepares. The Flutter one is FlutterNpuPlatform (flutter_npu_platform.dart), which reads the manifest through rootBundle and asks core's flutter_gemma_bundled channel.
NpuProbes
Each vendor's device probe: null when it passes, else why not. One member per probe, so a vendor whose probe is added to npuUnreachable is a compile error in every implementation until it answers.
NpuRefused
No NPU this host reaches can run the load; reason says why, worded for the person who has to fix it.
NpuStackCheck
Whether this build carries a vendor's NPU stack (NpuStack) where LiteRT will load it, and why not: from NativeAssetsManifest.json for a stack the build registers (checkNpuStackInManifest; Qualcomm on Android and Linux arm64), by file presence for Intel's beside the Windows executable (checkIntelNpuStack). The LLM engine and the EmbeddingGemma 2 placement both ask, through npuStackOf.
NpuStamp
The SoC a .litertlm bundle's compiled sections target: the LiteRtStamp the AOT compiler writes into each section it compiles, with the vendor and SoC spelled as the compiler stamped them.
RecoveringConversationHandle
The handle of a session's own native conversation — createSession's, and each openSession session's where they are native (OpenSessionBacking.native) — made to answer again after a turn is stopped or its input did not fit.
Sampler
The sampler sent to LiteRT-LM, every value set.
ScoringRun
What one option-scoring run returned.
ScoringStops
The stop flags of one owner's scoring runs — an option scorer's — so that closing it stops what it has in flight and refuses what it starts later. Main isolate only: the flags are set and freed there.
SendCallGate
One streamed send whose native call may run on another isolate. While that call is inside native, LiteRT-LM may already have called back with the turn's end (a prefill refused at once): ending the turn then would hand the engine lock on, let a close delete the conversation or a shutdown delete the engine, under a call still running on them. So an end that arrives during the call is held and applied once the call has returned (flushEnd); and the message strings the call reads are freed only when both the stream is done with them (release) and the call has returned.
SendOptions
One send's optional args beyond thinking and the constraint.
VirtualConversationHandle
A ConversationHandle backed by the virtual-session multiplexer.

Enums

ActivationDataType
The numeric type of a GPU model's activations, LiteRT-LM's ActivationDataType without I16/I8.
AndroidGpuFamily
The Android GPU families whose drivers this package tells apart, by the device node the driver exposes.
BackendKind
Where LiteRT-LM runs a model's main graph, or one of its encoders. Its name is LiteRT-LM's wire name (cpu, gpu, npu).
EmbedderModality
A kind of input an embedder takes.
EmbeddingPartKind
What one part of an embedding request carries.
GpuActivationReason
Why gpuActivationFor (Engine.create) chose the activation of a GPU load, for the engine's log line.
LiteRtLmBundleProblem
Why a .litertlm file cannot be read or loaded as asked.
LiteRtLmEg2Bundle
The generic EmbeddingGemma 2 bundles, told apart by the towers their container carries (readLiteRtLmMediaTowers).
LiteRtLmLogLevel
The minimum severity of LiteRT-LM's own native log: LiteRtLmLogSeverity of its C API (litert_lm_set_min_log_level).
LlmModelFamily
The model families whose LlmModelType the runtime acts on (the GPU activation rules of Engine.create); every other family is other.
MacOSLoadCause
Why LiteRT-LM did not load on macOS, read from dyld's error text.
MediaSide
Which side of a message's text a kind of media goes.
NativeLastTurnEnd
How native's last send ended (LiteRtLmTurnEnd in the patched c/conversation.h), as litert_lm_conversation_get_last_turn_end reports it after a send (§23) and litert_lm_conversation_rewind_cancelled_turn after waiting for one (§16). In the C enum's order: the index is the code.
NativeTurnEnd
How a native operation on a conversation ended.
NpuStack
A vendor whose NPU this package dispatches to.
OpenSessionBacking
How each openSession session of a loaded model holds its context.
RuntimeLogLevel
How much runtimeLog lets through: nothing, info messages, or info and verbose ones (model output, prompts, native log contents).
StoppedTurnRewind
What ConversationHandle.rewindStoppedTurn did.

Extensions

LiteRtLmLastErrorRead on LiteRtLmBindings
LiteRT-LM's thread-local last error, read on the calling thread.

Constants

absResourceExhausted → const int
The absl status code RESOURCE_EXHAUSTED.
gemma4Fp16MaxContextTokens → const int
The largest context Gemma 4 keeps at the GPU's FP16 default: above it a GPU attempt runs at FP32 (LiteRT-LM #4001), except on Android.
googleTensorNpuLibs → const List<String>
The complete app-bundled Google Tensor stack: the bridge only.
intelBrokenNpuMaskAux → const List<BrokenNpuMaskAux>
The tf_lite_aux sections whose NPU-compiled mask graphs are wrong, so npuHwMasking has the runtime compute the masks for them.
isDebugBuild → const bool
Neither a release nor a profile build. Flutter's kDebugMode, which is false in a profile build too.
isFlutterDebugBuild → const bool
A debug build of a Flutter app (flutter run, flutter test): Flutter's kDebugMode where the code runs under Flutter, false under dart run and dart test. dart.library.ui is defined only by Flutter's compilers.
isProductBuild → const bool
dart.vm.product: a release (product) build. Flutter's kReleaseMode.
isProfileBuild → const bool
dart.vm.profile: a profile build. Flutter's kProfileMode.
isReleaseBuild → const bool
A release build: isProductBuild under the name of Flutter's kReleaseMode. A profile build is not one.
liteRtLmAudioBundle → const String
The bundle with an audio encoder.
liteRtLmCApiMajor → const int
The LiteRT-LM C API major version the bindings are generated for (LITERT_LM_C_API_VERSION_MAJOR in the vendored c/engine.h).
liteRtLmConstrainedDecodingRows → const Set<LiteRtLmConstrainedDecodingRow>
Rows that passed the discriminating structured-output integration suite.
liteRtLmDefaultVisionTokensPerImage → const int
The vision budget an engine is created with unless the app asks for 70.
liteRtLmEmbeddingGpuAbis → const Set<String>
The hosts, in Abi.current() spelling, where a generic EmbeddingGemma 2 bundle runs on the GPU when preferredBackend: gpu asks for it: LiteRT-LM's GPU executor, over the WebGPU accelerator these archives ship on Linux and Windows and the Metal accelerator linked into libLiteRtLm on macOS.
liteRtLmEmbeddingGpuMeasured → const Map<String, Set<LiteRtLmEg2Bundle>>
Which generic EmbeddingGemma 2 bundles have run with their text tower on the GPU of each liteRtLmEmbeddingGpuAbis host; a gpu request for any other is refused there, naming the missing run. Linux x64: all three since litertlm 1.19.0 (measured on the 270m; the others run the same text tower with their media towers on CPU). Windows x64: the 270m on Panther Lake and Lunar Lake (L1, 2026-11-04). macOS: the 270m and the 740m text towers on Metal (L1 C probe; the 270m and 740m through this pass, L2).
liteRtLmEmbeddingGpuRefusedAbis → const Set<String>
The hosts where preferredBackend: gpu for a generic EmbeddingGemma 2 bundle is refused with an EmbeddingLoadException (liteRtLmEmbeddingGpuRefusal) instead of running anywhere: a host whose GPU path ends the process, which no Dart code can catch, so the request is refused rather than run on the CPU (the app asked for the GPU). None on native-v0.19.0-a: Windows x64 was here on native-v0.18.0-h (#744, an access violation in webgpu_dawn.dll at the first EmbeddingGemma 2 GPU load) and runs on 0.19 (liteRtLmEmbeddingGpuAbis).
liteRtLmEmbeddingSymbols → const List<String>
Every C function the pass calls. A libLiteRtLm that exports fewer is refused at load by checkLiteRtLmEmbeddingSymbols, by name, instead of failing on the first lookup somewhere inside the worker.
liteRtLmFreshConversationNpus → const Set<NpuStack>
The NPU stacks on which a native conversation answers one message only: on MediaTek (MT6989, LiteRT-LM 0.18.1 and 0.19) the second message in a conversation comes back empty (LiteRT-LM #4256; measured on a DDP Galaxy Tab S10+ 2026-11-01, doc/research/mediatek-dispatch-0.19.md). Every turn there runs in a fresh conversation that replays the history as full messages, as a rebuild after a stop does.
liteRtLmMaxInputLength → const int
The input length every EmbeddingGemma 2 engine is created with, in tokens.
liteRtLmMinWeightCacheBytes → const int
The smallest file that counts as a weight cache. XNNPack writes a header of about 100 bytes when it opens the file, then the packed weights (71 MB for the 270m's text tower); anything below this is what a failed or partial header write left. The threshold example/integration_test/embeddinggemma2_test.dart holds a device run to.
liteRtLmNativeAbis → const Set<String>
The hosts, in Abi.current() spelling, the native archive is published for (hook/build.dart, one litertlm-<host>.tar.gz each; the iOS simulator's ios_sim_arm64 archive reports ios_arm64). Anywhere else the app builds without libLiteRtLm, and nothing of this package loads.
liteRtLmNativeVersion → const String
The native-v* release suffix (LiteRT-LM 0.19.1, our build c).
liteRtLmOwnBuildSymbol → const String
A symbol every libLiteRtLm this package ships exports and upstream LiteRT-LM does not: patch_c_api.sh §4c adds it and the Windows .def there names it, and the release gate (check_embedding_exports.py, OWN_BUILD_SYMBOLS) refuses a native archive without it. Not tied to an optional feature (NPU, embeddings): it is an engine setting compiled on every platform.
liteRtLmVisionTokenBudgets → const List<int>
The vision budgets EmbeddingGemma 2 has signatures for.
maxGenericDataBytes → const int
The largest section readBundleGenericData reads by default: a bigger range is taken for a corrupt file rather than read into memory.
minCpuGpuContextTokens → const int
The smallest context window a CPU or GPU .litertlm engine is created with: the kv_cache_max_len the published CPU/GPU bundles bake. Below it the native KV-cache resize underflows and generation fails to allocate tensors (#318: 100, 256 and 512 crash on a Pixel 8a, 1024 works). An NPU bundle has one compiled cache length, which can be lower, so the NPU is left as asked.
qualcommNpuNotEnabledReason → const String
What an app that did not opt in sees when it asks for npu.
thoughtChunkPrefix → const String
How LiteRtLmFfiClient.extractTextFromResponse marks a thought chunk.

Properties

brokenNpuMaskAuxInUse → List<BrokenNpuMaskAux>
The list the engine's loads check: intelBrokenNpuMaskAux, unless a test replaced it (debugIntelBrokenNpuMaskAux).
no setter
capturesNativeStderr → bool
Whether loading the libraries moves native stderr to a log file (litertlm_native.log in the system temp directory): in a Flutter debug build on iOS, macOS and Linux, as before the runtime left the adapter; never in a plain Dart program or a profile or release build.
no setter
debugEngineHost ↔ EngineHost
The host every Engine.create goes through; a test replaces it.
getter/setter pair
debugIntelBrokenNpuMaskAux ← List<BrokenNpuMaskAux>?
Replaces brokenNpuMaskAuxInUse; null restores intelBrokenNpuMaskAux.
no getter
liteRtLmConstrainedDecodingCandidateRows → Set<LiteRtLmConstrainedDecodingRow>
Every row that may be qualified in the 2.6.0 device campaign.
final
liteRtLmNativeLogLevel ↔ LiteRtLmLogLevel?
The native log level Engine.create sets before every engine; null leaves the process's (LiteRT-LM's default until one is set). The level is process-wide, as LiteRT-LM's is.
getter/setter pair
runtimeLogLevel ↔ RuntimeLogLevel
The level of this isolate's default sink. Set from the spawner's currentRuntimeLogLevel in each isolate the runtime spawns.
getter/setter pair
runtimeLogLevelSetByApp ↔ bool
The app set the level through LiteRtLmLog.level on this isolate: the Flutter adapter then leaves runtimeLogLevelSource alone.
getter/setter pair
runtimeLogLevelSource ↔ RuntimeLogLevel Function()?
Where currentRuntimeLogLevel reads the level from, when not runtimeLogLevel: the adapter points it at core's edgeAiLogLevel.
getter/setter pair
runtimeLogSink ↔ RuntimeLogSink
The sink runtimeLog writes to in this isolate; print, filtered by currentRuntimeLogLevel, unless replaced.
getter/setter pair
runtimeLogSinkSetByApp ↔ bool
The app set the sink through LiteRtLmLog.sink on this isolate: the Flutter adapter then leaves runtimeLogSink alone.
getter/setter pair
sendOffMainIsolate ↔ Future<({int code, String? message, int micros})> Function(({int code, String? message, int micros}) work())
How a switching send reaches a spawned isolate: Isolate.run, replaced by tests.
getter/setter pair

Functions

activationDataTypeWireValue(ActivationDataType? type) → int?
The C API's number for type (ActivationDataType in LiteRT-LM's executor_settings_base.h: 0 F32, 1 F16, and 2 I16 / 3 I8 which the Dart enum does not offer), or null to leave the engine setting untouched. Spelled out rather than taken from index, so reordering the Dart enum cannot silently change what native receives.
androidGpuFamilyFrom(bool exists(String path)) → AndroidGpuFamily
The GPU family of an Android device whose device nodes exists reports: /dev/kgsl-3d0 Adreno, /dev/mali0 Mali, /dev/pvr_sync PowerVR, anything else (and a node the app may not stat, which reads as absent) AndroidGpuFamily.other.
androidLibFileName(String name) → String
The file name name gets on Android and Linux (lib<name>.so).
applyThinkingTokenBudget(LiteRtLmBindings b, Pointer<LiteRtLmConversationOptionalArgs> optionalArgs, {required bool thinking, required int? budget}) → void
Gives one send's optionalArgs a thinking token budget of budget when the send thinkings and a budget is set; otherwise touches nothing, so a send without a budget reaches LiteRT-LM exactly as before.
backendKindOf(Backend backend) → BackendKind
The runtime's name for backend.
bindLiteRtLmEmbeddings(DynamicLibrary library, {String label = 'libLiteRtLm'}) → LiteRtLmEmbeddingBindings
Binds library after checking it exports everything the pass calls.
bundleSamplerFrom({required int type, required double temperature, required int topK, required double topP, int? seed}) → BundleSampler
The bundle's SamplerParameters as BundleSampler.
checkEmbeddingRequest(List<Content> contents, {required Set<EmbedderModality> modalities, required bool hasText, required bool hasImages, required bool hasAudio}) → void
Refuses, as an ArgumentError, contents with no part, or with images or audio an engine of modalities was not loaded for.
checkIntelNpuStack(String dir) → NpuStackCheck
Whether dir holds intelNpuDispatchLib.
checkLiteRtLmCApiVersion(bool provides(String symbol), ({int code, int major, int minor, int patch}) readVersion(), {required String path}) → void
Throws StateError unless the library opened as path reports a C API version liteRtLmCApiMajor.x through litert_lm_get_api_version (readVersion, called only when provides has it).
checkLiteRtLmEmbeddingSymbols(bool provides(String symbol), {required String library}) → void
Throws EmbedderLoadException naming every symbol of liteRtLmEmbeddingSymbols that provides reports missing from library.
checkLiteRtLmLibrary(bool provides(String symbol), ({int code, int major, int minor, int patch}) readVersion(), {required String path}) → void
Both checks every open of libLiteRtLm makes, in order: checkOwnLiteRtLmBuild, then checkLiteRtLmCApiVersion.
checkNpuStackInManifest(NpuStack vendor, {required NativeAssetsManifestReader read, bool cache = false, String? abi, String? operatingSystem}) → Future<NpuStackCheck>
Whether this build registered vendor's NPU stack on operatingSystem (npuStackLibs).
checkOwnLiteRtLmBuild(bool provides(String symbol), {required String path}) → void
Throws StateError when provides reports liteRtLmOwnBuildSymbol missing from the library opened as path.
chooseNpuVendor({required NpuStamp? target, required LiteRtLmNpuHost host}) → Future<NpuChoice>
The one rule for which vendor's NPU a load runs on, for the LLM engine and the EmbeddingGemma 2 placement alike.
concatenatedJsonObjects(String text) → Iterable<Map<String, Object?>>
The JSON objects of text, a run of concatenated JSON values; anything that is not an object, or not JSON, is skipped.
contextTokensFor(int maxNumTokens, BackendKind backend) → int
The context maxNumTokens is created with on backend: raised to minCpuGpuContextTokens on the CPU and the GPU, with a log line saying that the context window is not the reply length.
conversationForTest(ConversationHandle handle, {ConversationConfig? config, bool engineClosed()?}) → Conversation
A conversation over handle, for tests that drive it with a fake.
conversationHandle(Conversation conversation) → ConversationHandle
The handle under conversation.
conversationTokenCount(Conversation conversation, String text) → Future<int?>
How many tokens text is, by conversation's model's own tokenizer; null when the engine cannot tell (closed, or the native call failed).
createConversationWithDefault(Engine engine, ConversationConfig config, SamplerConfig? defaultSamplerConfig) → Future<Conversation>
Engine.createConversation with the model's own sampler default when the config sets none: the values flutter_edge_ai_litertlm resolves (the file's sampler, then the family's published defaults, then the fallback), which never count as the caller's choice (LiteRT-LM#2080's one-time warning fires only for a sampler the caller set). Null takes the file's sampler, then 0.8 / 40 / 0.95 / seed 1.
createEmbeddingEngineWith(LiteRtLmEmbeddingConfig config, {LiteRtLmEmbeddingBindings openBindings()?, Allocator allocator = calloc, DateTime clock()?}) → Future<EmbeddingEngine>
An EmbeddingEngine over the embedder config config, already placed: for a caller that decides the placement itself, on another isolate (the Flutter adapter places on the app's isolate, where the platform channel that prepares an Android NPU dispatch directory is, and loads on a worker). openBindings, allocator and clock are for tests.
createEngineWithBundle(EngineConfig config, EngineBundle bundle, {ToolCallReader toolCallsOf = nativeToolCalls, LiteRtLmLogLevel? nativeLevel, GpuActivationDecision? gpuActivation}) → Future<Engine>
Engine.create with the file already read: a caller that reads it to plan its attempts does not pay for the read again in each one (an in-app read took ~0.57 s on a Linux VM). bundle must be config.modelPath's.
createOwnConversation(Engine engine, [ConversationConfig? config, SamplerConfig? defaultSamplerConfig]) → Future<Conversation>
A conversation of engine with a native conversation of its own whatever the engine's backing: what flutter_edge_ai_litertlm's singleton createSession always had, also where the engine's other conversations share one (one live conversation runs on every backend).
currentRuntimeLogLevel() → RuntimeLogLevel
The level this isolate logs at now, to hand to an isolate it spawns.
decodeNativeString(Pointer<Char> ptr, {int? maxBytes}) → String
Decode a NUL-terminated native string without throwing on malformed bytes.
embedderConfigFor(EmbeddingEngineConfig config) → Future<LiteRtLmEmbeddingConfig>
Where config runs, as the embedder's own config: the bundle's NPU stamp, read from its bytes, decides; nothing moves to another backend on its own.
embeddingParts(EmbedderInput input) → List<EmbeddingPart>
The parts of input, in request order: the text (its task prefix already in front), then the images, then the audio clip.
encoderBackendWireName(BackendKind? encoderBackend) → String
Wire name for an ENCODER backend (vision / audio), defaulting to cpu when unset. Shared by both because their defaulting is identical (null → cpu).
engineBundleSampler(Engine engine) → BundleSampler
The sampler engine's model file carries.
engineClient(Engine engine) → LiteRtLmFfiClient
The client under engine.
engineConversationBacking(Engine engine) → OpenSessionBacking
Whether engine's conversations have a native conversation each, or share one by replaying their messages.
engineFiltersChannelContent(Engine engine) → bool
Whether LiteRT-LM may strip a finished reply's thought from the KV cache of engine's conversations.
engineForTest(LiteRtLmFfiClient client, {Backend backend = const Backend.cpu(), int maxNumTokens = 4096, OpenSessionBacking backing = OpenSessionBacking.virtual, bool? linearAttentionState = false, NpuStack? npuVendor, String? constraintRefusal}) → Engine
An engine over client, for tests that drive conversations with a fake client.
engineNoRewindReason(Engine engine) → String?
Why engine's conversations are rebuilt instead of rewound after a stop or a full context; null when they rewind.
engineNpuVendor(Engine engine) → NpuStack?
The NPU vendor engine loaded on; null off the NPU.
engineRenderUserMessage(Engine engine, String content) → Future<String>
content rendered as a user message by engine's prompt template, with the turn's generation prompt: what option scoring continues.
executableDirOf([String? executable]) → String
The directory of executable — by default Platform.resolvedExecutable, the running app.
extractResponseText(String jsonStr) → String
The text of the LiteRT-LM JSON response chunk jsonStr: a thought as <|channel>thought\n…<channel|>, a string content as it is, the text parts of a content list joined, and anything that is not JSON as it came. A content of any other type is a contract change and throws StateError.
ffiBackendFallbackOrder(BackendKind? preferredBackend, {required bool npuAvailable}) → List<BackendKind>
The backends to try, in order, for a preferredBackend request, the NPU only when npuAvailable.
ffiBackendWireName(BackendKind backend) → String
genericDataRange(Uint8List header, String name, {required int headerEnd, int maxBytes = maxGenericDataBytes}) → (int, int)?
The [begin, end) byte range of the GenericBinaryData section named name, from the header flatbuffer that follows the 32-byte preamble (the file's header ends at headerEnd), or null when the bundle has none. A section that carries the name in another form — on a section of another data type, or as a value that is not a string — is a FormatException, not "none": it is data this release cannot read.
gpuActivationFor({required LlmModelFamily? family, required bool? declaresActivation, required int maxNumTokens, required bool android}) → ({GpuActivationReason? reason, ActivationDataType? type})
The activation type a GPU load runs at when the caller set none: float32 for Gemma 4 with a context above gemma4Fp16MaxContextTokens (its float16 rotary positions stop being distinct there, LiteRT-LM #4001), and for a FunctionGemma bundle that declares no prefer_activation_type (converted before litetune 0.1.4: at float16 it answers with <pad> or nothing). family is the bundle's llm_model_type; declaresActivation whether it declares a type (null: unread). Null leaves the bundle's own.
isUsableLiteRtLmBuild(DynamicLibrary library) → bool
Whether library passes requireOwnLiteRtLmBuild: this package's build, on C API liteRtLmCApiMajor.x. For choosing among development copies on a host, where a stale one is skipped rather than refused.
keepLastSendAfter(NativeTurnEnd how, NativeLastTurnEnd? end) → bool
Whether a conversation keeps the last message it sent once that send has ended as how, with native's turn-end record end: after a cancelled send, the one a rewind puts back, and after one native ended context_full (section 23), which L3 rewinds to before (core 2.6.0 design §6.4b).
liteRtLmCacheDirName(String modelFileName, {required int size, required DateTime modified}) → String
The cache directory name for one copy of a bundle: <liteRtLmCacheDirPrefix(file)>-<size>-<mtime ms>, both numbers in base 36, so a re-downloaded or replaced file never reads the weight cache of another (the engine's own cache file names carry the same pair).
liteRtLmCacheDirPrefix(String modelFileName) → String
The part of liteRtLmCacheDirName that names the file: the first 8 hex digits of the SHA-256 of its name. Directories with this prefix and another size or time are stale copies of the same file.
liteRtLmCacheFilePathLength(String cacheDir, String modelFileName, int modelSize, {String tower = _textTower}) → int
The length of the path LiteRT-LM gives tower's weight cache in cacheDir: <cacheDir>/<model file name>_<mtime s>_<size><tower>.xnnpack_cache (executor_settings_base.cc GetWeightCacheFile, file_util.cc GetFileCacheIdentifier), with the mtime's seconds taken as 10 digits (every date from 2001 to 2286). The text tower's by default — the vision tower's is 2 characters longer, the audio tower's 8 or 11 — so this is what Windows' MAX_PATH applies to, not the directory's length.
liteRtLmConstrainedDecodingRefusal({required String abi, required BackendKind backend, AndroidGpuFamily? androidGpu, NpuStack? npuVendor}) → String?
Why a response format is refused for a model loaded on backend on the host abi, or null where that row passed (liteRtLmConstrainedDecodingRows).
liteRtLmConstrainedDecodingRow({required String abi, required BackendKind backend, AndroidGpuFamily? androidGpu, NpuStack? npuVendor}) → LiteRtLmConstrainedDecodingRow
liteRtLmConstraintFailure(String? jsonSchema, String? nativeError) → ConstraintUnsupportedException?
The one native send failure that means the schema, rather than inference, was refused. CreateConstraint runs only when this send carries a schema; every other INTERNAL remains the native inference failure it is.
litertlmDeclaresActivationType(Uint8List header) → bool
Whether any TFLiteModel section in header declares prefer_activation_type: the LiteRTLMMetaData flatbuffer that follows the 32-byte preamble.
liteRtLmEncoderOnNpu({required BundleInfo? bundle, required bool supportImage, required bool supportAudio, BackendKind? preferredVisionBackend, BackendKind? preferredAudioBackend}) → bool
Whether an encoder the load turns on runs on the NPU of a bundle that was read, by the rule planLiteRtLmLoad applies (the request, else _encoderDefault). The engine asks it with liteRtLmTextCandidates, so that it looks for the app's NPU stack also when only an encoder needs the NPU. A request the plan then refuses may count: the plan throws before the stack's answer is used. A file that could not be read keeps the old rules.
liteRtLmFiltersChannelContent({required bool npu, required bool? linearAttentionState}) → bool
Whether LiteRT-LM may strip a completed reply's channel content (a thought) from the KV cache: only when the conversation is known to rewind, i.e. not on the NPU (npu) and on a model known to keep no linear attention state (linearAttentionState false; null, unread, counts as cannot rewind).
liteRtLmFreshConversationEachTurn({required BackendKind? backend, NpuStack? npuVendor}) → bool
Whether every turn of a model loaded on backend (the NPU stack npuVendor) needs a fresh native conversation that replays the history (liteRtLmFreshConversationNpus), in createSession and openSession sessions alike.
liteRtLmGpuCacheFileNames(String modelFileName, int modelSize) → ({String program, String weight})
The names LiteRT's WebGPU accelerator gives the text tower's GPU caches in a cache directory: LiteRT-LM sets the serialization directory and the model cache key <model file name>.text_encoder_<mtime s>_<size> (litert_compiled_model_executor_utils.cc, file_util.cc), and ML Drift appends _mldrift_weight_cache.bin / _mldrift_program_cache.bin. The mtime's seconds are taken as 10 digits, as in liteRtLmCacheFilePathLength. The program cache's is the longer, by one.
liteRtLmGpuCachePathCheck(String cacheDir, String modelFileName, int modelSize, {required bool windows, bool ansiCodePageIsUtf8 = false}) → LiteRtLmGpuCachePathCheck?
The check behind liteRtLmGpuCachePathProblem. On Windows the accelerator gets the path as UTF-8 and opens each cache through the UCRT _open, which reads it in the ANSI code page, then CreateFileW without the \\?\ prefix (read from the disassembly of the shipped LiteRtWebGpuAccelerator.dll). So, unless that code page is UTF-8 (ansiCodePageIsUtf8):
liteRtLmGpuCachePathProblem(String cacheDir, String modelFileName, int modelSize, {required bool windows, bool ansiCodePageIsUtf8 = false}) → String?
Why LiteRT's WebGPU accelerator cannot create the GPU caches of modelFileName in cacheDir, decided from the path alone before the engine exists; null when both fit. liteRtLmGpuCachePathCheck says which of the two each reason applies to.
litertlmHasAudioEncoder(Iterable<String> modelTypes) → bool
Whether a bundle whose TFLiteModel sections have modelTypes carries an audio encoder.
litertlmHasAudioTower(Iterable<String> modelTypes) → bool
Whether a bundle whose TFLiteModel sections have modelTypes carries an audio tower: a tf_lite_audio_encoder_hw section, the one key LiteRT-LM builds its audio executor for (ModelType::kTfLiteAudioEncoderHw); only the 740m has it, beside tf_lite_audio_adapter and tf_lite_end_of_audio. Compiled at create like the vision tower (litertlmHasVisionTower).
litertlmHasVisionTower(Iterable<String> modelTypes) → bool
Whether a bundle whose TFLiteModel sections have modelTypes carries a vision tower: a tf_lite_vision_encoder section, the one key LiteRT-LM builds its vision executor for (runtime/core/embedding_engine_impl.cc, ModelType::kTfLiteVisionEncoder). The 440m and the 740m have it beside tf_lite_vision_adapter and tf_lite_end_of_vision, neither of which gets an executor of its own; the 270m has only tf_lite_text_encoder and tf_lite_embedder. LiteRT-LM compiles a present tower at create, declared or not: on its own backend when one is named, else on the main one.
litertlmHeaderEnd(Uint8List preamble) → int
Where the header flatbuffer ends, from the 32-byte preamble: the magic LITERTLM, three uint32 version fields, 4 bytes of padding and a uint64 header end (LiteRT-LM schema/core/litertlm_read.cc, pin b2f686e2).
liteRtLmHostLibraryCandidates({String? envOverride}) → List<String>
macOS-only fallbacks for libLiteRtLm.dylib, used when there is no app bundle to resolve LiteRtLm.framework from — i.e. flutter test on the host. Never reached inside a shipped app.
liteRtLmLoadNpuTarget({required String path, required BundleInfo? bundle, required bool supportImage, required bool supportAudio, required String abi}) → NpuStamp?
The NPU a load of bundle runs on when it runs on one: the text model's (its own stamp, else tf_lite_aux's), else that of an encoder the load turns on. Null for a bundle compiled for no NPU, and for one that was not read. The engine asks it before the plan, to look up that vendor's stack.
liteRtLmNoAudioEncoderMessage(String file) → String
Why file cannot embed audio.
liteRtLmNoWeightCacheCauses({required int cachePathLength, required bool windows}) → String
Why the engine wrote no weight cache, for the refusal and the warnings, given the length of the cache file path (liteRtLmCacheFilePathLength). On Windows from 260 characters MAX_PATH is the likely cause; otherwise every cause that leaves a tower uncached is listed. A disk that fills while the cache is built is not one of them: XNNPack aborts the process then.
liteRtLmOpenSessionBacking({required String abi, required BackendKind? backend, NpuStack? npuVendor, AndroidGpuFamily? androidGpu}) → OpenSessionBacking
How the open sessions of a model loaded on backend on the host abi are backed: OpenSessionBacking.native only where that backend ran several live conversations correctly, OpenSessionBacking.virtual everywhere else. npuVendor is the stack an NPU load runs on; androidGpu the GPU family of an Android device, null when not read.
liteRtLmPathOutsideAscii(String path) → bool
Whether path has a character outside ASCII.
liteRtLmTextCandidates({required String path, required BundleInfo? bundle, required BackendKind? preferredBackend, required bool backendFallback, required String abi}) → ({List<BackendKind> backends, Map<BackendKind, String> dropped})
The text backends a load of path tries before it asks whether the NPU can be reached, and the backends of the request the file rules out (see LiteRtLmLoadPlan). planLiteRtLmLoad starts from it; the engine asks it first, so that it looks for the app's NPU stack only when the NPU is one of the backends.
liteRtLmUnreadableContainerMessage(String file, String reason, {String looking = 'its audio encoder'}) → String
Why file's container could not be read to find looking.
liteRtLmWrittenGpuCaches(Iterable<FileSystemEntity> entries) → ({bool program, bool weight})
The names of the real GPU cache files among entries, a cache directory's listing: *_mldrift_weight_cache.bin and *_mldrift_program_cache.bin of at least liteRtLmMinWeightCacheBytes (measured on the 270m: 68 MB and 9 MB).
liteRtLmWrittenWeightCaches(Iterable<FileSystemEntity> entries) → List<String>
The names of the real weight cache files among entries, a cache directory's listing: *.xnnpack_cache files of at least liteRtLmMinWeightCacheBytes. Each is stat'ed on its own, so a file that vanished or cannot be stat'ed is skipped instead of ending the scan.
llmModelFamilyFrom(Uint8List llmMetadata) → LlmModelFamily?
The family LlmMetadata.llm_model_type (field 6) names: its oneof gemma4 = 8, function_gemma = 3 (LiteRT-LM v0.19.1 runtime/proto/llm_model_type.proto), LlmModelFamily.other for any other case, and null when the bundle has no LlmMetadata or no model type.
markLibraryLoadFailure(Object error) → Object
error, thrown while a library loaded, marked as a LibraryLoadFailure of the same kind; anything else (an Error that is no load failure) as it is.
messageFromChunk(String chunk) → Message
The message one streamed chunk of a reply carries. A chunk that is not a JSON object is text.
messageFromJson(Map<String, Object?> json) → Message
A Conversation API JSON object as a Message. Parts of a type it does not know are left out.
messageJson(Message message) → Map<String, Object?>
message as the Conversation API JSON object native reads.
messageRawJson(Message message) → String?
The JSON native wrote for message, when it was parsed from a reply chunk: the Flutter adapter hands it to core's own parser, which reads what Message does not keep (a call's id).
nativeAssetId(String name) → String
The asset id the build hook registers name under — the key Flutter writes into NativeAssetsManifest.json.
nativeToolCalls(String rawReply) → List<({Map<String, Object?> arguments, String name})>
The calls native converted itself: the tool_calls of every JSON object in rawReply (a streamed reply is several, concatenated). A call the model wrote as text stays text, as the native conversation keeps it.
npuBackendOn(NpuPlatform platform) → NpuBackend
The NPU, with platform answering what plain Dart cannot: the build's NativeAssetsManifest.json, and on Android a directory prepared with the dispatch libraries of the vendor the bundle names. flutter_edge_ai_litertlm passes its Flutter implementation.
npuContextWindowNote({required int asked, required int window}) → String?
What an NPU engine says, once, when the context window its executor runs with (window, read after the first turn) is not the maxTokens it was asked for; null when they agree. An NPU bundle has one compiled KV cache, and LiteRT-LM lowers max_num_tokens to the bundle's metadata (or, without it, to the cache). When the metadata is above the cache (upstream #4184) the window read here is the metadata's, and a context past the cache fails natively; this note cannot see that case.
npuHwMasking(NpuStack vendor, BundleInfo? bundle, {required String path, List<BrokenNpuMaskAux> broken = intelBrokenNpuMaskAux, Future<String> sha256OfRange(String path, int begin, int end) = sha256OfFileRange}) → Future<bool?>
What an LLM engine on vendor's NPU sets use_hw_masking_for_npu to for the bundle at path (bundle: null when the file could not be read); null leaves LiteRT-LM's default (true on v0.19.x), which Qualcomm, Tensor and MediaTek run on.
npuNoVendorReason(String abi) → String
Why no NPU is on offer on abi: no vendor this package dispatches to runs there (NpuStack.forAbi is empty).
npuPlatformOf(Backend? backend) → NpuPlatform?
backend's platform layer (npuBackendOn); null for any other backend.
npuSentenceList(Iterable<String> items) → String
items joined for a sentence: A, A and B, A, B and C.
npuStackLibs(NpuStack vendor, String operatingSystem) → List<String>
Every library of vendor's stack on operatingSystem (Platform.operatingSystem spelling), as CodeAsset names: what NativeAssetsManifest.json lists when the build carries the stack, and what Android hands LiteRT as the dispatch directory.
npuStackOf(NpuStack vendor, NpuPlatform platform) → Future<NpuStackCheck>
Whether this build carries vendor's NPU stack where LiteRT will load it: for Qualcomm, what NativeAssetsManifest.json says the build bundled (checkNpuStackInManifest, read through platform and cached per vendor); for Intel, whether its dispatch is beside the executable (checkIntelNpuStack) — the directory LiteRT is pointed at, and a stack that answers NpuStackCheck.unknown there would go unchecked: Intel's prepareNpuDispatchDir checks nothing (it only sweeps stale weight banks).
npuStagesWeightBanks(NpuStack vendor) → bool
Whether a load on vendor's NPU writes weight banks to the temp directory that its process must claim (intel_weight_banks.dart). Intel: the Windows dispatch copies a bundle's shared weight pool there during the create.
npuVendorOf(NpuStamp target) → NpuStack?
The vendor this package dispatches a bundle stamped target to, or null when it has no dispatch for that vendor. Internal: not in the package's show list, so NpuStack stays out of the public API.
openBundledWindowsLibrary(String name, {void preload(String name)?}) → DynamicLibrary
Opens one of the bundled DLLs on Windows, after alignStdHandles has run once in this isolate. Every first open of a bundled library goes through here (a later plain open of one already loaded changes nothing): a library that loads first keeps the handles it copied. The alignment only moves where the native log goes, so a failure in it is printed and the library opens anyway. preload gets name after the alignment and before the open.
openLiteRtLmPreferringDefaultScope(String soname) → DynamicLibrary
Opens soname for a caller that resolves symbols through the returned handle and does not need them to be ambient.
openLiteRtLmRequiringDefaultScope(String soname) → DynamicLibrary
Opens soname for the .litertlm inference path, which does need the exports to be ambient, and throws when they are not.
openMacOSLiteRtLm({DynamicLibrary open(String path)?}) → DynamicLibrary
Opens LiteRtLm.framework in a macOS app, for the LLM and EmbeddingGemma 2.
planLiteRtLmLoad({required String path, required BundleInfo? bundle, required BackendKind? preferredBackend, required bool backendFallback, required bool supportImage, required bool supportAudio, BackendKind? preferredVisionBackend, BackendKind? preferredAudioBackend, required bool npuAvailable(), required String npuUnavailableReason(), required String abi}) → LiteRtLmLoadPlan
Decides a load of path from what the file says (bundle; null when it could not be read, which keeps the rules that need no file) and what the caller asked for.
prepareAndroidNpuDispatchDir(List<String> libs, NpuStack vendor, NpuPlatform platform) → Future<String>
Returns the directory to hand LiteRT as dispatch_lib_dir on Android for one vendor's stack, libs (CodeAsset names) — for the LLM engine and the EmbeddingGemma 2 embedding pass alike (the pass asks from LiteRtEmbeddingBackend.describe(), on the caller's isolate: this goes through a platform channel, which core's worker cannot reach).
prepareNpuDispatchDir(NpuStack vendor, NpuPlatform platform, {String? operatingSystem}) → Future<String>
The dispatch_lib_dir to hand LiteRT for vendor on this host: the prepared Qualcomm libraries on Android (prepareAndroidNpuDispatchDir) and on Linux (prepareLinuxNpuDispatchDir), windowsNpuDispatchDirOf for Intel, after deleting the weight banks of Intel loads whose process is gone. platform answers what only the app's platform layer can. Throws NpuDispatchUnavailableException when the stack cannot be prepared.
readBundleInfo(String path) → Future<BundleInfo>
What the .litertlm file at path says about itself: its container version, system metadata and sections, each TFLiteModel section with the NPU it was compiled for. Reads the header and a few hundred bytes per compiled section, never the weights.
readEngineBundle(String path) → Future<EngineBundle>
What Engine.create reads of path before any engine exists: its bundle info, or null when the file is not one this package reads; an incomplete file is refused.
readLiteRtLmHasLinearAttentionState(String path) → Future<bool>
Whether the main model of the .litertlm LLM at path keeps linear attention state (readLiteRtLmLinearAttentionState): a hybrid model such as LFM2.5: a TYPE_LINEAR_ATTENTION state buffer in the bundle's executor metadata, or else a main-model input named as LiteRT-LM names that state. A conversation on such a model is never rewound to an earlier step, which would restore its KV cache but not that state (core 2.6.0 design §6.4b); it is rebuilt from text instead. False when the bundle has no main section.
readLiteRtLmMediaTowers(String modelPath) → ({bool audio, bool vision})
The vision and audio towers the .litertlm bundle at modelPath carries, read from its container header. Called by LiteRtEmbeddingBackend.describe() for a GPU load: LiteRT-LM compiles every present tower at create, declared or not, on the main backend unless the tower's own is named, so the pass names each present tower cpu and keeps only the text tower on the GPU. A tower the bundle lacks is never named (measured: create fails with code 5; not found in upstream's source). A header this reader cannot parse is a load failure.
readLitertlmModelTypes(String path) → List<String>
The model_type of every TFLiteModel section of path, lowercased (as LiteRT-LM compares them), in file order. A section without one is left out.
readLitertlmModelTypesFrom(Future<Uint8List> prefix(int length)) → Future<List<String>>
The model_type of every TFLiteModel section of a .litertlm file, lowercased, in file order, from prefix — a reader of the file's first length bytes (fewer when the file is shorter). It is asked for the 32-byte preamble, then for everything up to the header end.
readLiteRtLmNpuTarget(String modelPath) → Future<NpuStamp?>
The NPU modelPath was compiled for, or null for a bundle with no compiled section (the generic CPU bundles).
readLlmModelFamily(String modelPath) → Future<LlmModelFamily?>
llmModelFamilyFrom of the bundle at modelPath; null when it has no LlmMetadata or no model type.
requireLiteRtLmAudioEncoder(String modelPath) → void
Throws EmbedderLoadException unless the .litertlm bundle at modelPath has an audio encoder, read from its container header (D23: ModelInfo claims audio on the 270m and the 440m). A header this reader cannot parse is a load failure, never "no audio".
requireOwnLiteRtLmBuild(DynamicLibrary library, String path) → DynamicLibrary
Returns library after checkLiteRtLmLibrary on it.
resetAndroidGemma4Float16Note() → void
Lets a test see noteAndroidGemma4Float16 print again.
resetRuntimeLogSink() → void
Restores runtimeLogSink and runtimeLogLevelSource to the defaults.
runtimeLog(String message, {bool verbose = false}) → void
Logs message through runtimeLogSink, in debug builds only: the constant check lets a release build drop the call and its message.
samplerFor(SamplerConfig? config, BundleSampler bundle) → Sampler
The sampler native gets for config, or for the model file's bundle sampler completed with 0.8 / 40 / 0.95 / seed 1 when config is null.
sendExtraContext(bool enableThinking, Map<String, Object?>? extra) → String
The per-send extra_context JSON: extra with the thinking switch of thinkingContext on top, so a template variable cannot turn thinking on or off behind the caller's back.
sha256OfFileRange(String path, int begin, int end) → Future<String>
The sha256 of bytes [begin, end) of the file at path, read and hashed on a spawned isolate (a 5.8 MB section is milliseconds, but not on the UI isolate). Throws FileSystemException when the file cannot be read or ends before end.
stoppedTurnRewindOutcome(int code, int lastTurnEnd, String? message) → StoppedTurnRewind
What litert_lm_conversation_rewind_cancelled_turn returning code, with the last send having ended as lastTurnEnd (a LiteRtLmTurnEnd; -1 when native wrote none), means: 0 with that send cancelled is a rewind, 0 with it completed (or none sent) an untouched conversation. Anything else — any non-zero code, or 0 with an end that native never pairs with it, as a library built before the contract would give — is a conversation to rebuild.
streamStartFailureTurnEnd(NativeLastTurnEnd? recorded) → NativeLastTurnEnd
The turn end to expose when send_message_stream itself returned an error. Preserve native's record: LiteRT-LM 0.19 can append a message to its history before media conversion fails, so a failed start must rebuild the conversation even though no stream task was started. A failure with no record is a refusal before native accepted a turn.
textBackendsAllowedForTesting(BundleInfo? bundle) → Set<BackendKind>
_textBackendsAllowed, for the catalog test that checks the generated catalog against it.
thinkingBudgetChannelWarning(int? budget, BundleSamplerRead read, String modelPath) → String?
The warning for a thinking budget that caps nothing on the bundle at modelPath, or null. LiteRT-LM caps the first channel a bundle declares; when read found none, a turn with thinking on can still think until maxOutputTokens. Null with no budget, and when the read failed: that read prints its own warning.
thinkingContext(bool enableThinking) → Map<String, Object>
The extra_context that sets thinking for a LiteRT-LM conversation, the one source for the native and the web path.
toolsJson(List<Tool> tools) → String?
The tools_json native reads for tools, or null for none.
tryReadBundleActivationDeclaration(String modelPath) → Future<BundleActivationRead>
litertlmDeclaresActivationType for the file at modelPath, with a file it cannot read or parse returned as BundleActivationRead.error instead of thrown, as tryReadBundleSampler does: the engine's own load reports a real failure, and what this decides is only a default. A caller must say that the answer is unknown, because the model then runs at the GPU default.
tryReadBundleSampler(String modelPath) → Future<BundleSamplerRead>
readBundleSampler, with a file it cannot read or parse returned instead of thrown: a FileSystemException, a FormatException, or an ArgumentError (a RangeError, or a value no engine can sample with). Any other error is a bug in this reader and is thrown.
turnFailureCause(TurnFailedException e) → Object?
What failed under e, for the Flutter adapter, which hands core the error core's apps always caught; null when nothing did.
virtualConversationHistoryForTest(ConversationHandle handle) → List<Map<String, Object?>>
Test seam for the history a virtual session would replay on its next turn.
virtualConversationTokenForTest(ConversationHandle handle) → Object
Test seam for driving the client's side of a consumer cancellation without calling VirtualConversationHandle.cancelGeneration, which deliberately marks the turn as an explicit stop instead.
whereNpuRuns(NpuStack vendor) → String
Where vendor's NPU runs, for the notice that the NPU was not tried.

Typedefs

BrokenNpuMaskAux = ({String file, String issue, int length, String sha256})
A tf_lite_aux section known to carry broken NPU-compiled attention-mask graphs, by its length and the sha256 of its bytes.
BundleActivationRead = ({bool? declaresKey, String? error})
What reading a bundle's header gave: declaresKey (null when it could not be read, with the reason in error).
EmbeddingPart = ({Uint8List bytes, EmbeddingPartKind kind})
One part of an embedding request: the UTF-8 of prefix + text for a text part, the encoded file for an image or audio part.
EngineCreateResult = ({int address, ({Set<String>? after, Set<String>? before})? banks, int code, String? message})
What the engine's isolate hands back: the engine's address (0 when litert_lm_engine_create returned NULL), LiteRT-LM's last error for a failed create, and the weight banks listed around a successful one.
GpuActivationDecision = ({GpuActivationReason? reason, ActivationDataType? type})
A GPU activation the caller decided by a rule of its own, for createEngineWithBundle: the type (null: the engine's default) and why it was chosen. Given, the engine does not apply gpuActivationFor.
LiteRtLmConstrainedDecodingRow = ({String abi, AndroidGpuFamily? androidGpu, BackendKind backend, NpuStack? npuVendor})
One native LiteRT-LM host/backend row whose constrained decoding needs its own discriminating run before it can be advertised.
LiteRtLmGpuCachePathCheck = ({bool cacheDirOutsideAnsi, Set<String> kinds, String problem})
What the path check before create found (liteRtLmGpuCachePathCheck): the reasons, the GPU cache kinds (weight, program) they apply to, and whether the cache directory itself is one the engine likely cannot use.
LiteRtLmLastError = String? Function()
Reads LiteRT-LM's thread-local last error on the calling thread.
LiteRtLmLoadPlan = ({BackendKind? audioBackend, Map<BackendKind, String> dropped, NpuStamp? npuTarget, List<BackendKind> textBackends, BackendKind? visionBackend})
What a .litertlm load does.
NativeAssetsManifestReader = Future<String> Function()
Reads flutter_assets/NativeAssetsManifest.json; in a Flutter app, FlutterNpuPlatform.readNativeAssetsManifest (rootBundle).
NpuDispatch = ({String dir, bool stagesWeightBanks, bool? useHwMasking})
What an NPU attempt hands LiteRT-LM: the dispatch_lib_dir its vendor prepared, what to set use_hw_masking_for_npu to (npuHwMasking; null leaves LiteRT-LM's default), and whether the load's weight banks are to be claimed (npuStagesWeightBanks).
RuntimeLogSink = void Function(String message, {bool verbose})
Where runtimeLog sends a message. verbose marks model output, prompts and native log contents; everything else is info.
StdHandleCalls = ({void Function(int handle) closeHandle, int Function(int fd) crtHandle, int Function(int fd) duplicate, int Function(int fd) getStdHandle, bool Function(int handle) isConsole, bool Function(int which, int handle) setStdHandle})
The Win32 and UCRT calls alignStdHandles makes, injectable for tests.
ToolCallReader = List<({Map<String, Object?> arguments, String name})> Function(String rawReply)
The tool calls in a raw LiteRT-LM reply: each call's name and arguments. Reading them is chat-template knowledge (a model can write a call as text), which the caller owns; the Flutter adapter passes core's parser.

Exceptions / Errors

ConstraintUnsupportedException
LiteRT-LM refused a response constraint (a JSON schema LLGuidance cannot compile). Core: ResponseFormatUnsupportedException.
EmbedderEngineException
A loaded embedder failed a request. Core: EmbeddingEngineException.
EmbedderInputTooLongException
An input longer than the embedder takes; never truncated. Core: EmbeddingInputTooLongException.
EmbedderLoadException
The embedder could not load. Core: EmbeddingLoadException.
LiteRtLmBundleException
A .litertlm file that cannot be read, or cannot run as asked — thrown before any engine is created.
LiteRtLmCallException
A C API call returned a status other than OK, or OK with no result.
LiteRtLmEngineCreateException
LiteRT-LM's engine_settings_create or engine_create returned NULL for a backend.
LiteRtLmTurnCancelledException
Native cancelled a turn that nobody on this side asked to cancel: the turn depended on an earlier cancelled one, so the conversation is poisoned and has to be rebuilt.
LiteRtLmTurnClosedException
A turn ended by closing its session or model before anyone asked to stop it (B90): a StateError, so a consumer of getResponseAsync sees one, and a type of its own, so the session's startTurn can report the close as TurnEnd.closed and end the text cleanly instead.
NpuDispatchUnavailableException
The NPU cannot be reached on this device: no dispatch stack in the build, no Qualcomm FastRPC or DSP on the device, or a dispatch directory that could not be prepared. message says which.
ScoringCancelledException
An option-scoring run was cancelled. Core: DeciderCancelledException.
ScoringStoppedError
A scoring run stopped on purpose — its scorer or its model closed — as opposed to a native call that failed (a plain StateError). Main isolate only, like ScoringStops.
StoppedTurnRewindException
The stopped turn could not be rewound, so the conversation must be rebuilt. code is the absl status code native returned (-1 when the call itself failed), lastTurnEnd how native's last send had ended.