internal library

Classes

BenchmarkInfo
What LiteRT-LM's benchmark reports about a conversation: its tokens over every turn so far, the last decode's speed, the first token's latency and the engine's load time. LiteRT-LM's Kotlin/Swift BenchmarkInfo, summed over the conversation rather than its last turn.
BundleInfo
What a .litertlm file says about itself, read before any engine exists: its container version, its system metadata, and its sections with their declared constraints and the NPU each compiled section targets.
BundleSampler
The sampler a .litertlm file declares: each value null when the file does not set it.
BundleSection
One section of a .litertlm container.
CancelSignal
A cancel the runtime can check and wait for. The adapter passes core's CancelToken through it.
EmbedderInfo
What a loaded embedder reports.
EmbedderInput
One embedding request: text, images and audio, embedded as one vector.
EngineSamplerLatch
LiteRT-LM builds an engine's sampler at its first generation and keeps it for every later conversation (google-ai-edge/LiteRT-LM#2080), so a later session that asks for another sampler does not get it. This records the first and says what a later generation should report.
LibraryLoadFailure
An error thrown while libLiteRtLm, or a library it needs, loads or is checked: what Engine.create reports as a LiteRtLmLibraryException. Each keeps the type the load threw (the Flutter adapter's backend fallback tells an Exception from an Error), so this marks it instead of replacing it.
MediaOrder
Where a message's images and audio go relative to its text.
NpuStamp
The SoC a .litertlm bundle's compiled sections target: the LiteRtStamp the AOT compiler writes into each section it compiles, with the vendor and SoC spelled as the compiler stamped them.
Sampler
The sampler sent to LiteRT-LM, every value set.
ScoringRun
What one option-scoring run returned.

Enums

ActivationDataType
The numeric type of a GPU model's activations, LiteRT-LM's ActivationDataType without I16/I8.
BackendKind
Where LiteRT-LM runs a model's main graph, or one of its encoders. Its name is LiteRT-LM's wire name (cpu, gpu, npu).
EmbedderModality
A kind of input an embedder takes.
LiteRtLmBundleProblem
Why a .litertlm file cannot be read or loaded as asked.
LiteRtLmEg2Bundle
The generic EmbeddingGemma 2 bundles, told apart by the towers their container carries (readLiteRtLmMediaTowers).
LiteRtLmLogLevel
The minimum severity of LiteRT-LM's own native log: LiteRtLmLogSeverity of its C API (litert_lm_set_min_log_level).
MediaSide
Which side of a message's text a kind of media goes.
NpuStack
A vendor whose NPU this package dispatches to.

Constants

googleTensorNpuLibs → const List<String>
The complete app-bundled Google Tensor stack: the bridge only.
liteRtLmAudioBundle → const String
The bundle with an audio encoder.
liteRtLmDefaultVisionTokensPerImage → const int
The vision budget an engine is created with unless the app asks for 70.
liteRtLmEmbeddingGpuAbis → const Set<String>
The hosts, in Abi.current() spelling, where a generic EmbeddingGemma 2 bundle runs on the GPU when preferredBackend: gpu asks for it: LiteRT-LM's GPU executor, over the WebGPU accelerator these archives ship on Linux and Windows and the Metal accelerator linked into libLiteRtLm on macOS.
liteRtLmEmbeddingGpuMeasured → const Map<String, Set<LiteRtLmEg2Bundle>>
Which generic EmbeddingGemma 2 bundles have run with their text tower on the GPU of each liteRtLmEmbeddingGpuAbis host; a gpu request for any other is refused there, naming the missing run. Linux x64: all three since litertlm 1.19.0 (measured on the 270m; the others run the same text tower with their media towers on CPU). Windows x64: the 270m on Panther Lake and Lunar Lake (L1, 2026-11-04). macOS: the 270m and the 740m text towers on Metal (L1 C probe; the 270m and 740m through this pass, L2).
liteRtLmEmbeddingGpuRefusedAbis → const Set<String>
The hosts where preferredBackend: gpu for a generic EmbeddingGemma 2 bundle is refused with an EmbeddingLoadException (liteRtLmEmbeddingGpuRefusal) instead of running anywhere: a host whose GPU path ends the process, which no Dart code can catch, so the request is refused rather than run on the CPU (the app asked for the GPU). None on native-v0.19.0-a: Windows x64 was here on native-v0.18.0-h (#744, an access violation in webgpu_dawn.dll at the first EmbeddingGemma 2 GPU load) and runs on 0.19 (liteRtLmEmbeddingGpuAbis).
liteRtLmMaxInputLength → const int
The input length every EmbeddingGemma 2 engine is created with, in tokens.
liteRtLmNativeAbis → const Set<String>
The hosts, in Abi.current() spelling, the native archive is published for (hook/build.dart, one litertlm-<host>.tar.gz each; the iOS simulator's ios_sim_arm64 archive reports ios_arm64). Anywhere else the app builds without libLiteRtLm, and nothing of this package loads.
liteRtLmVisionTokenBudgets → const List<int>
The vision budgets EmbeddingGemma 2 has signatures for.

Functions

androidLibFileName(String name) → String
The file name name gets on Android and Linux (lib<name>.so).
litertlmHasAudioEncoder(Iterable<String> modelTypes) → bool
Whether a bundle whose TFLiteModel sections have modelTypes carries an audio encoder.
litertlmHasAudioTower(Iterable<String> modelTypes) → bool
Whether a bundle whose TFLiteModel sections have modelTypes carries an audio tower: a tf_lite_audio_encoder_hw section, the one key LiteRT-LM builds its audio executor for (ModelType::kTfLiteAudioEncoderHw); only the 740m has it, beside tf_lite_audio_adapter and tf_lite_end_of_audio. Compiled at create like the vision tower (litertlmHasVisionTower).
litertlmHasVisionTower(Iterable<String> modelTypes) → bool
Whether a bundle whose TFLiteModel sections have modelTypes carries a vision tower: a tf_lite_vision_encoder section, the one key LiteRT-LM builds its vision executor for (runtime/core/embedding_engine_impl.cc, ModelType::kTfLiteVisionEncoder). The 440m and the 740m have it beside tf_lite_vision_adapter and tf_lite_end_of_vision, neither of which gets an executor of its own; the 270m has only tf_lite_text_encoder and tf_lite_embedder. LiteRT-LM compiles a present tower at create, declared or not: on its own backend when one is named, else on the main one.
liteRtLmNoAudioEncoderMessage(String file) → String
Why file cannot embed audio.
liteRtLmUnreadableContainerMessage(String file, String reason, {String looking = 'its audio encoder'}) → String
Why file's container could not be read to find looking.
markLibraryLoadFailure(Object error) → Object
error, thrown while a library loaded, marked as a LibraryLoadFailure of the same kind; anything else (an Error that is no load failure) as it is.
nativeAssetId(String name) → String
The asset id the build hook registers name under — the key Flutter writes into NativeAssetsManifest.json.
npuSentenceList(Iterable<String> items) → String
items joined for a sentence: A, A and B, A, B and C.
npuStackLibs(NpuStack vendor, String operatingSystem) → List<String>
Every library of vendor's stack on operatingSystem (Platform.operatingSystem spelling), as CodeAsset names: what NativeAssetsManifest.json lists when the build carries the stack, and what Android hands LiteRT as the dispatch directory.
npuVendorOf(NpuStamp target) → NpuStack?
The vendor this package dispatches a bundle stamped target to, or null when it has no dispatch for that vendor. Internal: not in the package's show list, so NpuStack stays out of the public API.
readLitertlmModelTypesFrom(Future<Uint8List> prefix(int length)) → Future<List<String>>
The model_type of every TFLiteModel section of a .litertlm file, lowercased, in file order, from prefix — a reader of the file's first length bytes (fewer when the file is shorter). It is asked for the 32-byte preamble, then for everything up to the header end.
thinkingContext(bool enableThinking) → Map<String, Object>
The extra_context that sets thinking for a LiteRT-LM conversation, the one source for the native and the web path.

Exceptions / Errors

ConstraintUnsupportedException
LiteRT-LM refused a response constraint (a JSON schema LLGuidance cannot compile). Core: ResponseFormatUnsupportedException.
EmbedderEngineException
A loaded embedder failed a request. Core: EmbeddingEngineException.
EmbedderInputTooLongException
An input longer than the embedder takes; never truncated. Core: EmbeddingInputTooLongException.
EmbedderLoadException
The embedder could not load. Core: EmbeddingLoadException.
LiteRtLmBundleException
A .litertlm file that cannot be read, or cannot run as asked — thrown before any engine is created.
LiteRtLmEngineCreateException
LiteRT-LM's engine_settings_create or engine_create returned NULL for a backend.
NpuDispatchUnavailableException
The NPU cannot be reached on this device: no dispatch stack in the build, no Qualcomm FastRPC or DSP on the device, or a dispatch directory that could not be prepared. message says which.
ScoringCancelledException
An option-scoring run was cancelled. Core: DeciderCancelledException.