What LiteRT-LM's benchmark reports about a conversation: its tokens over
every turn so far, the last decode's speed, the first token's latency and
the engine's load time. LiteRT-LM's Kotlin/Swift BenchmarkInfo, summed
over the conversation rather than its last turn.
What a .litertlm file says about itself, read before any engine exists:
its container version, its system metadata, and its sections with their
declared constraints and the NPU each compiled section targets.
Everything litert_lm_engine_create is made from, decided on the calling
isolate. Plain values only, so it crosses to the isolate that builds the
engine by copy: the settings are created, used and freed there
(createEngineOffMainIsolate), and no settings pointer is ever valid on
two isolates.
LiteRT-LM builds an engine's sampler at its first generation and keeps it
for every later conversation (google-ai-edge/LiteRT-LM#2080), so a later
session that asks for another sampler does not get it. This records the
first and says what a later generation should report.
Which conversation's turn LiteRT-LM's executor ran last, as far as the
client knows. A send to any other (or when it is not known) switches KV
caches inside the native send call, which then runs off the calling
isolate.
An error thrown while libLiteRtLm, or a library it needs, loads or is
checked: what Engine.create reports as a LiteRtLmLibraryException.
Each keeps the type the load threw (the Flutter adapter's backend fallback
tells an Exception from an Error), so this marks it instead of replacing
it.
One conversation owned by a LiteRtLmFfiClient. Each call to
LiteRtLmFfiClient.createConversationHandle returns a fresh handle:
the engine pointer is shared across handles, the conversation pointer
is private to this handle. The handle owns that pointer's lifetime and
routes every per-conversation native call through the client's private
_…On(conv, …) methods.
The .litertlm embedder: text, images on a bundle with a vision
encoder, audio on a bundle with an audio encoder, and any combination of
them in one vector; CPU, or the NPU for a per-SoC bundle.
Tables, vectors of tables, strings, union discriminators and scalar
fields, little-endian, over the header bytes. Out-of-range offsets throw
RangeError — checked here, before every read, rather than left to
ByteData: on the web the bytes are a JS-backed view, and an out-of-range
read there is the DataView's own JS RangeError, which dart2js delivers as
an ArgumentError and dart2wasm as a JS exception no on clause matches
(measured in the browser suite on Chrome 155).
What choosing an NPU vendor asks of the host. The real one is
LiteRtLmPlatformNpuHost; tests pass a fake, because every answer here is
a property of a device a test host is not.
The native operation running on each conversation. Nothing deletes a
conversation, and no later operation starts on it, before the one that runs
has ended: a cancel only asks native to stop, and the task — and its
callbacks — finish later.
What the NPU code needs from the app's platform layer that plain Dart
cannot answer: the build's NativeAssetsManifest.json, and on Android a
dispatch directory the app's native side prepares. The Flutter one is
FlutterNpuPlatform (flutter_npu_platform.dart), which reads the manifest
through rootBundle and asks core's flutter_gemma_bundled channel.
Each vendor's device probe: null when it passes, else why not. One member
per probe, so a vendor whose probe is added to npuUnreachable is a
compile error in every implementation until it answers.
Whether this build carries a vendor's NPU stack (NpuStack) where LiteRT
will load it, and why not: from NativeAssetsManifest.json for a stack the
build registers (checkNpuStackInManifest; Qualcomm on Android and Linux
arm64), by file presence for Intel's beside the Windows executable
(checkIntelNpuStack). The LLM engine and the EmbeddingGemma 2 placement
both ask, through npuStackOf.
The SoC a .litertlm bundle's compiled sections target: the
LiteRtStamp the AOT compiler writes into each section it compiles, with
the vendor and SoC spelled as the compiler stamped them.
The handle of a session's own native conversation — createSession's, and
each openSession session's where they are native
(OpenSessionBacking.native) — made to answer again after a turn is
stopped or its input did not fit.
The stop flags of one owner's scoring runs — an option scorer's — so that
closing it stops what it has in flight and refuses what it starts later.
Main isolate only: the flags are set and freed there.
One streamed send whose native call may run on another isolate. While
that call is inside native, LiteRT-LM may already have called back with
the turn's end (a prefill refused at once): ending the turn then would
hand the engine lock on, let a close delete the conversation or a
shutdown delete the engine, under a call still running on them. So an
end that arrives during the call is held and applied once the call has
returned (flushEnd); and the message strings the call reads are freed
only when both the stream is done with them (release) and the call has
returned.
How native's last send ended (LiteRtLmTurnEnd in the patched
c/conversation.h), as litert_lm_conversation_get_last_turn_end reports
it after a send (§23) and litert_lm_conversation_rewind_cancelled_turn
after waiting for one (§16). In the C enum's order: the index is the code.
A debug build of a Flutter app (flutter run, flutter test): Flutter's
kDebugMode where the code runs under Flutter, false under dart run and
dart test. dart.library.ui is defined only by Flutter's compilers.
The hosts, in Abi.current() spelling, where a generic EmbeddingGemma 2
bundle runs on the GPU when preferredBackend: gpu asks for it: LiteRT-LM's
GPU executor, over the WebGPU accelerator these archives ship on Linux and
Windows and the Metal accelerator linked into libLiteRtLm on macOS.
Which generic EmbeddingGemma 2 bundles have run with their text tower on
the GPU of each liteRtLmEmbeddingGpuAbis host; a gpu request for any
other is refused there, naming the missing run. Linux x64: all three since
litertlm 1.19.0 (measured on the 270m; the others run the same text tower
with their media towers on CPU). Windows x64: the 270m on Panther Lake and
Lunar Lake (L1, 2026-11-04). macOS: the 270m and the 740m text towers on
Metal (L1 C probe; the 270m and 740m through this pass, L2).
The hosts where preferredBackend: gpu for a generic EmbeddingGemma 2
bundle is refused with an EmbeddingLoadException
(liteRtLmEmbeddingGpuRefusal) instead of running anywhere: a host whose
GPU path ends the process, which no Dart code can catch, so the request is
refused rather than run on the CPU (the app asked for the GPU). None on
native-v0.19.0-a: Windows x64 was here on native-v0.18.0-h (#744, an
access violation in webgpu_dawn.dll at the first EmbeddingGemma 2 GPU
load) and runs on 0.19 (liteRtLmEmbeddingGpuAbis).
Every C function the pass calls. A libLiteRtLm that exports fewer is
refused at load by checkLiteRtLmEmbeddingSymbols, by name, instead of
failing on the first lookup somewhere inside the worker.
The NPU stacks on which a native conversation answers one message only:
on MediaTek (MT6989, LiteRT-LM 0.18.1 and 0.19) the second message in a
conversation comes back empty (LiteRT-LM #4256; measured on a DDP Galaxy
Tab S10+ 2026-11-01, doc/research/mediatek-dispatch-0.19.md). Every turn
there runs in a fresh conversation that replays the history as full
messages, as a rebuild after a stop does.
The smallest file that counts as a weight cache. XNNPack writes a header
of about 100 bytes when it opens the file, then the packed weights (71 MB
for the 270m's text tower); anything below this is what a failed or
partial header write left. The threshold
example/integration_test/embeddinggemma2_test.dart holds a device run to.
The hosts, in Abi.current() spelling, the native archive is published
for (hook/build.dart, one litertlm-<host>.tar.gz each; the iOS
simulator's ios_sim_arm64 archive reports ios_arm64). Anywhere else
the app builds without libLiteRtLm, and nothing of this package loads.
A symbol every libLiteRtLm this package ships exports and upstream
LiteRT-LM does not: patch_c_api.sh §4c adds it and the Windows .def
there names it, and the release gate (check_embedding_exports.py,
OWN_BUILD_SYMBOLS) refuses a native archive without it. Not tied to an
optional feature (NPU, embeddings): it is an engine setting compiled on
every platform.
The smallest context window a CPU or GPU .litertlm engine is created
with: the kv_cache_max_len the published CPU/GPU bundles bake. Below it
the native KV-cache resize underflows and generation fails to allocate
tensors (#318: 100, 256 and 512 crash on a Pixel 8a, 1024 works). An NPU
bundle has one compiled cache length, which can be lower, so the NPU is
left as asked.
Whether loading the libraries moves native stderr to a log file
(litertlm_native.log in the system temp directory): in a Flutter debug
build on iOS, macOS and Linux, as before the runtime left the adapter;
never in a plain Dart program or a profile or release build.
The native log level Engine.create sets before every engine; null
leaves the process's (LiteRT-LM's default until one is set). The level is
process-wide, as LiteRT-LM's is.
The C API's number for type (ActivationDataType in LiteRT-LM's
executor_settings_base.h: 0 F32, 1 F16, and 2 I16 / 3 I8 which the Dart
enum does not offer), or null to leave the engine setting untouched.
Spelled out rather than taken from index, so reordering the Dart enum
cannot silently change what native receives.
The GPU family of an Android device whose device nodes exists reports:
/dev/kgsl-3d0 Adreno, /dev/mali0 Mali, /dev/pvr_sync PowerVR,
anything else (and a node the app may not stat, which reads as absent)
AndroidGpuFamily.other.
Gives one send's optionalArgs a thinking token budget of budget when
the send thinkings and a budget is set; otherwise touches nothing, so a
send without a budget reaches LiteRT-LM exactly as before.
Throws StateError unless the library opened as path reports a C API
version liteRtLmCApiMajor.x through litert_lm_get_api_version
(readVersion, called only when provides has it).
The context maxNumTokens is created with on backend: raised to
minCpuGpuContextTokens on the CPU and the GPU, with a log line saying
that the context window is not the reply length.
Engine.createConversation with the model's own sampler default when the
config sets none: the values flutter_edge_ai_litertlm resolves (the file's
sampler, then the family's published defaults, then the fallback), which
never count as the caller's choice (LiteRT-LM#2080's one-time warning
fires only for a sampler the caller set). Null takes the file's sampler,
then 0.8 / 40 / 0.95 / seed 1.
An EmbeddingEngine over the embedder config config, already placed:
for a caller that decides the placement itself, on another isolate (the
Flutter adapter places on the app's isolate, where the platform channel
that prepares an Android NPU dispatch directory is, and loads on a worker).
openBindings, allocator and clock are for tests.
Engine.create with the file already read: a caller that reads it to plan
its attempts does not pay for the read again in each one (an in-app read
took ~0.57 s on a Linux VM). bundle must be config.modelPath's.
A conversation of engine with a native conversation of its own whatever
the engine's backing: what flutter_edge_ai_litertlm's singleton
createSession always had, also where the engine's other conversations
share one (one live conversation runs on every backend).
The text of the LiteRT-LM JSON response chunk jsonStr: a thought as
<|channel>thought\n…<channel|>, a string content as it is, the text
parts of a content list joined, and anything that is not JSON as it came.
A content of any other type is a contract change and throws
StateError.
The [begin, end) byte range of the GenericBinaryData section named
name, from the header flatbuffer that follows the 32-byte preamble (the
file's header ends at headerEnd), or null when the bundle has none. A
section that carries the name in another form — on a section of another
data type, or as a value that is not a string — is a FormatException,
not "none": it is data this release cannot read.
The activation type a GPU load runs at when the caller set none: float32
for Gemma 4 with a context above gemma4Fp16MaxContextTokens (its float16
rotary positions stop being distinct there, LiteRT-LM #4001), and for a
FunctionGemma bundle that declares no prefer_activation_type (converted
before litetune 0.1.4: at float16 it answers with <pad> or nothing).
family is the bundle's llm_model_type; declaresActivation whether it
declares a type (null: unread). Null leaves the bundle's own.
Whether library passes requireOwnLiteRtLmBuild: this package's build,
on C API liteRtLmCApiMajor.x. For choosing among development copies on a
host, where a stale one is skipped rather than refused.
Whether a conversation keeps the last message it sent once that send has
ended as how, with native's turn-end record end: after a cancelled
send, the one a rewind puts back, and after one native ended context_full
(section 23), which L3 rewinds to before (core 2.6.0 design §6.4b).
The cache directory name for one copy of a bundle:
<liteRtLmCacheDirPrefix(file)>-<size>-<mtime ms>, both numbers in base 36,
so a re-downloaded or replaced file never reads the weight cache of another
(the engine's own cache file names carry the same pair).
The part of liteRtLmCacheDirName that names the file: the first 8 hex
digits of the SHA-256 of its name. Directories with this prefix and another
size or time are stale copies of the same file.
The length of the path LiteRT-LM gives tower's weight cache in
cacheDir: <cacheDir>/<model file name>_<mtime s>_<size><tower>.xnnpack_cache
(executor_settings_base.ccGetWeightCacheFile, file_util.ccGetFileCacheIdentifier), with the mtime's seconds taken as 10 digits
(every date from 2001 to 2286). The text tower's by default — the vision
tower's is 2 characters longer, the audio tower's 8 or 11 — so this is what
Windows' MAX_PATH applies to, not the directory's length.
The one native send failure that means the schema, rather than inference,
was refused. CreateConstraint runs only when this send carries a schema;
every other INTERNAL remains the native inference failure it is.
Whether an encoder the load turns on runs on the NPU of a bundle that
was read, by the rule planLiteRtLmLoad applies (the request, else
_encoderDefault). The engine asks it with liteRtLmTextCandidates, so
that it looks for the app's NPU stack also when only an encoder needs the
NPU. A request the plan then refuses may count: the plan throws before the
stack's answer is used. A file that could not be read keeps the old rules.
Whether LiteRT-LM may strip a completed reply's channel content (a
thought) from the KV cache: only when the conversation is known to rewind,
i.e. not on the NPU (npu) and on a model known to keep no linear
attention state (linearAttentionState false; null, unread, counts as
cannot rewind).
Whether every turn of a model loaded on backend (the NPU stack
npuVendor) needs a fresh native conversation that replays the history
(liteRtLmFreshConversationNpus), in createSession and openSession
sessions alike.
The names LiteRT's WebGPU accelerator gives the text tower's GPU caches in
a cache directory: LiteRT-LM sets the serialization directory and the model
cache key <model file name>.text_encoder_<mtime s>_<size>
(litert_compiled_model_executor_utils.cc, file_util.cc), and ML Drift
appends _mldrift_weight_cache.bin / _mldrift_program_cache.bin. The
mtime's seconds are taken as 10 digits, as in
liteRtLmCacheFilePathLength. The program cache's is the longer, by one.
The check behind liteRtLmGpuCachePathProblem. On Windows the
accelerator gets the path as UTF-8 and opens each cache through the UCRT
_open, which reads it in the ANSI code page, then CreateFileW without
the \\?\ prefix (read from the disassembly of the shipped
LiteRtWebGpuAccelerator.dll). So, unless that code page is UTF-8
(ansiCodePageIsUtf8):
Why LiteRT's WebGPU accelerator cannot create the GPU caches of
modelFileName in cacheDir, decided from the path alone before the
engine exists; null when both fit. liteRtLmGpuCachePathCheck says which
of the two each reason applies to.
Whether a bundle whose TFLiteModel sections have modelTypes carries an
audio tower: a tf_lite_audio_encoder_hw section, the one key LiteRT-LM
builds its audio executor for (ModelType::kTfLiteAudioEncoderHw); only
the 740m has it, beside tf_lite_audio_adapter and tf_lite_end_of_audio.
Compiled at create like the vision tower (litertlmHasVisionTower).
Whether a bundle whose TFLiteModel sections have modelTypes carries a
vision tower: a tf_lite_vision_encoder section, the one key LiteRT-LM
builds its vision executor for (runtime/core/embedding_engine_impl.cc,
ModelType::kTfLiteVisionEncoder). The 440m and the 740m have it beside
tf_lite_vision_adapter and tf_lite_end_of_vision, neither of which
gets an executor of its own; the 270m has only tf_lite_text_encoder and
tf_lite_embedder. LiteRT-LM compiles a present tower at create, declared
or not: on its own backend when one is named, else on the main one.
Where the header flatbuffer ends, from the 32-byte preamble: the magic
LITERTLM, three uint32 version fields, 4 bytes of padding and a uint64
header end (LiteRT-LM schema/core/litertlm_read.cc, pin b2f686e2).
macOS-only fallbacks for libLiteRtLm.dylib, used when there is no app
bundle to resolve LiteRtLm.framework from — i.e. flutter test on the
host. Never reached inside a shipped app.
The NPU a load of bundle runs on when it runs on one: the text model's
(its own stamp, else tf_lite_aux's), else that of an encoder the load
turns on. Null for a bundle compiled for no NPU, and for one that was not
read. The engine asks it before the plan, to look up that vendor's stack.
Why the engine wrote no weight cache, for the refusal and the warnings,
given the length of the cache file path (liteRtLmCacheFilePathLength).
On Windows from 260 characters MAX_PATH is the likely cause; otherwise
every cause that leaves a tower uncached is listed. A disk that fills while
the cache is built is not one of them: XNNPack aborts the process then.
How the open sessions of a model loaded on backend on the host abi
are backed: OpenSessionBacking.native only where that backend ran
several live conversations correctly, OpenSessionBacking.virtual
everywhere else. npuVendor is the stack an NPU load runs on;
androidGpu the GPU family of an Android device, null when not read.
The text backends a load of path tries before it asks whether the NPU
can be reached, and the backends of the request the file rules out (see
LiteRtLmLoadPlan). planLiteRtLmLoad starts from it; the engine asks it
first, so that it looks for the app's NPU stack only when the NPU is one of
the backends.
The names of the real GPU cache files among entries, a cache directory's
listing: *_mldrift_weight_cache.bin and *_mldrift_program_cache.bin of
at least liteRtLmMinWeightCacheBytes (measured on the 270m: 68 MB and
9 MB).
The names of the real weight cache files among entries, a cache
directory's listing: *.xnnpack_cache files of at least
liteRtLmMinWeightCacheBytes. Each is stat'ed on its own, so a file that
vanished or cannot be stat'ed is skipped instead of ending the scan.
The family LlmMetadata.llm_model_type (field 6) names: its oneof
gemma4 = 8, function_gemma = 3 (LiteRT-LM v0.19.1
runtime/proto/llm_model_type.proto), LlmModelFamily.other for any other
case, and null when the bundle has no LlmMetadata or no model type.
The JSON native wrote for message, when it was parsed from a reply
chunk: the Flutter adapter hands it to core's own parser, which reads what
Message does not keep (a call's id).
The calls native converted itself: the tool_calls of every JSON object
in rawReply (a streamed reply is several, concatenated). A call the model
wrote as text stays text, as the native conversation keeps it.
The NPU, with platform answering what plain Dart cannot: the build's
NativeAssetsManifest.json, and on Android a directory prepared with the
dispatch libraries of the vendor the bundle names. flutter_edge_ai_litertlm
passes its Flutter implementation.
What an NPU engine says, once, when the context window its executor runs
with (window, read after the first turn) is not the maxTokens it was
asked for; null when they agree. An NPU bundle has one compiled KV cache,
and LiteRT-LM lowers max_num_tokens to the bundle's metadata (or, without
it, to the cache). When the metadata is above the cache (upstream #4184)
the window read here is the metadata's, and a context past the cache fails
natively; this note cannot see that case.
What an LLM engine on vendor's NPU sets use_hw_masking_for_npu to for
the bundle at path (bundle: null when the file could not be read); null
leaves LiteRT-LM's default (true on v0.19.x), which Qualcomm, Tensor and
MediaTek run on.
Every library of vendor's stack on operatingSystem
(Platform.operatingSystem spelling), as CodeAsset names: what
NativeAssetsManifest.json lists when the build carries the stack, and what
Android hands LiteRT as the dispatch directory.
Whether this build carries vendor's NPU stack where LiteRT will load it:
for Qualcomm, what NativeAssetsManifest.json says the build bundled
(checkNpuStackInManifest, read through platform and cached per
vendor); for Intel, whether its dispatch is beside the executable
(checkIntelNpuStack) — the directory LiteRT is pointed at, and a stack
that answers NpuStackCheck.unknown there would go unchecked: Intel's
prepareNpuDispatchDir checks nothing (it only sweeps stale weight banks).
Whether a load on vendor's NPU writes weight banks to the temp directory
that its process must claim (intel_weight_banks.dart). Intel: the Windows
dispatch copies a bundle's shared weight pool there during the create.
The vendor this package dispatches a bundle stamped target to, or null
when it has no dispatch for that vendor. Internal: not in the package's
show list, so NpuStack stays out of the public API.
Opens one of the bundled DLLs on Windows, after alignStdHandles has run
once in this isolate. Every first open of a bundled library goes through
here (a later plain open of one already loaded changes nothing): a library
that loads first keeps the handles it copied. The alignment only moves
where the native log goes, so a failure in it is printed and the library
opens anyway. preload gets name after the alignment and before the
open.
Decides a load of path from what the file says (bundle; null when it
could not be read, which keeps the rules that need no file) and what the
caller asked for.
Returns the directory to hand LiteRT as dispatch_lib_dir on Android for
one vendor's stack, libs (CodeAsset names) — for the LLM engine and the
EmbeddingGemma 2 embedding pass alike (the pass asks from
LiteRtEmbeddingBackend.describe(), on the caller's isolate: this goes
through a platform channel, which core's worker cannot reach).
The dispatch_lib_dir to hand LiteRT for vendor on this host: the
prepared Qualcomm libraries on Android (prepareAndroidNpuDispatchDir) and
on Linux (prepareLinuxNpuDispatchDir), windowsNpuDispatchDirOf for
Intel, after deleting the weight banks of Intel loads whose process is
gone. platform answers what only the app's platform layer can.
Throws NpuDispatchUnavailableException when the stack cannot be prepared.
What the .litertlm file at path says about itself: its container
version, system metadata and sections, each TFLiteModel section with the
NPU it was compiled for. Reads the header and a few hundred bytes per
compiled section, never the weights.
What Engine.create reads of path before any engine exists: its bundle
info, or null when the file is not one this package reads; an incomplete
file is refused.
Whether the main model of the .litertlm LLM at path keeps linear
attention state (readLiteRtLmLinearAttentionState): a hybrid model such
as LFM2.5: a TYPE_LINEAR_ATTENTION state buffer in the bundle's executor
metadata, or else a main-model input named as LiteRT-LM names that state.
A conversation on such a model is never rewound to an earlier
step, which would restore its KV cache but not that state (core 2.6.0
design §6.4b); it is rebuilt from text instead. False when the bundle has
no main section.
The vision and audio towers the .litertlm bundle at modelPath carries,
read from its container header. Called by LiteRtEmbeddingBackend.describe()
for a GPU load: LiteRT-LM compiles every present tower at create, declared
or not, on the main backend unless the tower's own is named, so the pass
names each present tower cpu and keeps only the text tower on the GPU.
A tower the bundle lacks is never named (measured: create fails with code
5; not found in upstream's source). A
header this reader cannot parse is a load failure.
The model_type of every TFLiteModel section of a .litertlm file,
lowercased, in file order, from prefix — a reader of the file's first
length bytes (fewer when the file is shorter). It is asked for the
32-byte preamble, then for everything up to the header end.
Throws EmbedderLoadException unless the .litertlm bundle at
modelPath has an audio encoder, read from its container header (D23:
ModelInfo claims audio on the 270m and the 440m). A header this reader
cannot parse is a load failure, never "no audio".
The per-send extra_context JSON: extra with the thinking switch of
thinkingContext on top, so a template variable cannot turn thinking on
or off behind the caller's back.
The sha256 of bytes [begin, end) of the file at path, read and hashed
on a spawned isolate (a 5.8 MB section is milliseconds, but not on the UI
isolate). Throws FileSystemException when the file cannot be read or ends
before end.
What litert_lm_conversation_rewind_cancelled_turn returning code, with
the last send having ended as lastTurnEnd (a LiteRtLmTurnEnd; -1 when
native wrote none), means: 0 with that send cancelled is a rewind, 0 with it
completed (or none sent) an untouched conversation. Anything else — any
non-zero code, or 0 with an end that native never pairs with it, as a
library built before the contract would give — is a conversation to
rebuild.
The turn end to expose when send_message_stream itself returned an
error. Preserve native's record: LiteRT-LM 0.19 can append a message to its
history before media conversion fails, so a failed start must rebuild the
conversation even though no stream task was started. A failure with no
record is a refusal before native accepted a turn.
The warning for a thinking budget that caps nothing on the bundle at
modelPath, or null. LiteRT-LM caps the first channel a bundle declares;
when read found none, a turn with thinking on can still think until
maxOutputTokens. Null with no budget, and when the read failed: that read
prints its own warning.
litertlmDeclaresActivationType for the file at modelPath, with a file it
cannot read or parse returned as BundleActivationRead.error instead of
thrown, as tryReadBundleSampler does: the engine's own load reports a
real failure, and what this decides is only a default. A caller must say
that the answer is unknown, because the model then runs at the GPU default.
readBundleSampler, with a file it cannot read or parse returned instead
of thrown: a FileSystemException, a FormatException, or an
ArgumentError (a RangeError, or a value no engine can sample with).
Any other error is a bug in this reader and is thrown.
Test seam for driving the client's side of a consumer cancellation without
calling VirtualConversationHandle.cancelGeneration, which deliberately
marks the turn as an explicit stop instead.
What the engine's isolate hands back: the engine's address (0 when
litert_lm_engine_create returned NULL), LiteRT-LM's last error for a
failed create, and the weight banks listed around a successful one.
A GPU activation the caller decided by a rule of its own, for
createEngineWithBundle: the type (null: the engine's default) and why it
was chosen. Given, the engine does not apply gpuActivationFor.
What the path check before create found (liteRtLmGpuCachePathCheck):
the reasons, the GPU cache kinds (weight, program) they apply to, and
whether the cache directory itself is one the engine likely cannot use.
What an NPU attempt hands LiteRT-LM: the dispatch_lib_dir its vendor
prepared, what to set use_hw_masking_for_npu to (npuHwMasking; null
leaves LiteRT-LM's default), and whether the load's weight banks are to be
claimed (npuStagesWeightBanks).
Where runtimeLog sends a message. verbose marks model output, prompts
and native log contents; everything else is info.
StdHandleCalls
= ({void Function(inthandle)closeHandle, int Function(intfd)crtHandle, int Function(intfd)duplicate, int Function(intfd)getStdHandle, bool Function(inthandle)isConsole, bool Function(intwhich, inthandle)setStdHandle})
The Win32 and UCRT calls alignStdHandles makes, injectable for tests.
The tool calls in a raw LiteRT-LM reply: each call's name and arguments.
Reading them is chat-template knowledge (a model can write a call as
text), which the caller owns; the Flutter adapter passes core's parser.
Native cancelled a turn that nobody on this side asked to cancel: the turn
depended on an earlier cancelled one, so the conversation is poisoned and
has to be rebuilt.
A turn ended by closing its session or model before anyone asked to stop
it (B90): a StateError, so a consumer of getResponseAsync sees one,
and a type of its own, so the session's startTurn can report the close
as TurnEnd.closed and end the text cleanly instead.
The NPU cannot be reached on this device: no dispatch stack in the build,
no Qualcomm FastRPC or DSP on the device, or a dispatch directory that
could not be prepared. message says which.
A scoring run stopped on purpose — its scorer or its model closed — as
opposed to a native call that failed (a plain StateError). Main isolate
only, like ScoringStops.
The stopped turn could not be rewound, so the conversation must be
rebuilt. code is the absl status code native returned (-1 when the call
itself failed), lastTurnEnd how native's last send had ended.