LiteRtLmRuntimeClient class

Web-safe placeholder for the native-only runtime client.

Constructors

LiteRtLmRuntimeClient({String? libraryPath})

Creates a placeholder client on platforms without dart:ffi.

Properties

hashCode → int

The hash code for this object.

no setter, inherited

runtimeType → Type

A representation of the runtime type of the object.

no setter, inherited

supportsAsrBridge → bool

Dedicated native ASR is unavailable without dart:ffi.

no setter

Methods

bundleCapabilities(String modelPath) → ({bool audio, bool speculativeDecoding, bool vision})?

Bundle capability queries are unavailable without dart:ffi.

cancel() → void

Cancels active native generation.

configureResponseThinkingTags({required String startTag, required String endTag}) → void

Configures how LiteRT-LM thought-channel chunks are exposed to parsers.

createAsrSession(LiteRtLmAsrRuntimeConfig config) → LiteRtLmAsrRuntimeSession

Dedicated native ASR is unavailable without dart:ffi.

createConversation({String? systemMessage, String? promptTemplate, List<Map<String, dynamic>>? messages, List<Map<String, dynamic>>? tools, Map<String, dynamic>? extraContext, double temperature = 0.8, int topK = 40, double topP = 0.95, int seed = 1, bool npuBackend = false, String? loraPath}) → void

Creates a conversation for generation/token operations.

detokenize(List<int> tokens) → String

Converts native LiteRT-LM token ids back to text.

dispose() → void

Releases native LiteRT-LM resources.

generate(String prompt, {int? maxOutputTokens}) → Stream<String>

Streams generated text from the active conversation.

generateMessageJson(String messageJson, {Map<String, dynamic>? extraContext, int? visualTokenBudget, int? maxOutputTokens}) → Stream<String>

Streams generated text from a native message JSON object.

initialize({required String modelPath, String backend = 'gpu', String? visionBackend, String? audioBackend, int maxTokens = 4096, int outputTokens = 256, int? prefillTokens, int? maxNumImages, String? cacheDir, bool speculativeDecoding = true, int minLogLevel = 3, LiteRtLmActivationDataType? activationDataType, int? prefillChunkSize, bool? parallelFileSectionLoading, String? dispatchLibDir, int? numberOfThreads}) → Future<void>

Initializes the native LiteRT-LM engine.

noSuchMethod(Invocation invocation) → dynamic

Invoked when a nonexistent method or property is accessed.

inherited

readMetrics({required int wallMilliseconds}) → LiteRtLmRuntimeMetrics

Reads runtime metrics for the active conversation.

renderMessageToString(Map<String, dynamic> message) → String

Renders a message with the active native conversation template.

run({required String prompt, int warmupRuns = 1, int measuredRuns = 3}) → Future<LiteRtLmRuntimeResult>

Runs a benchmark-style prompt loop and returns runtime metrics.

setMinLogLevel(int level) → void

Updates the native LiteRT-LM log level.

tokenize(String text, {bool addSpecial = true}) → List<int>

Tokenizes text with the native LiteRT-LM tokenizer.

toString() → String

A string representation of this object.

inherited

Operators

operator ==(Object other) → bool

The equality operator.

inherited