Internal bridge implemented by each LiteRT-LM platform runtime.
Properties
- hashCode → int
-
The hash code for this object.
no setterinherited
- runtimeType → Type
-
A representation of the runtime type of the object.
no setterinherited
Methods
-
benchmark(
EngineConfig config, {required int prefillTokens, required int decodeTokens}) → Future< BenchmarkInfo> - Runs a native benchmark.
-
cancelConversation(
ConversationHandle conversation) → void - Cancels generation for a native conversation.
-
cancelSession(
SessionHandle session) → void - Cancels generation for a native session.
-
createCapabilities(
String modelPath) → CapabilitiesHandle - Loads model capability information.
-
createConversation(
EngineHandle engine, ConversationConfig config) → Future< ConversationHandle> - Creates a native conversation handle.
-
createEngine(
EngineConfig config) → EngineHandle - Creates a native engine handle.
-
createSession(
EngineHandle engine, SessionConfig config) → Future< SessionHandle> - Creates a native session handle.
-
deleteCapabilities(
CapabilitiesHandle capabilities) → void - Deletes loaded capability information.
-
deleteConversation(
ConversationHandle conversation) → void - Deletes a native conversation handle.
-
deleteEngine(
EngineHandle engine) → void - Deletes a native engine handle.
-
deleteSession(
SessionHandle session) → void - Deletes a native session handle.
-
generateSessionContent(
SessionHandle session, List< InputData> inputData) → Future<String> - Generates session content.
-
generateSessionContentStream(
SessionHandle session, List< InputData> inputData) → Stream<String> - Generates session content as a text stream.
-
getBenchmarkInfo(
ConversationHandle conversation) → Future< BenchmarkInfo> - Gets benchmark information from a native conversation.
-
getTokenCount(
ConversationHandle conversation) → Future< int> - Gets the number of tokens in the conversation KV Cache.
-
hasSpeculativeDecodingSupport(
CapabilitiesHandle capabilities) → bool - Checks whether loaded capabilities support speculative decoding.
-
initializeEngine(
EngineHandle engine) → Future< void> - Initializes a native engine.
-
noSuchMethod(
Invocation invocation) → dynamic -
Invoked when a nonexistent method or property is accessed.
inherited
-
renderMessageIntoString(
ConversationHandle conversation, String messageJson) → Future< String> - Renders a message using the native prompt template.
-
runSessionDecode(
SessionHandle session) → Future< String> - Runs session decode.
-
runSessionPrefill(
SessionHandle session, List< InputData> inputData) → Future<void> - Runs session prefill.
-
sendMessage(
ConversationHandle conversation, String messageJson, {String? extraContextJson}) → Future< Message> - Sends a message and returns the model response.
-
sendMessageStream(
ConversationHandle conversation, String messageJson, {String? extraContextJson}) → Stream< Message> - Sends a message and streams response chunks.
-
setMinimumLogLevel(
LogSeverity severity) → void - Sets the minimum native log severity.
-
toString(
) → String -
A string representation of this object.
inherited
Operators
-
operator ==(
Object other) → bool -
The equality operator.
inherited
Static Properties
- instance → LiteRtLmNativeRuntime
-
Runtime implementation for the current platform.
final