LiteRtLmBackend class
Web-safe placeholder for the native-only LiteRT-LM backend.
- Implemented types
Constructors
- LiteRtLmBackend({Object? initialSendPort, @Deprecated('Use ModelParams.device. This will be removed in 1.0.') String? preferredBackend})
-
Creates a placeholder backend on platforms without
dart:ffi.
Properties
- hashCode → int
-
The hash code for this object.
no setterinherited
- isReady → bool
-
Whether the backend is currently initialized and ready for inference.
no setterinherited
- runtimeType → Type
-
A representation of the runtime type of the object.
no setterinherited
- supportsUrlLoading → bool
-
Whether this backend supports loading from URLs directly (e.g. WASM).
no setterinherited
Methods
-
applyChatTemplate(
int modelHandle, List< Map< messages, {String? customTemplate, bool addAssistant = true}) → Future<String, dynamic> >String> -
Applies the model's chat template to the given
messages.inherited -
cancelGeneration(
) → void -
Immediately cancels the current generation.
inherited
-
clearLoraAdapters(
int contextHandle) → Future< void> -
Removes all active LoRA adapters from the current context.
inherited
-
contextCreate(
int modelHandle, ModelParams params) → Future< int> -
Creates a new inference context for the given
modelHandle.inherited -
contextFree(
int contextHandle) → Future< void> -
Releases the allocated
contextHandle.inherited -
detokenize(
int modelHandle, List< int> tokens, {bool special = false}) → Future<String> -
Decodes a list of
tokensback into a human-readable string.inherited -
dispose(
) → Future< void> -
Releases all allocated backend resources.
inherited
-
generate(
int contextHandle, String prompt, GenerationParams params, {List< LlamaContentPart> ? parts}) → Stream<List< int> > -
Generates a stream of token bytes for a given prompt and context.
inherited
-
getBackendName(
) → Future< String> -
Returns the name of the active runtime backend.
inherited
-
getContextSize(
int contextHandle) → Future< int> -
Returns the actual context size used by the given
contextHandle.inherited -
getVramInfo(
) → Future< ({int free, int total})> -
Returns the total and free VRAM in bytes.
inherited
-
isGpuSupported(
) → Future< bool> -
Returns true if the hardware and backend support GPU acceleration.
inherited
-
modelFree(
int modelHandle) → Future< void> -
Releases the allocated
modelHandle.inherited -
modelLoad(
String path, ModelParams params) → Future< int> -
Initializes the model from a local file
path.inherited -
modelLoadFromUrl(
String url, ModelParams params, {dynamic onProgress(double progress)?}) → Future< int> -
Initializes the model from a remote
url.inherited -
modelMetadata(
int modelHandle) → Future< Map< String, String> > -
Retrieves all available metadata from the loaded model.
inherited
-
multimodalContextCreate(
int modelHandle, String mmProjPath) → Future< int?> -
Loads a multimodal projector for vision/audio support.
inherited
-
multimodalContextFree(
int mmContextHandle) → Future< void> -
Frees the multimodal context.
inherited
-
noSuchMethod(
Invocation invocation) → dynamic -
Invoked when a nonexistent method or property is accessed.
override
-
removeLoraAdapter(
int contextHandle, String path) → Future< void> -
Removes a specific LoRA adapter from the active session.
inherited
-
setLogLevel(
LlamaLogLevel level) → Future< void> -
Updates the minimum log level for the backend.
inherited
-
setLoraAdapter(
int contextHandle, String path, double scale) → Future< void> -
Dynamically loads or updates a LoRA adapter's scale.
inherited
-
supportsAudio(
int mmContextHandle) → Future< bool> -
Checks if the model supports audio input.
inherited
-
supportsVision(
int mmContextHandle) → Future< bool> -
Checks if the model supports vision input.
inherited
-
tokenize(
int modelHandle, String text, {bool addSpecial = true}) → Future< List< int> > -
Encodes the given
textinto a list of token IDs.inherited -
toString(
) → String -
A string representation of this object.
inherited
Operators
-
operator ==(
Object other) → bool -
The equality operator.
inherited