LlmInferenceOptions class
Configuration object for a MediaPipe text classifier.
See also:
- Inheritance
-
- Object
- BaseLlmInferenceOptions
- LlmInferenceOptions
Constructors
- LlmInferenceOptions.cpu({required String modelPath, required String cacheDir, required int maxTokens, required double temperature, required int topK, int? randomSeed})
-
{@macro LlmInferenceOptions}
factory
- LlmInferenceOptions.gpu({required String modelPath, required int sequenceBatchSize, required int maxTokens, required double temperature, required int topK, int decodeStepsPerSync = 3, int? randomSeed})
-
{@macro LlmInferenceOptions}
factory
Properties
- cacheDir → String
-
Directory path for storing model related tokenizer and cache weights. The
user is responsible for providing the directory that can be writable by the
program. Used by CPU only.
no setteroverride
- decodeStepsPerSync → int
-
Number of decode steps per sync. Used by GPU only. The default value is 3.
no setteroverride
- hashCode → int
-
The hash code for this object.
no setterinherited
- loraPath → String
-
Path to the LoRA tflite flatbuffer file. Optional (default is empty string).
This is only compatible with GPU models.
no setteroverride
- maxTokens → int
-
The total length of the kv-cache.
no setteroverride
- modelPath → String
-
The path that points to the tflite model file to use for inference.
no setteroverride
-
props
→ List<
Object?> -
The list of properties that will be used to determine whether
two instances are equal.
no setterinherited
- randomSeed → int
-
Random seed for sampling tokens.
no setteroverride
- runtimeType → Type
-
A representation of the runtime type of the object.
no setterinherited
- sequenceBatchSize → int
-
Sequence batch size for encoding. Used by GPU only. Number of input tokens
to process at a time for batch processing. Setting this value to 1 means
both the encoding and decoding share the same graph of sequence length
of 1. Setting this value to 0 means the batch size will be optimized
programmatically.
no setteroverride
- stringify → bool?
-
If set to
true, thetoStringmethod will be overridden to output this instance'sprops.no setterinherited - temperature → double
-
Randomness when decoding the next token.
no setteroverride
- topK → int
-
Top K number of tokens to be sampled from for each decoding step.
no setteroverride
Methods
-
noSuchMethod(
Invocation invocation) → dynamic -
Invoked when a nonexistent method or property is accessed.
inherited
-
toString(
) → String -
A string representation of this object.
inherited
Operators
-
operator ==(
Object other) → bool -
The equality operator.
inherited