llama_model_params class final
- Inheritance
- Implemented types
- Available extensions
Constructors
Properties
-
address
→ Pointer<
T> -
Available on T, provided by the StructAddress extension
The memory address of the underlying data.no setter - hashCode → int
-
The hash code for this object.
no setterinherited
-
kv_overrides
↔ Pointer<
llama_model_kv_override> -
override key-value pairs of the model meta data
getter/setter pair
- main_gpu ↔ int
-
main_gpu interpretation depends on split_mode:
LLAMA_SPLIT_NONE: the GPU that is used for the entire model
LLAMA_SPLIT_ROW: the GPU that is used for small tensors and intermediate results
LLAMA_SPLIT_LAYER: ignored
getter/setter pair
- n_gpu_layers ↔ int
-
number of layers to store in VRAM
getter/setter pair
- progress_callback ↔ llama_progress_callback
-
Called with a progress value between 0.0 and 1.0. Pass NULL to disable.
If the provided progress_callback returns true, model loading continues.
If it returns false, model loading is immediately aborted.
getter/setter pair
-
progress_callback_user_data
↔ Pointer<
Void> -
context pointer passed to the progress callback
getter/setter pair
- runtimeType → Type
-
A representation of the runtime type of the object.
no setterinherited
- split_mode ↔ int
-
how to split the model across multiple GPUs
getter/setter pair
-
tensor_split
↔ Pointer<
Float> -
proportion of the model (layers or rows) to offload to each GPU, size: llama_max_devices()
getter/setter pair
- use_mlock ↔ bool
-
force system to keep model in RAM
getter/setter pair
- use_mmap ↔ bool
-
use mmap if possible
getter/setter pair
- vocab_only ↔ bool
-
only load the vocabulary, no weights
getter/setter pair
Methods
-
noSuchMethod(
Invocation invocation) → dynamic -
Invoked when a nonexistent method or property is accessed.
inherited
-
toString(
) → String -
A string representation of this object.
inherited
Operators
-
operator ==(
Object other) → bool -
The equality operator.
inherited