LLamaModelParams class - model_params library

Constructors

LLamaModelParams(): General Library Documentation Undocument By General Corporation & Global Corporation & General Developer
LLamaModelParams.fromJson(Map<String, dynamic> json): Constructs a ModelParams instance from a JSON map

checkTensors ↔ bool: Validate model tensor data
getter/setter pair
formatter ↔ LLamaPromptFormat?: General Library Documentation Undocument By General Corporation & Global Corporation & General Developer
getter/setter pair
hashCode → int: The hash code for this object.
no setterinherited
kvOverrides ↔ Map<String, dynamic>: Override key-value pairs of the model metadata
getter/setter pair
mainGpu ↔ int: The GPU that is used for the entire model when split_mode is LLAMA_SPLIT_MODE_NONE
getter/setter pair
nGpuLayers ↔ int: Number of layers to store in VRAM
getter/setter pair
rpcServers ↔ String: Comma separated list of RPC servers to use for offloading
getter/setter pair
runtimeType → Type: A representation of the runtime type of the object.
no setterinherited
splitMode ↔ LlamaSplitMode: How to split the model across multiple GPUs
getter/setter pair
tensorSplit ↔ List<double>: Proportion of the model (layers or rows) to offload to each GPU
getter/setter pair
useMemoryLock ↔ bool: Force system to keep model in RAM
getter/setter pair
useMemorymap ↔ bool: Use mmap if possible
getter/setter pair
vocabOnly ↔ bool: Only load the vocabulary, no weights
getter/setter pair

dispose() → void: Free allocated memory
get({required LlamaLibrarySharedBindingsByGeneralDeveloper llama}) → llama_model_params: Constructs and returns a llama_model_params object with current settings
noSuchMethod(Invocation invocation) → dynamic: Invoked when a nonexistent method or property is accessed.
inherited
toJson() → Map<String, dynamic>: Converts the ModelParams instance to a JSON map
toString() → String: Returns a string representation of the ModelParams instance
override