LLamaContextParams class
ContextParams holds configuration settings for the Llama model context
Constructors
- LLamaContextParams()
- General Library Documentation Undocument By General Corporation & Global Corporation & General Developer
-
LLamaContextParams.fromJson(Map<
String, dynamic> json) -
Creates a ContextParams instance from JSON
factory
Properties
- attentionType ↔ LlamaAttentionType
-
Attention type to use for embeddings
getter/setter pair
- defragThold ↔ double
-
Defragment the KV cache if holes/size > thold, < 0 disabled
getter/setter pair
- embeddings ↔ bool
-
If true, extract embeddings (together with logits)
getter/setter pair
- flashAttn ↔ bool
-
Whether to use flash attention
EXPERIMENTALgetter/setter pair - hashCode → int
-
The hash code for this object.
no setterinherited
- logitsAll ↔ bool
-
The llama_decode() call computes all logits, not just the last one
getter/setter pair
- nBatch ↔ int
-
Logical maximum batch size that can be submitted to llama_decode
getter/setter pair
- nCtx ↔ int
-
Text context size. 0 = from model
getter/setter pair
- noPerfTimings ↔ bool
-
Whether to measure performance timings
getter/setter pair
- nPredit ↔ int
-
Maximum number of tokens to predict/generate in response
getter/setter pair
- nSeqMax ↔ int
-
Max number of sequences (i.e. distinct states for recurrent models)
getter/setter pair
- nThreads ↔ int
-
Number of threads to use for generation
getter/setter pair
- nThreadsBatch ↔ int
-
Number of threads to use for batch processing
getter/setter pair
- nUbatch ↔ int
-
Physical maximum batch size
getter/setter pair
- offloadKqv ↔ bool
-
Whether to offload the KQV ops (including the KV cache) to GPU
getter/setter pair
- poolingType ↔ LlamaPoolingType
-
Pooling type for embeddings
getter/setter pair
- ropeFreqBase ↔ double
-
RoPE base frequency, 0 = from model
getter/setter pair
- ropeFreqScale ↔ double
-
RoPE frequency scaling factor, 0 = from model
getter/setter pair
- ropeScalingType ↔ LlamaRopeScalingType
-
RoPE scaling type
getter/setter pair
- runtimeType → Type
-
A representation of the runtime type of the object.
no setterinherited
- yarnAttnFactor ↔ double
-
YaRN magnitude scaling factor
getter/setter pair
- yarnBetaFast ↔ double
-
YaRN low correction dim
getter/setter pair
- yarnBetaSlow ↔ double
-
YaRN high correction dim
getter/setter pair
- yarnExtFactor ↔ double
-
YaRN extrapolation mix factor, negative = from model
getter/setter pair
- yarnOrigCtx ↔ int
-
YaRN original context size
getter/setter pair
Methods
-
get(
{required LlamaLibrarySharedBindingsByGeneralDeveloper llama}) → llama_context_params -
Constructs and returns a
llama_context_paramsobject -
noSuchMethod(
Invocation invocation) → dynamic -
Invoked when a nonexistent method or property is accessed.
inherited
-
toJson(
) → Map< String, dynamic> - Converts to JSON
-
toString(
) → String -
A string representation of this object.
override
Operators
-
operator ==(
Object other) → bool -
The equality operator.
inherited