llama_context_params class final

Inheritance

• Object
• Struct
• llama_context_params

Implemented types

• NativeType

Available extensions

• StructAddress

Constructors

llama_context_params.new()

Properties

address → Pointer<T>

Available on T, provided by the StructAddress extension

The memory address of the underlying data.

no setter

cb_eval ↔ ggml_backend_sched_eval_callback

getter/setter pair

cb_eval_user_data ↔ Pointer<Void>

getter/setter pair

embedding ↔ bool

embedding mode only

getter/setter pair

hashCode → int

The hash code for this object.

no setter, inherited

logits_all ↔ bool

the llama_eval() call computes all logits, not just the last one (DEPRECATED - set llama_batch.logits instead)

getter/setter pair

mul_mat_q ↔ bool

if true, use experimental mul_mat_q kernels (DEPRECATED - always true)

getter/setter pair

n_batch ↔ int

prompt processing maximum batch size

getter/setter pair

n_ctx ↔ int

text context, 0 = from model

getter/setter pair

n_threads ↔ int

number of threads to use for generation

getter/setter pair

n_threads_batch ↔ int

number of threads to use for batch processing

getter/setter pair

offload_kqv ↔ bool

whether to offload the KQV ops (including the KV cache) to GPU

getter/setter pair

rope_freq_base ↔ double

RoPE base frequency, 0 = from model

getter/setter pair

rope_freq_scale ↔ double

RoPE frequency scaling factor, 0 = from model

getter/setter pair

rope_scaling_type ↔ int

RoPE scaling type, from enum llama_rope_scaling_type

getter/setter pair

runtimeType → Type

A representation of the runtime type of the object.

no setter, inherited

seed ↔ int

RNG seed, -1 for random

getter/setter pair

type_k ↔ int

data type for K cache

getter/setter pair

type_v ↔ int

data type for V cache

getter/setter pair

yarn_attn_factor ↔ double

YaRN magnitude scaling factor

getter/setter pair

yarn_beta_fast ↔ double

YaRN low correction dim

getter/setter pair

yarn_beta_slow ↔ double

YaRN high correction dim

getter/setter pair

yarn_ext_factor ↔ double

YaRN extrapolation mix factor, negative = from model

getter/setter pair

yarn_orig_ctx ↔ int

YaRN original context size

getter/setter pair

Methods

noSuchMethod(Invocation invocation) → dynamic

Invoked when a nonexistent method or property is accessed.

inherited

toString() → String

A string representation of this object.

inherited

Operators

operator ==(Object other) → bool

The equality operator.

inherited