LlamaController class

A class to manage the parameters for the llama model.

Inheritance

• Object
• ChangeNotifier
• LlamaController

Constructors

LlamaController({required String modelPath, bool? vocabOnly, bool? useMmap, bool? useMlock, bool? checkTensors, int? nCtx, int? nBatch, int? nUBatch, int? nSeqMax, int? nThreads, int? nThreadsBatch, RopeScalingType? ropeScalingType, PoolingType? poolingType, AttentionType? attentionType, double? ropeFrequencyBase, double? ropeFrequencyScale, double? yarnExtrapolationFactor, double? yarnAttenuationFactor, double? yarnBetaFast, double? yarnBetaSlow, int? yarnOriginalContext, double? defragmentationThreshold, GgmlType? typeK, GgmlType? typeV, bool? embeddings, bool? offloadKqv, bool? flashAttention, bool? noPerformance, bool? greedy, bool? infill, int? seed, int? topK, double? topP, int? minKeepTopP, double? minP, int? minKeepMinP, double? typicalP, int? minKeepTypicalP, double? temperature, double? temperatureDelta, double? temperatureExponent, double? xtcP, double? xtcT, int? minKeepXtc, int? xtcSeed, int? mirostatNVocab, int? mirostatSeed, double? mirostatTau, double? mirostatEta, int? mirostatM, int? mirostatV2Seed, double? mirostatV2Tau, double? mirostatV2Eta, String? grammarStr, String? grammarRoot, int? penaltiesLastN, double? penaltiesRepeat, double? penaltiesFrequency, double? penaltiesPresent, int? drySamplerNCtxTrain, double? drySamplerMultiplier, double? drySamplerDryBase, int? drySamplerAllowedLength})

Creates a new instance of LlamaController.

LlamaController.fromJson(String source)

Creates a new instance from a JSON string.

factory

LlamaController.fromMap(Map<String, dynamic> map)

Creates a new instance from a map.

factory

Properties

attentionType ↔ AttentionType?

attention type to use for embeddings

getter/setter pair

checkTensors ↔ bool?

A flag indicating whether to check tensors.

getter/setter pair

defragmentationThreshold ↔ double?

defragment the KV cache if holes/size > thold, < 0 disabled (default)

getter/setter pair

drySamplerAllowedLength ↔ int?

Dry sampler allowed length

getter/setter pair

drySamplerDryBase ↔ double?

Dry sampler dry base

getter/setter pair

drySamplerMultiplier ↔ double?

Dry sampler multiplier

getter/setter pair

drySamplerNCtxTrain ↔ int?

Dry sampler n ctx train

getter/setter pair

drySamplerPenaltyLastN ↔ int?

Dry sampler penalty last N

getter/setter pair

drySamplerSequenceBreakers ↔ List<String>?

Dry sampler sequence breakers

getter/setter pair

embeddings ↔ bool?

if true, extract embeddings (together with logits)

getter/setter pair

flashAttention ↔ bool?

whether to use flash attention

getter/setter pair

grammarRoot ↔ String?

Grammar-based sampling root

getter/setter pair

grammarStr ↔ String?

Grammar-based sampling string

getter/setter pair

greedy ↔ bool

Enables greedy decoding if set to true.

getter/setter pair

hashCode → int

The hash code for this object.

no setter, inherited

hasListeners → bool

Whether any listeners are currently registered.

no setter, inherited

infill ↔ bool

Enables infill sampling if set to true.

getter/setter pair

minKeepMinP ↔ int?

Minimum Probability sampling minimum keep

getter/setter pair

minKeepTopP ↔ int?

Top-P sampling minimum keep

getter/setter pair

minKeepTypicalP ↔ int?

Typical-P sampling minimum keep

getter/setter pair

minKeepXtc ↔ int?

XTC sampling minimum keep

getter/setter pair

minP ↔ double?

Minimum Probability sampling

getter/setter pair

mirostatEta ↔ double?

Mirostat sampling eta

getter/setter pair

mirostatM ↔ int?

Mirostat sampling M

getter/setter pair

mirostatNVocab ↔ int?

Mirostat sampling vocabulary size

getter/setter pair

mirostatSeed ↔ int?

Mirostat sampling seed

getter/setter pair

mirostatTau ↔ double?

Mirostat sampling tau

getter/setter pair

mirostatV2Eta ↔ double?

Mirostat v2 sampling eta

getter/setter pair

mirostatV2Seed ↔ int?

Mirostat v2 sampling seed

getter/setter pair

mirostatV2Tau ↔ double?

Mirostat v2 sampling tau

getter/setter pair

modelPath ↔ String

The path to the model file.

getter/setter pair

nBatch ↔ int?

logical maximum batch size that can be submitted to llama_decode

getter/setter pair

nCtx ↔ int

text context, 0 = from model

getter/setter pair

noPerformance ↔ bool?

whether to measure performance timings

getter/setter pair

nSeqMax ↔ int?

max number of sequences (i.e. distinct states for recurrent models)

getter/setter pair

nThreads ↔ int?

number of threads to use for generation

getter/setter pair

nThreadsBatch ↔ int?

number of threads to use for batch processing

getter/setter pair

nUBatch ↔ int?

physical maximum batch size

getter/setter pair

offloadKqv ↔ bool?

whether to offload the KQV ops (including the KV cache) to GPU

getter/setter pair

penaltiesFrequency ↔ double?

Penalties frequency

getter/setter pair

penaltiesLastN ↔ int?

Penalties last N

getter/setter pair

penaltiesPresent ↔ double?

Penalties present

getter/setter pair

penaltiesRepeat ↔ double?

Penalties repeat

getter/setter pair

poolingType ↔ PoolingType?

whether to pool (sum) embedding results by sequence id

getter/setter pair

ropeFrequencyBase ↔ double?

RoPE base frequency, 0 = from model

getter/setter pair

ropeFrequencyScale ↔ double?

RoPE frequency scaling factor, 0 = from model

getter/setter pair

ropeScalingType ↔ RopeScalingType?

RoPE scaling type, from enum llama_rope_scaling_type

getter/setter pair

runtimeType → Type

A representation of the runtime type of the object.

no setter, inherited

seed ↔ int?

Optional seed for random number generation to ensure reproducibility.

getter/setter pair

temperature ↔ double?

Temperature-based sampling

getter/setter pair

temperatureDelta ↔ double?

Temperature-based sampling delta

getter/setter pair

temperatureExponent ↔ double?

Temperature-based sampling exponent

getter/setter pair

topK ↔ int?

Limits the number of top candidates considered during sampling.

getter/setter pair

topP ↔ double?

Top-P sampling

getter/setter pair

typeK ↔ GgmlType?

data type for K cache

getter/setter pair

typeV ↔ GgmlType?

data type for V cache

getter/setter pair

typicalP ↔ double?

Typical-P sampling

getter/setter pair

useMlock ↔ bool?

Indicates whether memory locking (mlock) should be used.

getter/setter pair

useMmap ↔ bool?

Indicates whether memory-mapped files should be used.

getter/setter pair

vocabOnly ↔ bool?

Indicates whether only the vocabulary should be loaded.

getter/setter pair

xtcP ↔ double?

XTC sampling probability

getter/setter pair

xtcSeed ↔ int?

XTC sampling seed

getter/setter pair

xtcT ↔ double?

XTC sampling temperature

getter/setter pair

yarnAttenuationFactor ↔ double?

YaRN magnitude scaling factor

getter/setter pair

yarnBetaFast ↔ double?

YaRN low correction dim

getter/setter pair

yarnBetaSlow ↔ double?

YaRN high correction dim

getter/setter pair

yarnExtrapolationFactor ↔ double?

YaRN extrapolation mix factor, negative = from model

getter/setter pair

yarnOriginalContext ↔ int?

YaRN original context size

getter/setter pair

Methods

addListener(VoidCallback listener) → void

Register a closure to be called when the object changes.

inherited

dispose() → void

Discards any resources used by the object.

inherited

noSuchMethod(Invocation invocation) → dynamic

Invoked when a nonexistent method or property is accessed.

inherited

notifyListeners() → void

Call all the registered listeners.

inherited

removeListener(VoidCallback listener) → void

Remove a previously registered closure from the list of closures that are notified when the object changes.

inherited

toJson() → String

Converts the current instance to a JSON string.

toMap() → Map<String, dynamic>

Converts the current instance to a map.

toString() → String

A string representation of this object.

inherited

Operators

operator ==(Object other) → bool

The equality operator.

inherited