llama_batch class final

Input data for llama_encode/llama_decode A llama_batch object can contain input about one or many sequences The provided arrays (i.e. token, embd, pos, etc.) must have size of n_tokens

• token : the token ids of the input (used when embd is NULL)
• embd : token embeddings (i.e. float vector of size n_embd) (used when token is NULL)
• pos : the positions of the respective token in the sequence (if set to NULL, the token position will be tracked automatically by llama_encode/llama_decode)
• seq_id : the sequence to which the respective token belongs (if set to NULL, the sequence ID will be assumed to be 0)
• logits : if zero, the logits (and/or the embeddings) for the respective token will not be output (if set to NULL:
• if embeddings: all tokens are output
• if not: only the last token is output )

Inheritance

• Object
• Struct
• llama_batch

Implemented types

• NativeType

Available extensions

• StructAddress

Constructors

llama_batch()

Properties

address → Pointer<T>

Available on T, provided by the StructAddress extension

The memory address of the underlying data.

no setter

embd ↔ Pointer<Float>

getter/setter pair

hashCode → int

The hash code for this object.

no setter, inherited

logits ↔ Pointer<Int8>

TODO: rename this to "output"

getter/setter pair

n_seq_id ↔ Pointer<Int32>

getter/setter pair

n_tokens ↔ int

getter/setter pair

pos ↔ Pointer<Int32>

getter/setter pair

runtimeType → Type

A representation of the runtime type of the object.

no setter, inherited

seq_id ↔ Pointer<Pointer<Int32>>

getter/setter pair

token ↔ Pointer<Int32>

getter/setter pair

Methods

noSuchMethod(Invocation invocation) → dynamic

Invoked when a nonexistent method or property is accessed.

inherited

toString() → String

A string representation of this object.

inherited

Operators

operator ==(Object other) → bool

The equality operator.

inherited