llama_batch class final
Input data for llama_decode A llama_batch object can contain input about one or many sequences The provided arrays (i.e. token, embd, pos, etc.) must have size of n_tokens
- token : the token ids of the input (used when embd is NULL)
- embd : token embeddings (i.e. float vector of size n_embd) (used when token is NULL)
- pos : the positions of the respective token in the sequence
- seq_id : the sequence to which the respective token belongs
- logits : if zero, the logits for the respective token will not be output
- Inheritance
- Implemented types
- Available extensions
Constructors
Properties
- all_pos_0 ↔ int
-
used if pos == NULL
getter/setter pair
- all_pos_1 ↔ int
-
used if pos == NULL
getter/setter pair
- all_seq_id ↔ int
-
used if seq_id == NULL
getter/setter pair
-
embd
↔ Pointer<
Float> -
getter/setter pair
- hashCode → int
-
The hash code for this object.
no setterinherited
-
logits
↔ Pointer<
Int8> -
getter/setter pair
-
n_seq_id
↔ Pointer<
Int32> -
getter/setter pair
- n_tokens ↔ int
-
getter/setter pair
-
pos
↔ Pointer<
llama_pos> -
getter/setter pair
- runtimeType → Type
-
A representation of the runtime type of the object.
no setterinherited
-
seq_id
↔ Pointer<
Pointer< llama_seq_id> > -
getter/setter pair
-
token
↔ Pointer<
llama_token> -
getter/setter pair
Methods
-
noSuchMethod(
Invocation invocation) → dynamic -
Invoked when a nonexistent method or property is accessed.
inherited
-
toString(
) → String -
A string representation of this object.
inherited
Operators
-
operator ==(
Object other) → bool -
The equality operator.
inherited