llama_batch_init function

  1. @Native<llama_batch Function(Int32, Int32, Int32)>(ffi.Int32, ffi.Int32, ffi.Int32)>()
llama_batch llama_batch_init(
  1. int n_tokens,
  2. int embd,
  3. int n_seq_max
)

Allocates a batch of tokens on the heap that can hold a maximum of n_tokens Each token can be assigned up to n_seq_max sequence ids The batch has to be freed with llama_batch_free() If embd != 0, llama_batch.embd will be allocated with size of n_tokens * embd * sizeof(float) Otherwise, llama_batch.token will be allocated to store n_tokens llama_token The rest of the llama_batch members are allocated with size n_tokens All members are left uninitialized

Implementation

@ffi.Native<llama_batch Function(ffi.Int32, ffi.Int32, ffi.Int32)>()
external llama_batch llama_batch_init(
  int n_tokens,
  int embd,
  int n_seq_max,
);