llama_token_to_piece method

int llama_token_to_piece(
  1. Pointer<llama_model> model,
  2. int token,
  3. Pointer<Char> buf,
  4. int length,
)

Token Id -> Piece. Uses the vocabulary in the provided context. Does not write null terminator to the buffer. User code is responsible to remove the leading whitespace of the first non-BOS token when decoding multiple tokens.

Implementation

int llama_token_to_piece(
  ffi.Pointer<llama_model> model,
  int token,
  ffi.Pointer<ffi.Char> buf,
  int length,
) {
  return _llama_token_to_piece(
    model,
    token,
    buf,
    length,
  );
}