llama_kv_cache_seq_shift method

void llama_kv_cache_seq_shift(
  1. Pointer<llama_context> ctx,
  2. int seq_id,
  3. int p0,
  4. int p1,
  5. int delta,
)

Adds relative position "delta" to all tokens that belong to the specified sequence and have positions in [p0, p1) If the KV cache is RoPEd, the KV data is updated accordingly p0 < 0 : 0, p1 p1 < 0 : [p0, inf)

Implementation

void llama_kv_cache_seq_shift(
  ffi.Pointer<llama_context> ctx,
  int seq_id,
  int p0,
  int p1,
  int delta,
) {
  return _llama_kv_cache_seq_shift(
    ctx,
    seq_id,
    p0,
    p1,
    delta,
  );
}