kvBytesPerToken property

int kvBytesPerToken

KV-cache cost per context token (f16 K+V across all layers).

Conservative for models with interleaved sliding-window layers (Gemma 3+ style): those layers cap out below n_ctx, so the true cost is lower than this linear estimate.

Implementation

final int kvBytesPerToken;