maxContextForBudget method

int maxContextForBudget(
  1. int budgetBytes
)

The largest context that fits in budgetBytes, or 0 when even an empty context does not fit.

Implementation

int maxContextForBudget(int budgetBytes) {
  if (kvBytesPerToken <= 0) return 0;
  final kvBudget = budgetBytes - weightsBytes - fixedOverheadBytes;
  return kvBudget <= 0 ? 0 : kvBudget ~/ kvBytesPerToken;
}