planContextBudget function

PlannerDecision planContextBudget({ required MemorySnapshot memory, required ModelMemoryEstimate estimate, required int currentContextTokens, required int maxContextTokens, required MemoryPolicy policy, int reclaimableBytes = 0, bool underPressure = false, Duration? sinceLastResize, })

Plans the context size for the next planning period.

reclaimableBytes is what the currently loaded model + context occupy (zero when planning an initial load): it is added back to available memory, since a resize releases it before reallocating. maxContextTokens is the caller's cap — typically min(policy.maxContextTokens, spec.contextSize, trained context). sinceLastResize enables the cooldown; null means no resize happened yet. underPressure applies the pressure rules (immediate shrink, no growth).

Implementation

PlannerDecision planContextBudget({
  required MemorySnapshot memory,
  required ModelMemoryEstimate estimate,
  required int currentContextTokens,
  required int maxContextTokens,
  required MemoryPolicy policy,
  int reclaimableBytes = 0,
  bool underPressure = false,
  Duration? sinceLastResize,
}) {
  final budget =
      ((memory.availableBytes + reclaimableBytes) *
              (1 - policy.headroomFraction))
          .floor();
  var target = math.min(estimate.maxContextForBudget(budget), maxContextTokens);
  target = (target ~/ contextTokenGranularity) * contextTokenGranularity;

  if (target < policy.minContextTokens) {
    return MemoryCritical(
      availableBytes: memory.availableBytes,
      requiredBytes: estimate.bytesForContext(policy.minContextTokens),
    );
  }
  if (target == currentContextTokens) return const KeepContext();

  final growing = target > currentContextTokens;
  if (growing && underPressure) return const KeepContext();

  final delta = (target - currentContextTokens).abs();
  final withinHysteresis =
      delta / currentContextTokens <= policy.resizeHysteresisFraction;
  if (withinHysteresis && !underPressure) return const KeepContext();

  final coolingDown =
      sinceLastResize != null && sinceLastResize < policy.resizeCooldown;
  if (coolingDown && !underPressure) return const KeepContext();

  return ResizeContext(
    fromTokens: currentContextTokens,
    toTokens: target,
    reason: growing ? ResizeReason.grow : ResizeReason.shrink,
  );
}