contextTokenGranularity top-level constant

int const contextTokenGranularity

Context sizes are planned in steps of this many tokens, so measurement wobble cannot produce a stream of one-token resize proposals.

Implementation

const int contextTokenGranularity = 256;