minTokensForCache property

int minTokensForCache

Minimum number of tokens a cacheable prefix should have (API requires at least 1024 tokens for caching to be effective with Neomage models).

Implementation

final int minTokensForCache;