topK property

int? topK

The maximum number of tokens to consider when sampling. The model uses combined Top-k and nucleus sampling. Top-k sampling considers the set of top_k most probable tokens. Defaults to 40. Note:

The default value varies by model, see the Model.top_k attribute of the Model returned the getModel function.

Implementation

final int? topK;