scoreNextToken abstract method

Future<LlamaNextTokenScores> scoreNextToken(
  1. int contextHandle,
  2. String prompt, {
  3. required List<int> candidates,
  4. required int topK,
  5. required bool reusePromptPrefix,
})

Evaluates prompt on contextHandle as generation would, then returns the log-probabilities of candidates and of the topK most probable tokens at the next position. reusePromptPrefix reuses a matching prompt prefix already in the context, as GenerationParams.reusePromptPrefix does.

Implementation

Future<LlamaNextTokenScores> scoreNextToken(
  int contextHandle,
  String prompt, {
  required List<int> candidates,
  required int topK,
  required bool reusePromptPrefix,
});