BackendNextTokenScoring class abstract

Optional backend capability for scoring the token that would follow a prompt, without sampling.

Constructors

BackendNextTokenScoring()

Properties

hashCode → int

The hash code for this object.

no setter, inherited

runtimeType → Type

A representation of the runtime type of the object.

no setter, inherited

Methods

noSuchMethod(Invocation invocation) → dynamic

Invoked when a nonexistent method or property is accessed.

inherited

scoreNextToken(int contextHandle, String prompt, {required List<int> candidates, required int topK, required bool reusePromptPrefix}) → Future<LlamaNextTokenScores>

Evaluates prompt on contextHandle as generation would, then returns the log-probabilities of candidates and of the topK most probable tokens at the next position. reusePromptPrefix reuses a matching prompt prefix already in the context, as GenerationParams.reusePromptPrefix does.

toString() → String

A string representation of this object.

inherited

Operators

operator ==(Object other) → bool

The equality operator.

inherited