SpeculativeDecodingSpec_NgramSpeculation class final
N-Gram speculation works by trying to find matching tokens in the previous prompt sequence and use those as speculation for generating new tokens.
Constructors
- SpeculativeDecodingSpec_NgramSpeculation({int ngramSize = 0})
- SpeculativeDecodingSpec_NgramSpeculation.fromJson(Object? j)
-
factory
Properties
- hashCode → int
-
The hash code for this object.
no setterinherited
- ngramSize → int
-
The number of last N input tokens used as ngram to search/match
against the previous prompt sequence.
This is equal to the N in N-Gram.
The default value is 3 if not specified.
final
- qualifiedName → String
-
The fully qualified name of this message, i.e.,
google.protobuf.Durationorgoogle.rpc.ErrorInfo.finalinherited - runtimeType → Type
-
A representation of the runtime type of the object.
no setterinherited
Methods
-
noSuchMethod(
Invocation invocation) → dynamic -
Invoked when a nonexistent method or property is accessed.
inherited
-
toJson(
) → Object -
toString(
) → String -
A string representation of this object.
override
Operators
-
operator ==(
Object other) → bool -
The equality operator.
inherited
Constants
- fullyQualifiedName → const String