speculativeDecodingConfig property

SpeculativeDecodingConfig? speculativeDecodingConfig
final

Strategy and knobs for backend-native speculative decoding.

null disables speculative decoding unless speculativeDecoding is true. When speculativeDecoding is true and this is null, backends should treat the request as SpeculativeDecodingStrategy.backendDefault.

Implementation

final SpeculativeDecodingConfig? speculativeDecodingConfig;