enableSpeculativeDecoding property

bool? get enableSpeculativeDecoding

Whether speculative decoding is enabled.

If null, uses the model default. If true, enables speculative decoding; an error is thrown if the model does not support it. If false, disables speculative decoding.

This flag is read when a new Engine is created. Changing this value does not affect existing Engine or Conversation instances.

Implementation

static bool? get enableSpeculativeDecoding =>
    LiteRtLmNativeRuntime.instance.enableSpeculativeDecoding;

set enableSpeculativeDecoding (bool? value)

Implementation

static set enableSpeculativeDecoding(bool? value) {
  LiteRtLmNativeRuntime.instance.enableSpeculativeDecoding = value;
}