streamBatching property

bool streamBatching

Whether GenerationParams.streamBatchTokenThreshold and GenerationParams.streamBatchByteThreshold are applied.

Defaults to false so a backend that predates this field does not claim it.

Implementation

final bool streamBatching;