bundleSamplerFrom function

BundleSampler bundleSamplerFrom({
  1. required int type,
  2. required double temperature,
  3. required int topK,
  4. required double topP,
  5. int? seed,
})

The bundle's SamplerParameters as BundleSampler.

The engine is always sent a TOP_P sampler, because LiteRT-LM's CPU sampler rejects the other types, so the other types are written as TOP_P: a greedy bundle becomes topK: 1, and a TOP_K bundle gets topP: 1.0 so no nucleus cut is added to it. A zero field is unset: proto3 does not store one. Throws a FormatException for a type this reader does not know and an ArgumentError for a value no engine can sample with.

Implementation

BundleSampler bundleSamplerFrom({
  required int type,
  required double temperature,
  required int topK,
  required double topP,
  int? seed,
}) {
  final BundleSampler sampler;
  switch (type) {
    case _typeUnspecified:
      return const BundleSampler();
    case _typeGreedy:
      sampler = BundleSampler(topK: 1, randomSeed: seed);
    case _typeTopK || _typeTopP:
      sampler = BundleSampler(
        temperature: temperature > 0 ? temperature : null,
        topK: topK > 0 ? topK : null,
        topP: type == _typeTopK ? 1.0 : (topP > 0 ? topP : null),
        randomSeed: seed,
      );
    default:
      throw FormatException('unknown sampler type $type in the bundle');
  }
  sampler.validate();
  return sampler;
}