generate method

Stream<String> generate(
  1. String prompt, {
  2. int maxTokens = 256,
  3. double temperature = 0.8,
  4. int? topK,
  5. double? topP,
  6. int? seed,
  7. List<String> stopSequences = const <String>[],
  8. List<Uint8List>? images,
  9. int sequenceId = 0,
  10. void onStats(
    1. LlamaGenerationStats
    )?,
})

Generates text for prompt, yielding decoded tokens as they arrive.

Generation halts when any string in stopSequences appears in the output; the matched sequence is stripped from the stream. Cancel by cancelling the returned stream's subscription.

Pass encoded image bytes (PNG/JPEG) in images to feed a vision model; each entry corresponds, in order, to one media marker in prompt. Images require the session to have been loaded with a mmprojPath.

topK and topP add the corresponding sampler stages ahead of temperature sampling; null leaves each stage out. seed makes sampling reproducible; null draws a random seed.

onStats is invoked once when generation completes, with the run's prompt/cached/generated token counts, the reason it stopped, and its prefill/decode timings.

Implementation

Stream<String> generate(
  String prompt, {
  int maxTokens = 256,
  double temperature = 0.8,
  int? topK,
  double? topP,
  int? seed,
  List<String> stopSequences = const <String>[],
  List<Uint8List>? images,
  int sequenceId = 0,
  void Function(LlamaGenerationStats)? onStats,
}) {
  return _isolate.generate(
    GenerationRequest(
      sessionId: id,
      prompt: prompt,
      maxTokens: maxTokens,
      temperature: temperature,
      topK: topK,
      topP: topP,
      seed: seed,
      stopSequences: stopSequences,
      images: images,
      sequenceId: sequenceId,
    ),
    onStats: onStats,
  );
}