runStream method

Stream<SyniRuntimeStreamChunk> runStream(SyniRuntimeRequest request, { SyniPreset preset = SyniPreset.chat, int seed = 0, })

Run inference and stream token chunks as they are generated.

Returns a Stream of SyniRuntimeStreamChunk:

• Zero or more SyniRuntimeStreamDelta events as tokens arrive.
• Exactly one SyniRuntimeStreamFinal at the end carrying the schema-validated final JSON.
• On failure: a stream error.

V1 does not support mid-stream cancellation — let the stream complete or close it on the consumer side and ignore further deltas.

Implementation

Stream<SyniRuntimeStreamChunk> runStream(
  SyniRuntimeRequest request, {
  SyniPreset preset = SyniPreset.chat,
  int seed = 0,
}) async* {
  await initialize();
  if (_modelPath == null) {
    throw SyniRuntimeError(
      'Model not loaded. Call loadModel() or downloadModel() first.',
    );
  }
  await for (final chunk in _worker!.runStream(
    preset.value,
    seed,
    jsonEncode(request.toJson()),
  )) {
    // The final chunk carries the runtime's response JSON, which may be a
    // `{"ok":false,"error":{…}}` failure envelope. Surface it as a typed
    // stream error rather than emitting an envelope as if it were content.
    if (chunk is SyniRuntimeStreamFinal) {
      final decoded = jsonDecode(chunk.rawJson);
      if (decoded is Map && decoded['ok'] == false) {
        final error = decoded['error'];
        throw SyniRuntimeError.fromEnvelope(
          error is Map
              ? error.cast<String, dynamic>()
              : const <String, dynamic>{},
        );
      }
    }
    yield chunk;
  }
}