engineRunStreamJson static method

String? engineRunStreamJson(SyniEngineNative engine, SyniPreset preset, int seed, String requestJson, bool onChunk(String chunk),)

Streaming inference. Same shape as engineRunJson but fires onChunk for each emitted token chunk. Returns the final accumulated JSON.

onChunk runs synchronously on the worker isolate's thread (via NativeCallable.isolateLocal). It should be fast and non-blocking — typically just streamPort.send(chunk) to forward the chunk to the main isolate. Returning false from onChunk signals the runtime to stop generating early (cancellation).

MUST be called from the isolate that loaded the library — passing the resulting NativeCallable across isolate boundaries is unsupported.

Implementation

static String? engineRunStreamJson(
  SyniEngineNative engine,
  SyniPreset preset,
  int seed,
  String requestJson,
  bool Function(String chunk) onChunk,
) {
  initialize();

  final callable = NativeCallable<_SyniStreamCallbackC>.isolateLocal(
    (Pointer<Utf8> chunkPtr, Pointer<Void> _) {
      // chunkPtr is owned by the runtime — do NOT free.
      final s = chunkPtr.toDartString();
      return onChunk(s);
    },
    exceptionalReturn: false,
  );

  final jsonPtr = requestJson.toNativeUtf8();
  try {
    final resultPtr = _engineRunStreamJson!(
      engine,
      preset.value,
      seed,
      jsonPtr,
      callable.nativeFunction,
      nullptr,
    );
    if (resultPtr == nullptr) return null;
    final result = resultPtr.cast<Utf8>().toDartString();
    _stringFree!(resultPtr);
    return result;
  } finally {
    malloc.free(jsonPtr);
    callable.close();
  }
}