reembedAll method

Future<void> reembedAll({ void onProgress(RagReembedProgress progress)?, int batchSize = 32, })

Resolve an embeddingFingerprintLock by re-embedding every chunk with the currently loaded model.

The flow is resumable: progress is persisted per chunk on the Rust side, so killing the app mid-stream and restarting will let a subsequent reembedAll() pick up where this one left off without re-doing finished chunks. onProgress is invoked once per chunk with the cumulative (done, total) snapshot for the current run.

On success the lock is cleared and the HNSW index is rebuilt with the new embeddings. Throws StateError if no mismatch is active.

Cold-start latency budget for this call is "background, minutes-scale" — host apps SHOULD surface progress UI.

Implementation

Future<void> reembedAll({
  void Function(RagReembedProgress progress)? onProgress,
  int batchSize = 32,
}) async {
  final lock = _fingerprintLock;
  if (lock == null) {
    throw StateError(
      'reembedAll() called but no embedding fingerprint mismatch is active.',
    );
  }
  if (batchSize <= 0) {
    throw ArgumentError.value(batchSize, 'batchSize', 'must be positive');
  }
  final target = currentEmbeddingFingerprint;
  final initialRemaining = await migration_meta.beginEmbeddingReembed(
    targetFingerprint: target,
  );
  final total = initialRemaining;
  var done = 0;
  onProgress?.call(RagReembedProgress(done: done, total: total));

  while (true) {
    final batch = await rust_rag.listChunksNeedingReembed(
      targetFingerprint: target,
      limit: batchSize,
    );
    if (batch.isEmpty) break;
    for (final chunk in batch) {
      final embedding = await EmbeddingService.embed(chunk.content);
      await rust_rag.updateChunkReembedded(
        chunkId: chunk.chunkId,
        embedding: embedding,
        targetFingerprint: target,
      );
      done += 1;
      onProgress?.call(RagReembedProgress(done: done, total: total));
      // Yield so the embed loop never starves the event queue (e.g. so
      // host-app UI updates can land between chunks).
      await Future<void>.delayed(Duration.zero);
    }
  }

  await migration_meta.finalizeEmbeddingReembed(targetFingerprint: target);
  _fingerprintLock = null;
  debugPrint(
    '[RagEngine] reembedAll: completed $done/$total chunks; '
    'fingerprint now "$target". Rebuilding HNSW index...',
  );
  // The chunk embeddings just got rotated; force a rebuild so vector search
  // is consistent with the new fingerprint before we hand control back.
  await _ragService.rebuildIndex(force: true);
}