embedAll method

Future<List<Embedding>> embedAll(List<String> inputs, { required EmbeddingPurpose purpose, AgenticContext? context, void onProgress(int done, int total)?, })

Embeds any number of inputs, splitting into provider-sized batches.

The call an ingestion pipeline actually wants: hand it ten thousand chunks and let it deal with the batch limit. Batches run sequentially rather than in parallel, because embedding endpoints rate-limit aggressively and a parallel burst is the fastest way to a 429.

Implementation

Future<List<Embedding>> embedAll(
  List<String> inputs, {
  required EmbeddingPurpose purpose,
  AgenticContext? context,
  void Function(int done, int total)? onProgress,
}) async {
  if (inputs.isEmpty) return const <Embedding>[];

  final results = <Embedding>[];
  for (var start = 0; start < inputs.length; start += maxBatchSize) {
    context?.throwIfCancelled();
    final end = math.min(start + maxBatchSize, inputs.length);
    final batch = await embed(
      inputs.sublist(start, end),
      purpose: purpose,
      context: context,
    );
    // Re-index against the full input list; providers index within a batch.
    for (var i = 0; i < batch.length; i++) {
      results.add(
        Embedding(
          values: batch[i].values,
          index: start + i,
          text: batch[i].text,
        ),
      );
    }
    onProgress?.call(results.length, inputs.length);
  }
  return results;
}