search method

Future<RagSearchResult> search(
  1. String query, {
  2. int topK = 10,
  3. int tokenBudget = 2000,
  4. ContextStrategy strategy = ContextStrategy.relevanceFirst,
  5. int adjacentChunks = 0,
  6. bool singleSourceMode = false,
  7. List<int>? sourceIds,
})

Search for relevant chunks and assemble context for LLM.

ChunkSearchResult.similarity is the cosine similarity of the chunk to the query, with or without sourceIds.

With sourceIds, every chunk of those sources is compared, so scoping to most of a large collection is slower than searching without sourceIds: 20.5 ms against 2.7 ms for 18,000 of 20,000 chunks (9.7 ms before this was cosine), median on an M-series Mac with fake embeddings, not measured on a device. Scoping to a few sources is fast.

Implementation

Future<RagSearchResult> search(
  String query, {
  int topK = 10,
  int tokenBudget = 2000,
  ContextStrategy strategy = ContextStrategy.relevanceFirst,
  int adjacentChunks = 0,
  bool singleSourceMode = false,
  List<int>? sourceIds,
}) => _engine!.search(
  query,
  topK: topK,
  tokenBudget: tokenBudget,
  strategy: strategy,
  adjacentChunks: adjacentChunks,
  singleSourceMode: singleSourceMode,
  sourceIds: sourceIds,
);