core/cancellation library

Cancellation support for LLM operations

This module provides cancellation capabilities for all async operations in the library, including chat requests, streaming, embeddings, and audio.

Usage

import 'package:llm_dart/llm_dart.dart';

// Create a cancel token
final cancelToken = CancelToken();

// Start an operation
final future = provider.chat(messages, cancelToken: cancelToken);

// Cancel it later (e.g., user cancels)
cancelToken.cancel('User cancelled');

// Handle cancellation
try {
  final response = await future;
} catch (e) {
  if (CancellationHelper.isCancelled(e)) {
    print('Operation was cancelled');
  }
}

How it works

The library uses Dio's CancelToken internally to provide true cancellation of HTTP requests at the network level. When you cancel a token:

• In-flight HTTP requests are aborted immediately
• Streaming responses stop emitting events
• Providers stop generating tokens

The same token can be shared across multiple operations - cancelling it will abort all operations bound to that token.

Classes

CancellationHelper

Helper utilities for working with cancellation

CancelToken

Controls cancellation of Dio's requests.