ResponseCachingChatClient class

A CachingChatClient that persists responses to a ResponseCache and records per-turn latency and usage in a ChatDetails object.

Inheritance
Available extensions
Annotations
  • @Source(name: 'ResponseCachingChatClient.cs', namespace: 'Microsoft.Extensions.AI.Evaluation.Reporting', repository: 'dotnet/extensions', path: 'src/Libraries/Microsoft.Extensions.AI.Evaluation.Reporting/')

Constructors

ResponseCachingChatClient(ChatClient innerClient, {required ResponseCache cache, required ChatDetails chatDetails, List<String>? cachingKeys})
Creates a ResponseCachingChatClient.

Properties

hashCode → int
The hash code for this object.
no setterinherited
innerClient → ChatClient
The inner client to delegate to.
finalinherited
runtimeType → Type
A representation of the runtime type of the object.
no setterinherited

Methods

dispose() → void
Performs application-defined tasks associated with freeing, releasing, or resetting unmanaged resources.
inherited
getCachedResponse(String key) → Future<ChatResponse?>
Retrieves a cached response for the given key, or null if not found.
override
getCacheKey(Iterable<ChatMessage> messages, ChatOptions? options) → String
Gets a cache key for the given messages and options.
override
getChatResponseFromMessage(ChatMessage message, {ChatOptions? options, CancellationToken? cancellationToken}) → Future<ChatResponse>

Available on ChatClient, provided by the ChatClientExtensions extension

Sends a single message and returns the response.
getChatResponseFromText(String message, {ChatOptions? options, CancellationToken? cancellationToken}) → Future<ChatResponse>

Available on ChatClient, provided by the ChatClientExtensions extension

Sends a single user text message and returns the response.
getResponse({required Iterable<ChatMessage> messages, ChatOptions? options, CancellationToken? cancellationToken}) → Future<ChatResponse>
Sends a chat request and returns the complete response.
override
getService<T>({Object? key}) → T?
Gets a service of the specified type.
inherited
getStreamingChatResponseFromMessage(ChatMessage message, {ChatOptions? options, CancellationToken? cancellationToken}) → Stream<ChatResponseUpdate>

Available on ChatClient, provided by the ChatClientExtensions extension

Sends a single message and returns a streaming response.
getStreamingChatResponseFromText(String message, {ChatOptions? options, CancellationToken? cancellationToken}) → Stream<ChatResponseUpdate>

Available on ChatClient, provided by the ChatClientExtensions extension

Sends a single user text message and returns a streaming response.
getStreamingResponse({required Iterable<ChatMessage> messages, ChatOptions? options, CancellationToken? cancellationToken}) → Stream<ChatResponseUpdate>
Sends a chat request and returns a stream of response updates.
inherited
noSuchMethod(Invocation invocation) → dynamic
Invoked when a nonexistent method or property is accessed.
inherited
setCachedResponse(String key, ChatResponse response) → Future<void>
Stores a response in the cache with the given key.
override
toString() → String
A string representation of this object.
inherited

Operators

operator ==(Object other) → bool
The equality operator.
inherited