RetryMiddleware class

A middleware that retries model and tool requests on failure.

Only GenkitExceptions with specific status codes are retried. By default, it retries on StatusCodes.UNAVAILABLE, StatusCodes.DEADLINE_EXCEEDED, StatusCodes.RESOURCE_EXHAUSTED, StatusCodes.ABORTED, and StatusCodes.INTERNAL.

It uses exponential backoff with jitter to calculate the delay between retries.

Inheritance

Constructors

RetryMiddleware({int maxRetries = 3, List<StatusCodes> statuses = defaultRetryStatuses, int initialDelayMs = 1000, int maxDelayMs = 60000, double backoffFactor = 2.0, bool noJitter = false, bool onError(Object error, int attempt)?, bool retryModel = true, bool retryTools = false})
Creates a RetryMiddleware.

Properties

backoffFactor → double
The factor by which the delay increases with each retry.
final
hashCode → int
The hash code for this object.
no setterinherited
initialDelayMs → int
The initial delay in milliseconds for the first retry.
final
maxDelayMs → int
The maximum delay in milliseconds between retries.
final
maxRetries → int
The maximum number of retry attempts.
final
noJitter → bool
Whether to disable jitter. Jitter is enabled by default.
final
onError → bool Function(Object error, int attempt)?
An optional callback that is called on each error.
final
retryModel → bool
Whether to retry model requests. Defaults to true.
final
retryTools → bool
Whether to retry tool requests. Defaults to false.
final
runtimeType → Type
A representation of the runtime type of the object.
no setterinherited
statuses → List<StatusCodes>
The list of status codes that should trigger a retry.
final
tools → List<Tool>?
Middleware can act as a "kit" by providing tools directly. These tools will be added to the tool list of the generate call.
no setterinherited

Methods

generate(GenerateTurnState envelope, ActionFnArg<ModelResponseChunk, GenerateActionOptions, void> ctx, Future<GenerateResponseHelper> next(GenerateTurnState envelope, ActionFnArg<ModelResponseChunk, GenerateActionOptions, void> ctx)) → Future<GenerateResponseHelper>
Middleware for the top-level generate call.
inherited
model(ModelRequest request, ActionFnArg<ModelResponseChunk, ModelRequest, void> ctx, Future<ModelResponse> next(ModelRequest request, ActionFnArg<ModelResponseChunk, ModelRequest, void> ctx)) → Future<ModelResponse>
Middleware for the raw model call.
override
noSuchMethod(Invocation invocation) → dynamic
Invoked when a nonexistent method or property is accessed.
inherited
tool(ToolRequestPart request, ActionFnArg<void, dynamic, void> ctx, Future<ToolResponsePart> next(ToolRequestPart request, ActionFnArg<void, dynamic, void> ctx)) → Future<ToolResponsePart>
Middleware for tool execution.
override
toString() → String
A string representation of this object.
inherited

Operators

operator ==(Object other) → bool
The equality operator.
inherited

Constants

defaultRetryStatuses → const List<StatusCodes>
The default list of status codes that trigger a retry.