invokeEndpointWithBidirectionalStream method

Future<InvokeEndpointWithBidirectionalStreamOutput> invokeEndpointWithBidirectionalStream({ required RequestStreamEvent body, required String endpointName, String? modelInvocationPath, String? modelQueryString, String? targetVariant, })

Invokes a model endpoint with bidirectional streaming capabilities. This operation establishes a persistent connection that allows you to send multiple requests and receive streaming responses from the model in real-time.

Bidirectional streaming is useful for interactive applications such as chatbots, real-time translation, or any scenario where you need to maintain a conversation-like interaction with the model. The connection remains open, allowing you to send additional input and receive responses without establishing a new connection for each request.

For an overview of Amazon SageMaker AI, see How It Works.

Amazon SageMaker AI strips all POST headers except those supported by the API. Amazon SageMaker AI might add additional headers. You should not rely on the behavior of headers outside those enumerated in the request syntax.

Calls to InvokeEndpointWithBidirectionalStream are authenticated by using Amazon Web Services Signature Version 4. For information, see Authenticating Requests (Amazon Web Services Signature Version 4) in the Amazon S3 API Reference.

The bidirectional stream maintains the connection until either the client closes it or the model indicates completion. Each request and response in the stream is sent as an event with optional headers for data type and completion state.

May throw InputValidationError. May throw InternalServerError. May throw InternalStreamFailure. May throw ModelError. May throw ModelStreamError. May throw ServiceUnavailableError.

Parameter body : The request payload stream.

Parameter endpointName : The name of the endpoint to invoke.

Parameter modelInvocationPath : Model invocation path.

Parameter modelQueryString : Model query string.

Parameter targetVariant : Target variant for the request.

Implementation

Future<InvokeEndpointWithBidirectionalStreamOutput>
    invokeEndpointWithBidirectionalStream({
  required RequestStreamEvent body,
  required String endpointName,
  String? modelInvocationPath,
  String? modelQueryString,
  String? targetVariant,
}) async {
  final headers = <String, String>{
    if (modelInvocationPath != null)
      'X-Amzn-SageMaker-Model-Invocation-Path':
          modelInvocationPath.toString(),
    if (modelQueryString != null)
      'X-Amzn-SageMaker-Model-Query-String': modelQueryString.toString(),
    if (targetVariant != null)
      'X-Amzn-SageMaker-Target-Variant': targetVariant.toString(),
  };
  final response = await _protocol.sendRaw(
    payload: body,
    method: 'POST',
    requestUri:
        '/endpoints/${Uri.encodeComponent(endpointName)}/invocations-bidirectional-stream',
    headers: headers,
    exceptionFnMap: _exceptionFns,
  );
  final $json = await _s.jsonFromResponse(response);
  return InvokeEndpointWithBidirectionalStreamOutput(
    body: ResponseStreamEvent.fromJson($json),
    invokedProductionVariant: _s.extractHeaderStringValue(
        response.headers, 'X-Amzn-Invoked-Production-Variant'),
  );
}