createOnlineEvaluationConfig method

Future<CreateOnlineEvaluationConfigResponse> createOnlineEvaluationConfig({ required DataSourceConfig dataSourceConfig, required bool enableOnCreate, required String evaluationExecutionRoleArn, required List<EvaluatorReference> evaluators, required String onlineEvaluationConfigName, required Rule rule, String? clientToken, String? description, Map<String, String>? tags, })

Creates an online evaluation configuration for continuous monitoring of agent performance. Online evaluation automatically samples live traffic from CloudWatch logs at specified rates and applies evaluators to assess agent quality in production.

May throw AccessDeniedException. May throw ConflictException. May throw InternalServerException. May throw ServiceQuotaExceededException. May throw ThrottlingException. May throw ValidationException.

Parameter dataSourceConfig : The data source configuration that specifies CloudWatch log groups and service names to monitor for agent traces.

Parameter enableOnCreate : Whether to enable the online evaluation configuration immediately upon creation. If true, evaluation begins automatically.

Parameter evaluationExecutionRoleArn : The Amazon Resource Name (ARN) of the IAM role that grants permissions to read from CloudWatch logs, write evaluation results, and invoke Amazon Bedrock models for evaluation. If the configuration references evaluators encrypted with a customer managed KMS key, this role must also have kms:Decrypt permission on the KMS key. The service validates this permission at configuration creation time. For more information, see Encryption at rest for AgentCore Evaluations.

Parameter evaluators : The list of evaluators to apply during online evaluation. Can include both built-in evaluators and custom evaluators created with CreateEvaluator.

Parameter onlineEvaluationConfigName : The name of the online evaluation configuration. Must be unique within your account.

Parameter rule : The evaluation rule that defines sampling configuration, filters, and session detection settings for the online evaluation.

Parameter clientToken : A unique, case-sensitive identifier to ensure that the API request completes no more than one time. If you don't specify this field, a value is randomly generated for you. If this token matches a previous request, the service ignores the request, but doesn't return an error. For more information, see Ensuring idempotency.

Parameter description : The description of the online evaluation configuration that explains its monitoring purpose and scope.

Parameter tags : A map of tag keys and values to assign to an AgentCore Online Evaluation Config. Tags enable you to categorize your resources in different ways, for example, by purpose, owner, or environment.

Implementation

Future<CreateOnlineEvaluationConfigResponse> createOnlineEvaluationConfig({
  required DataSourceConfig dataSourceConfig,
  required bool enableOnCreate,
  required String evaluationExecutionRoleArn,
  required List<EvaluatorReference> evaluators,
  required String onlineEvaluationConfigName,
  required Rule rule,
  String? clientToken,
  String? description,
  Map<String, String>? tags,
}) async {
  final $payload = <String, dynamic>{
    'dataSourceConfig': dataSourceConfig,
    'enableOnCreate': enableOnCreate,
    'evaluationExecutionRoleArn': evaluationExecutionRoleArn,
    'evaluators': evaluators,
    'onlineEvaluationConfigName': onlineEvaluationConfigName,
    'rule': rule,
    'clientToken': clientToken ?? _s.generateIdempotencyToken(),
    if (description != null) 'description': description,
    if (tags != null) 'tags': tags,
  };
  final response = await _protocol.send(
    payload: $payload,
    method: 'POST',
    requestUri: '/online-evaluation-configs/create',
    exceptionFnMap: _exceptionFns,
  );
  return CreateOnlineEvaluationConfigResponse.fromJson(response);
}