online_evaluation_configs

Creates, updates, deletes, gets or lists an online_evaluation_configs resource.

Overview

Name	`online_evaluation_configs`
Type	Resource
Id	`aws.bedrock_agentcore_control.online_evaluation_configs`

Fields

The following fields are returned by SELECT queries:

get_online_evaluation_config
list_online_evaluation_configs

Name	Datatype	Description
`clustering_config`	`object`	Configuration for periodic batch evaluation clustering, specifying how often clustering jobs run.
`created_at`	`string (date-time)`	The timestamp when the online evaluation configuration was created.
`data_source_config`	`object`	The configuration that specifies where to read agent traces for online evaluation.
`description`	`string`	The description of the online evaluation configuration. (pattern: <code>.+</code>)
`evaluation_execution_role_arn`	`string`	The Amazon Resource Name (ARN) of the IAM role used for evaluation execution. (pattern: <code>arn:aws(-[^:]+)?:iam::([0-9]{12})?:role/.+</code>)
`evaluators`	`array`	The list of evaluators applied during online evaluation.
`execution_status`	`string`	The execution status indicating whether the online evaluation is currently running. (ENABLED, DISABLED)
`failure_reason`	`string`	The reason for failure if the online evaluation configuration execution failed.
`insights`	`array`	The list of insight types configured for this evaluation.
`online_evaluation_config_arn`	`string`	The Amazon Resource Name (ARN) of the online evaluation configuration. (pattern: <code>arn:aws[a-zA-Z-]*:bedrock-agentcore:[a-z0-9-]+:[0-9]{12}:online-evaluation-config/[a-zA-Z][a-zA-Z0-9-_]{0,99}-[a-zA-Z0-9]{10}</code>)
`online_evaluation_config_id`	`string`	The unique identifier of the online evaluation configuration. (pattern: <code>[a-zA-Z][a-zA-Z0-9-_]{0,99}-[a-zA-Z0-9]{10}</code>)
`online_evaluation_config_name`	`string`	The name of the online evaluation configuration. (pattern: <code>[a-zA-Z][a-zA-Z0-9_]{0,47}</code>)
`output_config`	`object`	The configuration that specifies where evaluation results should be written for monitoring and analysis.
`rule`	`object`	The evaluation rule that defines sampling configuration, filtering criteria, and session detection settings for online evaluation.
`status`	`string`	The status of the online evaluation configuration. (ACTIVE, CREATING, CREATE_FAILED, UPDATING, UPDATE_FAILED, DELETING, ERROR)
`updated_at`	`string (date-time)`	The timestamp when the online evaluation configuration was last updated.

Name	Datatype	Description
`clustering_config`	`object`	Configuration for periodic batch evaluation clustering, specifying how often clustering jobs run.
`created_at`	`string (date-time)`	The timestamp when the online evaluation configuration was created.
`description`	`string`	The description of the online evaluation configuration. (pattern: <code>.+</code>)
`execution_status`	`string`	The execution status indicating whether the online evaluation is currently running. (ENABLED, DISABLED)
`failure_reason`	`string`	The reason for failure if the online evaluation configuration execution failed.
`insights`	`array`	The list of insight types configured for this evaluation.
`online_evaluation_config_arn`	`string`	The Amazon Resource Name (ARN) of the online evaluation configuration. (pattern: <code>arn:aws[a-zA-Z-]*:bedrock-agentcore:[a-z0-9-]+:[0-9]{12}:online-evaluation-config/[a-zA-Z][a-zA-Z0-9-_]{0,99}-[a-zA-Z0-9]{10}</code>)
`online_evaluation_config_id`	`string`	The unique identifier of the online evaluation configuration. (pattern: <code>[a-zA-Z][a-zA-Z0-9-_]{0,99}-[a-zA-Z0-9]{10}</code>)
`online_evaluation_config_name`	`string`	The name of the online evaluation configuration. (pattern: <code>[a-zA-Z][a-zA-Z0-9_]{0,47}</code>)
`status`	`string`	The status of the online evaluation configuration. (ACTIVE, CREATING, CREATE_FAILED, UPDATING, UPDATE_FAILED, DELETING, ERROR)
`updated_at`	`string (date-time)`	The timestamp when the online evaluation configuration was last updated.

Methods

The following methods are available for this resource:

Name	Accessible by	Required Params	Optional Params	Description
`get_online_evaluation_config`	`select`	`online_evaluation_config_id`, `region`		Retrieves detailed information about an online evaluation configuration, including its rules, data sources, evaluators, and execution status.
`list_online_evaluation_configs`	`select`	`region`	`nextToken`, `maxResults`	Lists all online evaluation configurations in the account, providing summary information about each configuration's status and settings.
`create_online_evaluation_config`	`insert`	`region`, `onlineEvaluationConfigName`, `rule`, `dataSourceConfig`, `evaluationExecutionRoleArn`, `enableOnCreate`		Creates an online evaluation configuration for continuous monitoring of agent performance. Online evaluation automatically samples live traffic from CloudWatch logs at specified rates and applies evaluators to assess agent quality in production.
`update_online_evaluation_config`	`update`	`online_evaluation_config_id`, `region`		Updates an online evaluation configuration's settings, including rules, data sources, evaluators, and execution status. Changes take effect immediately for ongoing evaluations.
`delete_online_evaluation_config`	`delete`	`online_evaluation_config_id`, `region`		Deletes an online evaluation configuration and stops any ongoing evaluation processes associated with it.

Parameters

Parameters can be passed in the WHERE clause of a query. Check the Methods section to see which parameters are required or optional for each operation.

Name	Datatype	Description
`online_evaluation_config_id`	`string`	The unique identifier of the online evaluation configuration to delete.
`region`	`string`	AWS region (default: us-east-1)
`maxResults`	`integer`	The maximum number of online evaluation configurations to return in a single response.
`nextToken`	`string`	The pagination token from a previous request to retrieve the next page of results.

`SELECT` examples

get_online_evaluation_config
list_online_evaluation_configs

Retrieves detailed information about an online evaluation configuration, including its rules, data sources, evaluators, and execution status.

SELECT
clustering_config,
created_at,
data_source_config,
description,
evaluation_execution_role_arn,
evaluators,
execution_status,
failure_reason,
insights,
online_evaluation_config_arn,
online_evaluation_config_id,
online_evaluation_config_name,
output_config,
rule,
status,
updated_at
FROM aws.bedrock_agentcore_control.online_evaluation_configs
WHERE online_evaluation_config_id = '{{ online_evaluation_config_id }}' -- required
AND region = '{{ region }}' -- required
;

Lists all online evaluation configurations in the account, providing summary information about each configuration's status and settings.

SELECT
clustering_config,
created_at,
description,
execution_status,
failure_reason,
insights,
online_evaluation_config_arn,
online_evaluation_config_id,
online_evaluation_config_name,
status,
updated_at
FROM aws.bedrock_agentcore_control.online_evaluation_configs
WHERE region = '{{ region }}' -- required
AND nextToken = '{{ nextToken }}'
AND maxResults = '{{ maxResults }}'
;

`INSERT` examples

create_online_evaluation_config
Manifest

Creates an online evaluation configuration for continuous monitoring of agent performance. Online evaluation automatically samples live traffic from CloudWatch logs at specified rates and applies evaluators to assess agent quality in production.

INSERT INTO aws.bedrock_agentcore_control.online_evaluation_configs (
clientToken,
onlineEvaluationConfigName,
description,
rule,
dataSourceConfig,
evaluators,
insights,
clusteringConfig,
evaluationExecutionRoleArn,
enableOnCreate,
tags,
region
)
SELECT 
'{{ clientToken }}',
'{{ onlineEvaluationConfigName }}' /* required */,
'{{ description }}',
'{{ rule }}' /* required */,
'{{ dataSourceConfig }}' /* required */,
'{{ evaluators }}',
'{{ insights }}',
'{{ clusteringConfig }}',
'{{ evaluationExecutionRoleArn }}' /* required */,
{{ enableOnCreate }} /* required */,
'{{ tags }}',
'{{ region }}'
RETURNING
created_at,
execution_status,
failure_reason,
online_evaluation_config_arn,
online_evaluation_config_id,
output_config,
status
;

# Description fields are for documentation purposes
- name: online_evaluation_configs
props:
  - name: region
    value: "{{ region }}"
    description: Required parameter for the online_evaluation_configs resource.
  - name: clientToken
    value: "{{ clientToken }}"
  - name: onlineEvaluationConfigName
    value: "{{ onlineEvaluationConfigName }}"
  - name: description
    value: "{{ description }}"
  - name: rule
    description: |
      The evaluation rule that defines sampling configuration, filtering criteria, and session detection settings for online evaluation.
    value:
      samplingConfig:
        samplingPercentage: {{ samplingPercentage }}
      filters:
        - key: "{{ key }}"
          operator: "{{ operator }}"
          value:
            stringValue: "{{ stringValue }}"
            doubleValue: {{ doubleValue }}
            booleanValue: {{ booleanValue }}
      sessionConfig:
        sessionTimeoutMinutes: {{ sessionTimeoutMinutes }}
  - name: dataSourceConfig
    description: |
      The configuration that specifies where to read agent traces for online evaluation.
    value:
      cloudWatchLogs:
        logGroupNames:
          - "{{ logGroupNames }}"
        serviceNames:
          - "{{ serviceNames }}"
  - name: evaluators
    value:
      - evaluatorId: "{{ evaluatorId }}"
  - name: insights
    value:
      - insightId: "{{ insightId }}"
  - name: clusteringConfig
    description: |
      Configuration for periodic batch evaluation clustering, specifying how often clustering jobs run.
    value:
      frequencies:
        - "{{ frequencies }}"
  - name: evaluationExecutionRoleArn
    value: "{{ evaluationExecutionRoleArn }}"
  - name: enableOnCreate
    value: {{ enableOnCreate }}
  - name: tags
    value: "{{ tags }}"

`UPDATE` examples

update_online_evaluation_config

Updates an online evaluation configuration's settings, including rules, data sources, evaluators, and execution status. Changes take effect immediately for ongoing evaluations.

UPDATE aws.bedrock_agentcore_control.online_evaluation_configs
SET 
clientToken = '{{ clientToken }}',
description = '{{ description }}',
rule = '{{ rule }}',
dataSourceConfig = '{{ dataSourceConfig }}',
evaluators = '{{ evaluators }}',
insights = '{{ insights }}',
clusteringConfig = '{{ clusteringConfig }}',
evaluationExecutionRoleArn = '{{ evaluationExecutionRoleArn }}',
executionStatus = '{{ executionStatus }}'
WHERE 
online_evaluation_config_id = '{{ online_evaluation_config_id }}' --required
AND region = '{{ region }}' --required
RETURNING
execution_status,
failure_reason,
online_evaluation_config_arn,
online_evaluation_config_id,
status,
updated_at;

`DELETE` examples

delete_online_evaluation_config

Deletes an online evaluation configuration and stops any ongoing evaluation processes associated with it.

DELETE FROM aws.bedrock_agentcore_control.online_evaluation_configs
WHERE online_evaluation_config_id = '{{ online_evaluation_config_id }}' --required
AND region = '{{ region }}' --required
;

Overview​

Fields​

Methods​

Parameters​

SELECT examples​

INSERT examples​

UPDATE examples​

DELETE examples​