batch_evaluations

Creates, updates, deletes, gets or lists a batch_evaluations resource.

Overview

Name	batch_evaluations
Type	Resource
Id	aws.bedrock_agentcore.batch_evaluations

Fields

The following fields are returned by SELECT queries:

get_batch_evaluation

Name	Datatype	Description
batch_evaluation_arn	string	The Amazon Resource Name (ARN) of the batch evaluation.
batch_evaluation_id	string	The unique identifier for a batch evaluation. (pattern: <code>[a-zA-Z][a-zA-Z0-9-_]{0,99}-[a-zA-Z0-9]{10}</code>)
batch_evaluation_name	string	The name of the batch evaluation. (pattern: <code>[a-zA-Z][a-zA-Z0-9_]{0,47}</code>)
created_at	string (date-time)	The timestamp when the batch evaluation was created.
data_source_config	object	The data source configuration specifying where agent traces are pulled from.
description	string	The description of the batch evaluation.
error_details	array	The error details if the batch evaluation encountered failures.
evaluation_results	object	The aggregated evaluation results, including session completion counts and evaluator score summaries.
evaluators	array	The list of evaluators applied during the batch evaluation.
execution_summary_result	object	The execution summary clustering results from insights, containing grouped execution patterns across evaluated sessions.
failure_analysis_result	object	The failure analysis results from insights, containing categorized failure clusters with root causes and recommendations.
insights	array	The list of insight analyses applied during the batch evaluation.
kms_key_arn	string	The ARN of the KMS key used to encrypt evaluation data. (pattern: <code>arn:aws(|-cn|-us-gov):kms:[a-zA-Z0-9-]*:[0-9]{12}:key/[a-zA-Z0-9-]{36}</code>)
output_config	object	The output configuration specifying where evaluation results are written.
status	string	The current status of the batch evaluation. (PENDING, IN_PROGRESS, COMPLETED, COMPLETED_WITH_ERRORS, FAILED, STOPPING, STOPPED, DELETING)
updated_at	string (date-time)	The timestamp when the batch evaluation was last updated.
user_intent_result	object	The user intent clustering results from insights, containing grouped user intents across evaluated sessions.

list_batch_evaluations

Name	Datatype	Description
batch_evaluation_arn	string	The Amazon Resource Name (ARN) of the batch evaluation.
batch_evaluation_id	string	The unique identifier for a batch evaluation. (pattern: <code>[a-zA-Z][a-zA-Z0-9-_]{0,99}-[a-zA-Z0-9]{10}</code>)
batch_evaluation_name	string	The name of the batch evaluation. (pattern: <code>[a-zA-Z][a-zA-Z0-9_]{0,47}</code>)
created_at	string (date-time)	The timestamp when the batch evaluation was created.
description	string	The description of the batch evaluation.
error_details	array	The error details if the batch evaluation encountered failures.
evaluation_results	object	The aggregated evaluation results.
evaluators	array	The list of evaluators applied during the batch evaluation.
insights	array	The list of insight analyses applied during the batch evaluation.
kms_key_arn	string	The ARN of the KMS key used to encrypt evaluation data. (pattern: <code>arn:aws(|-cn|-us-gov):kms:[a-zA-Z0-9-]*:[0-9]{12}:key/[a-zA-Z0-9-]{36}</code>)
status	string	The current status of the batch evaluation. (PENDING, IN_PROGRESS, COMPLETED, COMPLETED_WITH_ERRORS, FAILED, STOPPING, STOPPED, DELETING)
updated_at	string (date-time)	The timestamp when the batch evaluation was last updated.

Methods

The following methods are available for this resource:

Name	Accessible by	Required Params	Optional Params	Description
get_batch_evaluation	select	batch_evaluation_id, region		Retrieves detailed information about a batch evaluation, including its status, configuration, results, and any error details.
list_batch_evaluations	select	region	maxResults, nextToken	Lists all batch evaluations in the account, providing summary information about each evaluation's status and configuration.
delete_batch_evaluation	delete	batch_evaluation_id, region		Deletes a batch evaluation and its associated results.
start_batch_evaluation	exec	region, batchEvaluationName, dataSourceConfig		Starts a batch evaluation job that evaluates agent performance across multiple sessions. Batch evaluations pull agent traces from CloudWatch Logs or an existing online evaluation configuration and run specified evaluators and insights against them.
stop_batch_evaluation	exec	batch_evaluation_id, region		Stops a running batch evaluation. Sessions that have already been evaluated retain their results.

Parameters

Parameters can be passed in the WHERE clause of a query. Check the Methods section to see which parameters are required or optional for each operation.

Name	Datatype	Description
batch_evaluation_id	string	The unique identifier of the batch evaluation to stop.
region	string	AWS region (default: us-east-1)
maxResults	integer	The maximum number of results to return in the response. If the total number of results is greater than this value, use the token returned in the response in the nextToken field when making another request to return the next batch of results.
nextToken	string	If the total number of results is greater than the maxResults value provided in the request, enter the token returned in the nextToken field in the response in this field to return the next batch of results.

SELECT examples

get_batch_evaluation

Retrieves detailed information about a batch evaluation, including its status, configuration, results, and any error details.

SELECT
batch_evaluation_arn,
batch_evaluation_id,
batch_evaluation_name,
created_at,
data_source_config,
description,
error_details,
evaluation_results,
evaluators,
execution_summary_result,
failure_analysis_result,
insights,
kms_key_arn,
output_config,
status,
updated_at,
user_intent_result
FROM aws.bedrock_agentcore.batch_evaluations
WHERE batch_evaluation_id = '{{ batch_evaluation_id }}' -- required
AND region = '{{ region }}' -- required
;

list_batch_evaluations

Lists all batch evaluations in the account, providing summary information about each evaluation's status and configuration.

SELECT
batch_evaluation_arn,
batch_evaluation_id,
batch_evaluation_name,
created_at,
description,
error_details,
evaluation_results,
evaluators,
insights,
kms_key_arn,
status,
updated_at
FROM aws.bedrock_agentcore.batch_evaluations
WHERE region = '{{ region }}' -- required
AND maxResults = '{{ maxResults }}'
AND nextToken = '{{ nextToken }}'
;

DELETE examples

delete_batch_evaluation

Deletes a batch evaluation and its associated results.

DELETE FROM aws.bedrock_agentcore.batch_evaluations
WHERE batch_evaluation_id = '{{ batch_evaluation_id }}' --required
AND region = '{{ region }}' --required
;

Lifecycle Methods

start_batch_evaluation

Starts a batch evaluation job that evaluates agent performance across multiple sessions. Batch evaluations pull agent traces from CloudWatch Logs or an existing online evaluation configuration and run specified evaluators and insights against them.

EXEC aws.bedrock_agentcore.batch_evaluations.start_batch_evaluation 
@region='{{ region }}' --required 
@@json=
'{
"batchEvaluationName": "{{ batchEvaluationName }}", 
"evaluators": "{{ evaluators }}", 
"insights": "{{ insights }}", 
"dataSourceConfig": "{{ dataSourceConfig }}", 
"clientToken": "{{ clientToken }}", 
"evaluationMetadata": "{{ evaluationMetadata }}", 
"tags": "{{ tags }}", 
"kmsKeyArn": "{{ kmsKeyArn }}", 
"description": "{{ description }}"
}'
;

stop_batch_evaluation

Stops a running batch evaluation. Sessions that have already been evaluated retain their results.

EXEC aws.bedrock_agentcore.batch_evaluations.stop_batch_evaluation 
@batch_evaluation_id='{{ batch_evaluation_id }}' --required, 
@region='{{ region }}' --required
;