ai_benchmark_jobs

Creates, updates, deletes, gets or lists an ai_benchmark_jobs resource.

Overview

Name	ai_benchmark_jobs
Type	Resource
Id	aws.sagemaker.ai_benchmark_jobs

Fields

The following fields are returned by SELECT queries:

describe_ai_benchmark_job

Name	Datatype	Description
ai_benchmark_job_arn	string	The Amazon Resource Name (ARN) of the AI benchmark job. (pattern: <code>arn:aws[a-z-]:sagemaker:[a-z0-9-]:[0-9]{12}:ai-benchmark-job/[a-zA-Z0-9](-*[a-zA-Z0-9]){0,62}</code>)
ai_benchmark_job_name	string	The name of the AI benchmark job. (pattern: <code>[a-zA-Z0-9](-*[a-zA-Z0-9]){0,62}</code>)
ai_benchmark_job_status	string	The status of the AI benchmark job. (InProgress, Completed, Failed, Stopping, Stopped)
ai_workload_config_identifier	string	The name or Amazon Resource Name (ARN) of the AI workload configuration used for this benchmark job. (pattern: <code>(arn:aws[a-z-]:sagemaker:[a-z0-9-]:[0-9]{12}:[a-z-]*/)?([a-zA-Z0-9]([a-zA-Z0-9-]){0,62})(?<!-)</code>)
benchmark_target	object	The target endpoint that was benchmarked.
creation_time	string (date-time)	A timestamp that indicates when the benchmark job was created.
end_time	string (date-time)	A timestamp that indicates when the benchmark job completed.
failure_reason	string	If the benchmark job failed, the reason it failed.
network_config	object	The network configuration for the benchmark job.
output_config	object	The output configuration for the benchmark job, including the Amazon S3 output location and CloudWatch log information.
role_arn	string	The Amazon Resource Name (ARN) of the IAM role used by the benchmark job. (pattern: <code>arn:aws[a-z-]*:iam::\d{12}:role/?[a-zA-Z_0-9+=,.@-_/]+</code>)
start_time	string (date-time)	A timestamp that indicates when the benchmark job started running.
tags	array	The tags associated with the benchmark job.

list_ai_benchmark_jobs

Name	Datatype	Description
ai_benchmark_job_arn	string	The Amazon Resource Name (ARN) of the benchmark job. (pattern: <code>arn:aws[a-z-]:sagemaker:[a-z0-9-]:[0-9]{12}:ai-benchmark-job/[a-zA-Z0-9](-*[a-zA-Z0-9]){0,62}</code>)
ai_benchmark_job_name	string	The name of the benchmark job. (pattern: <code>[a-zA-Z0-9](-*[a-zA-Z0-9]){0,62}</code>)
ai_benchmark_job_status	string	The status of the benchmark job. (InProgress, Completed, Failed, Stopping, Stopped)
ai_workload_config_name	string	The name of the AI workload configuration used by the benchmark job. (pattern: <code>[a-zA-Z0-9](-*[a-zA-Z0-9]){0,62}</code>)
creation_time	string (date-time)	A timestamp that indicates when the benchmark job was created.
end_time	string (date-time)	A timestamp that indicates when the benchmark job completed.

Methods

The following methods are available for this resource:

Name	Accessible by	Required Params	Optional Params	Description
describe_ai_benchmark_job	select	region		Returns details of an AI benchmark job, including its status, configuration, target endpoint, and timing information.
list_ai_benchmark_jobs	select	region		Returns a list of AI benchmark jobs in your account. You can filter the results by name, status, and creation time, and sort the results. The response is paginated.
create_ai_benchmark_job	insert	region, AIBenchmarkJobName, BenchmarkTarget, OutputConfig, AIWorkloadConfigIdentifier, RoleArn		Creates a benchmark job that runs performance benchmarks against inference infrastructure using a predefined AI workload configuration. The benchmark job measures metrics such as latency, throughput, and cost for your generative AI inference endpoints.
delete_ai_benchmark_job	delete	region		Deletes the specified AI benchmark job.
stop_ai_benchmark_job	exec	region, AIBenchmarkJobName		Stops a running AI benchmark job.

Parameters

Parameters can be passed in the WHERE clause of a query. Check the Methods section to see which parameters are required or optional for each operation.

Name	Datatype	Description
region	string	AWS region (default: us-east-1)

SELECT examples

describe_ai_benchmark_job

Returns details of an AI benchmark job, including its status, configuration, target endpoint, and timing information.

SELECT
ai_benchmark_job_arn,
ai_benchmark_job_name,
ai_benchmark_job_status,
ai_workload_config_identifier,
benchmark_target,
creation_time,
end_time,
failure_reason,
network_config,
output_config,
role_arn,
start_time,
tags
FROM aws.sagemaker.ai_benchmark_jobs
WHERE region = '{{ region }}' -- required
;

list_ai_benchmark_jobs

Returns a list of AI benchmark jobs in your account. You can filter the results by name, status, and creation time, and sort the results. The response is paginated.

SELECT
ai_benchmark_job_arn,
ai_benchmark_job_name,
ai_benchmark_job_status,
ai_workload_config_name,
creation_time,
end_time
FROM aws.sagemaker.ai_benchmark_jobs
WHERE region = '{{ region }}' -- required
;

INSERT examples

create_ai_benchmark_job

Creates a benchmark job that runs performance benchmarks against inference infrastructure using a predefined AI workload configuration. The benchmark job measures metrics such as latency, throughput, and cost for your generative AI inference endpoints.

INSERT INTO aws.sagemaker.ai_benchmark_jobs (
AIBenchmarkJobName,
BenchmarkTarget,
OutputConfig,
AIWorkloadConfigIdentifier,
RoleArn,
NetworkConfig,
Tags,
region
)
SELECT 
'{{ AIBenchmarkJobName }}' /* required */,
'{{ BenchmarkTarget }}' /* required */,
'{{ OutputConfig }}' /* required */,
'{{ AIWorkloadConfigIdentifier }}' /* required */,
'{{ RoleArn }}' /* required */,
'{{ NetworkConfig }}',
'{{ Tags }}',
'{{ region }}'
RETURNING
ai_benchmark_job_arn
;

Manifest

# Description fields are for documentation purposes
- name: ai_benchmark_jobs
props:
  - name: region
    value: "{{ region }}"
    description: Required parameter for the ai_benchmark_jobs resource.
  - name: AIBenchmarkJobName
    value: "{{ AIBenchmarkJobName }}"
    description: |
      The name of the AI benchmark job. The name must be unique within your Amazon Web Services account in the current Amazon Web Services Region.
  - name: BenchmarkTarget
    description: |
      The target endpoint to benchmark. Specify a SageMaker endpoint by providing its name or Amazon Resource Name (ARN).
    value:
      Endpoint:
        Identifier: "{{ Identifier }}"
        TargetContainerHostname: "{{ TargetContainerHostname }}"
        InferenceComponents:
          - Identifier: "{{ Identifier }}"
  - name: OutputConfig
    description: |
      The output configuration for the benchmark job, including the Amazon S3 location where benchmark results are stored.
    value:
      S3OutputLocation: "{{ S3OutputLocation }}"
      MlflowConfig:
        MlflowResourceArn: "{{ MlflowResourceArn }}"
        MlflowExperimentName: "{{ MlflowExperimentName }}"
        MlflowRunName: "{{ MlflowRunName }}"
  - name: AIWorkloadConfigIdentifier
    value: "{{ AIWorkloadConfigIdentifier }}"
    description: |
      The name or Amazon Resource Name (ARN) of the AI workload configuration to use for this benchmark job.
  - name: RoleArn
    value: "{{ RoleArn }}"
    description: |
      The Amazon Resource Name (ARN) of an IAM role that enables Amazon SageMaker AI to perform tasks on your behalf.
  - name: NetworkConfig
    description: |
      The network configuration for the benchmark job, including VPC settings.
    value:
      VpcConfig:
        SecurityGroupIds:
          - "{{ SecurityGroupIds }}"
        Subnets:
          - "{{ Subnets }}"
  - name: Tags
    description: |
      The metadata that you apply to Amazon Web Services resources to help you categorize and organize them. Each tag consists of a key and a value, both of which you define.
    value:
      - Key: "{{ Key }}"
        Value: "{{ Value }}"

DELETE examples

delete_ai_benchmark_job

Deletes the specified AI benchmark job.

DELETE FROM aws.sagemaker.ai_benchmark_jobs
WHERE region = '{{ region }}' --required
;

Lifecycle Methods

stop_ai_benchmark_job

Stops a running AI benchmark job.

EXEC aws.sagemaker.ai_benchmark_jobs.stop_ai_benchmark_job 
@region='{{ region }}' --required 
@@json=
'{
"AIBenchmarkJobName": "{{ AIBenchmarkJobName }}"
}'
;