Skip to main content

inference_experiments

Creates, updates, deletes, gets or lists an inference_experiments resource.

Overview​

Nameinference_experiments
TypeResource
Idaws.sagemaker.inference_experiments

Fields​

The following fields are returned by SELECT queries:

NameDatatypeDescription
arnstringThe ARN of the inference experiment being described. (pattern: <code>arn:aws[a-z-]:sagemaker:[a-z0-9-]:[0-9]{12}:inference-experiment/.*</code>)
completion_timestring (date-time)The timestamp at which the inference experiment was completed.
creation_timestring (date-time)The timestamp at which you created the inference experiment.
data_storage_configobjectThe Amazon S3 location and configuration for storing inference request and response data.
descriptionstringThe description of the inference experiment. (pattern: <code>.*</code>)
endpoint_metadataobjectThe metadata of the endpoint on which the inference experiment ran.
kms_keystringThe Amazon Web Services Key Management Service (Amazon Web Services KMS) key that Amazon SageMaker uses to encrypt data on the storage volume attached to the ML compute instance that hosts the endpoint. For more information, see CreateInferenceExperiment. (pattern: <code>[a-zA-Z0-9:/_-]*</code>)
last_modified_timestring (date-time)The timestamp at which you last modified the inference experiment.
model_variantsarrayAn array of ModelVariantConfigSummary objects. There is one for each variant in the inference experiment. Each ModelVariantConfigSummary object in the array describes the infrastructure configuration for deploying the corresponding variant.
namestringThe name of the inference experiment. (pattern: <code>[a-zA-Z0-9](-*[a-zA-Z0-9]){0,119}</code>)
role_arnstringThe ARN of the IAM role that Amazon SageMaker can assume to access model artifacts and container images, and manage Amazon SageMaker Inference endpoints for model deployment. (pattern: <code>arn:aws[a-z-]*:iam::\d{12}:role/?[a-zA-Z_0-9+=,.@-_/]+</code>)
scheduleobjectThe duration for which the inference experiment ran or will run.
shadow_mode_configobjectThe configuration of ShadowMode inference experiment type, which shows the production variant that takes all the inference requests, and the shadow variant to which Amazon SageMaker replicates a percentage of the inference requests. For the shadow variant it also shows the percentage of requests that Amazon SageMaker replicates.
statusstringThe status of the inference experiment. The following are the possible statuses for an inference experiment: Creating - Amazon SageMaker is creating your experiment. Created - Amazon SageMaker has finished the creation of your experiment and will begin the experiment at the scheduled time. Updating - When you make changes to your experiment, your experiment shows as updating. Starting - Amazon SageMaker is beginning your experiment. Running - Your experiment is in progress. Stopping - Amazon SageMaker is stopping your experiment. Completed - Your experiment has completed. Cancelled - When you conclude your experiment early using the StopInferenceExperiment API, or if any operation fails with an unexpected error, it shows as cancelled. (Creating, Created, Updating, Running, Starting, Stopping, Completed, Cancelled)
status_reasonstringThe error message or client-specified Reason from the StopInferenceExperiment API, that explains the status of the inference experiment. (pattern: <code>.*</code>)
typestringThe type of the inference experiment. (ShadowMode)

Methods​

The following methods are available for this resource:

NameAccessible byRequired ParamsOptional ParamsDescription
describe_inference_experimentselectregionReturns details about an inference experiment.
create_inference_experimentinsertregion, RoleArn, EndpointName, ModelVariants, ShadowModeConfigCreates an inference experiment using the configurations specified in the request. Use this API to setup and schedule an experiment to compare model variants on a Amazon SageMaker inference endpoint. For more information about inference experiments, see Shadow tests. Amazon SageMaker begins your experiment at the scheduled time and routes traffic to your endpoint's model variants based on your specified configuration. While the experiment is in progress or after it has concluded, you can view metrics that compare your model variants. For more information, see View, monitor, and edit shadow tests.
update_inference_experimentupdateregionUpdates an inference experiment that you created. The status of the inference experiment has to be either Created, Running. For more information on the status of an inference experiment, see DescribeInferenceExperiment.
delete_inference_experimentdeleteregionDeletes an inference experiment. This operation does not delete your endpoint, variants, or any underlying resources. This operation only deletes the metadata of your experiment.
list_inference_experimentsexecregionReturns the list of all inference experiments.

Parameters​

Parameters can be passed in the WHERE clause of a query. Check the Methods section to see which parameters are required or optional for each operation.

NameDatatypeDescription
regionstringAWS region (default: us-east-1)

SELECT examples​

Returns details about an inference experiment.

SELECT
arn,
completion_time,
creation_time,
data_storage_config,
description,
endpoint_metadata,
kms_key,
last_modified_time,
model_variants,
name,
role_arn,
schedule,
shadow_mode_config,
status,
status_reason,
type
FROM aws.sagemaker.inference_experiments
WHERE region = '{{ region }}' -- required
;

INSERT examples​

Creates an inference experiment using the configurations specified in the request. Use this API to setup and schedule an experiment to compare model variants on a Amazon SageMaker inference endpoint. For more information about inference experiments, see Shadow tests. Amazon SageMaker begins your experiment at the scheduled time and routes traffic to your endpoint's model variants based on your specified configuration. While the experiment is in progress or after it has concluded, you can view metrics that compare your model variants. For more information, see View, monitor, and edit shadow tests.

INSERT INTO aws.sagemaker.inference_experiments (
Name,
Type,
Schedule,
Description,
RoleArn,
EndpointName,
ModelVariants,
DataStorageConfig,
ShadowModeConfig,
KmsKey,
Tags,
region
)
SELECT 
'{{ Name }}',
'{{ Type }}',
'{{ Schedule }}',
'{{ Description }}',
'{{ RoleArn }}' /* required */,
'{{ EndpointName }}' /* required */,
'{{ ModelVariants }}' /* required */,
'{{ DataStorageConfig }}',
'{{ ShadowModeConfig }}' /* required */,
'{{ KmsKey }}',
'{{ Tags }}',
'{{ region }}'
RETURNING
inference_experiment_arn
;

UPDATE examples​

Updates an inference experiment that you created. The status of the inference experiment has to be either Created, Running. For more information on the status of an inference experiment, see DescribeInferenceExperiment.

UPDATE aws.sagemaker.inference_experiments
SET 
Name = '{{ Name }}',
Schedule = '{{ Schedule }}',
Description = '{{ Description }}',
ModelVariants = '{{ ModelVariants }}',
DataStorageConfig = '{{ DataStorageConfig }}',
ShadowModeConfig = '{{ ShadowModeConfig }}'
WHERE 
region = '{{ region }}' --required
RETURNING
inference_experiment_arn;

DELETE examples​

Deletes an inference experiment. This operation does not delete your endpoint, variants, or any underlying resources. This operation only deletes the metadata of your experiment.

DELETE FROM aws.sagemaker.inference_experiments
WHERE region = '{{ region }}' --required
;

Lifecycle Methods​

Returns the list of all inference experiments.

EXEC aws.sagemaker.inference_experiments.list_inference_experiments 
@region='{{ region }}' --required 
@@json=
'{
"NameContains": "{{ NameContains }}", 
"Type": "{{ Type }}", 
"StatusEquals": "{{ StatusEquals }}", 
"CreationTimeAfter": "{{ CreationTimeAfter }}", 
"CreationTimeBefore": "{{ CreationTimeBefore }}", 
"LastModifiedTimeAfter": "{{ LastModifiedTimeAfter }}", 
"LastModifiedTimeBefore": "{{ LastModifiedTimeBefore }}", 
"SortBy": "{{ SortBy }}", 
"SortOrder": "{{ SortOrder }}", 
"NextToken": "{{ NextToken }}", 
"MaxResults": {{ MaxResults }}
}'
;
Last updated on