inference_components

Creates, updates, deletes, gets or lists an inference_components resource.

Overview

Name	inference_components
Type	Resource
Id	aws.sagemaker.inference_components

Fields

The following fields are returned by SELECT queries:

describe_inference_component

Name	Datatype	Description
creation_time	string (date-time)	The time when the inference component was created.
endpoint_arn	string	The Amazon Resource Name (ARN) of the endpoint that hosts the inference component. (pattern: <code>arn:aws[a-z-]:sagemaker:[a-z0-9-]:[0-9]{12}:endpoint/.*</code>)
endpoint_name	string	The name of the endpoint that hosts the inference component. (pattern: <code>[a-zA-Z0-9](-*[a-zA-Z0-9]){0,62}</code>)
failure_reason	string	If the inference component status is Failed, the reason for the failure.
inference_component_arn	string	The Amazon Resource Name (ARN) of the inference component.
inference_component_name	string	The name of the inference component. (pattern: <code>[a-zA-Z0-9]([-a-zA-Z0-9]*[a-zA-Z0-9])?</code>)
inference_component_status	string	The status of the inference component. (InService, Creating, Updating, Failed, Deleting)
last_deployment_config	object	The deployment and rollback settings that you assigned to the inference component.
last_modified_time	string (date-time)	The time when the inference component was last updated.
runtime_config	object	Details about the runtime settings for the model that is deployed with the inference component.
specification	object	Details about the resources that are deployed with this inference component.
specifications	array	A list of specification summaries for the inference component, one per instance type. This parameter is populated when the inference component was created with multiple specifications. When this parameter is populated, the singular Specification parameter is not returned.
variant_name	string	The name of the production variant that hosts the inference component. (pattern: <code>[a-zA-Z0-9](-*[a-zA-Z0-9]){0,62}</code>)

list_inference_components

Name	Datatype	Description
creation_time	string (date-time)	The time when the inference component was created.
endpoint_arn	string	The Amazon Resource Name (ARN) of the endpoint that hosts the inference component. (pattern: <code>arn:aws[a-z-]:sagemaker:[a-z0-9-]:[0-9]{12}:endpoint/.*</code>)
endpoint_name	string	The name of the endpoint that hosts the inference component. (pattern: <code>[a-zA-Z0-9](-*[a-zA-Z0-9]){0,62}</code>)
inference_component_arn	string	The Amazon Resource Name (ARN) of the inference component.
inference_component_name	string	The name of the inference component. (pattern: <code>[a-zA-Z0-9]([-a-zA-Z0-9]*[a-zA-Z0-9])?</code>)
inference_component_status	string	The status of the inference component. (InService, Creating, Updating, Failed, Deleting)
last_modified_time	string (date-time)	The time when the inference component was last updated.
variant_name	string	The name of the production variant that hosts the inference component. (pattern: <code>[a-zA-Z0-9](-*[a-zA-Z0-9]){0,62}</code>)

Methods

The following methods are available for this resource:

Name	Accessible by	Required Params	Optional Params	Description
describe_inference_component	select	region		Returns information about an inference component.
list_inference_components	select	region		Lists the inference components in your account and their properties.
create_inference_component	insert	region, InferenceComponentName, EndpointName		Creates an inference component, which is a SageMaker AI hosting object that you can use to deploy a model to an endpoint. In the inference component settings, you specify the model, the endpoint, and how the model utilizes the resources that the endpoint hosts. You can optimize resource utilization by tailoring how the required CPU cores, accelerators, and memory are allocated. You can deploy multiple inference components to an endpoint, where each inference component contains one model and the resource utilization needs for that individual model. After you deploy an inference component, you can directly invoke the associated model when you use the InvokeEndpoint API action.
update_inference_component_runtime_config	update	region, InferenceComponentName, DesiredRuntimeConfig		Runtime settings for a model that is deployed with an inference component.
update_inference_component	update	region, InferenceComponentName		Updates an inference component.
delete_inference_component	delete	region		Deletes an inference component.

Parameters

Parameters can be passed in the WHERE clause of a query. Check the Methods section to see which parameters are required or optional for each operation.

Name	Datatype	Description
region	string	AWS region (default: us-east-1)

SELECT examples

describe_inference_component

Returns information about an inference component.

SELECT
creation_time,
endpoint_arn,
endpoint_name,
failure_reason,
inference_component_arn,
inference_component_name,
inference_component_status,
last_deployment_config,
last_modified_time,
runtime_config,
specification,
specifications,
variant_name
FROM aws.sagemaker.inference_components
WHERE region = '{{ region }}' -- required
;

list_inference_components

Lists the inference components in your account and their properties.

SELECT
creation_time,
endpoint_arn,
endpoint_name,
inference_component_arn,
inference_component_name,
inference_component_status,
last_modified_time,
variant_name
FROM aws.sagemaker.inference_components
WHERE region = '{{ region }}' -- required
;

INSERT examples

create_inference_component

Creates an inference component, which is a SageMaker AI hosting object that you can use to deploy a model to an endpoint. In the inference component settings, you specify the model, the endpoint, and how the model utilizes the resources that the endpoint hosts. You can optimize resource utilization by tailoring how the required CPU cores, accelerators, and memory are allocated. You can deploy multiple inference components to an endpoint, where each inference component contains one model and the resource utilization needs for that individual model. After you deploy an inference component, you can directly invoke the associated model when you use the InvokeEndpoint API action.

INSERT INTO aws.sagemaker.inference_components (
InferenceComponentName,
EndpointName,
VariantName,
Specification,
Specifications,
RuntimeConfig,
Tags,
region
)
SELECT 
'{{ InferenceComponentName }}' /* required */,
'{{ EndpointName }}' /* required */,
'{{ VariantName }}',
'{{ Specification }}',
'{{ Specifications }}',
'{{ RuntimeConfig }}',
'{{ Tags }}',
'{{ region }}'
RETURNING
inference_component_arn
;

Manifest

# Description fields are for documentation purposes
- name: inference_components
props:
  - name: region
    value: "{{ region }}"
    description: Required parameter for the inference_components resource.
  - name: InferenceComponentName
    value: "{{ InferenceComponentName }}"
    description: |
      A unique name to assign to the inference component.
  - name: EndpointName
    value: "{{ EndpointName }}"
    description: |
      The name of an existing endpoint where you host the inference component.
  - name: VariantName
    value: "{{ VariantName }}"
    description: |
      The name of an existing production variant where you host the inference component.
  - name: Specification
    description: |
      Details about the resources to deploy with this inference component, including the model, container, and compute resources.
    value:
      InstanceType: "{{ InstanceType }}"
      ModelName: "{{ ModelName }}"
      Container:
        Image: "{{ Image }}"
        ArtifactUrl: "{{ ArtifactUrl }}"
        Environment: "{{ Environment }}"
        ContainerMetricsConfig:
          MetricsEndpoints:
            - MetricsEndpointPath: "{{ MetricsEndpointPath }}"
              MetricPublishFrequencyInSeconds: {{ MetricPublishFrequencyInSeconds }}
      StartupParameters:
        ModelDataDownloadTimeoutInSeconds: {{ ModelDataDownloadTimeoutInSeconds }}
        ContainerStartupHealthCheckTimeoutInSeconds: {{ ContainerStartupHealthCheckTimeoutInSeconds }}
      ComputeResourceRequirements:
        NumberOfCpuCoresRequired: {{ NumberOfCpuCoresRequired }}
        NumberOfAcceleratorDevicesRequired: {{ NumberOfAcceleratorDevicesRequired }}
        MinMemoryRequiredInMb: {{ MinMemoryRequiredInMb }}
        MaxMemoryRequiredInMb: {{ MaxMemoryRequiredInMb }}
      BaseInferenceComponentName: "{{ BaseInferenceComponentName }}"
      DataCacheConfig:
        EnableCaching: {{ EnableCaching }}
      SchedulingConfig:
        PlacementStrategy: "{{ PlacementStrategy }}"
        AvailabilityZoneBalance:
          EnforcementMode: "{{ EnforcementMode }}"
          MaxImbalance: {{ MaxImbalance }}
  - name: Specifications
    description: |
      A list of specification objects for the inference component, one per instance type. Use this parameter when you want to deploy a different model or resource configuration for the inference component on each instance type. You can use either this parameter or the singular Specification parameter, but not both.
    value:
      - InstanceType: "{{ InstanceType }}"
        ModelName: "{{ ModelName }}"
        Container:
          Image: "{{ Image }}"
          ArtifactUrl: "{{ ArtifactUrl }}"
          Environment: "{{ Environment }}"
          ContainerMetricsConfig:
            MetricsEndpoints:
              - MetricsEndpointPath: "{{ MetricsEndpointPath }}"
                MetricPublishFrequencyInSeconds: {{ MetricPublishFrequencyInSeconds }}
        StartupParameters:
          ModelDataDownloadTimeoutInSeconds: {{ ModelDataDownloadTimeoutInSeconds }}
          ContainerStartupHealthCheckTimeoutInSeconds: {{ ContainerStartupHealthCheckTimeoutInSeconds }}
        ComputeResourceRequirements:
          NumberOfCpuCoresRequired: {{ NumberOfCpuCoresRequired }}
          NumberOfAcceleratorDevicesRequired: {{ NumberOfAcceleratorDevicesRequired }}
          MinMemoryRequiredInMb: {{ MinMemoryRequiredInMb }}
          MaxMemoryRequiredInMb: {{ MaxMemoryRequiredInMb }}
        BaseInferenceComponentName: "{{ BaseInferenceComponentName }}"
        DataCacheConfig:
          EnableCaching: {{ EnableCaching }}
        SchedulingConfig:
          PlacementStrategy: "{{ PlacementStrategy }}"
          AvailabilityZoneBalance:
            EnforcementMode: "{{ EnforcementMode }}"
            MaxImbalance: {{ MaxImbalance }}
  - name: RuntimeConfig
    description: |
      Runtime settings for a model that is deployed with an inference component.
    value:
      CopyCount: {{ CopyCount }}
  - name: Tags
    description: |
      A list of key-value pairs associated with the model. For more information, see Tagging Amazon Web Services resources in the Amazon Web Services General Reference.
    value:
      - Key: "{{ Key }}"
        Value: "{{ Value }}"

UPDATE examples

update_inference_component_runtime_config

Runtime settings for a model that is deployed with an inference component.

UPDATE aws.sagemaker.inference_components
SET 
InferenceComponentName = '{{ InferenceComponentName }}',
DesiredRuntimeConfig = '{{ DesiredRuntimeConfig }}'
WHERE 
region = '{{ region }}' --required
AND InferenceComponentName = '{{ InferenceComponentName }}' --required
AND DesiredRuntimeConfig = '{{ DesiredRuntimeConfig }}' --required
RETURNING
inference_component_arn;

update_inference_component

Updates an inference component.

UPDATE aws.sagemaker.inference_components
SET 
InferenceComponentName = '{{ InferenceComponentName }}',
Specification = '{{ Specification }}',
Specifications = '{{ Specifications }}',
RuntimeConfig = '{{ RuntimeConfig }}',
DeploymentConfig = '{{ DeploymentConfig }}'
WHERE 
region = '{{ region }}' --required
AND InferenceComponentName = '{{ InferenceComponentName }}' --required
RETURNING
inference_component_arn;

DELETE examples

delete_inference_component

Deletes an inference component.

DELETE FROM aws.sagemaker.inference_components
WHERE region = '{{ region }}' --required
;