data_quality_job_definitions

Creates, updates, deletes, gets or lists a data_quality_job_definitions resource.

Overview

Name	data_quality_job_definitions
Type	Resource
Id	aws.sagemaker.data_quality_job_definitions

Fields

The following fields are returned by SELECT queries:

describe_data_quality_job_definition

Name	Datatype	Description
creation_time	string (date-time)	The time that the data quality monitoring job definition was created.
data_quality_app_specification	object	Information about the container that runs the data quality monitoring job.
data_quality_baseline_config	object	The constraints and baselines for the data quality monitoring job definition.
data_quality_job_input	object	The list of inputs for the data quality monitoring job. Currently endpoints are supported.
data_quality_job_output_config	object	The output configuration for monitoring jobs.
job_definition_arn	string	The Amazon Resource Name (ARN) of the data quality monitoring job definition. (pattern: <code>.*</code>)
job_definition_name	string	The name of the data quality monitoring job definition. (pattern: <code>[a-zA-Z0-9](-*[a-zA-Z0-9]){0,62}</code>)
job_resources	object	Identifies the resources to deploy for a monitoring job.
network_config	object	The networking configuration for the data quality monitoring job.
role_arn	string	The Amazon Resource Name (ARN) of an IAM role that Amazon SageMaker AI can assume to perform tasks on your behalf. (pattern: <code>arn:aws[a-z-]*:iam::\d{12}:role/?[a-zA-Z_0-9+=,.@-_/]+</code>)
stopping_condition	object	A time limit for how long the monitoring job is allowed to run before stopping.

list_data_quality_job_definitions

Name	Datatype	Description
creation_time	string (date-time)	The time that the monitoring job was created.
endpoint_name	string	The name of the endpoint that the job monitors. (pattern: <code>[a-zA-Z0-9](-*[a-zA-Z0-9]){0,62}</code>)
monitoring_job_definition_arn	string	The Amazon Resource Name (ARN) of the monitoring job. (pattern: <code>.*</code>)
monitoring_job_definition_name	string	The name of the monitoring job. (pattern: <code>[a-zA-Z0-9](-*[a-zA-Z0-9]){0,62}</code>)

Methods

The following methods are available for this resource:

Name	Accessible by	Required Params	Optional Params	Description
describe_data_quality_job_definition	select	region		Gets the details of a data quality monitoring job definition.
list_data_quality_job_definitions	select	region		Lists the data quality job definitions in your account.
create_data_quality_job_definition	insert	region, JobDefinitionName, DataQualityAppSpecification, DataQualityJobInput, DataQualityJobOutputConfig, JobResources, RoleArn		Creates a definition for a job that monitors data quality and drift. For information about model monitor, see Amazon SageMaker AI Model Monitor.
delete_data_quality_job_definition	delete	region		Deletes a data quality monitoring job definition.

Parameters

Parameters can be passed in the WHERE clause of a query. Check the Methods section to see which parameters are required or optional for each operation.

Name	Datatype	Description
region	string	AWS region (default: us-east-1)

SELECT examples

describe_data_quality_job_definition

Gets the details of a data quality monitoring job definition.

SELECT
creation_time,
data_quality_app_specification,
data_quality_baseline_config,
data_quality_job_input,
data_quality_job_output_config,
job_definition_arn,
job_definition_name,
job_resources,
network_config,
role_arn,
stopping_condition
FROM aws.sagemaker.data_quality_job_definitions
WHERE region = '{{ region }}' -- required
;

list_data_quality_job_definitions

Lists the data quality job definitions in your account.

SELECT
creation_time,
endpoint_name,
monitoring_job_definition_arn,
monitoring_job_definition_name
FROM aws.sagemaker.data_quality_job_definitions
WHERE region = '{{ region }}' -- required
;

INSERT examples

create_data_quality_job_definition

Creates a definition for a job that monitors data quality and drift. For information about model monitor, see Amazon SageMaker AI Model Monitor.

INSERT INTO aws.sagemaker.data_quality_job_definitions (
JobDefinitionName,
DataQualityBaselineConfig,
DataQualityAppSpecification,
DataQualityJobInput,
DataQualityJobOutputConfig,
JobResources,
NetworkConfig,
RoleArn,
StoppingCondition,
Tags,
region
)
SELECT 
'{{ JobDefinitionName }}' /* required */,
'{{ DataQualityBaselineConfig }}',
'{{ DataQualityAppSpecification }}' /* required */,
'{{ DataQualityJobInput }}' /* required */,
'{{ DataQualityJobOutputConfig }}' /* required */,
'{{ JobResources }}' /* required */,
'{{ NetworkConfig }}',
'{{ RoleArn }}' /* required */,
'{{ StoppingCondition }}',
'{{ Tags }}',
'{{ region }}'
RETURNING
job_definition_arn
;

Manifest

# Description fields are for documentation purposes
- name: data_quality_job_definitions
props:
  - name: region
    value: "{{ region }}"
    description: Required parameter for the data_quality_job_definitions resource.
  - name: JobDefinitionName
    value: "{{ JobDefinitionName }}"
    description: |
      The name for the monitoring job definition.
  - name: DataQualityBaselineConfig
    description: |
      Configures the constraints and baselines for the monitoring job.
    value:
      BaseliningJobName: "{{ BaseliningJobName }}"
      ConstraintsResource:
        S3Uri: "{{ S3Uri }}"
      StatisticsResource:
        S3Uri: "{{ S3Uri }}"
  - name: DataQualityAppSpecification
    description: |
      Specifies the container that runs the monitoring job.
    value:
      ImageUri: "{{ ImageUri }}"
      ContainerEntrypoint:
        - "{{ ContainerEntrypoint }}"
      ContainerArguments:
        - "{{ ContainerArguments }}"
      RecordPreprocessorSourceUri: "{{ RecordPreprocessorSourceUri }}"
      PostAnalyticsProcessorSourceUri: "{{ PostAnalyticsProcessorSourceUri }}"
      Environment: "{{ Environment }}"
  - name: DataQualityJobInput
    description: |
      A list of inputs for the monitoring job. Currently endpoints are supported as monitoring inputs.
    value:
      EndpointInput:
        EndpointName: "{{ EndpointName }}"
        LocalPath: "{{ LocalPath }}"
        S3InputMode: "{{ S3InputMode }}"
        S3DataDistributionType: "{{ S3DataDistributionType }}"
        FeaturesAttribute: "{{ FeaturesAttribute }}"
        InferenceAttribute: "{{ InferenceAttribute }}"
        ProbabilityAttribute: "{{ ProbabilityAttribute }}"
        ProbabilityThresholdAttribute: {{ ProbabilityThresholdAttribute }}
        StartTimeOffset: "{{ StartTimeOffset }}"
        EndTimeOffset: "{{ EndTimeOffset }}"
        ExcludeFeaturesAttribute: "{{ ExcludeFeaturesAttribute }}"
      BatchTransformInput:
        DataCapturedDestinationS3Uri: "{{ DataCapturedDestinationS3Uri }}"
        DatasetFormat:
          Csv:
            Header: {{ Header }}
          Json:
            Line: {{ Line }}
          Parquet: "{{ Parquet }}"
        LocalPath: "{{ LocalPath }}"
        S3InputMode: "{{ S3InputMode }}"
        S3DataDistributionType: "{{ S3DataDistributionType }}"
        FeaturesAttribute: "{{ FeaturesAttribute }}"
        InferenceAttribute: "{{ InferenceAttribute }}"
        ProbabilityAttribute: "{{ ProbabilityAttribute }}"
        ProbabilityThresholdAttribute: {{ ProbabilityThresholdAttribute }}
        StartTimeOffset: "{{ StartTimeOffset }}"
        EndTimeOffset: "{{ EndTimeOffset }}"
        ExcludeFeaturesAttribute: "{{ ExcludeFeaturesAttribute }}"
  - name: DataQualityJobOutputConfig
    description: |
      The output configuration for monitoring jobs.
    value:
      MonitoringOutputs:
        - S3Output:
            S3Uri: "{{ S3Uri }}"
            LocalPath: "{{ LocalPath }}"
            S3UploadMode: "{{ S3UploadMode }}"
      KmsKeyId: "{{ KmsKeyId }}"
  - name: JobResources
    description: |
      Identifies the resources to deploy for a monitoring job.
    value:
      ClusterConfig:
        InstanceCount: {{ InstanceCount }}
        InstanceType: "{{ InstanceType }}"
        VolumeSizeInGB: {{ VolumeSizeInGB }}
        VolumeKmsKeyId: "{{ VolumeKmsKeyId }}"
  - name: NetworkConfig
    description: |
      Specifies networking configuration for the monitoring job.
    value:
      EnableInterContainerTrafficEncryption: {{ EnableInterContainerTrafficEncryption }}
      EnableNetworkIsolation: {{ EnableNetworkIsolation }}
      VpcConfig:
        SecurityGroupIds:
          - "{{ SecurityGroupIds }}"
        Subnets:
          - "{{ Subnets }}"
  - name: RoleArn
    value: "{{ RoleArn }}"
    description: |
      The Amazon Resource Name (ARN) of an IAM role that Amazon SageMaker AI can assume to perform tasks on your behalf.
  - name: StoppingCondition
    description: |
      A time limit for how long the monitoring job is allowed to run before stopping.
    value:
      MaxRuntimeInSeconds: {{ MaxRuntimeInSeconds }}
  - name: Tags
    description: |
      (Optional) An array of key-value pairs. For more information, see Using Cost Allocation Tags in the Amazon Web Services Billing and Cost Management User Guide.
    value:
      - Key: "{{ Key }}"
        Value: "{{ Value }}"

DELETE examples

delete_data_quality_job_definition

Deletes a data quality monitoring job definition.

DELETE FROM aws.sagemaker.data_quality_job_definitions
WHERE region = '{{ region }}' --required
;