data_quality_rulesets

Creates, updates, deletes, gets or lists a data_quality_rulesets resource.

Overview

Name	data_quality_rulesets
Type	Resource
Id	aws.glue.data_quality_rulesets

Fields

The following fields are returned by SELECT queries:

get_data_quality_ruleset

Name	Datatype	Description
created_on	string (date-time)	A timestamp. The time and date that this data quality ruleset was created.
data_quality_security_configuration	string	The name of the security configuration created with the data quality encryption option. (pattern: <code>[\u0020-\uD7FF\uE000-\uFFFD\uD800\uDC00-\uDBFF\uDFFF\t]*</code>)
description	string	A description of the ruleset. (pattern: <code>[\u0020-\uD7FF\uE000-\uFFFD\uD800\uDC00-\uDBFF\uDFFF\r\n\t]*</code>)
last_modified_on	string (date-time)	A timestamp. The last point in time when this data quality ruleset was modified.
name	string	The name of the ruleset. (pattern: <code>[\u0020-\uD7FF\uE000-\uFFFD\uD800\uDC00-\uDBFF\uDFFF\t]*</code>)
recommendation_run_id	string	When a ruleset was created from a recommendation run, this run ID is generated to link the two together. (pattern: <code>[\u0020-\uD7FF\uE000-\uFFFD\uD800\uDC00-\uDBFF\uDFFF\t]*</code>)
ruleset	string	A Data Quality Definition Language (DQDL) ruleset. For more information, see the Glue developer guide.
target_table	object	The name and database name of the target table.

Methods

The following methods are available for this resource:

Name	Accessible by	Required Params	Optional Params	Description
get_data_quality_ruleset	select	region		Returns an existing ruleset by identifier or name.
create_data_quality_ruleset	insert	region		Creates a data quality ruleset with DQDL rules applied to a specified Glue table. You create the ruleset using the Data Quality Definition Language (DQDL). For more information, see the Glue developer guide.
update_data_quality_ruleset	update	region		Updates the specified data quality ruleset.
delete_data_quality_ruleset	delete	region		Deletes a data quality ruleset.
cancel_data_quality_ruleset_evaluation_run	exec	region, RunId		Cancels a run where a ruleset is being evaluated against a data source.
list_data_quality_rulesets	exec	region		Returns a paginated list of rulesets for the specified list of Glue tables.
start_data_quality_ruleset_evaluation_run	exec	region, DataSource, Role, RulesetNames		Once you have a ruleset definition (either recommended or your own), you call this operation to evaluate the ruleset against a data source (Glue table). The evaluation computes results which you can retrieve with the GetDataQualityResult API.

Parameters

Parameters can be passed in the WHERE clause of a query. Check the Methods section to see which parameters are required or optional for each operation.

Name	Datatype	Description
region	string	AWS region (default: us-east-1)

SELECT examples

get_data_quality_ruleset

Returns an existing ruleset by identifier or name.

SELECT
created_on,
data_quality_security_configuration,
description,
last_modified_on,
name,
recommendation_run_id,
ruleset,
target_table
FROM aws.glue.data_quality_rulesets
WHERE region = '{{ region }}' -- required
;

INSERT examples

create_data_quality_ruleset

Creates a data quality ruleset with DQDL rules applied to a specified Glue table. You create the ruleset using the Data Quality Definition Language (DQDL). For more information, see the Glue developer guide.

INSERT INTO aws.glue.data_quality_rulesets (
Name,
Description,
Ruleset,
Tags,
TargetTable,
DataQualitySecurityConfiguration,
ClientToken,
region
)
SELECT 
'{{ Name }}',
'{{ Description }}',
'{{ Ruleset }}',
'{{ Tags }}',
'{{ TargetTable }}',
'{{ DataQualitySecurityConfiguration }}',
'{{ ClientToken }}',
'{{ region }}'
RETURNING
name
;

Manifest

# Description fields are for documentation purposes
- name: data_quality_rulesets
props:
  - name: region
    value: "{{ region }}"
    description: Required parameter for the data_quality_rulesets resource.
  - name: Name
    value: "{{ Name }}"
    description: |
      A unique name for the data quality ruleset.
  - name: Description
    value: "{{ Description }}"
    description: |
      A description of the data quality ruleset.
  - name: Ruleset
    value: "{{ Ruleset }}"
    description: |
      A Data Quality Definition Language (DQDL) ruleset. For more information, see the Glue developer guide.
  - name: Tags
    value: "{{ Tags }}"
    description: |
      A list of tags applied to the data quality ruleset.
  - name: TargetTable
    description: |
      A target table associated with the data quality ruleset.
    value:
      TableName: "{{ TableName }}"
      DatabaseName: "{{ DatabaseName }}"
      CatalogId: "{{ CatalogId }}"
  - name: DataQualitySecurityConfiguration
    value: "{{ DataQualitySecurityConfiguration }}"
    description: |
      The name of the security configuration created with the data quality encryption option.
  - name: ClientToken
    value: "{{ ClientToken }}"
    description: |
      Used for idempotency and is recommended to be set to a random ID (such as a UUID) to avoid creating or starting multiple instances of the same resource.

UPDATE examples

update_data_quality_ruleset

Updates the specified data quality ruleset.

UPDATE aws.glue.data_quality_rulesets
SET 
Name = '{{ Name }}',
Description = '{{ Description }}',
Ruleset = '{{ Ruleset }}'
WHERE 
region = '{{ region }}' --required
RETURNING
description,
name,
ruleset;

DELETE examples

delete_data_quality_ruleset

Deletes a data quality ruleset.

DELETE FROM aws.glue.data_quality_rulesets
WHERE region = '{{ region }}' --required
;

Lifecycle Methods

cancel_data_quality_ruleset_evaluation_run

Cancels a run where a ruleset is being evaluated against a data source.

EXEC aws.glue.data_quality_rulesets.cancel_data_quality_ruleset_evaluation_run 
@region='{{ region }}' --required 
@@json=
'{
"RunId": "{{ RunId }}"
}'
;

list_data_quality_rulesets

Returns a paginated list of rulesets for the specified list of Glue tables.

EXEC aws.glue.data_quality_rulesets.list_data_quality_rulesets 
@region='{{ region }}' --required 
@@json=
'{
"NextToken": "{{ NextToken }}", 
"MaxResults": {{ MaxResults }}, 
"Filter": "{{ Filter }}", 
"Tags": "{{ Tags }}"
}'
;

start_data_quality_ruleset_evaluation_run

Once you have a ruleset definition (either recommended or your own), you call this operation to evaluate the ruleset against a data source (Glue table). The evaluation computes results which you can retrieve with the GetDataQualityResult API.

EXEC aws.glue.data_quality_rulesets.start_data_quality_ruleset_evaluation_run 
@region='{{ region }}' --required 
@@json=
'{
"DataSource": "{{ DataSource }}", 
"Role": "{{ Role }}", 
"NumberOfWorkers": {{ NumberOfWorkers }}, 
"Timeout": {{ Timeout }}, 
"ClientToken": "{{ ClientToken }}", 
"AdditionalRunOptions": "{{ AdditionalRunOptions }}", 
"RulesetNames": "{{ RulesetNames }}", 
"AdditionalDataSources": "{{ AdditionalDataSources }}"
}'
;