data_sources

Creates, updates, deletes, gets or lists a data_sources resource.

Overview

Name	`data_sources`
Type	Resource
Id	`aws.datazone.data_sources`

Fields

The following fields are returned by SELECT queries:

list_data_sources
get_data_source

Name	Datatype	Description
`name`	`string`	The name of the data source.
`connection_id`	`string`	The connection ID that's part of the data source summary.
`created_at`	`string (date-time)`	The timestamp of when the data source was created.
`data_source_id`	`string`	The ID of the data source. (pattern: <code>[a-zA-Z0-9_-]{1,36}</code>)
`description`	`string`	The data source description.
`domain_id`	`string`	The ID of the Amazon DataZone domain in which the data source exists. (pattern: <code>dzd[-][a-zA-Z0-9-]{1,36}</code>)
`enable_setting`	`string`	Specifies whether the data source is enabled. (ENABLED, DISABLED)
`environment_id`	`string`	The ID of the environment in which the data source exists.
`last_run_asset_count`	`integer`	The count of the assets created during the last data source run.
`last_run_at`	`string (date-time)`	The timestamp of when the data source run was last performed.
`last_run_error_message`	`object`	The details of the error message that is returned if the operation cannot be successfully completed.
`last_run_status`	`string`	The status of the last data source run. (REQUESTED, RUNNING, FAILED, PARTIALLY_SUCCEEDED, SUCCESS)
`schedule`	`object`	The details of the schedule of the data source runs.
`status`	`string`	The status of the data source. (CREATING, FAILED_CREATION, READY, UPDATING, FAILED_UPDATE, RUNNING, DELETING, FAILED_DELETION)
`type_`	`string`	The type of the data source.
`updated_at`	`string (date-time)`	The timestamp of when the data source was updated.

Name	Datatype	Description
`id`	`string`	The ID of the data source. (pattern: <code>[a-zA-Z0-9_-]{1,36}</code>)
`name`	`string`	The name of the data source.
`asset_forms_output`	`array`	The metadata forms attached to the assets created by this data source.
`configuration`	`object`	The configuration of the data source.
`connection_id`	`string`	The ID of the connection.
`created_at`	`string (date-time)`	The timestamp of when the data source was created.
`description`	`string`	The description of the data source.
`domain_id`	`string`	The ID of the Amazon DataZone domain in which the data source exists. (pattern: <code>dzd[-][a-zA-Z0-9-]{1,36}</code>)
`enable_setting`	`string`	Specifies whether this data source is enabled or not. (ENABLED, DISABLED)
`environment_id`	`string`	The ID of the environment where this data source creates and publishes assets, (pattern: <code>[a-zA-Z0-9_-]{1,36}</code>)
`error_message`	`object`	The details of the error message that is returned if the operation cannot be successfully completed.
`last_run_asset_count`	`integer`	The number of assets created by the data source during its last run.
`last_run_at`	`string (date-time)`	The timestamp of the last run of the data source.
`last_run_error_message`	`object`	The details of the error message that is returned if the operation cannot be successfully completed.
`last_run_status`	`string`	The status of the last run of the data source. (REQUESTED, RUNNING, FAILED, PARTIALLY_SUCCEEDED, SUCCESS)
`project_id`	`string`	The ID of the project where the data source creates and publishes assets. (pattern: <code>[a-zA-Z0-9_-]{1,36}</code>)
`publish_on_import`	`boolean`	Specifies whether the assets that this data source creates in the inventory are to be also automatically published to the catalog.
`recommendation`	`object`	The recommendation to be updated as part of the UpdateDataSource action.
`schedule`	`object`	The details of the schedule of the data source runs.
`self_grant_status`	`object`	Specifies the status of the self-granting functionality.
`status`	`string`	The status of the data source. (CREATING, FAILED_CREATION, READY, UPDATING, FAILED_UPDATE, RUNNING, DELETING, FAILED_DELETION)
`type_`	`string`	The type of the data source.
`updated_at`	`string (date-time)`	The timestamp of when the data source was updated.

Methods

The following methods are available for this resource:

Name	Accessible by	Required Params	Optional Params	Description
`list_data_sources`	`select`	`domain_identifier`, `projectIdentifier`, `region`	`environmentIdentifier`, `connectionIdentifier`, `type`, `status`, `name`, `nextToken`, `maxResults`	Lists data sources in Amazon DataZone.
`get_data_source`	`select`	`domain_identifier`, `identifier`, `region`		Gets an Amazon DataZone data source.
`create_data_source`	`insert`	`domain_identifier`, `region`, `name`, `projectIdentifier`, `type`		Creates an Amazon DataZone data source.
`update_data_source`	`update`	`domain_identifier`, `identifier`, `region`		Updates the specified data source in Amazon DataZone.
`delete_data_source`	`delete`	`domain_identifier`, `identifier`, `region`	`clientToken`, `retainPermissionsOnRevokeFailure`	Deletes a data source in Amazon DataZone.
`start_data_source_run`	`exec`	`domain_identifier`, `data_source_identifier`, `region`		Start the run of the specified data source in Amazon DataZone.

Parameters

Parameters can be passed in the WHERE clause of a query. Check the Methods section to see which parameters are required or optional for each operation.

Name	Datatype	Description
`data_source_identifier`	`string`	The identifier of the data source.
`domain_identifier`	`string`	The identifier of the Amazon DataZone domain in which to start a data source run.
`identifier`	`string`	The identifier of the data source that is deleted.
`projectIdentifier`	`string`	The identifier of the project in which to list data sources.
`region`	`string`	AWS region (default: us-east-1)
`clientToken`	`string`	A unique, case-sensitive identifier that is provided to ensure the idempotency of the request.
`connectionIdentifier`	`string`	The ID of the connection.
`environmentIdentifier`	`string`	The identifier of the environment in which to list the data sources.
`maxResults`	`integer`	The maximum number of data sources to return in a single call to ListDataSources. When the number of data sources to be listed is greater than the value of MaxResults, the response contains a NextToken value that you can use in a subsequent call to ListDataSources to list the next set of data sources.
`name`	`string`	The name of the data source.
`nextToken`	`string`	When the number of data sources is greater than the default value for the MaxResults parameter, or if you explicitly specify a value for MaxResults that is less than the number of data sources, the response includes a pagination token named NextToken. You can specify this NextToken value in a subsequent call to ListDataSources to list the next set of data sources.
`retainPermissionsOnRevokeFailure`	`boolean`	Specifies that the granted permissions are retained in case of a self-subscribe functionality failure for a data source.
`status`	`string`	The status of the data source.
`type`	`string`	The type of the data source.

`SELECT` examples

list_data_sources
get_data_source

Lists data sources in Amazon DataZone.

SELECT
name,
connection_id,
created_at,
data_source_id,
description,
domain_id,
enable_setting,
environment_id,
last_run_asset_count,
last_run_at,
last_run_error_message,
last_run_status,
schedule,
status,
type_,
updated_at
FROM aws.datazone.data_sources
WHERE domain_identifier = '{{ domain_identifier }}' -- required
AND projectIdentifier = '{{ projectIdentifier }}' -- required
AND region = '{{ region }}' -- required
AND environmentIdentifier = '{{ environmentIdentifier }}'
AND connectionIdentifier = '{{ connectionIdentifier }}'
AND type = '{{ type }}'
AND status = '{{ status }}'
AND name = '{{ name }}'
AND nextToken = '{{ nextToken }}'
AND maxResults = '{{ maxResults }}'
;

Gets an Amazon DataZone data source.

SELECT
id,
name,
asset_forms_output,
configuration,
connection_id,
created_at,
description,
domain_id,
enable_setting,
environment_id,
error_message,
last_run_asset_count,
last_run_at,
last_run_error_message,
last_run_status,
project_id,
publish_on_import,
recommendation,
schedule,
self_grant_status,
status,
type_,
updated_at
FROM aws.datazone.data_sources
WHERE domain_identifier = '{{ domain_identifier }}' -- required
AND identifier = '{{ identifier }}' -- required
AND region = '{{ region }}' -- required
;

`INSERT` examples

create_data_source
Manifest

Creates an Amazon DataZone data source.

INSERT INTO aws.datazone.data_sources (
name,
description,
projectIdentifier,
environmentIdentifier,
connectionIdentifier,
type,
configuration,
recommendation,
enableSetting,
schedule,
publishOnImport,
assetFormsInput,
clientToken,
domain_identifier,
region
)
SELECT 
'{{ name }}' /* required */,
'{{ description }}',
'{{ projectIdentifier }}' /* required */,
'{{ environmentIdentifier }}',
'{{ connectionIdentifier }}',
'{{ type }}' /* required */,
'{{ configuration }}',
'{{ recommendation }}',
'{{ enableSetting }}',
'{{ schedule }}',
{{ publishOnImport }},
'{{ assetFormsInput }}',
'{{ clientToken }}',
'{{ domain_identifier }}',
'{{ region }}'
RETURNING
id,
name,
asset_forms_output,
configuration,
connection_id,
created_at,
description,
domain_id,
enable_setting,
environment_id,
error_message,
last_run_at,
last_run_error_message,
last_run_status,
project_id,
publish_on_import,
recommendation,
schedule,
status,
type_,
updated_at
;

# Description fields are for documentation purposes
- name: data_sources
props:
  - name: domain_identifier
    value: "{{ domain_identifier }}"
    description: Required parameter for the data_sources resource.
  - name: region
    value: "{{ region }}"
    description: Required parameter for the data_sources resource.
  - name: name
    value: "{{ name }}"
  - name: description
    value: "{{ description }}"
  - name: projectIdentifier
    value: "{{ projectIdentifier }}"
  - name: environmentIdentifier
    value: "{{ environmentIdentifier }}"
  - name: connectionIdentifier
    value: "{{ connectionIdentifier }}"
  - name: type
    value: "{{ type }}"
  - name: configuration
    description: |
      The configuration of the data source.
    value:
      glueRunConfiguration:
        dataAccessRole: "{{ dataAccessRole }}"
        relationalFilterConfigurations:
          - databaseName: "{{ databaseName }}"
            schemaName: "{{ schemaName }}"
            filterExpressions: "{{ filterExpressions }}"
        autoImportDataQualityResult: {{ autoImportDataQualityResult }}
        catalogName: "{{ catalogName }}"
      redshiftRunConfiguration:
        dataAccessRole: "{{ dataAccessRole }}"
        relationalFilterConfigurations:
          - databaseName: "{{ databaseName }}"
            schemaName: "{{ schemaName }}"
            filterExpressions: "{{ filterExpressions }}"
        redshiftCredentialConfiguration:
          secretManagerArn: "{{ secretManagerArn }}"
        redshiftStorage:
          redshiftClusterSource:
            clusterName: "{{ clusterName }}"
          redshiftServerlessSource:
            workgroupName: "{{ workgroupName }}"
      sageMakerRunConfiguration:
        trackingAssets: "{{ trackingAssets }}"
  - name: recommendation
    description: |
      The recommendation to be updated as part of the UpdateDataSource action.
    value:
      enableBusinessNameGeneration: {{ enableBusinessNameGeneration }}
  - name: enableSetting
    value: "{{ enableSetting }}"
    valid_values: ['ENABLED', 'DISABLED']
  - name: schedule
    description: |
      The details of the schedule of the data source runs.
    value:
      timezone: "{{ timezone }}"
      schedule: "{{ schedule }}"
  - name: publishOnImport
    value: {{ publishOnImport }}
  - name: assetFormsInput
    value:
      - formName: "{{ formName }}"
        typeIdentifier: "{{ typeIdentifier }}"
        typeRevision: "{{ typeRevision }}"
        content: "{{ content }}"
  - name: clientToken
    value: "{{ clientToken }}"

`UPDATE` examples

update_data_source

Updates the specified data source in Amazon DataZone.

UPDATE aws.datazone.data_sources
SET 
name = '{{ name }}',
description = '{{ description }}',
enableSetting = '{{ enableSetting }}',
publishOnImport = {{ publishOnImport }},
assetFormsInput = '{{ assetFormsInput }}',
schedule = '{{ schedule }}',
configuration = '{{ configuration }}',
recommendation = '{{ recommendation }}',
retainPermissionsOnRevokeFailure = {{ retainPermissionsOnRevokeFailure }}
WHERE 
domain_identifier = '{{ domain_identifier }}' --required
AND identifier = '{{ identifier }}' --required
AND region = '{{ region }}' --required
RETURNING
id,
name,
asset_forms_output,
configuration,
connection_id,
created_at,
description,
domain_id,
enable_setting,
environment_id,
error_message,
last_run_at,
last_run_error_message,
last_run_status,
project_id,
publish_on_import,
recommendation,
retain_permissions_on_revoke_failure,
schedule,
self_grant_status,
status,
type_,
updated_at;

`DELETE` examples

delete_data_source

Deletes a data source in Amazon DataZone.

DELETE FROM aws.datazone.data_sources
WHERE domain_identifier = '{{ domain_identifier }}' --required
AND identifier = '{{ identifier }}' --required
AND region = '{{ region }}' --required
AND clientToken = '{{ clientToken }}'
AND retainPermissionsOnRevokeFailure = '{{ retainPermissionsOnRevokeFailure }}'
;

Lifecycle Methods

start_data_source_run

Start the run of the specified data source in Amazon DataZone.

EXEC aws.datazone.data_sources.start_data_source_run 
@domain_identifier='{{ domain_identifier }}' --required, 
@data_source_identifier='{{ data_source_identifier }}' --required, 
@region='{{ region }}' --required 
@@json=
'{
"clientToken": "{{ clientToken }}"
}'
;

Overview​

Fields​

Methods​

Parameters​

SELECT examples​

INSERT examples​

UPDATE examples​

DELETE examples​

Lifecycle Methods​