flows

Creates, updates, deletes, gets or lists a flows resource.

Overview

Name	flows
Type	Resource
Id	aws.appflow.flows

Fields

The following fields are returned by SELECT queries:

describe_flow

Name	Datatype	Description
created_at	string (date-time)	Specifies when the flow was created.
created_by	string	The ARN of the user who created the flow. (pattern: <code>\S+</code>)
description	string	A description of the flow. (pattern: <code>[\w!@#-.?,\s]*</code>)
destination_flow_config_list	array	The configuration that controls how Amazon AppFlow transfers data to the destination connector.
flow_arn	string	The flow's Amazon Resource Name (ARN). (pattern: <code>arn:aws:appflow:.:[0-9]+:.</code>)
flow_name	string	The specified name of the flow. Spaces are not allowed. Use underscores (_) or hyphens (-) only. (pattern: <code>[a-zA-Z0-9][\w!@#.-]+</code>)
flow_status	string	Indicates the current status of the flow. (Active, Deprecated, Deleted, Draft, Errored, Suspended)
flow_status_message	string	Contains an error message if the flow status is in a suspended or error state. This applies only to scheduled or event-triggered flows. (pattern: <code>[\s\w/!@#+=.-]*</code>)
kms_arn	string	The ARN (Amazon Resource Name) of the Key Management Service (KMS) key you provide for encryption. This is required if you do not want to use the Amazon AppFlow-managed KMS key. If you don't provide anything here, Amazon AppFlow uses the Amazon AppFlow-managed KMS key. (pattern: <code>arn:aws:kms:.:[0-9]+:.</code>)
last_run_execution_details	object	Describes the details of the most recent flow run.
last_run_metadata_catalog_details	array	Describes the metadata catalog, metadata table, and data partitions that Amazon AppFlow used for the associated flow run.
last_updated_at	string (date-time)	Specifies when the flow was last updated.
last_updated_by	string	Specifies the user name of the account that performed the most recent update. (pattern: <code>\S+</code>)
metadata_catalog_config	object	Specifies the configuration that Amazon AppFlow uses when it catalogs your data. When Amazon AppFlow catalogs your data, it stores metadata in a data catalog.
schema_version	integer (int64)	The version number of your data schema. Amazon AppFlow assigns this version number. The version number increases by one when you change any of the following settings in your flow configuration: Source-to-destination field mappings Field data types Partition keys
source_flow_config	object	Contains information about the configuration of the source connector used in the flow.
tags	object	The tags used to organize, track, or control access for your flow.
tasks	array	A list of tasks that Amazon AppFlow performs while transferring the data in the flow run.
trigger_config	object	The trigger settings that determine how and when Amazon AppFlow runs the specified flow.

list_flows

Name	Datatype	Description
flows	array	The list of flows associated with your account.
next_token	string	The pagination token for next page of data. (pattern: <code>\S+</code>)

Methods

The following methods are available for this resource:

Name	Accessible by	Required Params	Optional Params	Description
describe_flow	select	region		Provides a description of the specified flow.
list_flows	select	region		Lists all of the flows associated with your account.
create_flow	insert	region, flowName, triggerConfig, sourceFlowConfig, destinationFlowConfigList, tasks		Enables your application to create a new flow using Amazon AppFlow. You must create a connector profile before calling this API. Please note that the Request Syntax below shows syntax for multiple destinations, however, you can only transfer data to one item in this list at a time. Amazon AppFlow does not currently support flows to multiple destinations at once.
update_flow	update	region, flowName, triggerConfig, sourceFlowConfig, destinationFlowConfigList, tasks		Updates an existing flow.
delete_flow	delete	region		Enables your application to delete an existing flow. Before deleting the flow, Amazon AppFlow validates the request by checking the flow configuration and status. You can delete flows one at a time.
cancel_flow_executions	exec	region, flowName		Cancels active runs for a flow. You can cancel all of the active runs for a flow, or you can cancel specific runs by providing their IDs. You can cancel a flow run only when the run is in progress. You can't cancel a run that has already completed or failed. You also can't cancel a run that's scheduled to occur but hasn't started yet. To prevent a scheduled run, you can deactivate the flow with the StopFlow action. You cannot resume a run after you cancel it. When you send your request, the status for each run becomes CancelStarted. When the cancellation completes, the status becomes Canceled. When you cancel a run, you still incur charges for any data that the run already processed before the cancellation. If the run had already written some data to the flow destination, then that data remains in the destination. If you configured the flow to use a batch API (such as the Salesforce Bulk API 2.0), then the run will finish reading or writing its entire batch of data after the cancellation. For these operations, the data processing charges for Amazon AppFlow apply. For the pricing information, see Amazon AppFlow pricing.
start_flow	exec	region, flowName		Activates an existing flow. For on-demand flows, this operation runs the flow immediately. For schedule and event-triggered flows, this operation activates the flow.
stop_flow	exec	region, flowName		Deactivates the existing flow. For on-demand flows, this operation returns an unsupportedOperationException error message. For schedule and event-triggered flows, this operation deactivates the flow.

Parameters

Parameters can be passed in the WHERE clause of a query. Check the Methods section to see which parameters are required or optional for each operation.

Name	Datatype	Description
region	string	AWS region (default: us-east-1)

SELECT examples

describe_flow

Provides a description of the specified flow.

SELECT
created_at,
created_by,
description,
destination_flow_config_list,
flow_arn,
flow_name,
flow_status,
flow_status_message,
kms_arn,
last_run_execution_details,
last_run_metadata_catalog_details,
last_updated_at,
last_updated_by,
metadata_catalog_config,
schema_version,
source_flow_config,
tags,
tasks,
trigger_config
FROM aws.appflow.flows
WHERE region = '{{ region }}' -- required
;

list_flows

Lists all of the flows associated with your account.

SELECT
flows,
next_token
FROM aws.appflow.flows
WHERE region = '{{ region }}' -- required
;

INSERT examples

create_flow

Enables your application to create a new flow using Amazon AppFlow. You must create a connector profile before calling this API. Please note that the Request Syntax below shows syntax for multiple destinations, however, you can only transfer data to one item in this list at a time. Amazon AppFlow does not currently support flows to multiple destinations at once.

INSERT INTO aws.appflow.flows (
flowName,
description,
kmsArn,
triggerConfig,
sourceFlowConfig,
destinationFlowConfigList,
tasks,
tags,
metadataCatalogConfig,
clientToken,
region
)
SELECT 
'{{ flowName }}' /* required */,
'{{ description }}',
'{{ kmsArn }}',
'{{ triggerConfig }}' /* required */,
'{{ sourceFlowConfig }}' /* required */,
'{{ destinationFlowConfigList }}' /* required */,
'{{ tasks }}' /* required */,
'{{ tags }}',
'{{ metadataCatalogConfig }}',
'{{ clientToken }}',
'{{ region }}'
RETURNING
flow_arn,
flow_status
;

Manifest

# Description fields are for documentation purposes
- name: flows
props:
  - name: region
    value: "{{ region }}"
    description: Required parameter for the flows resource.
  - name: flowName
    value: "{{ flowName }}"
  - name: description
    value: "{{ description }}"
  - name: kmsArn
    value: "{{ kmsArn }}"
  - name: triggerConfig
    description: |
      The trigger settings that determine how and when Amazon AppFlow runs the specified flow.
    value:
      triggerType: "{{ triggerType }}"
      triggerProperties:
        Scheduled:
          scheduleExpression: "{{ scheduleExpression }}"
          dataPullMode: "{{ dataPullMode }}"
          scheduleStartTime: "{{ scheduleStartTime }}"
          scheduleEndTime: "{{ scheduleEndTime }}"
          timezone: "{{ timezone }}"
          scheduleOffset: {{ scheduleOffset }}
          firstExecutionFrom: "{{ firstExecutionFrom }}"
          flowErrorDeactivationThreshold: {{ flowErrorDeactivationThreshold }}
  - name: sourceFlowConfig
    description: |
      Contains information about the configuration of the source connector used in the flow.
    value:
      connectorType: "{{ connectorType }}"
      apiVersion: "{{ apiVersion }}"
      connectorProfileName: "{{ connectorProfileName }}"
      sourceConnectorProperties:
        Amplitude:
          object: "{{ object }}"
        Datadog:
          object: "{{ object }}"
        Dynatrace:
          object: "{{ object }}"
        GoogleAnalytics:
          object: "{{ object }}"
        InforNexus:
          object: "{{ object }}"
        Marketo:
          object: "{{ object }}"
        S3:
          bucketName: "{{ bucketName }}"
          bucketPrefix: "{{ bucketPrefix }}"
          s3InputFormatConfig:
            s3InputFileType: "{{ s3InputFileType }}"
        Salesforce:
          object: "{{ object }}"
          enableDynamicFieldUpdate: {{ enableDynamicFieldUpdate }}
          includeDeletedRecords: {{ includeDeletedRecords }}
          dataTransferApi: "{{ dataTransferApi }}"
        ServiceNow:
          object: "{{ object }}"
        Singular:
          object: "{{ object }}"
        Slack:
          object: "{{ object }}"
        Trendmicro:
          object: "{{ object }}"
        Veeva:
          object: "{{ object }}"
          documentType: "{{ documentType }}"
          includeSourceFiles: {{ includeSourceFiles }}
          includeRenditions: {{ includeRenditions }}
          includeAllVersions: {{ includeAllVersions }}
        Zendesk:
          object: "{{ object }}"
        SAPOData:
          objectPath: "{{ objectPath }}"
          parallelismConfig:
            maxParallelism: {{ maxParallelism }}
          paginationConfig:
            maxPageSize: {{ maxPageSize }}
        CustomConnector:
          entityName: "{{ entityName }}"
          customProperties: "{{ customProperties }}"
          dataTransferApi:
            Name: "{{ Name }}"
            Type: "{{ Type }}"
        Pardot:
          object: "{{ object }}"
      incrementalPullConfig:
        datetimeTypeFieldName: "{{ datetimeTypeFieldName }}"
  - name: destinationFlowConfigList
    value:
      - connectorType: "{{ connectorType }}"
        apiVersion: "{{ apiVersion }}"
        connectorProfileName: "{{ connectorProfileName }}"
        destinationConnectorProperties:
          Redshift:
            object: "{{ object }}"
            intermediateBucketName: "{{ intermediateBucketName }}"
            bucketPrefix: "{{ bucketPrefix }}"
            errorHandlingConfig:
              failOnFirstDestinationError: {{ failOnFirstDestinationError }}
              bucketPrefix: "{{ bucketPrefix }}"
              bucketName: "{{ bucketName }}"
          S3:
            bucketName: "{{ bucketName }}"
            bucketPrefix: "{{ bucketPrefix }}"
            s3OutputFormatConfig:
              fileType: "{{ fileType }}"
              prefixConfig:
                prefixType: "{{ prefixType }}"
                prefixFormat: "{{ prefixFormat }}"
                pathPrefixHierarchy: "{{ pathPrefixHierarchy }}"
              aggregationConfig:
                aggregationType: "{{ aggregationType }}"
                targetFileSize: {{ targetFileSize }}
              preserveSourceDataTyping: {{ preserveSourceDataTyping }}
          Salesforce:
            object: "{{ object }}"
            idFieldNames:
              - "{{ idFieldNames }}"
            errorHandlingConfig:
              failOnFirstDestinationError: {{ failOnFirstDestinationError }}
              bucketPrefix: "{{ bucketPrefix }}"
              bucketName: "{{ bucketName }}"
            writeOperationType: "{{ writeOperationType }}"
            dataTransferApi: "{{ dataTransferApi }}"
          Snowflake:
            object: "{{ object }}"
            intermediateBucketName: "{{ intermediateBucketName }}"
            bucketPrefix: "{{ bucketPrefix }}"
            errorHandlingConfig:
              failOnFirstDestinationError: {{ failOnFirstDestinationError }}
              bucketPrefix: "{{ bucketPrefix }}"
              bucketName: "{{ bucketName }}"
          EventBridge:
            object: "{{ object }}"
            errorHandlingConfig:
              failOnFirstDestinationError: {{ failOnFirstDestinationError }}
              bucketPrefix: "{{ bucketPrefix }}"
              bucketName: "{{ bucketName }}"
          LookoutMetrics: "{{ LookoutMetrics }}"
          Upsolver:
            bucketName: "{{ bucketName }}"
            bucketPrefix: "{{ bucketPrefix }}"
            s3OutputFormatConfig:
              fileType: "{{ fileType }}"
              prefixConfig:
                prefixType: "{{ prefixType }}"
                prefixFormat: "{{ prefixFormat }}"
                pathPrefixHierarchy: "{{ pathPrefixHierarchy }}"
              aggregationConfig:
                aggregationType: "{{ aggregationType }}"
                targetFileSize: {{ targetFileSize }}
          Honeycode:
            object: "{{ object }}"
            errorHandlingConfig:
              failOnFirstDestinationError: {{ failOnFirstDestinationError }}
              bucketPrefix: "{{ bucketPrefix }}"
              bucketName: "{{ bucketName }}"
          CustomerProfiles:
            domainName: "{{ domainName }}"
            objectTypeName: "{{ objectTypeName }}"
          Zendesk:
            object: "{{ object }}"
            idFieldNames:
              - "{{ idFieldNames }}"
            errorHandlingConfig:
              failOnFirstDestinationError: {{ failOnFirstDestinationError }}
              bucketPrefix: "{{ bucketPrefix }}"
              bucketName: "{{ bucketName }}"
            writeOperationType: "{{ writeOperationType }}"
          Marketo:
            object: "{{ object }}"
            errorHandlingConfig:
              failOnFirstDestinationError: {{ failOnFirstDestinationError }}
              bucketPrefix: "{{ bucketPrefix }}"
              bucketName: "{{ bucketName }}"
          CustomConnector:
            entityName: "{{ entityName }}"
            errorHandlingConfig:
              failOnFirstDestinationError: {{ failOnFirstDestinationError }}
              bucketPrefix: "{{ bucketPrefix }}"
              bucketName: "{{ bucketName }}"
            writeOperationType: "{{ writeOperationType }}"
            idFieldNames:
              - "{{ idFieldNames }}"
            customProperties: "{{ customProperties }}"
          SAPOData:
            objectPath: "{{ objectPath }}"
            successResponseHandlingConfig:
              bucketPrefix: "{{ bucketPrefix }}"
              bucketName: "{{ bucketName }}"
            idFieldNames:
              - "{{ idFieldNames }}"
            errorHandlingConfig:
              failOnFirstDestinationError: {{ failOnFirstDestinationError }}
              bucketPrefix: "{{ bucketPrefix }}"
              bucketName: "{{ bucketName }}"
            writeOperationType: "{{ writeOperationType }}"
  - name: tasks
    value:
      - sourceFields: "{{ sourceFields }}"
        connectorOperator:
          Amplitude: "{{ Amplitude }}"
          Datadog: "{{ Datadog }}"
          Dynatrace: "{{ Dynatrace }}"
          GoogleAnalytics: "{{ GoogleAnalytics }}"
          InforNexus: "{{ InforNexus }}"
          Marketo: "{{ Marketo }}"
          S3: "{{ S3 }}"
          Salesforce: "{{ Salesforce }}"
          ServiceNow: "{{ ServiceNow }}"
          Singular: "{{ Singular }}"
          Slack: "{{ Slack }}"
          Trendmicro: "{{ Trendmicro }}"
          Veeva: "{{ Veeva }}"
          Zendesk: "{{ Zendesk }}"
          SAPOData: "{{ SAPOData }}"
          CustomConnector: "{{ CustomConnector }}"
          Pardot: "{{ Pardot }}"
        destinationField: "{{ destinationField }}"
        taskType: "{{ taskType }}"
        taskProperties: "{{ taskProperties }}"
  - name: tags
    value: "{{ tags }}"
  - name: metadataCatalogConfig
    description: |
      Specifies the configuration that Amazon AppFlow uses when it catalogs your data. When Amazon AppFlow catalogs your data, it stores metadata in a data catalog.
    value:
      glueDataCatalog:
        roleArn: "{{ roleArn }}"
        databaseName: "{{ databaseName }}"
        tablePrefix: "{{ tablePrefix }}"
  - name: clientToken
    value: "{{ clientToken }}"

UPDATE examples

update_flow

Updates an existing flow.

UPDATE aws.appflow.flows
SET 
flowName = '{{ flowName }}',
description = '{{ description }}',
triggerConfig = '{{ triggerConfig }}',
sourceFlowConfig = '{{ sourceFlowConfig }}',
destinationFlowConfigList = '{{ destinationFlowConfigList }}',
tasks = '{{ tasks }}',
metadataCatalogConfig = '{{ metadataCatalogConfig }}',
clientToken = '{{ clientToken }}'
WHERE 
region = '{{ region }}' --required
AND flowName = '{{ flowName }}' --required
AND triggerConfig = '{{ triggerConfig }}' --required
AND sourceFlowConfig = '{{ sourceFlowConfig }}' --required
AND destinationFlowConfigList = '{{ destinationFlowConfigList }}' --required
AND tasks = '{{ tasks }}' --required
RETURNING
flow_status;

DELETE examples

delete_flow

Enables your application to delete an existing flow. Before deleting the flow, Amazon AppFlow validates the request by checking the flow configuration and status. You can delete flows one at a time.

DELETE FROM aws.appflow.flows
WHERE region = '{{ region }}' --required
;

Lifecycle Methods

cancel_flow_executions

Cancels active runs for a flow. You can cancel all of the active runs for a flow, or you can cancel specific runs by providing their IDs. You can cancel a flow run only when the run is in progress. You can't cancel a run that has already completed or failed. You also can't cancel a run that's scheduled to occur but hasn't started yet. To prevent a scheduled run, you can deactivate the flow with the StopFlow action. You cannot resume a run after you cancel it. When you send your request, the status for each run becomes CancelStarted. When the cancellation completes, the status becomes Canceled. When you cancel a run, you still incur charges for any data that the run already processed before the cancellation. If the run had already written some data to the flow destination, then that data remains in the destination. If you configured the flow to use a batch API (such as the Salesforce Bulk API 2.0), then the run will finish reading or writing its entire batch of data after the cancellation. For these operations, the data processing charges for Amazon AppFlow apply. For the pricing information, see Amazon AppFlow pricing.

EXEC aws.appflow.flows.cancel_flow_executions 
@region='{{ region }}' --required 
@@json=
'{
"flowName": "{{ flowName }}", 
"executionIds": "{{ executionIds }}"
}'
;

start_flow

Activates an existing flow. For on-demand flows, this operation runs the flow immediately. For schedule and event-triggered flows, this operation activates the flow.

EXEC aws.appflow.flows.start_flow 
@region='{{ region }}' --required 
@@json=
'{
"flowName": "{{ flowName }}", 
"clientToken": "{{ clientToken }}"
}'
;

stop_flow

Deactivates the existing flow. For on-demand flows, this operation returns an unsupportedOperationException error message. For schedule and event-triggered flows, this operation deactivates the flow.

EXEC aws.appflow.flows.stop_flow 
@region='{{ region }}' --required 
@@json=
'{
"flowName": "{{ flowName }}"
}'
;