tasks

Creates, updates, deletes, gets or lists a tasks resource.

Overview

Name	tasks
Type	Resource
Id	aws.datasync.tasks

Fields

The following fields are returned by SELECT queries:

describe_task

Name	Datatype	Description
cloud_watch_log_group_arn	string	The Amazon Resource Name (ARN) of an Amazon CloudWatch log group for monitoring your task. For more information, see Monitoring data transfers with CloudWatch Logs. (pattern: <code>^arn:(aws|aws-cn|aws-us-gov|aws-eusc|aws-iso|aws-iso-b):logs:[a-z-0-9]+:[0-9]{12}:log-group:([^:*]*)(:*)?$</code>)
creation_time	string (date-time)	The time that the task was created.
current_task_execution_arn	string	The ARN of the most recent task execution. (pattern: <code>^arn:(aws|aws-cn|aws-us-gov|aws-eusc|aws-iso|aws-iso-b):datasync:[a-z-0-9]+:[0-9]{12}:task/task-[0-9a-f]{17}/execution/exec-[0-9a-f]{17}$</code>)
destination_location_arn	string	The ARN of your transfer's destination location. (pattern: <code>^arn:(aws|aws-cn|aws-us-gov|aws-eusc|aws-iso|aws-iso-b):datasync:[a-z-0-9]+:[0-9]{12}:location/loc-[0-9a-z]{17}$</code>)
destination_network_interface_arns	array	The ARNs of the network interfaces that DataSync created for your destination location.
error_code	string	If there's an issue with your task, you can use the error code to help you troubleshoot the problem. For more information, see Troubleshooting issues with DataSync transfers.
error_detail	string	If there's an issue with your task, you can use the error details to help you troubleshoot the problem. For more information, see Troubleshooting issues with DataSync transfers.
excludes	array	The exclude filters that define the files, objects, and folders in your source location that you don't want DataSync to transfer. For more information and examples, see Specifying what DataSync transfers by using filters.
includes	array	The include filters that define the files, objects, and folders in your source location that you want DataSync to transfer. For more information and examples, see Specifying what DataSync transfers by using filters.
manifest_config	object	The configuration of the manifest that lists the files or objects that you want DataSync to transfer. For more information, see Specifying what DataSync transfers by using a manifest.
name	string	The name of your task. (pattern: <code>^[a-zA-Z0-9\s+=._:@/-]+$</code>)
options	object	The task's settings. For example, what file metadata gets preserved, how data integrity gets verified at the end of your transfer, bandwidth limits, among other options.
schedule	object	The schedule for when you want your task to run. For more information, see Scheduling your task.
schedule_details	object	The details about your task schedule.
source_location_arn	string	The ARN of your transfer's source location. (pattern: <code>^arn:(aws|aws-cn|aws-us-gov|aws-eusc|aws-iso|aws-iso-b):datasync:[a-z-0-9]+:[0-9]{12}:location/loc-[0-9a-z]{17}$</code>)
source_network_interface_arns	array	The ARNs of the network interfaces that DataSync created for your source location.
status	string	The status of your task. For information about what each status means, see Task statuses. (AVAILABLE, CREATING, QUEUED, RUNNING, UNAVAILABLE)
task_arn	string	The ARN of your task. (pattern: <code>^arn:(aws|aws-cn|aws-us-gov|aws-eusc|aws-iso|aws-iso-b):datasync:[a-z-0-9]+:[0-9]{12}:task/task-[0-9a-f]{17}$</code>)
task_mode	string	The task mode that you're using. For more information, see Choosing a task mode for your data transfer. (BASIC, ENHANCED)
task_report_config	object	The configuration of your task report, which provides detailed information about your DataSync transfer. For more information, see Monitoring your DataSync transfers with task reports.

list_tasks

Name	Datatype	Description
name	string	The name of the task. (pattern: <code>^[a-zA-Z0-9\s+=._:@/-]+$</code>)
status	string	The status of the task. (AVAILABLE, CREATING, QUEUED, RUNNING, UNAVAILABLE)
task_arn	string	The Amazon Resource Name (ARN) of the task. (pattern: <code>^arn:(aws|aws-cn|aws-us-gov|aws-eusc|aws-iso|aws-iso-b):datasync:[a-z-0-9]+:[0-9]{12}:task/task-[0-9a-f]{17}$</code>)
task_mode	string	The task mode that you're using. For more information, see Choosing a task mode for your data transfer. (BASIC, ENHANCED)

Methods

The following methods are available for this resource:

Name	Accessible by	Required Params	Optional Params	Description
describe_task	select	region		Provides information about a task, which defines where and how DataSync transfers your data.
list_tasks	select	region		Returns a list of the DataSync tasks you created.
create_task	insert	region, SourceLocationArn, DestinationLocationArn		Configures a task, which defines where and how DataSync transfers your data. A task includes a source location, destination location, and transfer options (such as bandwidth limits, scheduling, and more). If you're planning to transfer data to or from an Amazon S3 location, review how DataSync can affect your S3 request charges and the DataSync pricing page before you begin.
update_task	update	region, TaskArn		Updates the configuration of a task, which defines where and how DataSync transfers your data.
delete_task	delete	region		Deletes a transfer task resource from DataSync.
cancel_task_execution	exec	region, TaskExecutionArn		Stops an DataSync task execution that's in progress. The transfer of some files are abruptly interrupted. File contents that're transferred to the destination might be incomplete or inconsistent with the source files. However, if you start a new task execution using the same task and allow it to finish, file content on the destination will be complete and consistent. This applies to other unexpected failures that interrupt a task execution. In all of these cases, DataSync successfully completes the transfer when you start the next task execution.
start_task_execution	exec	region, TaskArn		Starts an DataSync transfer task. For each task, you can only run one task execution at a time. There are several steps to a task execution. For more information, see Task execution statuses. If you're planning to transfer data to or from an Amazon S3 location, review how DataSync can affect your S3 request charges and the DataSync pricing page before you begin.

Parameters

Parameters can be passed in the WHERE clause of a query. Check the Methods section to see which parameters are required or optional for each operation.

Name	Datatype	Description
region	string	AWS region (default: us-east-1)

SELECT examples

describe_task

Provides information about a task, which defines where and how DataSync transfers your data.

SELECT
cloud_watch_log_group_arn,
creation_time,
current_task_execution_arn,
destination_location_arn,
destination_network_interface_arns,
error_code,
error_detail,
excludes,
includes,
manifest_config,
name,
options,
schedule,
schedule_details,
source_location_arn,
source_network_interface_arns,
status,
task_arn,
task_mode,
task_report_config
FROM aws.datasync.tasks
WHERE region = '{{ region }}' -- required
;

list_tasks

Returns a list of the DataSync tasks you created.

SELECT
name,
status,
task_arn,
task_mode
FROM aws.datasync.tasks
WHERE region = '{{ region }}' -- required
;

INSERT examples

create_task

Configures a task, which defines where and how DataSync transfers your data. A task includes a source location, destination location, and transfer options (such as bandwidth limits, scheduling, and more). If you're planning to transfer data to or from an Amazon S3 location, review how DataSync can affect your S3 request charges and the DataSync pricing page before you begin.

INSERT INTO aws.datasync.tasks (
SourceLocationArn,
DestinationLocationArn,
CloudWatchLogGroupArn,
Name,
Options,
Excludes,
Schedule,
Tags,
Includes,
ManifestConfig,
TaskReportConfig,
TaskMode,
region
)
SELECT 
'{{ SourceLocationArn }}' /* required */,
'{{ DestinationLocationArn }}' /* required */,
'{{ CloudWatchLogGroupArn }}',
'{{ Name }}',
'{{ Options }}',
'{{ Excludes }}',
'{{ Schedule }}',
'{{ Tags }}',
'{{ Includes }}',
'{{ ManifestConfig }}',
'{{ TaskReportConfig }}',
'{{ TaskMode }}',
'{{ region }}'
RETURNING
task_arn
;

Manifest

# Description fields are for documentation purposes
- name: tasks
props:
  - name: region
    value: "{{ region }}"
    description: Required parameter for the tasks resource.
  - name: SourceLocationArn
    value: "{{ SourceLocationArn }}"
    description: |
      Specifies the ARN of your transfer's source location.
  - name: DestinationLocationArn
    value: "{{ DestinationLocationArn }}"
    description: |
      Specifies the ARN of your transfer's destination location.
  - name: CloudWatchLogGroupArn
    value: "{{ CloudWatchLogGroupArn }}"
    description: |
      Specifies the Amazon Resource Name (ARN) of an Amazon CloudWatch log group for monitoring your task. For Enhanced mode tasks, you don't need to specify anything. DataSync automatically sends logs to a CloudWatch log group named /aws/datasync.
  - name: Name
    value: "{{ Name }}"
    description: |
      Specifies the name of your task.
  - name: Options
    description: |
      Specifies your task's settings, such as preserving file metadata, verifying data integrity, among other options.
    value:
      VerifyMode: "{{ VerifyMode }}"
      OverwriteMode: "{{ OverwriteMode }}"
      Atime: "{{ Atime }}"
      Mtime: "{{ Mtime }}"
      Uid: "{{ Uid }}"
      Gid: "{{ Gid }}"
      PreserveDeletedFiles: "{{ PreserveDeletedFiles }}"
      PreserveDevices: "{{ PreserveDevices }}"
      PosixPermissions: "{{ PosixPermissions }}"
      BytesPerSecond: {{ BytesPerSecond }}
      TaskQueueing: "{{ TaskQueueing }}"
      LogLevel: "{{ LogLevel }}"
      TransferMode: "{{ TransferMode }}"
      SecurityDescriptorCopyFlags: "{{ SecurityDescriptorCopyFlags }}"
      ObjectTags: "{{ ObjectTags }}"
  - name: Excludes
    description: |
      Specifies exclude filters that define the files, objects, and folders in your source location that you don't want DataSync to transfer. For more information and examples, see Specifying what DataSync transfers by using filters.
    value:
      - FilterType: "{{ FilterType }}"
        Value: "{{ Value }}"
  - name: Schedule
    description: |
      Specifies a schedule for when you want your task to run. For more information, see Scheduling your task.
    value:
      ScheduleExpression: "{{ ScheduleExpression }}"
      Status: "{{ Status }}"
  - name: Tags
    description: |
      Specifies the tags that you want to apply to your task. Tags are key-value pairs that help you manage, filter, and search for your DataSync resources.
    value:
      - Key: "{{ Key }}"
        Value: "{{ Value }}"
  - name: Includes
    description: |
      Specifies include filters that define the files, objects, and folders in your source location that you want DataSync to transfer. For more information and examples, see Specifying what DataSync transfers by using filters.
    value:
      - FilterType: "{{ FilterType }}"
        Value: "{{ Value }}"
  - name: ManifestConfig
    description: |
      Configures a manifest, which is a list of files or objects that you want DataSync to transfer. For more information and configuration examples, see Specifying what DataSync transfers by using a manifest. When using this parameter, your caller identity (the role that you're using DataSync with) must have the iam:PassRole permission. The AWSDataSyncFullAccess policy includes this permission.
    value:
      Action: "{{ Action }}"
      Format: "{{ Format }}"
      Source:
        S3:
          ManifestObjectPath: "{{ ManifestObjectPath }}"
          BucketAccessRoleArn: "{{ BucketAccessRoleArn }}"
          S3BucketArn: "{{ S3BucketArn }}"
          ManifestObjectVersionId: "{{ ManifestObjectVersionId }}"
  - name: TaskReportConfig
    description: |
      Specifies how you want to configure a task report, which provides detailed information about your DataSync transfer. For more information, see Monitoring your DataSync transfers with task reports. When using this parameter, your caller identity (the role that you're using DataSync with) must have the iam:PassRole permission. The AWSDataSyncFullAccess policy includes this permission.
    value:
      Destination:
        S3:
          Subdirectory: "{{ Subdirectory }}"
          S3BucketArn: "{{ S3BucketArn }}"
          BucketAccessRoleArn: "{{ BucketAccessRoleArn }}"
      OutputType: "{{ OutputType }}"
      ReportLevel: "{{ ReportLevel }}"
      ObjectVersionIds: "{{ ObjectVersionIds }}"
      Overrides:
        Transferred:
          ReportLevel: "{{ ReportLevel }}"
        Verified:
          ReportLevel: "{{ ReportLevel }}"
        Deleted:
          ReportLevel: "{{ ReportLevel }}"
        Skipped:
          ReportLevel: "{{ ReportLevel }}"
  - name: TaskMode
    value: "{{ TaskMode }}"
    description: |
      Specifies one of the following task modes for your data transfer: ENHANCED - Transfer virtually unlimited numbers of objects with higher performance than Basic mode. Enhanced mode tasks optimize the data transfer process by listing, preparing, transferring, and verifying data in parallel. Enhanced mode is currently available for transfers between Amazon S3 locations, transfers between Azure Blob and Amazon S3 without an agent, and transfers between other clouds and Amazon S3 without an agent. To create an Enhanced mode task, the IAM role that you use to call the CreateTask operation must have the iam:CreateServiceLinkedRole permission. BASIC (default) - Transfer files or objects between Amazon Web Services storage and all other supported DataSync locations. Basic mode tasks are subject to quotas on the number of files, objects, and directories in a dataset. Basic mode sequentially prepares, transfers, and verifies data, making it slower than Enhanced mode for most workloads. For more information, see Understanding task mode differences.
    valid_values: ['BASIC', 'ENHANCED']

UPDATE examples

update_task

Updates the configuration of a task, which defines where and how DataSync transfers your data.

UPDATE aws.datasync.tasks
SET 
TaskArn = '{{ TaskArn }}',
Options = '{{ Options }}',
Excludes = '{{ Excludes }}',
Schedule = '{{ Schedule }}',
Name = '{{ Name }}',
CloudWatchLogGroupArn = '{{ CloudWatchLogGroupArn }}',
Includes = '{{ Includes }}',
ManifestConfig = '{{ ManifestConfig }}',
TaskReportConfig = '{{ TaskReportConfig }}'
WHERE 
region = '{{ region }}' --required
AND TaskArn = '{{ TaskArn }}' --required;

DELETE examples

delete_task

Deletes a transfer task resource from DataSync.

DELETE FROM aws.datasync.tasks
WHERE region = '{{ region }}' --required
;

Lifecycle Methods

cancel_task_execution

Stops an DataSync task execution that's in progress. The transfer of some files are abruptly interrupted. File contents that're transferred to the destination might be incomplete or inconsistent with the source files. However, if you start a new task execution using the same task and allow it to finish, file content on the destination will be complete and consistent. This applies to other unexpected failures that interrupt a task execution. In all of these cases, DataSync successfully completes the transfer when you start the next task execution.

EXEC aws.datasync.tasks.cancel_task_execution 
@region='{{ region }}' --required 
@@json=
'{
"TaskExecutionArn": "{{ TaskExecutionArn }}"
}'
;

start_task_execution

Starts an DataSync transfer task. For each task, you can only run one task execution at a time. There are several steps to a task execution. For more information, see Task execution statuses. If you're planning to transfer data to or from an Amazon S3 location, review how DataSync can affect your S3 request charges and the DataSync pricing page before you begin.

EXEC aws.datasync.tasks.start_task_execution 
@region='{{ region }}' --required 
@@json=
'{
"TaskArn": "{{ TaskArn }}", 
"OverrideOptions": "{{ OverrideOptions }}", 
"Includes": "{{ Includes }}", 
"Excludes": "{{ Excludes }}", 
"ManifestConfig": "{{ ManifestConfig }}", 
"TaskReportConfig": "{{ TaskReportConfig }}", 
"Tags": "{{ Tags }}"
}'
;