bulk_import_jobs

Creates, updates, deletes, gets or lists a bulk_import_jobs resource.

Overview

Name	`bulk_import_jobs`
Type	Resource
Id	`aws.iotsitewise.bulk_import_jobs`

Fields

The following fields are returned by SELECT queries:

describe_bulk_import_job
list_bulk_import_jobs

Name	Datatype	Description
`adaptive_ingestion`	`boolean`	If set to true, ingest new data into IoT SiteWise storage. Measurements with notifications, metrics and transforms are computed. If set to false, historical data is ingested into IoT SiteWise as is.
`dataset_id`	`string`	The ID of the dataset. (pattern: <code>^(?!00000000-0000-0000-0000-000000000000)[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$</code>)
`delete_files_after_import`	`boolean`	If set to true, your data files is deleted from S3, after ingestion into IoT SiteWise storage.
`error_report_location`	`object`	The Amazon S3 destination where errors associated with the job creation request are saved.
`files`	`array`	The files in the specified Amazon S3 bucket that contain your data. You can specify up to 100 files for each bulk import job. Each file supports the following size limits: Parquet files – Up to 256 MiB. Other file formats – Up to 5 GiB.
`job_configuration`	`object`	Contains the configuration information of a job, such as the file format used to save data in Amazon S3.
`job_creation_date`	`string (date-time)`	The date the job was created, in Unix epoch TIME.
`job_id`	`string`	The ID of the job. (pattern: <code>^(?!00000000-0000-0000-0000-000000000000)[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$</code>)
`job_last_update_date`	`string (date-time)`	The date the job was last updated, in Unix epoch time.
`job_name`	`string`	The unique name that helps identify the job request. (pattern: <code>[^\u0000-\u001F\u007F]+</code>)
`job_role_arn`	`string`	The ARN of the IAM role that allows IoT SiteWise to read Amazon S3 data. (pattern: <code>^arn:aws(-cn\|-us-gov)?:[a-zA-Z0-9-:/_.]+$</code>)
`job_status`	`string`	The status of the bulk import job can be one of following values: PENDING – IoT SiteWise is waiting for the current bulk import job to finish. CANCELLED – The bulk import job has been canceled. RUNNING – IoT SiteWise is processing your request to import your data from Amazon S3. COMPLETED – IoT SiteWise successfully completed your request to import data from Amazon S3. FAILED – IoT SiteWise couldn't process your request to import data from Amazon S3. You can use logs saved in the specified error report location in Amazon S3 to troubleshoot issues. COMPLETED_WITH_FAILURES – IoT SiteWise completed your request to import data from Amazon S3 with errors. You can use logs saved in the specified error report location in Amazon S3 to troubleshoot issues. (PENDING, CANCELLED, RUNNING, COMPLETED, FAILED, COMPLETED_WITH_FAILURES)
`workspace_name`	`string`	The name of the workspace. (pattern: <code>^[a-zA-Z0-9_-]+$</code>)

Name	Datatype	Description
`id`	`string`	The ID of the job. (pattern: <code>^(?!00000000-0000-0000-0000-000000000000)[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$</code>)
`name`	`string`	The unique name that helps identify the job request. (pattern: <code>[^\u0000-\u001F\u007F]+</code>)
`status`	`string`	The status of the bulk import job can be one of following values: PENDING – IoT SiteWise is waiting for the current bulk import job to finish. CANCELLED – The bulk import job has been canceled. RUNNING – IoT SiteWise is processing your request to import your data from Amazon S3. COMPLETED – IoT SiteWise successfully completed your request to import data from Amazon S3. FAILED – IoT SiteWise couldn't process your request to import data from Amazon S3. You can use logs saved in the specified error report location in Amazon S3 to troubleshoot issues. COMPLETED_WITH_FAILURES – IoT SiteWise completed your request to import data from Amazon S3 with errors. You can use logs saved in the specified error report location in Amazon S3 to troubleshoot issues. (PENDING, CANCELLED, RUNNING, COMPLETED, FAILED, COMPLETED_WITH_FAILURES)

Methods

The following methods are available for this resource:

Name	Accessible by	Required Params	Optional Params	Description
`describe_bulk_import_job`	`select`	`job_id`, `region`	`workspaceName`	Retrieves information about a bulk import job request. For more information, see Describe a bulk import job (CLI) in the Amazon Simple Storage Service User Guide.
`list_bulk_import_jobs`	`select`	`region`	`nextToken`, `maxResults`, `filter`, `workspaceName`	Retrieves a paginated list of bulk import job requests. For more information, see List bulk import jobs (CLI) in the IoT SiteWise User Guide.
`create_bulk_import_job`	`insert`	`region`, `jobName`, `jobRoleArn`, `files`, `errorReportLocation`		Defines a job to ingest data to IoT SiteWise from Amazon S3. For more information, see Create a bulk import job (CLI) in the Amazon Simple Storage Service User Guide. Before you create a bulk import job that ingests data into time series outside of a workspace, you must enable IoT SiteWise warm tier or IoT SiteWise cold tier. For more information about how to configure storage settings, see PutStorageConfiguration. This requirement doesn't apply to bulk import jobs that ingest data into a session dataset in a workspace (jobs that specify a workspaceName and datasetId). Those jobs don't use IoT SiteWise warm or cold tier storage. Bulk import is designed to store historical data to IoT SiteWise. Newly ingested data in the hot tier triggers notifications and computations. After data moves from the hot tier to the warm or cold tier based on retention settings, it does not trigger computations or notifications. Data older than 7 days does not trigger computations or notifications.

Parameters

Parameters can be passed in the WHERE clause of a query. Check the Methods section to see which parameters are required or optional for each operation.

Name	Datatype	Description
`job_id`	`string`	The ID of the job.
`region`	`string`	AWS region (default: us-east-1)
`filter`	`string`	You can use a filter to select the bulk import jobs that you want to retrieve.
`maxResults`	`integer`	The maximum number of results to return for each paginated request.
`nextToken`	`string`	The token to be used for the next set of paginated results.
`workspaceName`	`string`	The name of the workspace.

`SELECT` examples

describe_bulk_import_job
list_bulk_import_jobs

Retrieves information about a bulk import job request. For more information, see Describe a bulk import job (CLI) in the Amazon Simple Storage Service User Guide.

SELECT
adaptive_ingestion,
dataset_id,
delete_files_after_import,
error_report_location,
files,
job_configuration,
job_creation_date,
job_id,
job_last_update_date,
job_name,
job_role_arn,
job_status,
workspace_name
FROM aws.iotsitewise.bulk_import_jobs
WHERE job_id = '{{ job_id }}' -- required
AND region = '{{ region }}' -- required
AND workspaceName = '{{ workspaceName }}'
;

Retrieves a paginated list of bulk import job requests. For more information, see List bulk import jobs (CLI) in the IoT SiteWise User Guide.

SELECT
id,
name,
status
FROM aws.iotsitewise.bulk_import_jobs
WHERE region = '{{ region }}' -- required
AND nextToken = '{{ nextToken }}'
AND maxResults = '{{ maxResults }}'
AND filter = '{{ filter }}'
AND workspaceName = '{{ workspaceName }}'
;

`INSERT` examples

create_bulk_import_job
Manifest

Defines a job to ingest data to IoT SiteWise from Amazon S3. For more information, see Create a bulk import job (CLI) in the Amazon Simple Storage Service User Guide. Before you create a bulk import job that ingests data into time series outside of a workspace, you must enable IoT SiteWise warm tier or IoT SiteWise cold tier. For more information about how to configure storage settings, see PutStorageConfiguration. This requirement doesn't apply to bulk import jobs that ingest data into a session dataset in a workspace (jobs that specify a workspaceName and datasetId). Those jobs don't use IoT SiteWise warm or cold tier storage. Bulk import is designed to store historical data to IoT SiteWise. Newly ingested data in the hot tier triggers notifications and computations. After data moves from the hot tier to the warm or cold tier based on retention settings, it does not trigger computations or notifications. Data older than 7 days does not trigger computations or notifications.

INSERT INTO aws.iotsitewise.bulk_import_jobs (
jobName,
jobRoleArn,
files,
errorReportLocation,
jobConfiguration,
adaptiveIngestion,
deleteFilesAfterImport,
datasetId,
workspaceName,
region
)
SELECT 
'{{ jobName }}' /* required */,
'{{ jobRoleArn }}' /* required */,
'{{ files }}' /* required */,
'{{ errorReportLocation }}' /* required */,
'{{ jobConfiguration }}',
{{ adaptiveIngestion }},
{{ deleteFilesAfterImport }},
'{{ datasetId }}',
'{{ workspaceName }}',
'{{ region }}'
RETURNING
job_id,
job_name,
job_status
;

# Description fields are for documentation purposes
- name: bulk_import_jobs
props:
  - name: region
    value: "{{ region }}"
    description: Required parameter for the bulk_import_jobs resource.
  - name: jobName
    value: "{{ jobName }}"
  - name: jobRoleArn
    value: "{{ jobRoleArn }}"
  - name: files
    value:
      - bucket: "{{ bucket }}"
        key: "{{ key }}"
        versionId: "{{ versionId }}"
        alias: "{{ alias }}"
        startTime:
          timeInSeconds: {{ timeInSeconds }}
          offsetInNanos: {{ offsetInNanos }}
        fileFormat:
          csv:
            columnNames:
              - "{{ columnNames }}"
          parquet: "{{ parquet }}"
          mp4: "{{ mp4 }}"
          annotation: "{{ annotation }}"
  - name: errorReportLocation
    description: |
      The Amazon S3 destination where errors associated with the job creation request are saved.
    value:
      bucket: "{{ bucket }}"
      prefix: "{{ prefix }}"
  - name: jobConfiguration
    description: |
      Contains the configuration information of a job, such as the file format used to save data in Amazon S3.
    value:
      fileFormat:
        csv:
          columnNames:
            - "{{ columnNames }}"
        parquet: "{{ parquet }}"
        mp4: "{{ mp4 }}"
        annotation: "{{ annotation }}"
  - name: adaptiveIngestion
    value: {{ adaptiveIngestion }}
  - name: deleteFilesAfterImport
    value: {{ deleteFilesAfterImport }}
  - name: datasetId
    value: "{{ datasetId }}"
  - name: workspaceName
    value: "{{ workspaceName }}"

Overview​

Fields​

Methods​

Parameters​

SELECT examples​

INSERT examples​