data_ingestion_jobs

Creates, updates, deletes, gets or lists a data_ingestion_jobs resource.

Overview

Name	data_ingestion_jobs
Type	Resource
Id	aws.lookoutequipment.data_ingestion_jobs

Fields

The following fields are returned by SELECT queries:

describe_data_ingestion_job

Name	Datatype	Description
created_at	string (date-time)	The time at which the data ingestion job was created.
data_end_time	string (date-time)	Indicates the latest timestamp corresponding to data that was successfully ingested during this specific ingestion job.
data_quality_summary	object	Gives statistics about a completed ingestion job. These statistics primarily relate to quantifying incorrect data such as MissingCompleteSensorData, MissingSensorData, UnsupportedDateFormats, InsufficientSensorData, and DuplicateTimeStamps.
data_start_time	string (date-time)	Indicates the earliest timestamp corresponding to data that was successfully ingested during this specific ingestion job.
dataset_arn	string	The Amazon Resource Name (ARN) of the dataset being used in the data ingestion job. (pattern: <code>arn:aws(-[^:]+)?:lookoutequipment:[a-zA-Z0-9-]*:[0-9]{12}:dataset/[0-9a-zA-Z_-]{1,200}/.+</code>)
failed_reason	string	Specifies the reason for failure when a data ingestion job has failed. (pattern: <code>[\P{M}\p{M}]{1,5000}</code>)
ingested_data_size	integer (int64)	Indicates the size of the ingested dataset.
ingested_files_summary	object	Gives statistics about how many files have been ingested, and which files have not been ingested, for a particular ingestion job.
ingestion_input_configuration	object	Specifies the S3 location configuration for the data input for the data ingestion job.
job_id	string	Indicates the job ID of the data ingestion job. (pattern: <code>[A-Fa-f0-9]{0,32}</code>)
role_arn	string	The Amazon Resource Name (ARN) of an IAM role with permission to access the data source being ingested. (pattern: <code>arn:aws(-[^:]+)?:iam::[0-9]{12}:role/.+</code>)
source_dataset_arn	string	The Amazon Resource Name (ARN) of the source dataset from which the data used for the data ingestion job was imported from. (pattern: <code>arn:aws(-[^:]+)?:lookoutequipment:[a-zA-Z0-9-]*:[0-9]{12}:dataset/[0-9a-zA-Z_-]{1,200}/.+</code>)
status	string	Indicates the status of the DataIngestionJob operation. (IN_PROGRESS, SUCCESS, FAILED, IMPORT_IN_PROGRESS)
status_detail	string	Provides details about status of the ingestion job that is currently in progress. (pattern: <code>[\P{M}\p{M}]{1,5000}</code>)

list_data_ingestion_jobs

Name	Datatype	Description
data_ingestion_job_summaries	array	Specifies information about the specific data ingestion job, including dataset name and status.
next_token	string	An opaque pagination token indicating where to continue the listing of data ingestion jobs. (pattern: <code>\p{ASCII}{0,8192}</code>)

Methods

The following methods are available for this resource:

Name	Accessible by	Required Params	Optional Params	Description
describe_data_ingestion_job	select	region		Provides information on a specific data ingestion job such as creation time, dataset ARN, and status.
list_data_ingestion_jobs	select	region		Provides a list of all data ingestion jobs, including dataset name and ARN, S3 location of the input data, status, and so on.

Parameters

Parameters can be passed in the WHERE clause of a query. Check the Methods section to see which parameters are required or optional for each operation.

Name	Datatype	Description
region	string	AWS region (default: us-east-1)

SELECT examples

describe_data_ingestion_job

Provides information on a specific data ingestion job such as creation time, dataset ARN, and status.

SELECT
created_at,
data_end_time,
data_quality_summary,
data_start_time,
dataset_arn,
failed_reason,
ingested_data_size,
ingested_files_summary,
ingestion_input_configuration,
job_id,
role_arn,
source_dataset_arn,
status,
status_detail
FROM aws.lookoutequipment.data_ingestion_jobs
WHERE region = '{{ region }}' -- required
;

list_data_ingestion_jobs

Provides a list of all data ingestion jobs, including dataset name and ARN, S3 location of the input data, status, and so on.

SELECT
data_ingestion_job_summaries,
next_token
FROM aws.lookoutequipment.data_ingestion_jobs
WHERE region = '{{ region }}' -- required
;