Skip to main content

dataset_import_jobs

Creates, updates, deletes, gets or lists a dataset_import_jobs resource.

Overview​

Namedataset_import_jobs
TypeResource
Idaws.forecast.dataset_import_jobs

Fields​

The following fields are returned by SELECT queries:

NameDatatypeDescription
creation_timestring (date-time)When the dataset import job was created.
data_sizenumber (double)The size of the dataset in gigabytes (GB) after the import job has finished.
data_sourceobjectThe source of your data, an Identity and Access Management (IAM) role that allows Amazon Forecast to access the data and, optionally, an Key Management Service (KMS) key.
dataset_arnstringThe Amazon Resource Name (ARN) of the dataset that the training data was imported to. (pattern: <code>arn:([a-z\d-]+):forecast:.:.:.+</code>)
dataset_import_job_arnstringThe ARN of the dataset import job. (pattern: <code>arn:([a-z\d-]+):forecast:.:.:.+</code>)
dataset_import_job_namestringThe name of the dataset import job. (pattern: <code>^[a-zA-Z][a-zA-Z0-9_]*</code>)
estimated_time_remaining_in_minutesinteger (int64)The estimated time remaining in minutes for the dataset import job to complete.
field_statisticsobjectStatistical information about each field in the input data.
formatstringThe format of the imported data, CSV or PARQUET. (pattern: <code>^CSV|PARQUET$</code>)
geolocation_formatstringThe format of the geolocation attribute. Valid Values:"LAT_LONG" and "CC_POSTALCODE". (pattern: <code>^[a-zA-Z0-9_]+$</code>)
import_modestringThe import mode of the dataset import job, FULL or INCREMENTAL. (FULL, INCREMENTAL)
last_modification_timestring (date-time)The last time the resource was modified. The timestamp depends on the status of the job: CREATE_PENDING - The CreationTime. CREATE_IN_PROGRESS - The current timestamp. CREATE_STOPPING - The current timestamp. CREATE_STOPPED - When the job stopped. ACTIVE or CREATE_FAILED - When the job finished or failed.
messagestringIf an error occurred, an informational message about the error.
statusstringThe status of the dataset import job. States include: ACTIVE CREATE_PENDING, CREATE_IN_PROGRESS, CREATE_FAILED DELETE_PENDING, DELETE_IN_PROGRESS, DELETE_FAILED CREATE_STOPPING, CREATE_STOPPED
time_zonestringThe single time zone applied to every item in the dataset (pattern: <code>^[a-zA-Z0-9/+-_]+$</code>)
timestamp_formatstringThe format of timestamps in the dataset. The format that you specify depends on the DataFrequency specified when the dataset was created. The following formats are supported "yyyy-MM-dd" For the following data frequencies: Y, M, W, and D "yyyy-MM-dd HH:mm:ss" For the following data frequencies: H, 30min, 15min, and 1min; and optionally, for: Y, M, W, and D (pattern: <code>^[a-zA-Z0-9-:.,'\s]+$</code>)
use_geolocation_for_time_zonebooleanWhether TimeZone is automatically derived from the geolocation attribute.

Methods​

The following methods are available for this resource:

NameAccessible byRequired ParamsOptional ParamsDescription
describe_dataset_import_jobselectregionDescribes a dataset import job created using the CreateDatasetImportJob operation. In addition to listing the parameters provided in the CreateDatasetImportJob request, this operation includes the following properties: CreationTime LastModificationTime DataSize FieldStatistics Status Message - If an error occurred, information about the error.
list_dataset_import_jobsselectregionReturns a list of dataset import jobs created using the CreateDatasetImportJob operation. For each import job, this operation returns a summary of its properties, including its Amazon Resource Name (ARN). You can retrieve the complete set of properties by using the ARN with the DescribeDatasetImportJob operation. You can filter the list by providing an array of Filter objects.
create_dataset_import_jobinsertregion, DatasetImportJobName, DatasetArn, DataSourceImports your training data to an Amazon Forecast dataset. You provide the location of your training data in an Amazon Simple Storage Service (Amazon S3) bucket and the Amazon Resource Name (ARN) of the dataset that you want to import the data to. You must specify a DataSource object that includes an Identity and Access Management (IAM) role that Amazon Forecast can assume to access the data, as Amazon Forecast makes a copy of your data and processes it in an internal Amazon Web Services system. For more information, see Set up permissions. The training data must be in CSV or Parquet format. The delimiter must be a comma (,). You can specify the path to a specific file, the S3 bucket, or to a folder in the S3 bucket. For the latter two cases, Amazon Forecast imports all files up to the limit of 10,000 files. Because dataset imports are not aggregated, your most recent dataset import is the one that is used when training a predictor or generating a forecast. Make sure that your most recent dataset import contains all of the data you want to model off of, and not just the new data collected since the previous import. To get a list of all your dataset import jobs, filtered by specified criteria, use the ListDatasetImportJobs operation.
delete_dataset_import_jobdeleteregionDeletes a dataset import job created using the CreateDatasetImportJob operation. You can delete only dataset import jobs that have a status of ACTIVE or CREATE_FAILED. To get the status, use the DescribeDatasetImportJob operation.

Parameters​

Parameters can be passed in the WHERE clause of a query. Check the Methods section to see which parameters are required or optional for each operation.

NameDatatypeDescription
regionstringAWS region (default: us-east-1)

SELECT examples​

Describes a dataset import job created using the CreateDatasetImportJob operation. In addition to listing the parameters provided in the CreateDatasetImportJob request, this operation includes the following properties: CreationTime LastModificationTime DataSize FieldStatistics Status Message - If an error occurred, information about the error.

SELECT
creation_time,
data_size,
data_source,
dataset_arn,
dataset_import_job_arn,
dataset_import_job_name,
estimated_time_remaining_in_minutes,
field_statistics,
format,
geolocation_format,
import_mode,
last_modification_time,
message,
status,
time_zone,
timestamp_format,
use_geolocation_for_time_zone
FROM aws.forecast.dataset_import_jobs
WHERE region = '{{ region }}' -- required
;

INSERT examples​

Imports your training data to an Amazon Forecast dataset. You provide the location of your training data in an Amazon Simple Storage Service (Amazon S3) bucket and the Amazon Resource Name (ARN) of the dataset that you want to import the data to. You must specify a DataSource object that includes an Identity and Access Management (IAM) role that Amazon Forecast can assume to access the data, as Amazon Forecast makes a copy of your data and processes it in an internal Amazon Web Services system. For more information, see Set up permissions. The training data must be in CSV or Parquet format. The delimiter must be a comma (,). You can specify the path to a specific file, the S3 bucket, or to a folder in the S3 bucket. For the latter two cases, Amazon Forecast imports all files up to the limit of 10,000 files. Because dataset imports are not aggregated, your most recent dataset import is the one that is used when training a predictor or generating a forecast. Make sure that your most recent dataset import contains all of the data you want to model off of, and not just the new data collected since the previous import. To get a list of all your dataset import jobs, filtered by specified criteria, use the ListDatasetImportJobs operation.

INSERT INTO aws.forecast.dataset_import_jobs (
DatasetImportJobName,
DatasetArn,
DataSource,
TimestampFormat,
TimeZone,
UseGeolocationForTimeZone,
GeolocationFormat,
Tags,
Format,
ImportMode,
region
)
SELECT 
'{{ DatasetImportJobName }}' /* required */,
'{{ DatasetArn }}' /* required */,
'{{ DataSource }}' /* required */,
'{{ TimestampFormat }}',
'{{ TimeZone }}',
{{ UseGeolocationForTimeZone }},
'{{ GeolocationFormat }}',
'{{ Tags }}',
'{{ Format }}',
'{{ ImportMode }}',
'{{ region }}'
RETURNING
dataset_import_job_arn
;

DELETE examples​

Deletes a dataset import job created using the CreateDatasetImportJob operation. You can delete only dataset import jobs that have a status of ACTIVE or CREATE_FAILED. To get the status, use the DescribeDatasetImportJob operation.

DELETE FROM aws.forecast.dataset_import_jobs
WHERE region = '{{ region }}' --required
;
Last updated on