telemetry_pipelines

Creates, updates, deletes, gets or lists a telemetry_pipelines resource.

Overview

Name	telemetry_pipelines
Type	Resource
Id	aws.observabilityadmin.telemetry_pipelines

Fields

The following fields are returned by SELECT queries:

get_telemetry_pipeline

Name	Datatype	Description
arn	string	The Amazon Resource Name (ARN) of the telemetry pipeline. (pattern: <code>arn:aws([a-z0-9-]+)?:([a-zA-Z0-9-]+):([a-z0-9-]+)?:([0-9]{12})?:(.+)</code>)
configuration	object	Defines the configuration for a pipeline, including how data flows from sources through processors to destinations. The configuration is specified in YAML format and must include a valid pipeline definition with required source and sink components. This pipeline enables end-to-end telemetry data collection, transformation, and delivery while supporting optional processing steps and extensions for enhanced functionality. The primary pipeline configuration section are: Source: Defines where log data originates from (S3 buckets, CloudWatch Logs, third-party APIs). Each pipeline must have exactly one source. Processors (optional): Transform, parse, and enrich log data as it flows through the pipeline. Processors are applied sequentially in the order they are defined. Sink: Defines the destination where processed log data is sent. Each pipeline must have exactly one sink. Extensions (optional): Provide additional functionality such as Amazon Web Services Secrets Manager integration for credential management. For more details on each configuration section see CloudWatch pipelines User Guide. Additional comprehensive configuration examples can be found in the CreateTelemetryPipeline API docs.
created_time_stamp	integer (int64)	The timestamp when the telemetry pipeline was created.
last_update_time_stamp	integer (int64)	The timestamp when the telemetry pipeline was last updated.
name	string	The name of the telemetry pipeline. (pattern: <code>.[a-z][a-z0-9-]+.</code>)
status	string	The current status of the telemetry pipeline. (CREATING, ACTIVE, UPDATING, DELETING, CREATE_FAILED, UPDATE_FAILED)
status_reason	object	Additional information about the pipeline status, including reasons for failure states.
tags	object	The key-value pairs associated with the telemetry pipeline resource.

list_telemetry_pipelines

Name	Datatype	Description
arn	string	The Amazon Resource Name (ARN) of the telemetry pipeline. (pattern: <code>arn:aws([a-z0-9-]+)?:([a-zA-Z0-9-]+):([a-z0-9-]+)?:([0-9]{12})?:(.+)</code>)
configuration_summary	object	A summary of the pipeline configuration components.
created_time_stamp	integer (int64)	The timestamp when the telemetry pipeline was created.
last_update_time_stamp	integer (int64)	The timestamp when the telemetry pipeline was last updated.
name	string	The name of the telemetry pipeline. (pattern: <code>.[a-z][a-z0-9-]+.</code>)
status	string	The current status of the telemetry pipeline. (CREATING, ACTIVE, UPDATING, DELETING, CREATE_FAILED, UPDATE_FAILED)
tags	object	The key-value pairs associated with the telemetry pipeline resource.

Methods

The following methods are available for this resource:

Name	Accessible by	Required Params	Optional Params	Description
get_telemetry_pipeline	select	region		Retrieves information about a specific telemetry pipeline, including its configuration, status, and metadata.
list_telemetry_pipelines	select	region		Returns a list of telemetry pipelines in your account. Returns up to 100 results. If more than 100 telemetry pipelines exist, include the NextToken value from the response to retrieve the next set of results.
create_telemetry_pipeline	insert	region, Configuration		Creates a telemetry pipeline for processing and transforming telemetry data. The pipeline defines how data flows from sources through processors to destinations, enabling data transformation and delivering capabilities.
update_telemetry_pipeline	update	region, PipelineIdentifier, Configuration		Updates the configuration of an existing telemetry pipeline. The following attributes cannot be updated after pipeline creation: Pipeline name - The pipeline name is immutable Pipeline ARN - The ARN is automatically generated and cannot be changed Source type - Once a pipeline is created with a specific source type (such as S3, CloudWatch Logs, GitHub, or third-party sources), it cannot be changed to a different source type Processors can be added, removed, or modified. However, some processors are not supported for third-party pipelines and cannot be added through updates. Source-Specific Update Rules CloudWatch Logs Sources (Vended and Custom) Updatable: sts_role_arn Fixed: data_source_name, data_source_type, sink (must remain @original) S3 Sources (Crowdstrike, Zscaler, SentinelOne, Custom) Updatable: All SQS configuration parameters, sts_role_arn, codec settings, compression type, bucket ownership settings, sink log group Fixed: notification_type, aws.region GitHub Audit Logs Updatable: All Amazon Web Services Secrets Manager attributes, scope (can switch between ORGANIZATION/ENTERPRISE), organization or enterprise name, range, authentication credentials (PAT or GitHub App) Microsoft Sources (Entra ID, Office365, Windows) Updatable: All Amazon Web Services Secrets Manager attributes, tenant_id, workspace_id (Windows only), OAuth2 credentials (client_id, client_secret) Okta Sources (SSO, Auth0) Updatable: All Amazon Web Services Secrets Manager attributes, domain, range, OAuth2 credentials (client_id, client_secret) Palo Alto Networks Updatable: All Amazon Web Services Secrets Manager attributes, hostname, basic authentication credentials (username, password) ServiceNow CMDB Updatable: All Amazon Web Services Secrets Manager attributes, instance_url, range, OAuth2 credentials (client_id, client_secret) Wiz CNAPP Updatable: All Amazon Web Services Secrets Manager attributes, region, range, OAuth2 credentials (client_id, client_secret)
delete_telemetry_pipeline	delete	region		Deletes a telemetry pipeline and its associated resources. This operation stops data processing and removes the pipeline configuration.
validate_telemetry_pipeline_configuration	exec	region, Configuration		Validates a pipeline configuration without creating the pipeline. This operation checks the configuration for syntax errors and compatibility issues.

Parameters

Parameters can be passed in the WHERE clause of a query. Check the Methods section to see which parameters are required or optional for each operation.

Name	Datatype	Description
region	string	AWS region (default: us-east-1)

SELECT examples

get_telemetry_pipeline

Retrieves information about a specific telemetry pipeline, including its configuration, status, and metadata.

SELECT
arn,
configuration,
created_time_stamp,
last_update_time_stamp,
name,
status,
status_reason,
tags
FROM aws.observabilityadmin.telemetry_pipelines
WHERE region = '{{ region }}' -- required
;

list_telemetry_pipelines

Returns a list of telemetry pipelines in your account. Returns up to 100 results. If more than 100 telemetry pipelines exist, include the NextToken value from the response to retrieve the next set of results.

SELECT
arn,
configuration_summary,
created_time_stamp,
last_update_time_stamp,
name,
status,
tags
FROM aws.observabilityadmin.telemetry_pipelines
WHERE region = '{{ region }}' -- required
;

INSERT examples

create_telemetry_pipeline

Creates a telemetry pipeline for processing and transforming telemetry data. The pipeline defines how data flows from sources through processors to destinations, enabling data transformation and delivering capabilities.

INSERT INTO aws.observabilityadmin.telemetry_pipelines (
Name,
Configuration,
Tags,
region
)
SELECT 
'{{ Name }}',
'{{ Configuration }}' /* required */,
'{{ Tags }}',
'{{ region }}'
RETURNING
arn
;

Manifest

# Description fields are for documentation purposes
- name: telemetry_pipelines
props:
  - name: region
    value: "{{ region }}"
    description: Required parameter for the telemetry_pipelines resource.
  - name: Name
    value: "{{ Name }}"
  - name: Configuration
    description: |
      Defines the configuration for a pipeline, including how data flows from sources through processors to destinations. The configuration is specified in YAML format and must include a valid pipeline definition with required source and sink components. This pipeline enables end-to-end telemetry data collection, transformation, and delivery while supporting optional processing steps and extensions for enhanced functionality. The primary pipeline configuration section are: Source: Defines where log data originates from (S3 buckets, CloudWatch Logs, third-party APIs). Each pipeline must have exactly one source. Processors (optional): Transform, parse, and enrich log data as it flows through the pipeline. Processors are applied sequentially in the order they are defined. Sink: Defines the destination where processed log data is sent. Each pipeline must have exactly one sink. Extensions (optional): Provide additional functionality such as Amazon Web Services Secrets Manager integration for credential management. For more details on each configuration section see CloudWatch pipelines User Guide. Additional comprehensive configuration examples can be found in the CreateTelemetryPipeline API docs.
    value:
      Body: "{{ Body }}"
  - name: Tags
    value: "{{ Tags }}"

UPDATE examples

update_telemetry_pipeline

Updates the configuration of an existing telemetry pipeline. The following attributes cannot be updated after pipeline creation: Pipeline name - The pipeline name is immutable Pipeline ARN - The ARN is automatically generated and cannot be changed Source type - Once a pipeline is created with a specific source type (such as S3, CloudWatch Logs, GitHub, or third-party sources), it cannot be changed to a different source type Processors can be added, removed, or modified. However, some processors are not supported for third-party pipelines and cannot be added through updates. Source-Specific Update Rules CloudWatch Logs Sources (Vended and Custom) Updatable: sts_role_arn Fixed: data_source_name, data_source_type, sink (must remain @original) S3 Sources (Crowdstrike, Zscaler, SentinelOne, Custom) Updatable: All SQS configuration parameters, sts_role_arn, codec settings, compression type, bucket ownership settings, sink log group Fixed: notification_type, aws.region GitHub Audit Logs Updatable: All Amazon Web Services Secrets Manager attributes, scope (can switch between ORGANIZATION/ENTERPRISE), organization or enterprise name, range, authentication credentials (PAT or GitHub App) Microsoft Sources (Entra ID, Office365, Windows) Updatable: All Amazon Web Services Secrets Manager attributes, tenant_id, workspace_id (Windows only), OAuth2 credentials (client_id, client_secret) Okta Sources (SSO, Auth0) Updatable: All Amazon Web Services Secrets Manager attributes, domain, range, OAuth2 credentials (client_id, client_secret) Palo Alto Networks Updatable: All Amazon Web Services Secrets Manager attributes, hostname, basic authentication credentials (username, password) ServiceNow CMDB Updatable: All Amazon Web Services Secrets Manager attributes, instance_url, range, OAuth2 credentials (client_id, client_secret) Wiz CNAPP Updatable: All Amazon Web Services Secrets Manager attributes, region, range, OAuth2 credentials (client_id, client_secret)

UPDATE aws.observabilityadmin.telemetry_pipelines
SET 
PipelineIdentifier = '{{ PipelineIdentifier }}',
Configuration = '{{ Configuration }}'
WHERE 
region = '{{ region }}' --required
AND PipelineIdentifier = '{{ PipelineIdentifier }}' --required
AND Configuration = '{{ Configuration }}' --required;

DELETE examples

delete_telemetry_pipeline

Deletes a telemetry pipeline and its associated resources. This operation stops data processing and removes the pipeline configuration.

DELETE FROM aws.observabilityadmin.telemetry_pipelines
WHERE region = '{{ region }}' --required
;

Lifecycle Methods

validate_telemetry_pipeline_configuration

Validates a pipeline configuration without creating the pipeline. This operation checks the configuration for syntax errors and compatibility issues.

EXEC aws.observabilityadmin.telemetry_pipelines.validate_telemetry_pipeline_configuration 
@region='{{ region }}' --required 
@@json=
'{
"Configuration": "{{ Configuration }}"
}'
;