stream_consumers

Creates, updates, deletes, gets or lists a stream_consumers resource.

Overview

Name	stream_consumers
Type	Resource
Id	aws.kinesis.stream_consumers

Fields

The following fields are returned by SELECT queries:

list_stream_consumers

Name	Datatype	Description
consumer_arn	string	When you register a consumer, Kinesis Data Streams generates an ARN for it. You need this ARN to be able to call SubscribeToShard. If you delete a consumer and then create a new one with the same name, it won't have the same ARN. That's because consumer ARNs contain the creation timestamp. This is important to keep in mind if you have IAM policies that reference consumer ARNs. (pattern: <code>^(arn):aws.:kinesis:.:\d{12}:.*stream/[a-zA-Z0-9_.-]+/consumer/[a-zA-Z0-9_.-]+:[0-9]+</code>)
consumer_creation_timestamp	string (date-time)
consumer_name	string	The name of the consumer is something you choose when you register the consumer. (pattern: <code>[a-zA-Z0-9_.-]+</code>)
consumer_status	string	A consumer can't read data while in the CREATING or DELETING states. (CREATING, DELETING, ACTIVE)

describe_stream_consumer

Name	Datatype	Description
consumer_arn	string	When you register a consumer, Kinesis Data Streams generates an ARN for it. You need this ARN to be able to call SubscribeToShard. If you delete a consumer and then create a new one with the same name, it won't have the same ARN. That's because consumer ARNs contain the creation timestamp. This is important to keep in mind if you have IAM policies that reference consumer ARNs. (pattern: <code>^(arn):aws.:kinesis:.:\d{12}:.*stream/[a-zA-Z0-9_.-]+/consumer/[a-zA-Z0-9_.-]+:[0-9]+</code>)
consumer_creation_timestamp	string (date-time)
consumer_name	string	The name of the consumer is something you choose when you register the consumer. (pattern: <code>[a-zA-Z0-9_.-]+</code>)
consumer_status	string	A consumer can't read data while in the CREATING or DELETING states. (CREATING, DELETING, ACTIVE)
stream_arn	string	The ARN of the stream with which you registered the consumer. (pattern: <code>arn:aws.:kinesis:.:\d{12}:stream/\S+</code>)

Methods

The following methods are available for this resource:

Name	Accessible by	Required Params	Optional Params	Description
list_stream_consumers	select	region		Lists the consumers registered to receive data from a stream using enhanced fan-out, and provides information about each consumer. This operation has a limit of 5 transactions per second per stream.
describe_stream_consumer	select	region		To get the description of a registered consumer, provide the ARN of the consumer. Alternatively, you can provide the ARN of the data stream and the name you gave the consumer when you registered it. You may also provide all three parameters, as long as they don't conflict with each other. If you don't know the name or ARN of the consumer that you want to describe, you can use the ListStreamConsumers operation to get a list of the descriptions of all the consumers that are currently registered with a given data stream. This operation has a limit of 20 transactions per second per stream. When making a cross-account call with DescribeStreamConsumer, make sure to provide the ARN of the consumer.
register_stream_consumer	insert	region, StreamARN, ConsumerName		Registers a consumer with a Kinesis data stream. When you use this operation, the consumer you register can then call SubscribeToShard to receive data from the stream using enhanced fan-out, at a rate of up to 2 MiB per second for every shard you subscribe to. This rate is unaffected by the total number of consumers that read from the same stream. You can add tags to the registered consumer when making a RegisterStreamConsumer request by setting the Tags parameter. If you pass the Tags parameter, in addition to having the kinesis:RegisterStreamConsumer permission, you must also have the kinesis:TagResource permission for the consumer that will be registered. Tags will take effect from the CREATING status of the consumer. With On-demand Advantage streams, you can register up to 50 consumers per stream to use Enhanced Fan-out. With On-demand Standard and Provisioned streams, you can register up to 20 consumers per stream to use Enhanced Fan-out. A given consumer can only be registered with one stream at a time. For an example of how to use this operation, see Enhanced Fan-Out Using the Kinesis Data Streams API. The use of this operation has a limit of five transactions per second per account. Also, only 5 consumers can be created simultaneously. In other words, you cannot have more than 5 consumers in a CREATING status at the same time. Registering a 6th consumer while there are 5 in a CREATING status results in a LimitExceededException.
deregister_stream_consumer	delete	region		To deregister a consumer, provide its ARN. Alternatively, you can provide the ARN of the data stream and the name you gave the consumer when you registered it. You may also provide all three parameters, as long as they don't conflict with each other. If you don't know the name or ARN of the consumer that you want to deregister, you can use the ListStreamConsumers operation to get a list of the descriptions of all the consumers that are currently registered with a given data stream. The description of a consumer contains its name and ARN. This operation has a limit of five transactions per second per stream.

Parameters

Parameters can be passed in the WHERE clause of a query. Check the Methods section to see which parameters are required or optional for each operation.

Name	Datatype	Description
region	string	AWS region (default: us-east-1)

SELECT examples

list_stream_consumers

Lists the consumers registered to receive data from a stream using enhanced fan-out, and provides information about each consumer. This operation has a limit of 5 transactions per second per stream.

SELECT
consumer_arn,
consumer_creation_timestamp,
consumer_name,
consumer_status
FROM aws.kinesis.stream_consumers
WHERE region = '{{ region }}' -- required
;

describe_stream_consumer

To get the description of a registered consumer, provide the ARN of the consumer. Alternatively, you can provide the ARN of the data stream and the name you gave the consumer when you registered it. You may also provide all three parameters, as long as they don't conflict with each other. If you don't know the name or ARN of the consumer that you want to describe, you can use the ListStreamConsumers operation to get a list of the descriptions of all the consumers that are currently registered with a given data stream. This operation has a limit of 20 transactions per second per stream. When making a cross-account call with DescribeStreamConsumer, make sure to provide the ARN of the consumer.

SELECT
consumer_arn,
consumer_creation_timestamp,
consumer_name,
consumer_status,
stream_arn
FROM aws.kinesis.stream_consumers
WHERE region = '{{ region }}' -- required
;

INSERT examples

register_stream_consumer

Registers a consumer with a Kinesis data stream. When you use this operation, the consumer you register can then call SubscribeToShard to receive data from the stream using enhanced fan-out, at a rate of up to 2 MiB per second for every shard you subscribe to. This rate is unaffected by the total number of consumers that read from the same stream. You can add tags to the registered consumer when making a RegisterStreamConsumer request by setting the Tags parameter. If you pass the Tags parameter, in addition to having the kinesis:RegisterStreamConsumer permission, you must also have the kinesis:TagResource permission for the consumer that will be registered. Tags will take effect from the CREATING status of the consumer. With On-demand Advantage streams, you can register up to 50 consumers per stream to use Enhanced Fan-out. With On-demand Standard and Provisioned streams, you can register up to 20 consumers per stream to use Enhanced Fan-out. A given consumer can only be registered with one stream at a time. For an example of how to use this operation, see Enhanced Fan-Out Using the Kinesis Data Streams API. The use of this operation has a limit of five transactions per second per account. Also, only 5 consumers can be created simultaneously. In other words, you cannot have more than 5 consumers in a CREATING status at the same time. Registering a 6th consumer while there are 5 in a CREATING status results in a LimitExceededException.

INSERT INTO aws.kinesis.stream_consumers (
StreamARN,
ConsumerName,
StreamId,
Tags,
region
)
SELECT 
'{{ StreamARN }}' /* required */,
'{{ ConsumerName }}' /* required */,
'{{ StreamId }}',
'{{ Tags }}',
'{{ region }}'
RETURNING
consumer
;

Manifest

# Description fields are for documentation purposes
- name: stream_consumers
props:
  - name: region
    value: "{{ region }}"
    description: Required parameter for the stream_consumers resource.
  - name: StreamARN
    value: "{{ StreamARN }}"
    description: |
      The ARN of the Kinesis data stream that you want to register the consumer with. For more info, see Amazon Resource Names (ARNs) and Amazon Web Services Service Namespaces.
  - name: ConsumerName
    value: "{{ ConsumerName }}"
    description: |
      For a given Kinesis data stream, each consumer must have a unique name. However, consumer names don't have to be unique across data streams.
  - name: StreamId
    value: "{{ StreamId }}"
    description: |
      Not Implemented. Reserved for future use.
  - name: Tags
    value: "{{ Tags }}"
    description: |
      A set of up to 50 key-value pairs. A tag consists of a required key and an optional value.

DELETE examples

deregister_stream_consumer

To deregister a consumer, provide its ARN. Alternatively, you can provide the ARN of the data stream and the name you gave the consumer when you registered it. You may also provide all three parameters, as long as they don't conflict with each other. If you don't know the name or ARN of the consumer that you want to deregister, you can use the ListStreamConsumers operation to get a list of the descriptions of all the consumers that are currently registered with a given data stream. The description of a consumer contains its name and ARN. This operation has a limit of five transactions per second per stream.

DELETE FROM aws.kinesis.stream_consumers
WHERE region = '{{ region }}' --required
;