connectors

Creates, updates, deletes, gets or lists a connectors resource.

Overview

Name	connectors
Type	Resource
Id	aws.kafkaconnect.connectors

Fields

The following fields are returned by SELECT queries:

describe_connector

Name	Datatype	Description
capacity	object	Information about the capacity of the connector, whether it is auto scaled or provisioned.
connector_arn	string	The Amazon Resource Name (ARN) of the connector.
connector_configuration	object	A map of keys to values that represent the configuration for the connector.
connector_description	string	A summary description of the connector.
connector_name	string	The name of the connector.
connector_state	string	The state of the connector. (RUNNING, CREATING, UPDATING, DELETING, FAILED)
creation_time	string (date-time)	The time the connector was created.
current_version	string	The current version of the connector.
kafka_cluster	object	The Apache Kafka cluster that the connector is connected to.
kafka_cluster_client_authentication	object	The type of client authentication used to connect to the Apache Kafka cluster. The value is NONE when no client authentication is used.
kafka_cluster_encryption_in_transit	object	Details of encryption in transit to the Apache Kafka cluster.
kafka_connect_version	string	The version of Kafka Connect. It has to be compatible with both the Apache Kafka cluster's version and the plugins.
log_delivery	object	Details about delivering logs to Amazon CloudWatch Logs.
network_type	string	The network type of a connector. (IPV4, DUAL)
plugins	array	Specifies which plugins were used for this connector.
service_execution_role_arn	string	The Amazon Resource Name (ARN) of the IAM role used by the connector to access Amazon Web Services resources.
state_description	object	Details about the state of a connector.
worker_configuration	object	Specifies which worker configuration was used for the connector.

list_connectors

Name	Datatype	Description
capacity	object	The connector's compute capacity settings.
connector_arn	string	The Amazon Resource Name (ARN) of the connector.
connector_description	string	The description of the connector.
connector_name	string	The name of the connector.
connector_state	string	The state of the connector. (RUNNING, CREATING, UPDATING, DELETING, FAILED)
creation_time	string (date-time)	The time that the connector was created.
current_version	string	The current version of the connector.
kafka_cluster	object	The details of the Apache Kafka cluster to which the connector is connected.
kafka_cluster_client_authentication	object	The type of client authentication used to connect to the Apache Kafka cluster. The value is NONE when no client authentication is used.
kafka_cluster_encryption_in_transit	object	Details of encryption in transit to the Apache Kafka cluster.
kafka_connect_version	string	The version of Kafka Connect. It has to be compatible with both the Apache Kafka cluster's version and the plugins.
log_delivery	object	The settings for delivering connector logs to Amazon CloudWatch Logs.
network_type	string	The network type of a connector. (IPV4, DUAL)
plugins	array	Specifies which plugins were used for this connector.
service_execution_role_arn	string	The Amazon Resource Name (ARN) of the IAM role used by the connector to access Amazon Web Services resources.
worker_configuration	object	The worker configurations that are in use with the connector.

Methods

The following methods are available for this resource:

Name	Accessible by	Required Params	Optional Params	Description
describe_connector	select	connector_arn, region		Returns summary information about the connector.
list_connectors	select	region	connectorNamePrefix, maxResults, nextToken	Returns a list of all the connectors in this account and Region. The list is limited to connectors whose name starts with the specified prefix. The response also includes a description of each of the listed connectors.
create_connector	insert	region, capacity, connectorConfiguration, connectorName, kafkaCluster, kafkaClusterClientAuthentication, kafkaClusterEncryptionInTransit, kafkaConnectVersion, plugins, serviceExecutionRoleArn		Creates a connector using the specified properties.
update_connector	update	connector_arn, currentVersion, region		Updates the specified connector. For request body, specify only one parameter: either capacity or connectorConfiguration.
delete_connector	delete	connector_arn, region	currentVersion	Deletes the specified connector.

Parameters

Parameters can be passed in the WHERE clause of a query. Check the Methods section to see which parameters are required or optional for each operation.

Name	Datatype	Description
connector_arn	string	The Amazon Resource Name (ARN) of the connector that you want to delete.
currentVersion	string	The current version of the connector that you want to update.
region	string	AWS region (default: us-east-1)
connectorNamePrefix	string	The name prefix that you want to use to search for and list connectors.
currentVersion	string	The current version of the connector that you want to delete.
maxResults	integer	The maximum number of connectors to list in one response.
nextToken	string	If the response of a ListConnectors operation is truncated, it will include a NextToken. Send this NextToken in a subsequent request to continue listing from where the previous operation left off.

SELECT examples

describe_connector

Returns summary information about the connector.

SELECT
capacity,
connector_arn,
connector_configuration,
connector_description,
connector_name,
connector_state,
creation_time,
current_version,
kafka_cluster,
kafka_cluster_client_authentication,
kafka_cluster_encryption_in_transit,
kafka_connect_version,
log_delivery,
network_type,
plugins,
service_execution_role_arn,
state_description,
worker_configuration
FROM aws.kafkaconnect.connectors
WHERE connector_arn = '{{ connector_arn }}' -- required
AND region = '{{ region }}' -- required
;

list_connectors

Returns a list of all the connectors in this account and Region. The list is limited to connectors whose name starts with the specified prefix. The response also includes a description of each of the listed connectors.

SELECT
capacity,
connector_arn,
connector_description,
connector_name,
connector_state,
creation_time,
current_version,
kafka_cluster,
kafka_cluster_client_authentication,
kafka_cluster_encryption_in_transit,
kafka_connect_version,
log_delivery,
network_type,
plugins,
service_execution_role_arn,
worker_configuration
FROM aws.kafkaconnect.connectors
WHERE region = '{{ region }}' -- required
AND connectorNamePrefix = '{{ connectorNamePrefix }}'
AND maxResults = '{{ maxResults }}'
AND nextToken = '{{ nextToken }}'
;

INSERT examples

create_connector

Creates a connector using the specified properties.

INSERT INTO aws.kafkaconnect.connectors (
capacity,
connectorConfiguration,
connectorDescription,
connectorName,
kafkaCluster,
kafkaClusterClientAuthentication,
kafkaClusterEncryptionInTransit,
kafkaConnectVersion,
logDelivery,
networkType,
plugins,
serviceExecutionRoleArn,
workerConfiguration,
tags,
region
)
SELECT 
'{{ capacity }}' /* required */,
'{{ connectorConfiguration }}' /* required */,
'{{ connectorDescription }}',
'{{ connectorName }}' /* required */,
'{{ kafkaCluster }}' /* required */,
'{{ kafkaClusterClientAuthentication }}' /* required */,
'{{ kafkaClusterEncryptionInTransit }}' /* required */,
'{{ kafkaConnectVersion }}' /* required */,
'{{ logDelivery }}',
'{{ networkType }}',
'{{ plugins }}' /* required */,
'{{ serviceExecutionRoleArn }}' /* required */,
'{{ workerConfiguration }}',
'{{ tags }}',
'{{ region }}'
RETURNING
connector_arn,
connector_name,
connector_state
;

Manifest

# Description fields are for documentation purposes
- name: connectors
props:
  - name: region
    value: "{{ region }}"
    description: Required parameter for the connectors resource.
  - name: capacity
    description: |
      Information about the capacity of the connector, whether it is auto scaled or provisioned.
    value:
      autoScaling:
        maxWorkerCount: {{ maxWorkerCount }}
        mcuCount: {{ mcuCount }}
        minWorkerCount: {{ minWorkerCount }}
        scaleInPolicy:
          cpuUtilizationPercentage: {{ cpuUtilizationPercentage }}
        scaleOutPolicy:
          cpuUtilizationPercentage: {{ cpuUtilizationPercentage }}
        maxAutoscalingTaskCount: {{ maxAutoscalingTaskCount }}
      provisionedCapacity:
        mcuCount: {{ mcuCount }}
        workerCount: {{ workerCount }}
  - name: connectorConfiguration
    value: "{{ connectorConfiguration }}"
  - name: connectorDescription
    value: "{{ connectorDescription }}"
  - name: connectorName
    value: "{{ connectorName }}"
  - name: kafkaCluster
    description: |
      The details of the Apache Kafka cluster to which the connector is connected.
    value:
      apacheKafkaCluster:
        bootstrapServers: "{{ bootstrapServers }}"
        vpc:
          securityGroups:
            - "{{ securityGroups }}"
          subnets:
            - "{{ subnets }}"
  - name: kafkaClusterClientAuthentication
    description: |
      The client authentication information used in order to authenticate with the Apache Kafka cluster.
    value:
      authenticationType: "{{ authenticationType }}"
  - name: kafkaClusterEncryptionInTransit
    description: |
      Details of encryption in transit to the Apache Kafka cluster.
    value:
      encryptionType: "{{ encryptionType }}"
  - name: kafkaConnectVersion
    value: "{{ kafkaConnectVersion }}"
  - name: logDelivery
    description: |
      Details about log delivery.
    value:
      workerLogDelivery:
        cloudWatchLogs:
          enabled: {{ enabled }}
          logGroup: "{{ logGroup }}"
        firehose:
          deliveryStream: "{{ deliveryStream }}"
          enabled: {{ enabled }}
        s3:
          bucket: "{{ bucket }}"
          enabled: {{ enabled }}
          prefix: "{{ prefix }}"
  - name: networkType
    value: "{{ networkType }}"
    description: |
      The network type of a connector.
    valid_values: ['IPV4', 'DUAL']
  - name: plugins
    value:
      - customPlugin:
          customPluginArn: "{{ customPluginArn }}"
          revision: {{ revision }}
  - name: serviceExecutionRoleArn
    value: "{{ serviceExecutionRoleArn }}"
  - name: workerConfiguration
    description: |
      The configuration of the workers, which are the processes that run the connector logic.
    value:
      revision: {{ revision }}
      workerConfigurationArn: "{{ workerConfigurationArn }}"
  - name: tags
    value: "{{ tags }}"

UPDATE examples

update_connector

Updates the specified connector. For request body, specify only one parameter: either capacity or connectorConfiguration.

UPDATE aws.kafkaconnect.connectors
SET 
capacity = '{{ capacity }}',
connectorConfiguration = '{{ connectorConfiguration }}'
WHERE 
connector_arn = '{{ connector_arn }}' --required
AND currentVersion = '{{ currentVersion }}' --required
AND region = '{{ region }}' --required
RETURNING
connector_arn,
connector_operation_arn,
connector_state;

DELETE examples

delete_connector

Deletes the specified connector.

DELETE FROM aws.kafkaconnect.connectors
WHERE connector_arn = '{{ connector_arn }}' --required
AND region = '{{ region }}' --required
AND currentVersion = '{{ currentVersion }}'
;