virtual_clusters

Creates, updates, deletes, gets or lists a virtual_clusters resource.

Overview

Name	virtual_clusters
Type	Resource
Id	aws.emr_containers.virtual_clusters

Fields

The following fields are returned by SELECT queries:

describe_virtual_cluster

Name	Datatype	Description
id	string	The ID of the virtual cluster. (pattern: <code>[0-9a-z]+</code>)
name	string	The name of the virtual cluster. (pattern: <code>[.-_/#A-Za-z0-9]+</code>)
arn	string	The ARN of the virtual cluster. (pattern: <code>^arn:(aws[a-zA-Z0-9-]*):emr-containers:.+:(\d{12}):/virtualclusters/[0-9a-zA-Z]+$</code>)
container_provider	object	The information about the container provider.
created_at	string (date-time)	The date and time when the virtual cluster is created.
scheduler_configuration	object	The scheduler configuration for a virtual cluster on Amazon EMR on EKS. It controls how many job runs can run concurrently and how many can wait in the queue. When not set, no concurrency or queue limits are applied.
scheduler_status	object	The current in-queue and concurrent job-run counts for the virtual cluster.
security_configuration_id	string	The ID of the security configuration. (pattern: <code>[0-9a-z]+</code>)
session_enabled	boolean	Specifies whether the virtual cluster has session support enabled.
state	string	The state of the virtual cluster. (RUNNING, TERMINATING, TERMINATED, ARRESTED)
tags	object	The assigned tags of the virtual cluster.

list_virtual_clusters

Name	Datatype	Description
id	string	The ID of the virtual cluster. (pattern: <code>[0-9a-z]+</code>)
name	string	The name of the virtual cluster. (pattern: <code>[.-_/#A-Za-z0-9]+</code>)
arn	string	The ARN of the virtual cluster. (pattern: <code>^arn:(aws[a-zA-Z0-9-]*):emr-containers:.+:(\d{12}):/virtualclusters/[0-9a-zA-Z]+$</code>)
container_provider	object	The information about the container provider.
created_at	string (date-time)	The date and time when the virtual cluster is created.
scheduler_configuration	object	The scheduler configuration for a virtual cluster on Amazon EMR on EKS. It controls how many job runs can run concurrently and how many can wait in the queue. When not set, no concurrency or queue limits are applied.
scheduler_status	object	The current in-queue and concurrent job-run counts for the virtual cluster.
security_configuration_id	string	The ID of the security configuration. (pattern: <code>[0-9a-z]+</code>)
session_enabled	boolean	Specifies whether the virtual cluster has session support enabled.
state	string	The state of the virtual cluster. (RUNNING, TERMINATING, TERMINATED, ARRESTED)
tags	object	The assigned tags of the virtual cluster.

Methods

The following methods are available for this resource:

Name	Accessible by	Required Params	Optional Params	Description
describe_virtual_cluster	select	virtual_cluster_id, region		Displays detailed information about a specified virtual cluster. Virtual cluster is a managed entity on Amazon EMR on EKS. You can create, update, describe, list and delete virtual clusters. They do not consume any additional resource in your system. A single virtual cluster maps to a single Kubernetes namespace. Given this relationship, you can model virtual clusters the same way you model Kubernetes namespaces to meet your requirements.
list_virtual_clusters	select	region	containerProviderId, containerProviderType, createdAfter, createdBefore, states, maxResults, nextToken, eksAccessEntryIntegrated	Lists information about the specified virtual cluster. Virtual cluster is a managed entity on Amazon EMR on EKS. You can create, update, describe, list and delete virtual clusters. They do not consume any additional resource in your system. A single virtual cluster maps to a single Kubernetes namespace. Given this relationship, you can model virtual clusters the same way you model Kubernetes namespaces to meet your requirements.
create_virtual_cluster	insert	region, name, containerProvider, clientToken		Creates a virtual cluster. Virtual cluster is a managed entity on Amazon EMR on EKS. You can create, update, describe, list and delete virtual clusters. They do not consume any additional resource in your system. A single virtual cluster maps to a single Kubernetes namespace. Given this relationship, you can model virtual clusters the same way you model Kubernetes namespaces to meet your requirements.
update_virtual_cluster	update	virtual_cluster_id, region, clientToken		Updates a virtual cluster. Virtual cluster is a managed entity on Amazon EMR on EKS. You can create, update, describe, list and delete virtual clusters. They do not consume any additional resource in your system. A single virtual cluster maps to a single Kubernetes namespace. Given this relationship, you can model virtual clusters the same way you model Kubernetes namespaces to meet your requirements.
delete_virtual_cluster	delete	virtual_cluster_id, region		Deletes a virtual cluster. Virtual cluster is a managed entity on Amazon EMR on EKS. You can create, update, describe, list and delete virtual clusters. They do not consume any additional resource in your system. A single virtual cluster maps to a single Kubernetes namespace. Given this relationship, you can model virtual clusters the same way you model Kubernetes namespaces to meet your requirements.
cancel_job_run	exec	job_run_id, virtual_cluster_id, region		Cancels a job run. A job run is a unit of work, such as a Spark jar, PySpark script, or SparkSQL query, that you submit to Amazon EMR on EKS.
start_job_run	exec	virtual_cluster_id, region, clientToken		Starts a job run. A job run is a unit of work, such as a Spark jar, PySpark script, or SparkSQL query, that you submit to Amazon EMR on EKS.

Parameters

Parameters can be passed in the WHERE clause of a query. Check the Methods section to see which parameters are required or optional for each operation.

Name	Datatype	Description
job_run_id	string	The ID of the job run to cancel.
region	string	AWS region (default: us-east-1)
virtual_cluster_id	string	The virtual cluster ID for which the job run request is submitted.
containerProviderId	string	The container provider ID of the virtual cluster.
containerProviderType	string	The container provider type of the virtual cluster. Amazon EKS is the only supported type as of now.
createdAfter	string (date-time)	The date and time after which the virtual clusters are created.
createdBefore	string (date-time)	The date and time before which the virtual clusters are created.
eksAccessEntryIntegrated	boolean	Optional Boolean that specifies whether the operation should return the virtual clusters that have the access entry integration enabled or disabled. If not specified, the operation returns all applicable virtual clusters.
maxResults	integer	The maximum number of virtual clusters that can be listed.
nextToken	string	The token for the next set of virtual clusters to return.
states	array	The states of the requested virtual clusters.

SELECT examples

describe_virtual_cluster

Displays detailed information about a specified virtual cluster. Virtual cluster is a managed entity on Amazon EMR on EKS. You can create, update, describe, list and delete virtual clusters. They do not consume any additional resource in your system. A single virtual cluster maps to a single Kubernetes namespace. Given this relationship, you can model virtual clusters the same way you model Kubernetes namespaces to meet your requirements.

SELECT
id,
name,
arn,
container_provider,
created_at,
scheduler_configuration,
scheduler_status,
security_configuration_id,
session_enabled,
state,
tags
FROM aws.emr_containers.virtual_clusters
WHERE virtual_cluster_id = '{{ virtual_cluster_id }}' -- required
AND region = '{{ region }}' -- required
;

list_virtual_clusters

Lists information about the specified virtual cluster. Virtual cluster is a managed entity on Amazon EMR on EKS. You can create, update, describe, list and delete virtual clusters. They do not consume any additional resource in your system. A single virtual cluster maps to a single Kubernetes namespace. Given this relationship, you can model virtual clusters the same way you model Kubernetes namespaces to meet your requirements.

SELECT
id,
name,
arn,
container_provider,
created_at,
scheduler_configuration,
scheduler_status,
security_configuration_id,
session_enabled,
state,
tags
FROM aws.emr_containers.virtual_clusters
WHERE region = '{{ region }}' -- required
AND containerProviderId = '{{ containerProviderId }}'
AND containerProviderType = '{{ containerProviderType }}'
AND createdAfter = '{{ createdAfter }}'
AND createdBefore = '{{ createdBefore }}'
AND states = '{{ states }}'
AND maxResults = '{{ maxResults }}'
AND nextToken = '{{ nextToken }}'
AND eksAccessEntryIntegrated = '{{ eksAccessEntryIntegrated }}'
;

INSERT examples

create_virtual_cluster

Creates a virtual cluster. Virtual cluster is a managed entity on Amazon EMR on EKS. You can create, update, describe, list and delete virtual clusters. They do not consume any additional resource in your system. A single virtual cluster maps to a single Kubernetes namespace. Given this relationship, you can model virtual clusters the same way you model Kubernetes namespaces to meet your requirements.

INSERT INTO aws.emr_containers.virtual_clusters (
name,
containerProvider,
clientToken,
tags,
securityConfigurationId,
sessionEnabled,
schedulerConfiguration,
region
)
SELECT 
'{{ name }}' /* required */,
'{{ containerProvider }}' /* required */,
'{{ clientToken }}' /* required */,
'{{ tags }}',
'{{ securityConfigurationId }}',
{{ sessionEnabled }},
'{{ schedulerConfiguration }}',
'{{ region }}'
RETURNING
id,
name,
arn
;

Manifest

# Description fields are for documentation purposes
- name: virtual_clusters
props:
  - name: region
    value: "{{ region }}"
    description: Required parameter for the virtual_clusters resource.
  - name: name
    value: "{{ name }}"
  - name: containerProvider
    description: |
      The information about the container provider.
    value:
      type_: "{{ type_ }}"
      id: "{{ id }}"
      info:
        eksInfo:
          namespace: "{{ namespace }}"
          nodeLabel: "{{ nodeLabel }}"
  - name: clientToken
    value: "{{ clientToken }}"
  - name: tags
    value: "{{ tags }}"
  - name: securityConfigurationId
    value: "{{ securityConfigurationId }}"
  - name: sessionEnabled
    value: {{ sessionEnabled }}
  - name: schedulerConfiguration
    description: |
      The scheduler configuration for a virtual cluster on Amazon EMR on EKS. It controls how many job runs can run concurrently and how many can wait in the queue. When not set, no concurrency or queue limits are applied.
    value:
      maxInQueueJobRuns: {{ maxInQueueJobRuns }}
      maxConcurrentJobRuns: {{ maxConcurrentJobRuns }}

UPDATE examples

update_virtual_cluster

Updates a virtual cluster. Virtual cluster is a managed entity on Amazon EMR on EKS. You can create, update, describe, list and delete virtual clusters. They do not consume any additional resource in your system. A single virtual cluster maps to a single Kubernetes namespace. Given this relationship, you can model virtual clusters the same way you model Kubernetes namespaces to meet your requirements.

UPDATE aws.emr_containers.virtual_clusters
SET 
schedulerConfiguration = '{{ schedulerConfiguration }}',
clientToken = '{{ clientToken }}'
WHERE 
virtual_cluster_id = '{{ virtual_cluster_id }}' --required
AND region = '{{ region }}' --required
AND clientToken = '{{ clientToken }}' --required
RETURNING
virtual_cluster;

DELETE examples

delete_virtual_cluster

Deletes a virtual cluster. Virtual cluster is a managed entity on Amazon EMR on EKS. You can create, update, describe, list and delete virtual clusters. They do not consume any additional resource in your system. A single virtual cluster maps to a single Kubernetes namespace. Given this relationship, you can model virtual clusters the same way you model Kubernetes namespaces to meet your requirements.

DELETE FROM aws.emr_containers.virtual_clusters
WHERE virtual_cluster_id = '{{ virtual_cluster_id }}' --required
AND region = '{{ region }}' --required
;

Lifecycle Methods

cancel_job_run

Cancels a job run. A job run is a unit of work, such as a Spark jar, PySpark script, or SparkSQL query, that you submit to Amazon EMR on EKS.

EXEC aws.emr_containers.virtual_clusters.cancel_job_run 
@job_run_id='{{ job_run_id }}' --required, 
@virtual_cluster_id='{{ virtual_cluster_id }}' --required, 
@region='{{ region }}' --required
;

start_job_run

Starts a job run. A job run is a unit of work, such as a Spark jar, PySpark script, or SparkSQL query, that you submit to Amazon EMR on EKS.

EXEC aws.emr_containers.virtual_clusters.start_job_run 
@virtual_cluster_id='{{ virtual_cluster_id }}' --required, 
@region='{{ region }}' --required 
@@json=
'{
"name": "{{ name }}", 
"clientToken": "{{ clientToken }}", 
"executionRoleArn": "{{ executionRoleArn }}", 
"releaseLabel": "{{ releaseLabel }}", 
"jobDriver": "{{ jobDriver }}", 
"configurationOverrides": "{{ configurationOverrides }}", 
"tags": "{{ tags }}", 
"jobTemplateId": "{{ jobTemplateId }}", 
"jobTemplateParameters": "{{ jobTemplateParameters }}", 
"retryPolicyConfiguration": "{{ retryPolicyConfiguration }}"
}'
;