Table of Contents
Batch.Client¶A low-level client representing AWS Batch
Using Batch, you can run batch computing workloads on the Amazon Web Services Cloud. Batch computing is a common means for developers, scientists, and engineers to access large amounts of compute resources. Batch uses the advantages of this computing workload to remove the undifferentiated heavy lifting of configuring and managing required infrastructure. At the same time, it also adopts a familiar batch computing software approach. Given these advantages, Batch can help you to efficiently provision resources in response to jobs submitted, thus effectively helping you to eliminate capacity constraints, reduce compute costs, and deliver your results more quickly.
As a fully managed service, Batch can run batch computing workloads of any scale. Batch automatically provisions compute resources and optimizes workload distribution based on the quantity and scale of your specific workloads. With Batch, there's no need to install or manage batch computing software. This means that you can focus your time and energy on analyzing results and solving your specific problems.
import boto3
client = boto3.client('batch')
These are the available methods:
can_paginate()cancel_job()close()create_compute_environment()create_job_queue()create_scheduling_policy()delete_compute_environment()delete_job_queue()delete_scheduling_policy()deregister_job_definition()describe_compute_environments()describe_job_definitions()describe_job_queues()describe_jobs()describe_scheduling_policies()get_paginator()get_waiter()list_jobs()list_scheduling_policies()list_tags_for_resource()register_job_definition()submit_job()tag_resource()terminate_job()untag_resource()update_compute_environment()update_job_queue()update_scheduling_policy()can_paginate(operation_name)¶Check if an operation can be paginated.
create_foo, and you'd normally invoke the
operation as client.create_foo(**kwargs), if the
create_foo operation can be paginated, you can use the
call client.get_paginator("create_foo").True if the operation can be paginated,
False otherwise.cancel_job(**kwargs)¶Cancels a job in an Batch job queue. Jobs that are in the SUBMITTED , PENDING , or RUNNABLE state are canceled. Jobs that have progressed to STARTING or RUNNING aren't canceled, but the API operation still succeeds, even if no job is canceled. These jobs must be terminated with the TerminateJob operation.
See also: AWS API Documentation
Request Syntax
response = client.cancel_job(
jobId='string',
reason='string'
)
[REQUIRED]
The Batch job ID of the job to cancel.
[REQUIRED]
A message to attach to the job that explains the reason for canceling it. This message is returned by future DescribeJobs operations on the job. This message is also recorded in the Batch activity logs.
dict
Response Syntax
{}
Response Structure
Exceptions
Batch.Client.exceptions.ClientExceptionBatch.Client.exceptions.ServerExceptionExamples
This example cancels a job with the specified job ID.
response = client.cancel_job(
jobId='1d828f65-7a4d-42e8-996d-3b900ed59dc4',
reason='Cancelling job.',
)
print(response)
Expected Output:
{
'ResponseMetadata': {
'...': '...',
},
}
close()¶Closes underlying endpoint connections.
create_compute_environment(**kwargs)¶Creates an Batch compute environment. You can create MANAGED or UNMANAGED compute environments. MANAGED compute environments can use Amazon EC2 or Fargate resources. UNMANAGED compute environments can only use EC2 resources.
In a managed compute environment, Batch manages the capacity and instance types of the compute resources within the environment. This is based on the compute resource specification that you define or the launch template that you specify when you create the compute environment. Either, you can choose to use EC2 On-Demand Instances and EC2 Spot Instances. Or, you can use Fargate and Fargate Spot capacity in your managed compute environment. You can optionally set a maximum price so that Spot Instances only launch when the Spot Instance price is less than a specified percentage of the On-Demand price.
Note
Multi-node parallel jobs aren't supported on Spot Instances.
In an unmanaged compute environment, you can manage your own EC2 compute resources and have a lot of flexibility with how you configure your compute resources. For example, you can use custom AMIs. However, you must verify that each of your AMIs meet the Amazon ECS container instance AMI specification. For more information, see container instance AMIs in the Amazon Elastic Container Service Developer Guide . After you created your unmanaged compute environment, you can use the DescribeComputeEnvironments operation to find the Amazon ECS cluster that's associated with it. Then, launch your container instances into that Amazon ECS cluster. For more information, see Launching an Amazon ECS container instance in the Amazon Elastic Container Service Developer Guide .
Note
Batch doesn't automatically upgrade the AMIs in a compute environment after it's created. For example, it also doesn't update the AMIs in your compute environment when a newer version of the Amazon ECS optimized AMI is available. You're responsible for the management of the guest operating system. This includes any updates and security patches. You're also responsible for any additional application software or utilities that you install on the compute resources. There are two ways to use a new AMI for your Batch jobs. The original method is to complete these steps:
In April 2022, Batch added enhanced support for updating compute environments. For more information, see Updating compute environments . To use the enhanced updating of compute environments to update AMIs, follow these rules:
serviceRole ) parameter or set it to the AWSBatchServiceRole service-linked role.allocationStrategy ) parameter to BEST_FIT_PROGRESSIVE or SPOT_CAPACITY_OPTIMIZED .updateToLatestImageVersion ) parameter to true .imageId , imageIdOverride (in ` ec2Configuration https://docs.aws.amazon.com/batch/latest/APIReference/API_Ec2Configuration.html`__ ), or in the launch template (launchTemplate ). In that case Batch will select the latest Amazon ECS optimized AMI supported by Batch at the time the infrastructure update is initiated. Alternatively you can specify the AMI ID in the imageId or imageIdOverride parameters, or the launch template identified by the LaunchTemplate properties. Changing any of these properties will trigger an infrastructure update. If the AMI ID is specified in the launch template, it can not be replaced by specifying an AMI ID in either the imageId or imageIdOverride parameters. It can only be replaced by specifying a different launch template, or if the launch template version is set to $Default or $Latest , by setting either a new default version for the launch template (if $Default )or by adding a new version to the launch template (if $Latest ).If these rules are followed, any update that triggers an infrastructure update will cause the AMI ID to be re-selected. If the version setting in the launch template (launchTemplate ) is set to $Latest or $Default , the latest or default version of the launch template will be evaluated up at the time of the infrastructure update, even if the launchTemplate was not updated.
See also: AWS API Documentation
Request Syntax
response = client.create_compute_environment(
computeEnvironmentName='string',
type='MANAGED'|'UNMANAGED',
state='ENABLED'|'DISABLED',
unmanagedvCpus=123,
computeResources={
'type': 'EC2'|'SPOT'|'FARGATE'|'FARGATE_SPOT',
'allocationStrategy': 'BEST_FIT'|'BEST_FIT_PROGRESSIVE'|'SPOT_CAPACITY_OPTIMIZED',
'minvCpus': 123,
'maxvCpus': 123,
'desiredvCpus': 123,
'instanceTypes': [
'string',
],
'imageId': 'string',
'subnets': [
'string',
],
'securityGroupIds': [
'string',
],
'ec2KeyPair': 'string',
'instanceRole': 'string',
'tags': {
'string': 'string'
},
'placementGroup': 'string',
'bidPercentage': 123,
'spotIamFleetRole': 'string',
'launchTemplate': {
'launchTemplateId': 'string',
'launchTemplateName': 'string',
'version': 'string'
},
'ec2Configuration': [
{
'imageType': 'string',
'imageIdOverride': 'string'
},
]
},
serviceRole='string',
tags={
'string': 'string'
}
)
[REQUIRED]
The name for your compute environment. It can be up to 128 letters long. It can contain uppercase and lowercase letters, numbers, hyphens (-), and underscores (_).
[REQUIRED]
The type of the compute environment: MANAGED or UNMANAGED . For more information, see Compute Environments in the Batch User Guide .
The state of the compute environment. If the state is ENABLED , then the compute environment accepts jobs from a queue and can scale out automatically based on queues.
If the state is ENABLED , then the Batch scheduler can attempt to place jobs from an associated job queue on the compute resources within the environment. If the compute environment is managed, then it can scale its instances out or in automatically, based on the job queue demand.
If the state is DISABLED , then the Batch scheduler doesn't attempt to place jobs within the environment. Jobs in a STARTING or RUNNING state continue to progress normally. Managed compute environments in the DISABLED state don't scale out. However, they scale in to minvCpus value after instances become idle.
The maximum number of vCPUs for an unmanaged compute environment. This parameter is only used for fair share scheduling to reserve vCPU capacity for new share identifiers. If this parameter isn't provided for a fair share job queue, no vCPU capacity is reserved.
Note
This parameter is only supported when the type parameter is set to UNMANAGED .
Details about the compute resources managed by the compute environment. This parameter is required for managed compute environments. For more information, see Compute Environments in the Batch User Guide .
The type of compute environment: EC2 , SPOT , FARGATE , or FARGATE_SPOT . For more information, see Compute environments in the Batch User Guide .
If you choose SPOT , you must also specify an Amazon EC2 Spot Fleet role with the spotIamFleetRole parameter. For more information, see Amazon EC2 spot fleet role in the Batch User Guide .
The allocation strategy to use for the compute resource if not enough instances of the best fitting instance type can be allocated. This might be because of availability of the instance type in the Region or Amazon EC2 service limits . For more information, see Allocation strategies in the Batch User Guide .
Note
This parameter isn't applicable to jobs that are running on Fargate resources, and shouldn't be specified.
BEST_FIT (default)
Batch selects an instance type that best fits the needs of the jobs with a preference for the lowest-cost instance type. If additional instances of the selected instance type aren't available, Batch waits for the additional instances to be available. If there aren't enough instances available, or if the user is reaching Amazon EC2 service limits then additional jobs aren't run until the currently running jobs have completed. This allocation strategy keeps costs lower but can limit scaling. If you are using Spot Fleets with BEST_FIT then the Spot Fleet IAM Role must be specified. Compute resources that use a BEST_FIT allocation strategy don't support infrastructure updates and can't update some parameters. For more information, see Updating compute environments in the Batch User Guide .
BEST_FIT_PROGRESSIVE
Batch will select additional instance types that are large enough to meet the requirements of the jobs in the queue, with a preference for instance types with a lower cost per unit vCPU. If additional instances of the previously selected instance types aren't available, Batch will select new instance types.
SPOT_CAPACITY_OPTIMIZED
Batch will select one or more instance types that are large enough to meet the requirements of the jobs in the queue, with a preference for instance types that are less likely to be interrupted. This allocation strategy is only available for Spot Instance compute resources.
With both BEST_FIT_PROGRESSIVE and SPOT_CAPACITY_OPTIMIZED strategies, Batch might need to go above maxvCpus to meet your capacity requirements. In this event, Batch never exceeds maxvCpus by more than a single instance.
The minimum number of Amazon EC2 vCPUs that an environment should maintain (even if the compute environment is DISABLED ).
Note
This parameter isn't applicable to jobs that are running on Fargate resources, and shouldn't be specified.
The maximum number of Amazon EC2 vCPUs that a compute environment can reach.
Note
With both BEST_FIT_PROGRESSIVE and SPOT_CAPACITY_OPTIMIZED allocation strategies, Batch might need to exceed maxvCpus to meet your capacity requirements. In this event, Batch never exceeds maxvCpus by more than a single instance. For example, no more than a single instance from among those specified in your compute environment is allocated.
The desired number of Amazon EC2 vCPUS in the compute environment. Batch modifies this value between the minimum and maximum values, based on job queue demand.
Note
This parameter isn't applicable to jobs that are running on Fargate resources, and shouldn't be specified.
The instances types that can be launched. You can specify instance families to launch any instance type within those families (for example, c5 or p3 ), or you can specify specific sizes within a family (such as c5.8xlarge ). You can also choose optimal to select instance types (from the C4, M4, and R4 instance families) that match the demand of your job queues.
Note
This parameter isn't applicable to jobs that are running on Fargate resources, and shouldn't be specified.
Note
When you create a compute environment, the instance types that you select for the compute environment must share the same architecture. For example, you can't mix x86 and ARM instances in the same compute environment.
Note
Currently, optimal uses instance types from the C4, M4, and R4 instance families. In Regions that don't have instance types from those instance families, instance types from the C5, M5. and R5 instance families are used.
The Amazon Machine Image (AMI) ID used for instances launched in the compute environment. This parameter is overridden by the imageIdOverride member of the Ec2Configuration structure.
Note
This parameter isn't applicable to jobs that are running on Fargate resources, and shouldn't be specified.
Note
The AMI that you choose for a compute environment must match the architecture of the instance types that you intend to use for that compute environment. For example, if your compute environment uses A1 instance types, the compute resource AMI that you choose must support ARM instances. Amazon ECS vends both x86 and ARM versions of the Amazon ECS-optimized Amazon Linux 2 AMI. For more information, see Amazon ECS-optimized Amazon Linux 2 AMI in the Amazon Elastic Container Service Developer Guide .
The VPC subnets where the compute resources are launched. These subnets must be within the same VPC. Fargate compute resources can contain up to 16 subnets. For more information, see VPCs and subnets in the Amazon VPC User Guide .
The Amazon EC2 security groups associated with instances launched in the compute environment. One or more security groups must be specified, either in securityGroupIds or using a launch template referenced in launchTemplate . This parameter is required for jobs that are running on Fargate resources and must contain at least one security group. Fargate doesn't support launch templates. If security groups are specified using both securityGroupIds and launchTemplate , the values in securityGroupIds are used.
The Amazon EC2 key pair that's used for instances launched in the compute environment. You can use this key pair to log in to your instances with SSH.
Note
This parameter isn't applicable to jobs that are running on Fargate resources, and shouldn't be specified.
The Amazon ECS instance profile applied to Amazon EC2 instances in a compute environment. You can specify the short name or full Amazon Resource Name (ARN) of an instance profile. For example, `` ecsInstanceRole `` or ``arn:aws:iam::<aws_account_id> :instance-profile/ecsInstanceRole `` . For more information, see Amazon ECS instance role in the Batch User Guide .
Note
This parameter isn't applicable to jobs that are running on Fargate resources, and shouldn't be specified.
Key-value pair tags to be applied to EC2 resources that are launched in the compute environment. For Batch, these take the form of "String1": "String2", where String1 is the tag key and String2 is the tag value−for example, { "Name": "Batch Instance - C4OnDemand" } . This is helpful for recognizing your Batch instances in the Amazon EC2 console. Updating these tags requires an infrastructure update to the compute environment. For more information, see Updating compute environments in the Batch User Guide . These tags aren't seen when using the Batch ListTagsForResource API operation.
Note
This parameter isn't applicable to jobs that are running on Fargate resources, and shouldn't be specified.
The Amazon EC2 placement group to associate with your compute resources. If you intend to submit multi-node parallel jobs to your compute environment, you should consider creating a cluster placement group and associate it with your compute resources. This keeps your multi-node parallel job on a logical grouping of instances within a single Availability Zone with high network flow potential. For more information, see Placement groups in the Amazon EC2 User Guide for Linux Instances .
Note
This parameter isn't applicable to jobs that are running on Fargate resources, and shouldn't be specified.
The maximum percentage that a Spot Instance price can be when compared with the On-Demand price for that instance type before instances are launched. For example, if your maximum percentage is 20%, then the Spot price must be less than 20% of the current On-Demand price for that Amazon EC2 instance. You always pay the lowest (market) price and never more than your maximum percentage. If you leave this field empty, the default value is 100% of the On-Demand price.
Note
This parameter isn't applicable to jobs that are running on Fargate resources, and shouldn't be specified.
The Amazon Resource Name (ARN) of the Amazon EC2 Spot Fleet IAM role applied to a SPOT compute environment. This role is required if the allocation strategy set to BEST_FIT or if the allocation strategy isn't specified. For more information, see Amazon EC2 spot fleet role in the Batch User Guide .
Note
This parameter isn't applicable to jobs that are running on Fargate resources, and shouldn't be specified.
Warning
To tag your Spot Instances on creation, the Spot Fleet IAM role specified here must use the newer AmazonEC2SpotFleetTaggingRole managed policy. The previously recommended AmazonEC2SpotFleetRole managed policy doesn't have the required permissions to tag Spot Instances. For more information, see Spot instances not tagged on creation in the Batch User Guide .
The launch template to use for your compute resources. Any other compute resource parameters that you specify in a CreateComputeEnvironment API operation override the same parameters in the launch template. You must specify either the launch template ID or launch template name in the request, but not both. For more information, see Launch template support in the Batch User Guide .
Note
This parameter isn't applicable to jobs that are running on Fargate resources, and shouldn't be specified.
The ID of the launch template.
The name of the launch template.
The version number of the launch template, $Latest , or $Default .
If the value is $Latest , the latest version of the launch template is used. If the value is $Default , the default version of the launch template is used.
Warning
If the AMI ID that's used in a compute environment is from the launch template, the AMI isn't changed when the compute environment is updated. It's only changed if the updateToLatestImageVersion parameter for the compute environment is set to true . During an infrastructure update, if either $Latest or $Default is specified, Batch re-evaluates the launch template version, and it might use a different version of the launch template. This is the case even if the launch template isn't specified in the update. When updating a compute environment, changing the launch template requires an infrastructure update of the compute environment. For more information, see Updating compute environments in the Batch User Guide .
Default: $Default .
Provides information used to select Amazon Machine Images (AMIs) for EC2 instances in the compute environment. If Ec2Configuration isn't specified, the default is ECS_AL2 .
One or two values can be provided.
Note
This parameter isn't applicable to jobs that are running on Fargate resources, and shouldn't be specified.
Provides information used to select Amazon Machine Images (AMIs) for instances in the compute environment. If Ec2Configuration isn't specified, the default is ECS_AL2 (Amazon Linux 2 ).
Note
This object isn't applicable to jobs that are running on Fargate resources.
The image type to match with the instance type to select an AMI. If the imageIdOverride parameter isn't specified, then a recent Amazon ECS-optimized Amazon Linux 2 AMI (ECS_AL2 ) is used. If a new image type is specified in an update, but neither an imageId nor a imageIdOverride parameter is specified, then the latest Amazon ECS optimized AMI for that image type that's supported by Batch is used.
ECS_AL2Amazon Linux 2 − Default for all non-GPU instance families.
ECS_AL2_NVIDIAAmazon Linux 2 (GPU) −Default for all GPU instance families (for example
P4andG4) and can be used for all non Amazon Web Services Graviton-based instance types.ECS_AL1Amazon Linux . Amazon Linux is reaching the end-of-life of standard support. For more information, see Amazon Linux AMI .
The AMI ID used for instances launched in the compute environment that match the image type. This setting overrides the imageId set in the computeResource object.
Note
The AMI that you choose for a compute environment must match the architecture of the instance types that you intend to use for that compute environment. For example, if your compute environment uses A1 instance types, the compute resource AMI that you choose must support ARM instances. Amazon ECS vends both x86 and ARM versions of the Amazon ECS-optimized Amazon Linux 2 AMI. For more information, see Amazon ECS-optimized Amazon Linux 2 AMI in the Amazon Elastic Container Service Developer Guide .
The full Amazon Resource Name (ARN) of the IAM role that allows Batch to make calls to other Amazon Web Services services on your behalf. For more information, see Batch service IAM role in the Batch User Guide .
Warning
If your account already created the Batch service-linked role, that role is used by default for your compute environment unless you specify a different role here. If the Batch service-linked role doesn't exist in your account, and no role is specified here, the service attempts to create the Batch service-linked role in your account.
If your specified role has a path other than / , then you must specify either the full role ARN (recommended) or prefix the role name with the path. For example, if a role with the name bar has a path of /foo/ then you would specify /foo/bar as the role name. For more information, see Friendly names and paths in the IAM User Guide .
Note
Depending on how you created your Batch service role, its ARN might contain the service-role path prefix. When you only specify the name of the service role, Batch assumes that your ARN doesn't use the service-role path prefix. Because of this, we recommend that you specify the full ARN of your service role when you create compute environments.
The tags that you apply to the compute environment to help you categorize and organize your resources. Each tag consists of a key and an optional value. For more information, see Tagging Amazon Web Services Resources in Amazon Web Services General Reference .
These tags can be updated or removed using the TagResource and UntagResource API operations. These tags don't propagate to the underlying compute resources.
dict
Response Syntax
{
'computeEnvironmentName': 'string',
'computeEnvironmentArn': 'string'
}
Response Structure
(dict) --
computeEnvironmentName (string) --
The name of the compute environment. It can be up to 128 letters long. It can contain uppercase and lowercase letters, numbers, hyphens (-), and underscores (_).
computeEnvironmentArn (string) --
The Amazon Resource Name (ARN) of the compute environment.
Exceptions
Batch.Client.exceptions.ClientExceptionBatch.Client.exceptions.ServerExceptionExamples
This example creates a managed compute environment with specific C4 instance types that are launched on demand. The compute environment is called C4OnDemand.
response = client.create_compute_environment(
type='MANAGED',
computeEnvironmentName='C4OnDemand',
computeResources={
'type': 'EC2',
'desiredvCpus': 48,
'ec2KeyPair': 'id_rsa',
'instanceRole': 'ecsInstanceRole',
'instanceTypes': [
'c4.large',
'c4.xlarge',
'c4.2xlarge',
'c4.4xlarge',
'c4.8xlarge',
],
'maxvCpus': 128,
'minvCpus': 0,
'securityGroupIds': [
'sg-cf5093b2',
],
'subnets': [
'subnet-220c0e0a',
'subnet-1a95556d',
'subnet-978f6dce',
],
'tags': {
'Name': 'Batch Instance - C4OnDemand',
},
},
serviceRole='arn:aws:iam::012345678910:role/AWSBatchServiceRole',
state='ENABLED',
)
print(response)
Expected Output:
{
'computeEnvironmentArn': 'arn:aws:batch:us-east-1:012345678910:compute-environment/C4OnDemand',
'computeEnvironmentName': 'C4OnDemand',
'ResponseMetadata': {
'...': '...',
},
}
This example creates a managed compute environment with the M4 instance type that is launched when the Spot bid price is at or below 20% of the On-Demand price for the instance type. The compute environment is called M4Spot.
response = client.create_compute_environment(
type='MANAGED',
computeEnvironmentName='M4Spot',
computeResources={
'type': 'SPOT',
'bidPercentage': 20,
'desiredvCpus': 4,
'ec2KeyPair': 'id_rsa',
'instanceRole': 'ecsInstanceRole',
'instanceTypes': [
'm4',
],
'maxvCpus': 128,
'minvCpus': 0,
'securityGroupIds': [
'sg-cf5093b2',
],
'spotIamFleetRole': 'arn:aws:iam::012345678910:role/aws-ec2-spot-fleet-role',
'subnets': [
'subnet-220c0e0a',
'subnet-1a95556d',
'subnet-978f6dce',
],
'tags': {
'Name': 'Batch Instance - M4Spot',
},
},
serviceRole='arn:aws:iam::012345678910:role/AWSBatchServiceRole',
state='ENABLED',
)
print(response)
Expected Output:
{
'computeEnvironmentArn': 'arn:aws:batch:us-east-1:012345678910:compute-environment/M4Spot',
'computeEnvironmentName': 'M4Spot',
'ResponseMetadata': {
'...': '...',
},
}
create_job_queue(**kwargs)¶Creates an Batch job queue. When you create a job queue, you associate one or more compute environments to the queue and assign an order of preference for the compute environments.
You also set a priority to the job queue that determines the order that the Batch scheduler places jobs onto its associated compute environments. For example, if a compute environment is associated with more than one job queue, the job queue with a higher priority is given preference for scheduling jobs to that compute environment.
See also: AWS API Documentation
Request Syntax
response = client.create_job_queue(
jobQueueName='string',
state='ENABLED'|'DISABLED',
schedulingPolicyArn='string',
priority=123,
computeEnvironmentOrder=[
{
'order': 123,
'computeEnvironment': 'string'
},
],
tags={
'string': 'string'
}
)
[REQUIRED]
The name of the job queue. It can be up to 128 letters long. It can contain uppercase and lowercase letters, numbers, hyphens (-), and underscores (_).
ENABLED , it is able to accept jobs. If the job queue state is DISABLED , new jobs can't be added to the queue, but jobs already in the queue can finish.aws:*Partition* :batch:*Region* :*Account* :scheduling-policy/*Name* `` . An example is ``aws:aws:batch:us-west-2:012345678910:scheduling-policy/MySchedulingPolicy .[REQUIRED]
The priority of the job queue. Job queues with a higher priority (or a higher integer value for the priority parameter) are evaluated first when associated with the same compute environment. Priority is determined in descending order. For example, a job queue with a priority value of 10 is given scheduling preference over a job queue with a priority value of 1 . All of the compute environments must be either EC2 (EC2 or SPOT ) or Fargate (FARGATE or FARGATE_SPOT ); EC2 and Fargate compute environments can't be mixed.
[REQUIRED]
The set of compute environments mapped to a job queue and their order relative to each other. The job scheduler uses this parameter to determine which compute environment runs a specific job. Compute environments must be in the VALID state before you can associate them with a job queue. You can associate up to three compute environments with a job queue. All of the compute environments must be either EC2 (EC2 or SPOT ) or Fargate (FARGATE or FARGATE_SPOT ); EC2 and Fargate compute environments can't be mixed.
Note
All compute environments that are associated with a job queue must share the same architecture. Batch doesn't support mixing compute environment architecture types in a single job queue.
The order in which compute environments are tried for job placement within a queue. Compute environments are tried in ascending order. For example, if two compute environments are associated with a job queue, the compute environment with a lower order integer value is tried for job placement first. Compute environments must be in the VALID state before you can associate them with a job queue. All of the compute environments must be either EC2 (EC2 or SPOT ) or Fargate (FARGATE or FARGATE_SPOT ); EC2 and Fargate compute environments can't be mixed.
Note
All compute environments that are associated with a job queue must share the same architecture. Batch doesn't support mixing compute environment architecture types in a single job queue.
The order of the compute environment. Compute environments are tried in ascending order. For example, if two compute environments are associated with a job queue, the compute environment with a lower order integer value is tried for job placement first.
The Amazon Resource Name (ARN) of the compute environment.
The tags that you apply to the job queue to help you categorize and organize your resources. Each tag consists of a key and an optional value. For more information, see Tagging your Batch resources in Batch User Guide .
dict
Response Syntax
{
'jobQueueName': 'string',
'jobQueueArn': 'string'
}
Response Structure
(dict) --
jobQueueName (string) --
The name of the job queue.
jobQueueArn (string) --
The Amazon Resource Name (ARN) of the job queue.
Exceptions
Batch.Client.exceptions.ClientExceptionBatch.Client.exceptions.ServerExceptionExamples
This example creates a job queue called LowPriority that uses the M4Spot compute environment.
response = client.create_job_queue(
computeEnvironmentOrder=[
{
'computeEnvironment': 'M4Spot',
'order': 1,
},
],
jobQueueName='LowPriority',
priority=1,
state='ENABLED',
)
print(response)
Expected Output:
{
'jobQueueArn': 'arn:aws:batch:us-east-1:012345678910:job-queue/LowPriority',
'jobQueueName': 'LowPriority',
'ResponseMetadata': {
'...': '...',
},
}
This example creates a job queue called HighPriority that uses the C4OnDemand compute environment with an order of 1 and the M4Spot compute environment with an order of 2.
response = client.create_job_queue(
computeEnvironmentOrder=[
{
'computeEnvironment': 'C4OnDemand',
'order': 1,
},
{
'computeEnvironment': 'M4Spot',
'order': 2,
},
],
jobQueueName='HighPriority',
priority=10,
state='ENABLED',
)
print(response)
Expected Output:
{
'jobQueueArn': 'arn:aws:batch:us-east-1:012345678910:job-queue/HighPriority',
'jobQueueName': 'HighPriority',
'ResponseMetadata': {
'...': '...',
},
}
create_scheduling_policy(**kwargs)¶Creates an Batch scheduling policy.
See also: AWS API Documentation
Request Syntax
response = client.create_scheduling_policy(
name='string',
fairsharePolicy={
'shareDecaySeconds': 123,
'computeReservation': 123,
'shareDistribution': [
{
'shareIdentifier': 'string',
'weightFactor': ...
},
]
},
tags={
'string': 'string'
}
)
[REQUIRED]
The name of the scheduling policy. It can be up to 128 letters long. It can contain uppercase and lowercase letters, numbers, hyphens (-), and underscores (_).
The fair share policy of the scheduling policy.
The time period to use to calculate a fair share percentage for each fair share identifier in use, in seconds. A value of zero (0) indicates that only current usage should be measured. The decay allows for more recently run jobs to have more weight than jobs that ran earlier. The maximum supported value is 604800 (1 week).
A value used to reserve some of the available maximum vCPU for fair share identifiers that have not yet been used.
The reserved ratio is ``(computeReservation /100)^*ActiveFairShares* `` where `` ActiveFairShares `` is the number of active fair share identifiers.
For example, a computeReservation value of 50 indicates that Batch should reserve 50% of the maximum available vCPU if there is only one fair share identifier, 25% if there are two fair share identifiers, and 12.5% if there are three fair share identifiers. A computeReservation value of 25 indicates that Batch should reserve 25% of the maximum available vCPU if there is only one fair share identifier, 6.25% if there are two fair share identifiers, and 1.56% if there are three fair share identifiers.
The minimum value is 0 and the maximum value is 99.
An array of SharedIdentifier objects that contain the weights for the fair share identifiers for the fair share policy. Fair share identifiers that aren't included have a default weight of 1.0 .
Specifies the weights for the fair share identifiers for the fair share policy. Fair share identifiers that aren't included have a default weight of 1.0 .
A fair share identifier or fair share identifier prefix. If the string ends with an asterisk (*), this entry specifies the weight factor to use for fair share identifiers that start with that prefix. The list of fair share identifiers in a fair share policy cannot overlap. For example, you can't have one that specifies a shareIdentifier of UserA* and another that specifies a shareIdentifier of UserA-1 .
There can be no more than 500 fair share identifiers active in a job queue.
The string is limited to 255 alphanumeric characters, optionally followed by an asterisk (*).
The weight factor for the fair share identifier. The default value is 1.0. A lower value has a higher priority for compute resources. For example, jobs that use a share identifier with a weight factor of 0.125 (1/8) get 8 times the compute resources of jobs that use a share identifier with a weight factor of 1.
The smallest supported value is 0.0001, and the largest supported value is 999.9999.
The tags that you apply to the scheduling policy to help you categorize and organize your resources. Each tag consists of a key and an optional value. For more information, see Tagging Amazon Web Services Resources in Amazon Web Services General Reference .
These tags can be updated or removed using the TagResource and UntagResource API operations.
dict
Response Syntax
{
'name': 'string',
'arn': 'string'
}
Response Structure
(dict) --
name (string) --
The name of the scheduling policy.
arn (string) --
The Amazon Resource Name (ARN) of the scheduling policy. The format is aws:*Partition* :batch:*Region* :*Account* :scheduling-policy/*Name* `` . For example, ``aws:aws:batch:us-west-2:012345678910:scheduling-policy/MySchedulingPolicy .
Exceptions
Batch.Client.exceptions.ClientExceptionBatch.Client.exceptions.ServerExceptiondelete_compute_environment(**kwargs)¶Deletes an Batch compute environment.
Before you can delete a compute environment, you must set its state to DISABLED with the UpdateComputeEnvironment API operation and disassociate it from any job queues with the UpdateJobQueue API operation. Compute environments that use Fargate resources must terminate all active jobs on that compute environment before deleting the compute environment. If this isn't done, the compute environment enters an invalid state.
See also: AWS API Documentation
Request Syntax
response = client.delete_compute_environment(
computeEnvironment='string'
)
[REQUIRED]
The name or Amazon Resource Name (ARN) of the compute environment to delete.
{}
Response Structure
Exceptions
Batch.Client.exceptions.ClientExceptionBatch.Client.exceptions.ServerExceptionExamples
This example deletes the P2OnDemand compute environment.
response = client.delete_compute_environment(
computeEnvironment='P2OnDemand',
)
print(response)
Expected Output:
{
'ResponseMetadata': {
'...': '...',
},
}
delete_job_queue(**kwargs)¶Deletes the specified job queue. You must first disable submissions for a queue with the UpdateJobQueue operation. All jobs in the queue are eventually terminated when you delete a job queue. The jobs are terminated at a rate of about 16 jobs each second.
It's not necessary to disassociate compute environments from a queue before submitting a DeleteJobQueue request.
See also: AWS API Documentation
Request Syntax
response = client.delete_job_queue(
jobQueue='string'
)
[REQUIRED]
The short name or full Amazon Resource Name (ARN) of the queue to delete.
{}
Response Structure
Exceptions
Batch.Client.exceptions.ClientExceptionBatch.Client.exceptions.ServerExceptionExamples
This example deletes the GPGPU job queue.
response = client.delete_job_queue(
jobQueue='GPGPU',
)
print(response)
Expected Output:
{
'ResponseMetadata': {
'...': '...',
},
}
delete_scheduling_policy(**kwargs)¶Deletes the specified scheduling policy.
You can't delete a scheduling policy that's used in any job queues.
See also: AWS API Documentation
Request Syntax
response = client.delete_scheduling_policy(
arn='string'
)
[REQUIRED]
The Amazon Resource Name (ARN) of the scheduling policy to delete.
{}
Response Structure
Exceptions
Batch.Client.exceptions.ClientExceptionBatch.Client.exceptions.ServerExceptionderegister_job_definition(**kwargs)¶Deregisters an Batch job definition. Job definitions are permanently deleted after 180 days.
See also: AWS API Documentation
Request Syntax
response = client.deregister_job_definition(
jobDefinition='string'
)
[REQUIRED]
The name and revision (name:revision ) or full Amazon Resource Name (ARN) of the job definition to deregister.
{}
Response Structure
Exceptions
Batch.Client.exceptions.ClientExceptionBatch.Client.exceptions.ServerExceptionExamples
This example deregisters a job definition called sleep10.
response = client.deregister_job_definition(
jobDefinition='sleep10',
)
print(response)
Expected Output:
{
'ResponseMetadata': {
'...': '...',
},
}
describe_compute_environments(**kwargs)¶Describes one or more of your compute environments.
If you're using an unmanaged compute environment, you can use the DescribeComputeEnvironment operation to determine the ecsClusterArn that you launch your Amazon ECS container instances into.
See also: AWS API Documentation
Request Syntax
response = client.describe_compute_environments(
computeEnvironments=[
'string',
],
maxResults=123,
nextToken='string'
)
A list of up to 100 compute environment names or full Amazon Resource Name (ARN) entries.
DescribeComputeEnvironments in paginated output. When this parameter is used, DescribeComputeEnvironments only returns maxResults results in a single page along with a nextToken response element. The remaining results of the initial request can be seen by sending another DescribeComputeEnvironments request with the returned nextToken value. This value can be between 1 and 100. If this parameter isn't used, then DescribeComputeEnvironments returns up to 100 results and a nextToken value if applicable.The nextToken value returned from a previous paginated DescribeComputeEnvironments request where maxResults was used and the results exceeded the value of that parameter. Pagination continues from the end of the previous results that returned the nextToken value. This value is null when there are no more results to return.
Note
This token should be treated as an opaque identifier that's only used to retrieve the next items in a list and not for other programmatic purposes.
dict
Response Syntax
{
'computeEnvironments': [
{
'computeEnvironmentName': 'string',
'computeEnvironmentArn': 'string',
'unmanagedvCpus': 123,
'ecsClusterArn': 'string',
'tags': {
'string': 'string'
},
'type': 'MANAGED'|'UNMANAGED',
'state': 'ENABLED'|'DISABLED',
'status': 'CREATING'|'UPDATING'|'DELETING'|'DELETED'|'VALID'|'INVALID',
'statusReason': 'string',
'computeResources': {
'type': 'EC2'|'SPOT'|'FARGATE'|'FARGATE_SPOT',
'allocationStrategy': 'BEST_FIT'|'BEST_FIT_PROGRESSIVE'|'SPOT_CAPACITY_OPTIMIZED',
'minvCpus': 123,
'maxvCpus': 123,
'desiredvCpus': 123,
'instanceTypes': [
'string',
],
'imageId': 'string',
'subnets': [
'string',
],
'securityGroupIds': [
'string',
],
'ec2KeyPair': 'string',
'instanceRole': 'string',
'tags': {
'string': 'string'
},
'placementGroup': 'string',
'bidPercentage': 123,
'spotIamFleetRole': 'string',
'launchTemplate': {
'launchTemplateId': 'string',
'launchTemplateName': 'string',
'version': 'string'
},
'ec2Configuration': [
{
'imageType': 'string',
'imageIdOverride': 'string'
},
]
},
'serviceRole': 'string',
'updatePolicy': {
'terminateJobsOnUpdate': True|False,
'jobExecutionTimeoutMinutes': 123
}
},
],
'nextToken': 'string'
}
Response Structure
(dict) --
computeEnvironments (list) --
The list of compute environments.
(dict) --
An object representing an Batch compute environment.
computeEnvironmentName (string) --
The name of the compute environment. It can be up to 128 letters long. It can contain uppercase and lowercase letters, numbers, hyphens (-), and underscores (_).
computeEnvironmentArn (string) --
The Amazon Resource Name (ARN) of the compute environment.
unmanagedvCpus (integer) --
The maximum number of VCPUs expected to be used for an unmanaged compute environment.
ecsClusterArn (string) --
The Amazon Resource Name (ARN) of the underlying Amazon ECS cluster used by the compute environment.
tags (dict) --
The tags applied to the compute environment.
type (string) --
The type of the compute environment: MANAGED or UNMANAGED . For more information, see Compute environments in the Batch User Guide .
state (string) --
The state of the compute environment. The valid values are ENABLED or DISABLED .
If the state is ENABLED , then the Batch scheduler can attempt to place jobs from an associated job queue on the compute resources within the environment. If the compute environment is managed, then it can scale its instances out or in automatically, based on the job queue demand.
If the state is DISABLED , then the Batch scheduler doesn't attempt to place jobs within the environment. Jobs in a STARTING or RUNNING state continue to progress normally. Managed compute environments in the DISABLED state don't scale out. However, they scale in to minvCpus value after instances become idle.
status (string) --
The current status of the compute environment (for example, CREATING or VALID ).
statusReason (string) --
A short, human-readable string to provide additional details about the current status of the compute environment.
computeResources (dict) --
The compute resources defined for the compute environment. For more information, see Compute environments in the Batch User Guide .
type (string) --
The type of compute environment: EC2 , SPOT , FARGATE , or FARGATE_SPOT . For more information, see Compute environments in the Batch User Guide .
If you choose SPOT , you must also specify an Amazon EC2 Spot Fleet role with the spotIamFleetRole parameter. For more information, see Amazon EC2 spot fleet role in the Batch User Guide .
allocationStrategy (string) --
The allocation strategy to use for the compute resource if not enough instances of the best fitting instance type can be allocated. This might be because of availability of the instance type in the Region or Amazon EC2 service limits . For more information, see Allocation strategies in the Batch User Guide .
Note
This parameter isn't applicable to jobs that are running on Fargate resources, and shouldn't be specified.
BEST_FIT (default)
Batch selects an instance type that best fits the needs of the jobs with a preference for the lowest-cost instance type. If additional instances of the selected instance type aren't available, Batch waits for the additional instances to be available. If there aren't enough instances available, or if the user is reaching Amazon EC2 service limits then additional jobs aren't run until the currently running jobs have completed. This allocation strategy keeps costs lower but can limit scaling. If you are using Spot Fleets with BEST_FIT then the Spot Fleet IAM Role must be specified. Compute resources that use a BEST_FIT allocation strategy don't support infrastructure updates and can't update some parameters. For more information, see Updating compute environments in the Batch User Guide .
BEST_FIT_PROGRESSIVE
Batch will select additional instance types that are large enough to meet the requirements of the jobs in the queue, with a preference for instance types with a lower cost per unit vCPU. If additional instances of the previously selected instance types aren't available, Batch will select new instance types.
SPOT_CAPACITY_OPTIMIZED
Batch will select one or more instance types that are large enough to meet the requirements of the jobs in the queue, with a preference for instance types that are less likely to be interrupted. This allocation strategy is only available for Spot Instance compute resources.
With both BEST_FIT_PROGRESSIVE and SPOT_CAPACITY_OPTIMIZED strategies, Batch might need to go above maxvCpus to meet your capacity requirements. In this event, Batch never exceeds maxvCpus by more than a single instance.
minvCpus (integer) --
The minimum number of Amazon EC2 vCPUs that an environment should maintain (even if the compute environment is DISABLED ).
Note
This parameter isn't applicable to jobs that are running on Fargate resources, and shouldn't be specified.
maxvCpus (integer) --
The maximum number of Amazon EC2 vCPUs that a compute environment can reach.
Note
With both BEST_FIT_PROGRESSIVE and SPOT_CAPACITY_OPTIMIZED allocation strategies, Batch might need to exceed maxvCpus to meet your capacity requirements. In this event, Batch never exceeds maxvCpus by more than a single instance. For example, no more than a single instance from among those specified in your compute environment is allocated.
desiredvCpus (integer) --
The desired number of Amazon EC2 vCPUS in the compute environment. Batch modifies this value between the minimum and maximum values, based on job queue demand.
Note
This parameter isn't applicable to jobs that are running on Fargate resources, and shouldn't be specified.
instanceTypes (list) --
The instances types that can be launched. You can specify instance families to launch any instance type within those families (for example, c5 or p3 ), or you can specify specific sizes within a family (such as c5.8xlarge ). You can also choose optimal to select instance types (from the C4, M4, and R4 instance families) that match the demand of your job queues.
Note
This parameter isn't applicable to jobs that are running on Fargate resources, and shouldn't be specified.
Note
When you create a compute environment, the instance types that you select for the compute environment must share the same architecture. For example, you can't mix x86 and ARM instances in the same compute environment.
Note
Currently, optimal uses instance types from the C4, M4, and R4 instance families. In Regions that don't have instance types from those instance families, instance types from the C5, M5. and R5 instance families are used.
imageId (string) --
The Amazon Machine Image (AMI) ID used for instances launched in the compute environment. This parameter is overridden by the imageIdOverride member of the Ec2Configuration structure.
Note
This parameter isn't applicable to jobs that are running on Fargate resources, and shouldn't be specified.
Note
The AMI that you choose for a compute environment must match the architecture of the instance types that you intend to use for that compute environment. For example, if your compute environment uses A1 instance types, the compute resource AMI that you choose must support ARM instances. Amazon ECS vends both x86 and ARM versions of the Amazon ECS-optimized Amazon Linux 2 AMI. For more information, see Amazon ECS-optimized Amazon Linux 2 AMI in the Amazon Elastic Container Service Developer Guide .
subnets (list) --
The VPC subnets where the compute resources are launched. These subnets must be within the same VPC. Fargate compute resources can contain up to 16 subnets. For more information, see VPCs and subnets in the Amazon VPC User Guide .
securityGroupIds (list) --
The Amazon EC2 security groups associated with instances launched in the compute environment. One or more security groups must be specified, either in securityGroupIds or using a launch template referenced in launchTemplate . This parameter is required for jobs that are running on Fargate resources and must contain at least one security group. Fargate doesn't support launch templates. If security groups are specified using both securityGroupIds and launchTemplate , the values in securityGroupIds are used.
ec2KeyPair (string) --
The Amazon EC2 key pair that's used for instances launched in the compute environment. You can use this key pair to log in to your instances with SSH.
Note
This parameter isn't applicable to jobs that are running on Fargate resources, and shouldn't be specified.
instanceRole (string) --
The Amazon ECS instance profile applied to Amazon EC2 instances in a compute environment. You can specify the short name or full Amazon Resource Name (ARN) of an instance profile. For example, `` ecsInstanceRole `` or ``arn:aws:iam::<aws_account_id> :instance-profile/ecsInstanceRole `` . For more information, see Amazon ECS instance role in the Batch User Guide .
Note
This parameter isn't applicable to jobs that are running on Fargate resources, and shouldn't be specified.
tags (dict) --
Key-value pair tags to be applied to EC2 resources that are launched in the compute environment. For Batch, these take the form of "String1": "String2", where String1 is the tag key and String2 is the tag value−for example, { "Name": "Batch Instance - C4OnDemand" } . This is helpful for recognizing your Batch instances in the Amazon EC2 console. Updating these tags requires an infrastructure update to the compute environment. For more information, see Updating compute environments in the Batch User Guide . These tags aren't seen when using the Batch ListTagsForResource API operation.
Note
This parameter isn't applicable to jobs that are running on Fargate resources, and shouldn't be specified.
placementGroup (string) --
The Amazon EC2 placement group to associate with your compute resources. If you intend to submit multi-node parallel jobs to your compute environment, you should consider creating a cluster placement group and associate it with your compute resources. This keeps your multi-node parallel job on a logical grouping of instances within a single Availability Zone with high network flow potential. For more information, see Placement groups in the Amazon EC2 User Guide for Linux Instances .
Note
This parameter isn't applicable to jobs that are running on Fargate resources, and shouldn't be specified.
bidPercentage (integer) --
The maximum percentage that a Spot Instance price can be when compared with the On-Demand price for that instance type before instances are launched. For example, if your maximum percentage is 20%, then the Spot price must be less than 20% of the current On-Demand price for that Amazon EC2 instance. You always pay the lowest (market) price and never more than your maximum percentage. If you leave this field empty, the default value is 100% of the On-Demand price.
Note
This parameter isn't applicable to jobs that are running on Fargate resources, and shouldn't be specified.
spotIamFleetRole (string) --
The Amazon Resource Name (ARN) of the Amazon EC2 Spot Fleet IAM role applied to a SPOT compute environment. This role is required if the allocation strategy set to BEST_FIT or if the allocation strategy isn't specified. For more information, see Amazon EC2 spot fleet role in the Batch User Guide .
Note
This parameter isn't applicable to jobs that are running on Fargate resources, and shouldn't be specified.
Warning
To tag your Spot Instances on creation, the Spot Fleet IAM role specified here must use the newer AmazonEC2SpotFleetTaggingRole managed policy. The previously recommended AmazonEC2SpotFleetRole managed policy doesn't have the required permissions to tag Spot Instances. For more information, see Spot instances not tagged on creation in the Batch User Guide .
launchTemplate (dict) --
The launch template to use for your compute resources. Any other compute resource parameters that you specify in a CreateComputeEnvironment API operation override the same parameters in the launch template. You must specify either the launch template ID or launch template name in the request, but not both. For more information, see Launch template support in the Batch User Guide .
Note
This parameter isn't applicable to jobs that are running on Fargate resources, and shouldn't be specified.
launchTemplateId (string) --
The ID of the launch template.
launchTemplateName (string) --
The name of the launch template.
version (string) --
The version number of the launch template, $Latest , or $Default .
If the value is $Latest , the latest version of the launch template is used. If the value is $Default , the default version of the launch template is used.
Warning
If the AMI ID that's used in a compute environment is from the launch template, the AMI isn't changed when the compute environment is updated. It's only changed if the updateToLatestImageVersion parameter for the compute environment is set to true . During an infrastructure update, if either $Latest or $Default is specified, Batch re-evaluates the launch template version, and it might use a different version of the launch template. This is the case even if the launch template isn't specified in the update. When updating a compute environment, changing the launch template requires an infrastructure update of the compute environment. For more information, see Updating compute environments in the Batch User Guide .
Default: $Default .
ec2Configuration (list) --
Provides information used to select Amazon Machine Images (AMIs) for EC2 instances in the compute environment. If Ec2Configuration isn't specified, the default is ECS_AL2 .
One or two values can be provided.
Note
This parameter isn't applicable to jobs that are running on Fargate resources, and shouldn't be specified.
(dict) --
Provides information used to select Amazon Machine Images (AMIs) for instances in the compute environment. If Ec2Configuration isn't specified, the default is ECS_AL2 (Amazon Linux 2 ).
Note
This object isn't applicable to jobs that are running on Fargate resources.
imageType (string) --
The image type to match with the instance type to select an AMI. If the imageIdOverride parameter isn't specified, then a recent Amazon ECS-optimized Amazon Linux 2 AMI (ECS_AL2 ) is used. If a new image type is specified in an update, but neither an imageId nor a imageIdOverride parameter is specified, then the latest Amazon ECS optimized AMI for that image type that's supported by Batch is used.
ECS_AL2
Amazon Linux 2 − Default for all non-GPU instance families.
ECS_AL2_NVIDIA
Amazon Linux 2 (GPU) −Default for all GPU instance families (for example
P4andG4) and can be used for all non Amazon Web Services Graviton-based instance types.ECS_AL1
Amazon Linux . Amazon Linux is reaching the end-of-life of standard support. For more information, see Amazon Linux AMI .
imageIdOverride (string) --
The AMI ID used for instances launched in the compute environment that match the image type. This setting overrides the imageId set in the computeResource object.
Note
The AMI that you choose for a compute environment must match the architecture of the instance types that you intend to use for that compute environment. For example, if your compute environment uses A1 instance types, the compute resource AMI that you choose must support ARM instances. Amazon ECS vends both x86 and ARM versions of the Amazon ECS-optimized Amazon Linux 2 AMI. For more information, see Amazon ECS-optimized Amazon Linux 2 AMI in the Amazon Elastic Container Service Developer Guide .
serviceRole (string) --
The service role associated with the compute environment that allows Batch to make calls to Amazon Web Services API operations on your behalf. For more information, see Batch service IAM role in the Batch User Guide .
updatePolicy (dict) --
Specifies the infrastructure update policy for the compute environment. For more information about infrastructure updates, see Updating compute environments in the Batch User Guide .
terminateJobsOnUpdate (boolean) --
Specifies whether jobs are automatically terminated when the computer environment infrastructure is updated. The default value is false .
jobExecutionTimeoutMinutes (integer) --
Specifies the job timeout, in minutes, when the compute environment infrastructure is updated. The default value is 30.
nextToken (string) --
The nextToken value to include in a future DescribeComputeEnvironments request. When the results of a DescribeComputeEnvironments request exceed maxResults , this value can be used to retrieve the next page of results. This value is null when there are no more results to return.
Exceptions
Batch.Client.exceptions.ClientExceptionBatch.Client.exceptions.ServerExceptionExamples
This example describes the P2OnDemand compute environment.
response = client.describe_compute_environments(
computeEnvironments=[
'P2OnDemand',
],
)
print(response)
Expected Output:
{
'computeEnvironments': [
{
'type': 'MANAGED',
'computeEnvironmentArn': 'arn:aws:batch:us-east-1:012345678910:compute-environment/P2OnDemand',
'computeEnvironmentName': 'P2OnDemand',
'computeResources': {
'type': 'EC2',
'desiredvCpus': 48,
'ec2KeyPair': 'id_rsa',
'instanceRole': 'ecsInstanceRole',
'instanceTypes': [
'p2',
],
'maxvCpus': 128,
'minvCpus': 0,
'securityGroupIds': [
'sg-cf5093b2',
],
'subnets': [
'subnet-220c0e0a',
'subnet-1a95556d',
'subnet-978f6dce',
],
'tags': {
'Name': 'Batch Instance - P2OnDemand',
},
},
'ecsClusterArn': 'arn:aws:ecs:us-east-1:012345678910:cluster/P2OnDemand_Batch_2c06f29d-d1fe-3a49-879d-42394c86effc',
'serviceRole': 'arn:aws:iam::012345678910:role/AWSBatchServiceRole',
'state': 'ENABLED',
'status': 'VALID',
'statusReason': 'ComputeEnvironment Healthy',
},
],
'ResponseMetadata': {
'...': '...',
},
}
describe_job_definitions(**kwargs)¶Describes a list of job definitions. You can specify a status (such as ACTIVE ) to only return job definitions that match that status.
See also: AWS API Documentation
Request Syntax
response = client.describe_job_definitions(
jobDefinitions=[
'string',
],
maxResults=123,
jobDefinitionName='string',
status='string',
nextToken='string'
)
A list of up to 100 job definitions. Each entry in the list can either be an ARN in the format arn:aws:batch:${Region}:${Account}:job-definition/${JobDefinitionName}:${Revision} or a short version using the form ${JobDefinitionName}:${Revision} .
DescribeJobDefinitions in paginated output. When this parameter is used, DescribeJobDefinitions only returns maxResults results in a single page and a nextToken response element. The remaining results of the initial request can be seen by sending another DescribeJobDefinitions request with the returned nextToken value. This value can be between 1 and 100. If this parameter isn't used, then DescribeJobDefinitions returns up to 100 results and a nextToken value if applicable.The nextToken value returned from a previous paginated DescribeJobDefinitions request where maxResults was used and the results exceeded the value of that parameter. Pagination continues from the end of the previous results that returned the nextToken value. This value is null when there are no more results to return.
Note
This token should be treated as an opaque identifier that's only used to retrieve the next items in a list and not for other programmatic purposes.
dict
Response Syntax
{
'jobDefinitions': [
{
'jobDefinitionName': 'string',
'jobDefinitionArn': 'string',
'revision': 123,
'status': 'string',
'type': 'string',
'schedulingPriority': 123,
'parameters': {
'string': 'string'
},
'retryStrategy': {
'attempts': 123,
'evaluateOnExit': [
{
'onStatusReason': 'string',
'onReason': 'string',
'onExitCode': 'string',
'action': 'RETRY'|'EXIT'
},
]
},
'containerProperties': {
'image': 'string',
'vcpus': 123,
'memory': 123,
'command': [
'string',
],
'jobRoleArn': 'string',
'executionRoleArn': 'string',
'volumes': [
{
'host': {
'sourcePath': 'string'
},
'name': 'string',
'efsVolumeConfiguration': {
'fileSystemId': 'string',
'rootDirectory': 'string',
'transitEncryption': 'ENABLED'|'DISABLED',
'transitEncryptionPort': 123,
'authorizationConfig': {
'accessPointId': 'string',
'iam': 'ENABLED'|'DISABLED'
}
}
},
],
'environment': [
{
'name': 'string',
'value': 'string'
},
],
'mountPoints': [
{
'containerPath': 'string',
'readOnly': True|False,
'sourceVolume': 'string'
},
],
'readonlyRootFilesystem': True|False,
'privileged': True|False,
'ulimits': [
{
'hardLimit': 123,
'name': 'string',
'softLimit': 123
},
],
'user': 'string',
'instanceType': 'string',
'resourceRequirements': [
{
'value': 'string',
'type': 'GPU'|'VCPU'|'MEMORY'
},
],
'linuxParameters': {
'devices': [
{
'hostPath': 'string',
'containerPath': 'string',
'permissions': [
'READ'|'WRITE'|'MKNOD',
]
},
],
'initProcessEnabled': True|False,
'sharedMemorySize': 123,
'tmpfs': [
{
'containerPath': 'string',
'size': 123,
'mountOptions': [
'string',
]
},
],
'maxSwap': 123,
'swappiness': 123
},
'logConfiguration': {
'logDriver': 'json-file'|'syslog'|'journald'|'gelf'|'fluentd'|'awslogs'|'splunk',
'options': {
'string': 'string'
},
'secretOptions': [
{
'name': 'string',
'valueFrom': 'string'
},
]
},
'secrets': [
{
'name': 'string',
'valueFrom': 'string'
},
],
'networkConfiguration': {
'assignPublicIp': 'ENABLED'|'DISABLED'
},
'fargatePlatformConfiguration': {
'platformVersion': 'string'
}
},
'timeout': {
'attemptDurationSeconds': 123
},
'nodeProperties': {
'numNodes': 123,
'mainNode': 123,
'nodeRangeProperties': [
{
'targetNodes': 'string',
'container': {
'image': 'string',
'vcpus': 123,
'memory': 123,
'command': [
'string',
],
'jobRoleArn': 'string',
'executionRoleArn': 'string',
'volumes': [
{
'host': {
'sourcePath': 'string'
},
'name': 'string',
'efsVolumeConfiguration': {
'fileSystemId': 'string',
'rootDirectory': 'string',
'transitEncryption': 'ENABLED'|'DISABLED',
'transitEncryptionPort': 123,
'authorizationConfig': {
'accessPointId': 'string',
'iam': 'ENABLED'|'DISABLED'
}
}
},
],
'environment': [
{
'name': 'string',
'value': 'string'
},
],
'mountPoints': [
{
'containerPath': 'string',
'readOnly': True|False,
'sourceVolume': 'string'
},
],
'readonlyRootFilesystem': True|False,
'privileged': True|False,
'ulimits': [
{
'hardLimit': 123,
'name': 'string',
'softLimit': 123
},
],
'user': 'string',
'instanceType': 'string',
'resourceRequirements': [
{
'value': 'string',
'type': 'GPU'|'VCPU'|'MEMORY'
},
],
'linuxParameters': {
'devices': [
{
'hostPath': 'string',
'containerPath': 'string',
'permissions': [
'READ'|'WRITE'|'MKNOD',
]
},
],
'initProcessEnabled': True|False,
'sharedMemorySize': 123,
'tmpfs': [
{
'containerPath': 'string',
'size': 123,
'mountOptions': [
'string',
]
},
],
'maxSwap': 123,
'swappiness': 123
},
'logConfiguration': {
'logDriver': 'json-file'|'syslog'|'journald'|'gelf'|'fluentd'|'awslogs'|'splunk',
'options': {
'string': 'string'
},
'secretOptions': [
{
'name': 'string',
'valueFrom': 'string'
},
]
},
'secrets': [
{
'name': 'string',
'valueFrom': 'string'
},
],
'networkConfiguration': {
'assignPublicIp': 'ENABLED'|'DISABLED'
},
'fargatePlatformConfiguration': {
'platformVersion': 'string'
}
}
},
]
},
'tags': {
'string': 'string'
},
'propagateTags': True|False,
'platformCapabilities': [
'EC2'|'FARGATE',
]
},
],
'nextToken': 'string'
}
Response Structure
(dict) --
jobDefinitions (list) --
The list of job definitions.
(dict) --
An object representing an Batch job definition.
jobDefinitionName (string) --
The name of the job definition.
jobDefinitionArn (string) --
The Amazon Resource Name (ARN) for the job definition.
revision (integer) --
The revision of the job definition.
status (string) --
The status of the job definition.
type (string) --
The type of job definition, either container or multinode . If the job is run on Fargate resources, then multinode isn't supported. For more information about multi-node parallel jobs, see Creating a multi-node parallel job definition in the Batch User Guide .
schedulingPriority (integer) --
The scheduling priority of the job definition. This only affects jobs in job queues with a fair share policy. Jobs with a higher scheduling priority are scheduled before jobs with a lower scheduling priority.
parameters (dict) --
Default parameters or parameter substitution placeholders that are set in the job definition. Parameters are specified as a key-value pair mapping. Parameters in a SubmitJob request override any corresponding parameter defaults from the job definition. For more information about specifying parameters, see Job definition parameters in the Batch User Guide .
retryStrategy (dict) --
The retry strategy to use for failed jobs that are submitted with this job definition.
attempts (integer) --
The number of times to move a job to the RUNNABLE status. You can specify between 1 and 10 attempts. If the value of attempts is greater than one, the job is retried on failure the same number of attempts as the value.
evaluateOnExit (list) --
Array of up to 5 objects that specify conditions under which the job should be retried or failed. If this parameter is specified, then the attempts parameter must also be specified.
(dict) --
Specifies a set of conditions to be met, and an action to take (RETRY or EXIT ) if all conditions are met.
onStatusReason (string) --
Contains a glob pattern to match against the StatusReason returned for a job. The pattern can be up to 512 characters in length. It can contain letters, numbers, periods (.), colons (:), and white space (including spaces or tabs). It can optionally end with an asterisk (*) so that only the start of the string needs to be an exact match.
The string can be between 1 and 512 characters in length.
onReason (string) --
Contains a glob pattern to match against the Reason returned for a job. The pattern can be up to 512 characters in length. It can contain letters, numbers, periods (.), colons (:), and white space (including spaces and tabs). It can optionally end with an asterisk (*) so that only the start of the string needs to be an exact match.
The string can be between 1 and 512 characters in length.
onExitCode (string) --
Contains a glob pattern to match against the decimal representation of the ExitCode returned for a job. The pattern can be up to 512 characters in length. It can contain only numbers, and can optionally end with an asterisk (*) so that only the start of the string needs to be an exact match.
The string can be between 1 and 512 characters in length.
action (string) --
Specifies the action to take if all of the specified conditions (onStatusReason , onReason , and onExitCode ) are met. The values aren't case sensitive.
containerProperties (dict) --
An object with various properties specific to container-based jobs.
image (string) --
The image used to start a container. This string is passed directly to the Docker daemon. Images in the Docker Hub registry are available by default. Other repositories are specified with `` repository-url /image :tag `` . Up to 255 letters (uppercase and lowercase), numbers, hyphens, underscores, colons, periods, forward slashes, and number signs are allowed. This parameter maps to Image in the Create a container section of the Docker Remote API and the IMAGE parameter of docker run .
Note
Docker image architecture must match the processor architecture of the compute resources that they're scheduled on. For example, ARM-based Docker images can only run on ARM-based compute resources.
registry/repository[:tag] or registry/repository[@digest] naming conventions. For example, ``public.ecr.aws/registry_alias /my-web-app :latest `` .012345678910.dkr.ecr.<region-name>.amazonaws.com/<repository-name> ).ubuntu or mongo ).amazon/amazon-ecs-agent ).quay.io/assemblyline/ubuntu ).vcpus (integer) --
This parameter is deprecated, use resourceRequirements to specify the vCPU requirements for the job definition. It's not supported for jobs running on Fargate resources. For jobs running on EC2 resources, it specifies the number of vCPUs reserved for the job.
Each vCPU is equivalent to 1,024 CPU shares. This parameter maps to CpuShares in the Create a container section of the Docker Remote API and the --cpu-shares option to docker run . The number of vCPUs must be specified but can be specified in several places. You must specify it at least once for each node.
memory (integer) --
This parameter is deprecated, use resourceRequirements to specify the memory requirements for the job definition. It's not supported for jobs running on Fargate resources. For jobs running on EC2 resources, it specifies the memory hard limit (in MiB) for a container. If your container attempts to exceed the specified number, it's terminated. You must specify at least 4 MiB of memory for a job using this parameter. The memory hard limit can be specified in several places. It must be specified for each node at least once.
command (list) --
The command that's passed to the container. This parameter maps to Cmd in the Create a container section of the Docker Remote API and the COMMAND parameter to docker run . For more information, see https://docs.docker.com/engine/reference/builder/#cmd .
jobRoleArn (string) --
The Amazon Resource Name (ARN) of the IAM role that the container can assume for Amazon Web Services permissions. For more information, see IAM roles for tasks in the Amazon Elastic Container Service Developer Guide .
executionRoleArn (string) --
The Amazon Resource Name (ARN) of the execution role that Batch can assume. For jobs that run on Fargate resources, you must provide an execution role. For more information, see Batch execution IAM role in the Batch User Guide .
volumes (list) --
A list of data volumes used in a job.
(dict) --
A data volume used in a job's container properties.
host (dict) --
The contents of the host parameter determine whether your data volume persists on the host container instance and where it is stored. If the host parameter is empty, then the Docker daemon assigns a host path for your data volume. However, the data isn't guaranteed to persist after the containers associated with it stop running.
Note
This parameter isn't applicable to jobs that are running on Fargate resources and shouldn't be provided.
sourcePath (string) --
The path on the host container instance that's presented to the container. If this parameter is empty, then the Docker daemon has assigned a host path for you. If this parameter contains a file location, then the data volume persists at the specified location on the host container instance until you delete it manually. If the source path location doesn't exist on the host container instance, the Docker daemon creates it. If the location does exist, the contents of the source path folder are exported.
Note
This parameter isn't applicable to jobs that run on Fargate resources and shouldn't be provided.
name (string) --
The name of the volume. It can be up to 255 letters long. It can contain uppercase and lowercase letters, numbers, hyphens (-), and underscores (_). This name is referenced in the sourceVolume parameter of container definition mountPoints .
efsVolumeConfiguration (dict) --
This parameter is specified when you are using an Amazon Elastic File System file system for job storage. Jobs that are running on Fargate resources must specify a platformVersion of at least 1.4.0 .
fileSystemId (string) --
The Amazon EFS file system ID to use.
rootDirectory (string) --
The directory within the Amazon EFS file system to mount as the root directory inside the host. If this parameter is omitted, the root of the Amazon EFS volume is used instead. Specifying / has the same effect as omitting this parameter. The maximum length is 4,096 characters.
Warning
If an EFS access point is specified in the authorizationConfig , the root directory parameter must either be omitted or set to / , which enforces the path set on the Amazon EFS access point.
transitEncryption (string) --
Determines whether to enable encryption for Amazon EFS data in transit between the Amazon ECS host and the Amazon EFS server. Transit encryption must be enabled if Amazon EFS IAM authorization is used. If this parameter is omitted, the default value of DISABLED is used. For more information, see Encrypting data in transit in the Amazon Elastic File System User Guide .
transitEncryptionPort (integer) --
The port to use when sending encrypted data between the Amazon ECS host and the Amazon EFS server. If you don't specify a transit encryption port, it uses the port selection strategy that the Amazon EFS mount helper uses. The value must be between 0 and 65,535. For more information, see EFS mount helper in the Amazon Elastic File System User Guide .
authorizationConfig (dict) --
The authorization configuration details for the Amazon EFS file system.
accessPointId (string) --
The Amazon EFS access point ID to use. If an access point is specified, the root directory value specified in the EFSVolumeConfiguration must either be omitted or set to / which will enforce the path set on the EFS access point. If an access point is used, transit encryption must be enabled in the EFSVolumeConfiguration . For more information, see Working with Amazon EFS access points in the Amazon Elastic File System User Guide .
iam (string) --
Whether or not to use the Batch job IAM role defined in a job definition when mounting the Amazon EFS file system. If enabled, transit encryption must be enabled in the EFSVolumeConfiguration . If this parameter is omitted, the default value of DISABLED is used. For more information, see Using Amazon EFS access points in the Batch User Guide . EFS IAM authorization requires that TransitEncryption be ENABLED and that a JobRoleArn is specified.
environment (list) --
The environment variables to pass to a container. This parameter maps to Env in the Create a container section of the Docker Remote API and the --env option to docker run .
Warning
We don't recommend using plaintext environment variables for sensitive information, such as credential data.
Note
Environment variables must not start with AWS_BATCH ; this naming convention is reserved for variables that are set by the Batch service.
(dict) --
A key-value pair object.
name (string) --
The name of the key-value pair. For environment variables, this is the name of the environment variable.
value (string) --
The value of the key-value pair. For environment variables, this is the value of the environment variable.
mountPoints (list) --
The mount points for data volumes in your container. This parameter maps to Volumes in the Create a container section of the Docker Remote API and the --volume option to docker run .
(dict) --
Details on a Docker volume mount point that's used in a job's container properties. This parameter maps to Volumes in the Create a container section of the Docker Remote API and the --volume option to docker run.
containerPath (string) --
The path on the container where the host volume is mounted.
readOnly (boolean) --
If this value is true , the container has read-only access to the volume. Otherwise, the container can write to the volume. The default value is false .
sourceVolume (string) --
The name of the volume to mount.
readonlyRootFilesystem (boolean) --
When this parameter is true, the container is given read-only access to its root file system. This parameter maps to ReadonlyRootfs in the Create a container section of the Docker Remote API and the --read-only option to docker run .
privileged (boolean) --
When this parameter is true, the container is given elevated permissions on the host container instance (similar to the root user). This parameter maps to Privileged in the Create a container section of the Docker Remote API and the --privileged option to docker run . The default value is false.
Note
This parameter isn't applicable to jobs that are running on Fargate resources and shouldn't be provided, or specified as false.
ulimits (list) --
A list of ulimits to set in the container. This parameter maps to Ulimits in the Create a container section of the Docker Remote API and the --ulimit option to docker run .
Note
This parameter isn't applicable to jobs that are running on Fargate resources and shouldn't be provided.
(dict) --
The ulimit settings to pass to the container.
Note
This object isn't applicable to jobs that are running on Fargate resources.
hardLimit (integer) --
The hard limit for the ulimit type.
name (string) --
The type of the ulimit .
softLimit (integer) --
The soft limit for the ulimit type.
user (string) --
The user name to use inside the container. This parameter maps to User in the Create a container section of the Docker Remote API and the --user option to docker run .
instanceType (string) --
The instance type to use for a multi-node parallel job. All node groups in a multi-node parallel job must use the same instance type.
Note
This parameter isn't applicable to single-node container jobs or jobs that run on Fargate resources, and shouldn't be provided.
resourceRequirements (list) --
The type and amount of resources to assign to a container. The supported resources include GPU , MEMORY , and VCPU .
(dict) --
The type and amount of a resource to assign to a container. The supported resources include GPU , MEMORY , and VCPU .
value (string) --
The quantity of the specified resource to reserve for the container. The values vary based on the type specified.
type="GPU"
The number of physical GPUs to reserve for the container. The number of GPUs reserved for all containers in a job shouldn't exceed the number of available GPUs on the compute resource that the job is launched on.
Note
GPUs are not available for jobs that are running on Fargate resources.
type="MEMORY"
The memory hard limit (in MiB) present to the container. This parameter is supported for jobs that are running on EC2 resources. If your container attempts to exceed the memory specified, the container is terminated. This parameter maps to Memory in the Create a container section of the Docker Remote API and the --memory option to docker run . You must specify at least 4 MiB of memory for a job. This is required but can be specified in several places for multi-node parallel (MNP) jobs. It must be specified for each node at least once. This parameter maps to Memory in the Create a container section of the Docker Remote API and the --memory option to docker run .
Note
If you're trying to maximize your resource utilization by providing your jobs as much memory as possible for a particular instance type, see Memory management in the Batch User Guide .
For jobs that are running on Fargate resources, then value is the hard limit (in MiB), and must match one of the supported values and the VCPU values must be one of the values supported for that memory value.
value = 512
VCPU= 0.25value = 1024
VCPU= 0.25 or 0.5value = 2048
VCPU= 0.25, 0.5, or 1value = 3072
VCPU= 0.5, or 1value = 4096
VCPU= 0.5, 1, or 2value = 5120, 6144, or 7168
VCPU= 1 or 2value = 8192
VCPU= 1, 2, or 4value = 9216, 10240, 11264, 12288, 13312, 14336, 15360, or 16384
VCPU= 2 or 4value = 17408, 18432, 19456, 20480, 21504, 22528, 23552, 24576, 25600, 26624, 27648, 28672, 29696, or 30720
VCPU= 4type="VCPU"
The number of vCPUs reserved for the container. This parameter maps to CpuShares in the Create a container section of the Docker Remote API and the --cpu-shares option to docker run . Each vCPU is equivalent to 1,024 CPU shares. For EC2 resources, you must specify at least one vCPU. This is required but can be specified in several places; it must be specified for each node at least once.
For jobs that are running on Fargate resources, then value must match one of the supported values and the MEMORY values must be one of the values supported for that VCPU value. The supported values are 0.25, 0.5, 1, 2, and 4
value = 0.25
MEMORY= 512, 1024, or 2048value = 0.5
MEMORY= 1024, 2048, 3072, or 4096value = 1
MEMORY= 2048, 3072, 4096, 5120, 6144, 7168, or 8192value = 2
MEMORY= 4096, 5120, 6144, 7168, 8192, 9216, 10240, 11264, 12288, 13312, 14336, 15360, or 16384value = 4
MEMORY= 8192, 9216, 10240, 11264, 12288, 13312, 14336, 15360, 16384, 17408, 18432, 19456, 20480, 21504, 22528, 23552, 24576, 25600, 26624, 27648, 28672, 29696, or 30720
type (string) --
The type of resource to assign to a container. The supported resources include GPU , MEMORY , and VCPU .
linuxParameters (dict) --
Linux-specific modifications that are applied to the container, such as details for device mappings.
devices (list) --
Any host devices to expose to the container. This parameter maps to Devices in the Create a container section of the Docker Remote API and the --device option to docker run .
Note
This parameter isn't applicable to jobs that are running on Fargate resources and shouldn't be provided.
(dict) --
An object representing a container instance host device.
Note
This object isn't applicable to jobs that are running on Fargate resources and shouldn't be provided.
hostPath (string) --
The path for the device on the host container instance.
containerPath (string) --
The path inside the container that's used to expose the host device. By default, the hostPath value is used.
permissions (list) --
The explicit permissions to provide to the container for the device. By default, the container has permissions for read , write , and mknod for the device.
initProcessEnabled (boolean) --
If true, run an init process inside the container that forwards signals and reaps processes. This parameter maps to the --init option to docker run . This parameter requires version 1.25 of the Docker Remote API or greater on your container instance. To check the Docker Remote API version on your container instance, log into your container instance and run the following command: sudo docker version | grep "Server API version"
sharedMemorySize (integer) --
The value for the size (in MiB) of the /dev/shm volume. This parameter maps to the --shm-size option to docker run .
Note
This parameter isn't applicable to jobs that are running on Fargate resources and shouldn't be provided.
tmpfs (list) --
The container path, mount options, and size (in MiB) of the tmpfs mount. This parameter maps to the --tmpfs option to docker run .
Note
This parameter isn't applicable to jobs that are running on Fargate resources and shouldn't be provided.
(dict) --
The container path, mount options, and size of the tmpfs mount.
Note
This object isn't applicable to jobs that are running on Fargate resources.
containerPath (string) --
The absolute file path in the container where the tmpfs volume is mounted.
size (integer) --
The size (in MiB) of the tmpfs volume.
mountOptions (list) --
The list of tmpfs volume mount options.
Valid values: "defaults " | "ro " | "rw " | "suid " | "nosuid " | "dev " | "nodev " | "exec " | "noexec " | "sync " | "async " | "dirsync " | "remount " | "mand " | "nomand " | "atime " | "noatime " | "diratime " | "nodiratime " | "bind " | "rbind" | "unbindable" | "runbindable" | "private" | "rprivate" | "shared" | "rshared" | "slave" | "rslave" | "relatime " | "norelatime " | "strictatime " | "nostrictatime " | "mode " | "uid " | "gid " | "nr_inodes " | "nr_blocks " | "mpol "
maxSwap (integer) --
The total amount of swap memory (in MiB) a container can use. This parameter is translated to the --memory-swap option to docker run where the value is the sum of the container memory plus the maxSwap value. For more information, see ` --memory-swap details <https://docs.docker.com/config/containers/resource_constraints/#--memory-swap-details>`__ in the Docker documentation.
If a maxSwap value of 0 is specified, the container doesn't use swap. Accepted values are 0 or any positive integer. If the maxSwap parameter is omitted, the container doesn't use the swap configuration for the container instance it is running on. A maxSwap value must be set for the swappiness parameter to be used.
Note
This parameter isn't applicable to jobs that are running on Fargate resources and shouldn't be provided.
swappiness (integer) --
This allows you to tune a container's memory swappiness behavior. A swappiness value of 0 causes swapping not to happen unless absolutely necessary. A swappiness value of 100 causes pages to be swapped very aggressively. Accepted values are whole numbers between 0 and 100 . If the swappiness parameter isn't specified, a default value of 60 is used. If a value isn't specified for maxSwap , then this parameter is ignored. If maxSwap is set to 0, the container doesn't use swap. This parameter maps to the --memory-swappiness option to docker run .
Consider the following when you use a per-container swap configuration.
Note
The Amazon ECS optimized AMIs don't have swap enabled by default. You must enable swap on the instance to use this feature. For more information, see Instance store swap volumes in the Amazon EC2 User Guide for Linux Instances or How do I allocate memory to work as swap space in an Amazon EC2 instance by using a swap file?
maxSwap and swappiness parameters are omitted from a job definition, each container will have a default swappiness value of 60, and the total swap usage will be limited to two times the memory reservation of the container.Note
This parameter isn't applicable to jobs that are running on Fargate resources and shouldn't be provided.
logConfiguration (dict) --
The log configuration specification for the container.
This parameter maps to LogConfig in the Create a container section of the Docker Remote API and the --log-driver option to docker run . By default, containers use the same logging driver that the Docker daemon uses. However the container might use a different logging driver than the Docker daemon by specifying a log driver with this parameter in the container definition. To use a different logging driver for a container, the log system must be configured properly on the container instance (or on a different log server for remote logging options). For more information on the options for different supported log drivers, see Configure logging drivers in the Docker documentation.
Note
Batch currently supports a subset of the logging drivers available to the Docker daemon (shown in the LogConfiguration data type).
This parameter requires version 1.18 of the Docker Remote API or greater on your container instance. To check the Docker Remote API version on your container instance, log into your container instance and run the following command: sudo docker version | grep "Server API version"
Note
The Amazon ECS container agent running on a container instance must register the logging drivers available on that instance with the ECS_AVAILABLE_LOGGING_DRIVERS environment variable before containers placed on that instance can use these log configuration options. For more information, see Amazon ECS container agent configuration in the Amazon Elastic Container Service Developer Guide .
logDriver (string) --
The log driver to use for the container. The valid values listed for this parameter are log drivers that the Amazon ECS container agent can communicate with by default.
The supported log drivers are awslogs , fluentd , gelf , json-file , journald , logentries , syslog , and splunk .
Note
Jobs that are running on Fargate resources are restricted to the awslogs and splunk log drivers.
awslogs
Specifies the Amazon CloudWatch Logs logging driver. For more information, see Using the awslogs log driver in the Batch User Guide and Amazon CloudWatch Logs logging driver in the Docker documentation.
fluentd
Specifies the Fluentd logging driver. For more information, including usage and options, see Fluentd logging driver in the Docker documentation.
gelf
Specifies the Graylog Extended Format (GELF) logging driver. For more information, including usage and options, see Graylog Extended Format logging driver in the Docker documentation.
journald
Specifies the journald logging driver. For more information, including usage and options, see Journald logging driver in the Docker documentation.
json-file
Specifies the JSON file logging driver. For more information, including usage and options, see JSON File logging driver in the Docker documentation.
splunk
Specifies the Splunk logging driver. For more information, including usage and options, see Splunk logging driver in the Docker documentation.
syslog
Specifies the syslog logging driver. For more information, including usage and options, see Syslog logging driver in the Docker documentation.
Note
If you have a custom driver that's not listed earlier that you want to work with the Amazon ECS container agent, you can fork the Amazon ECS container agent project that's available on GitHub and customize it to work with that driver. We encourage you to submit pull requests for changes that you want to have included. However, Amazon Web Services doesn't currently support running modified copies of this software.
This parameter requires version 1.18 of the Docker Remote API or greater on your container instance. To check the Docker Remote API version on your container instance, log into your container instance and run the following command: sudo docker version | grep "Server API version"
options (dict) --
The configuration options to send to the log driver. This parameter requires version 1.19 of the Docker Remote API or greater on your container instance. To check the Docker Remote API version on your container instance, log into your container instance and run the following command: sudo docker version | grep "Server API version"
secretOptions (list) --
The secrets to pass to the log configuration. For more information, see Specifying sensitive data in the Batch User Guide .
(dict) --
An object representing the secret to expose to your container. Secrets can be exposed to a container in the following ways:
secrets container definition parameter.secretOptions container definition parameter.For more information, see Specifying sensitive data in the Batch User Guide .
name (string) --
The name of the secret.
valueFrom (string) --
The secret to expose to the container. The supported values are either the full ARN of the Secrets Manager secret or the full ARN of the parameter in the Amazon Web Services Systems Manager Parameter Store.
Note
If the Amazon Web Services Systems Manager Parameter Store parameter exists in the same Region as the job you're launching, then you can use either the full ARN or name of the parameter. If the parameter exists in a different Region, then the full ARN must be specified.
secrets (list) --
The secrets for the container. For more information, see Specifying sensitive data in the Batch User Guide .
(dict) --
An object representing the secret to expose to your container. Secrets can be exposed to a container in the following ways:
secrets container definition parameter.secretOptions container definition parameter.For more information, see Specifying sensitive data in the Batch User Guide .
name (string) --
The name of the secret.
valueFrom (string) --
The secret to expose to the container. The supported values are either the full ARN of the Secrets Manager secret or the full ARN of the parameter in the Amazon Web Services Systems Manager Parameter Store.
Note
If the Amazon Web Services Systems Manager Parameter Store parameter exists in the same Region as the job you're launching, then you can use either the full ARN or name of the parameter. If the parameter exists in a different Region, then the full ARN must be specified.
networkConfiguration (dict) --
The network configuration for jobs that are running on Fargate resources. Jobs that are running on EC2 resources must not specify this parameter.
assignPublicIp (string) --
Indicates whether the job should have a public IP address. For a job that is running on Fargate resources in a private subnet to send outbound traffic to the internet (for example, to pull container images), the private subnet requires a NAT gateway be attached to route requests to the internet. For more information, see Amazon ECS task networking . The default value is "DISABLED".
fargatePlatformConfiguration (dict) --
The platform configuration for jobs that are running on Fargate resources. Jobs that are running on EC2 resources must not specify this parameter.
platformVersion (string) --
The Fargate platform version where the jobs are running. A platform version is specified only for jobs that are running on Fargate resources. If one isn't specified, the LATEST platform version is used by default. This uses a recent, approved version of the Fargate platform for compute resources. For more information, see Fargate platform versions in the Amazon Elastic Container Service Developer Guide .
timeout (dict) --
The timeout configuration for jobs that are submitted with this job definition. You can specify a timeout duration after which Batch terminates your jobs if they haven't finished.
attemptDurationSeconds (integer) --
The time duration in seconds (measured from the job attempt's startedAt timestamp) after which Batch terminates your jobs if they have not finished. The minimum value for the timeout is 60 seconds.
nodeProperties (dict) --
An object with various properties specific to multi-node parallel jobs.
Note
If the job runs on Fargate resources, then you must not specify nodeProperties ; use containerProperties instead.
numNodes (integer) --
The number of nodes associated with a multi-node parallel job.
mainNode (integer) --
Specifies the node index for the main node of a multi-node parallel job. This node index value must be fewer than the number of nodes.
nodeRangeProperties (list) --
A list of node ranges and their properties associated with a multi-node parallel job.
(dict) --
An object representing the properties of the node range for a multi-node parallel job.
targetNodes (string) --
The range of nodes, using node index values. A range of 0:3 indicates nodes with index values of 0 through 3 . If the starting range value is omitted (:n ), then 0 is used to start the range. If the ending range value is omitted (n: ), then the highest possible node index is used to end the range. Your accumulative node ranges must account for all nodes (0:n ). You can nest node ranges, for example 0:10 and 4:5 , in which case the 4:5 range properties override the 0:10 properties.
container (dict) --
The container details for the node range.
image (string) --
The image used to start a container. This string is passed directly to the Docker daemon. Images in the Docker Hub registry are available by default. Other repositories are specified with `` repository-url /image :tag `` . Up to 255 letters (uppercase and lowercase), numbers, hyphens, underscores, colons, periods, forward slashes, and number signs are allowed. This parameter maps to Image in the Create a container section of the Docker Remote API and the IMAGE parameter of docker run .
Note
Docker image architecture must match the processor architecture of the compute resources that they're scheduled on. For example, ARM-based Docker images can only run on ARM-based compute resources.
registry/repository[:tag] or registry/repository[@digest] naming conventions. For example, ``public.ecr.aws/registry_alias /my-web-app :latest `` .012345678910.dkr.ecr.<region-name>.amazonaws.com/<repository-name> ).ubuntu or mongo ).amazon/amazon-ecs-agent ).quay.io/assemblyline/ubuntu ).vcpus (integer) --
This parameter is deprecated, use resourceRequirements to specify the vCPU requirements for the job definition. It's not supported for jobs running on Fargate resources. For jobs running on EC2 resources, it specifies the number of vCPUs reserved for the job.
Each vCPU is equivalent to 1,024 CPU shares. This parameter maps to CpuShares in the Create a container section of the Docker Remote API and the --cpu-shares option to docker run . The number of vCPUs must be specified but can be specified in several places. You must specify it at least once for each node.
memory (integer) --
This parameter is deprecated, use resourceRequirements to specify the memory requirements for the job definition. It's not supported for jobs running on Fargate resources. For jobs running on EC2 resources, it specifies the memory hard limit (in MiB) for a container. If your container attempts to exceed the specified number, it's terminated. You must specify at least 4 MiB of memory for a job using this parameter. The memory hard limit can be specified in several places. It must be specified for each node at least once.
command (list) --
The command that's passed to the container. This parameter maps to Cmd in the Create a container section of the Docker Remote API and the COMMAND parameter to docker run . For more information, see https://docs.docker.com/engine/reference/builder/#cmd .
jobRoleArn (string) --
The Amazon Resource Name (ARN) of the IAM role that the container can assume for Amazon Web Services permissions. For more information, see IAM roles for tasks in the Amazon Elastic Container Service Developer Guide .
executionRoleArn (string) --
The Amazon Resource Name (ARN) of the execution role that Batch can assume. For jobs that run on Fargate resources, you must provide an execution role. For more information, see Batch execution IAM role in the Batch User Guide .
volumes (list) --
A list of data volumes used in a job.
(dict) --
A data volume used in a job's container properties.
host (dict) --
The contents of the host parameter determine whether your data volume persists on the host container instance and where it is stored. If the host parameter is empty, then the Docker daemon assigns a host path for your data volume. However, the data isn't guaranteed to persist after the containers associated with it stop running.
Note
This parameter isn't applicable to jobs that are running on Fargate resources and shouldn't be provided.
sourcePath (string) --
The path on the host container instance that's presented to the container. If this parameter is empty, then the Docker daemon has assigned a host path for you. If this parameter contains a file location, then the data volume persists at the specified location on the host container instance until you delete it manually. If the source path location doesn't exist on the host container instance, the Docker daemon creates it. If the location does exist, the contents of the source path folder are exported.
Note
This parameter isn't applicable to jobs that run on Fargate resources and shouldn't be provided.
name (string) --
The name of the volume. It can be up to 255 letters long. It can contain uppercase and lowercase letters, numbers, hyphens (-), and underscores (_). This name is referenced in the sourceVolume parameter of container definition mountPoints .
efsVolumeConfiguration (dict) --
This parameter is specified when you are using an Amazon Elastic File System file system for job storage. Jobs that are running on Fargate resources must specify a platformVersion of at least 1.4.0 .
fileSystemId (string) --
The Amazon EFS file system ID to use.
rootDirectory (string) --
The directory within the Amazon EFS file system to mount as the root directory inside the host. If this parameter is omitted, the root of the Amazon EFS volume is used instead. Specifying / has the same effect as omitting this parameter. The maximum length is 4,096 characters.
Warning
If an EFS access point is specified in the authorizationConfig , the root directory parameter must either be omitted or set to / , which enforces the path set on the Amazon EFS access point.
transitEncryption (string) --
Determines whether to enable encryption for Amazon EFS data in transit between the Amazon ECS host and the Amazon EFS server. Transit encryption must be enabled if Amazon EFS IAM authorization is used. If this parameter is omitted, the default value of DISABLED is used. For more information, see Encrypting data in transit in the Amazon Elastic File System User Guide .
transitEncryptionPort (integer) --
The port to use when sending encrypted data between the Amazon ECS host and the Amazon EFS server. If you don't specify a transit encryption port, it uses the port selection strategy that the Amazon EFS mount helper uses. The value must be between 0 and 65,535. For more information, see EFS mount helper in the Amazon Elastic File System User Guide .
authorizationConfig (dict) --
The authorization configuration details for the Amazon EFS file system.
accessPointId (string) --
The Amazon EFS access point ID to use. If an access point is specified, the root directory value specified in the EFSVolumeConfiguration must either be omitted or set to / which will enforce the path set on the EFS access point. If an access point is used, transit encryption must be enabled in the EFSVolumeConfiguration . For more information, see Working with Amazon EFS access points in the Amazon Elastic File System User Guide .
iam (string) --
Whether or not to use the Batch job IAM role defined in a job definition when mounting the Amazon EFS file system. If enabled, transit encryption must be enabled in the EFSVolumeConfiguration . If this parameter is omitted, the default value of DISABLED is used. For more information, see Using Amazon EFS access points in the Batch User Guide . EFS IAM authorization requires that TransitEncryption be ENABLED and that a JobRoleArn is specified.
environment (list) --
The environment variables to pass to a container. This parameter maps to Env in the Create a container section of the Docker Remote API and the --env option to docker run .
Warning
We don't recommend using plaintext environment variables for sensitive information, such as credential data.
Note
Environment variables must not start with AWS_BATCH ; this naming convention is reserved for variables that are set by the Batch service.
(dict) --
A key-value pair object.
name (string) --
The name of the key-value pair. For environment variables, this is the name of the environment variable.
value (string) --
The value of the key-value pair. For environment variables, this is the value of the environment variable.
mountPoints (list) --
The mount points for data volumes in your container. This parameter maps to Volumes in the Create a container section of the Docker Remote API and the --volume option to docker run .
(dict) --
Details on a Docker volume mount point that's used in a job's container properties. This parameter maps to Volumes in the Create a container section of the Docker Remote API and the --volume option to docker run.
containerPath (string) --
The path on the container where the host volume is mounted.
readOnly (boolean) --
If this value is true , the container has read-only access to the volume. Otherwise, the container can write to the volume. The default value is false .
sourceVolume (string) --
The name of the volume to mount.
readonlyRootFilesystem (boolean) --
When this parameter is true, the container is given read-only access to its root file system. This parameter maps to ReadonlyRootfs in the Create a container section of the Docker Remote API and the --read-only option to docker run .
privileged (boolean) --
When this parameter is true, the container is given elevated permissions on the host container instance (similar to the root user). This parameter maps to Privileged in the Create a container section of the Docker Remote API and the --privileged option to docker run . The default value is false.
Note
This parameter isn't applicable to jobs that are running on Fargate resources and shouldn't be provided, or specified as false.
ulimits (list) --
A list of ulimits to set in the container. This parameter maps to Ulimits in the Create a container section of the Docker Remote API and the --ulimit option to docker run .
Note
This parameter isn't applicable to jobs that are running on Fargate resources and shouldn't be provided.
(dict) --
The ulimit settings to pass to the container.
Note
This object isn't applicable to jobs that are running on Fargate resources.
hardLimit (integer) --
The hard limit for the ulimit type.
name (string) --
The type of the ulimit .
softLimit (integer) --
The soft limit for the ulimit type.
user (string) --
The user name to use inside the container. This parameter maps to User in the Create a container section of the Docker Remote API and the --user option to docker run .
instanceType (string) --
The instance type to use for a multi-node parallel job. All node groups in a multi-node parallel job must use the same instance type.
Note
This parameter isn't applicable to single-node container jobs or jobs that run on Fargate resources, and shouldn't be provided.
resourceRequirements (list) --
The type and amount of resources to assign to a container. The supported resources include GPU , MEMORY , and VCPU .
(dict) --
The type and amount of a resource to assign to a container. The supported resources include GPU , MEMORY , and VCPU .
value (string) --
The quantity of the specified resource to reserve for the container. The values vary based on the type specified.
type="GPU"
The number of physical GPUs to reserve for the container. The number of GPUs reserved for all containers in a job shouldn't exceed the number of available GPUs on the compute resource that the job is launched on.
Note
GPUs are not available for jobs that are running on Fargate resources.
type="MEMORY"
The memory hard limit (in MiB) present to the container. This parameter is supported for jobs that are running on EC2 resources. If your container attempts to exceed the memory specified, the container is terminated. This parameter maps to Memory in the Create a container section of the Docker Remote API and the --memory option to docker run . You must specify at least 4 MiB of memory for a job. This is required but can be specified in several places for multi-node parallel (MNP) jobs. It must be specified for each node at least once. This parameter maps to Memory in the Create a container section of the Docker Remote API and the --memory option to docker run .
Note
If you're trying to maximize your resource utilization by providing your jobs as much memory as possible for a particular instance type, see Memory management in the Batch User Guide .
For jobs that are running on Fargate resources, then value is the hard limit (in MiB), and must match one of the supported values and the VCPU values must be one of the values supported for that memory value.
value = 512
VCPU= 0.25value = 1024
VCPU= 0.25 or 0.5value = 2048
VCPU= 0.25, 0.5, or 1value = 3072
VCPU= 0.5, or 1value = 4096
VCPU= 0.5, 1, or 2value = 5120, 6144, or 7168
VCPU= 1 or 2value = 8192
VCPU= 1, 2, or 4value = 9216, 10240, 11264, 12288, 13312, 14336, 15360, or 16384
VCPU= 2 or 4value = 17408, 18432, 19456, 20480, 21504, 22528, 23552, 24576, 25600, 26624, 27648, 28672, 29696, or 30720
VCPU= 4type="VCPU"
The number of vCPUs reserved for the container. This parameter maps to CpuShares in the Create a container section of the Docker Remote API and the --cpu-shares option to docker run . Each vCPU is equivalent to 1,024 CPU shares. For EC2 resources, you must specify at least one vCPU. This is required but can be specified in several places; it must be specified for each node at least once.
For jobs that are running on Fargate resources, then value must match one of the supported values and the MEMORY values must be one of the values supported for that VCPU value. The supported values are 0.25, 0.5, 1, 2, and 4
value = 0.25
MEMORY= 512, 1024, or 2048value = 0.5
MEMORY= 1024, 2048, 3072, or 4096value = 1
MEMORY= 2048, 3072, 4096, 5120, 6144, 7168, or 8192value = 2
MEMORY= 4096, 5120, 6144, 7168, 8192, 9216, 10240, 11264, 12288, 13312, 14336, 15360, or 16384value = 4
MEMORY= 8192, 9216, 10240, 11264, 12288, 13312, 14336, 15360, 16384, 17408, 18432, 19456, 20480, 21504, 22528, 23552, 24576, 25600, 26624, 27648, 28672, 29696, or 30720
type (string) --
The type of resource to assign to a container. The supported resources include GPU , MEMORY , and VCPU .
linuxParameters (dict) --
Linux-specific modifications that are applied to the container, such as details for device mappings.
devices (list) --
Any host devices to expose to the container. This parameter maps to Devices in the Create a container section of the Docker Remote API and the --device option to docker run .
Note
This parameter isn't applicable to jobs that are running on Fargate resources and shouldn't be provided.
(dict) --
An object representing a container instance host device.
Note
This object isn't applicable to jobs that are running on Fargate resources and shouldn't be provided.
hostPath (string) --
The path for the device on the host container instance.
containerPath (string) --
The path inside the container that's used to expose the host device. By default, the hostPath value is used.
permissions (list) --
The explicit permissions to provide to the container for the device. By default, the container has permissions for read , write , and mknod for the device.
initProcessEnabled (boolean) --
If true, run an init process inside the container that forwards signals and reaps processes. This parameter maps to the --init option to docker run . This parameter requires version 1.25 of the Docker Remote API or greater on your container instance. To check the Docker Remote API version on your container instance, log into your container instance and run the following command: sudo docker version | grep "Server API version"
sharedMemorySize (integer) --
The value for the size (in MiB) of the /dev/shm volume. This parameter maps to the --shm-size option to docker run .
Note
This parameter isn't applicable to jobs that are running on Fargate resources and shouldn't be provided.
tmpfs (list) --
The container path, mount options, and size (in MiB) of the tmpfs mount. This parameter maps to the --tmpfs option to docker run .
Note
This parameter isn't applicable to jobs that are running on Fargate resources and shouldn't be provided.
(dict) --
The container path, mount options, and size of the tmpfs mount.
Note
This object isn't applicable to jobs that are running on Fargate resources.
containerPath (string) --
The absolute file path in the container where the tmpfs volume is mounted.
size (integer) --
The size (in MiB) of the tmpfs volume.
mountOptions (list) --
The list of tmpfs volume mount options.
Valid values: "defaults " | "ro " | "rw " | "suid " | "nosuid " | "dev " | "nodev " | "exec " | "noexec " | "sync " | "async " | "dirsync " | "remount " | "mand " | "nomand " | "atime " | "noatime " | "diratime " | "nodiratime " | "bind " | "rbind" | "unbindable" | "runbindable" | "private" | "rprivate" | "shared" | "rshared" | "slave" | "rslave" | "relatime " | "norelatime " | "strictatime " | "nostrictatime " | "mode " | "uid " | "gid " | "nr_inodes " | "nr_blocks " | "mpol "
maxSwap (integer) --
The total amount of swap memory (in MiB) a container can use. This parameter is translated to the --memory-swap option to docker run where the value is the sum of the container memory plus the maxSwap value. For more information, see ` --memory-swap details <https://docs.docker.com/config/containers/resource_constraints/#--memory-swap-details>`__ in the Docker documentation.
If a maxSwap value of 0 is specified, the container doesn't use swap. Accepted values are 0 or any positive integer. If the maxSwap parameter is omitted, the container doesn't use the swap configuration for the container instance it is running on. A maxSwap value must be set for the swappiness parameter to be used.
Note
This parameter isn't applicable to jobs that are running on Fargate resources and shouldn't be provided.
swappiness (integer) --
This allows you to tune a container's memory swappiness behavior. A swappiness value of 0 causes swapping not to happen unless absolutely necessary. A swappiness value of 100 causes pages to be swapped very aggressively. Accepted values are whole numbers between 0 and 100 . If the swappiness parameter isn't specified, a default value of 60 is used. If a value isn't specified for maxSwap , then this parameter is ignored. If maxSwap is set to 0, the container doesn't use swap. This parameter maps to the --memory-swappiness option to docker run .
Consider the following when you use a per-container swap configuration.
Note
The Amazon ECS optimized AMIs don't have swap enabled by default. You must enable swap on the instance to use this feature. For more information, see Instance store swap volumes in the Amazon EC2 User Guide for Linux Instances or How do I allocate memory to work as swap space in an Amazon EC2 instance by using a swap file?
maxSwap and swappiness parameters are omitted from a job definition, each container will have a default swappiness value of 60, and the total swap usage will be limited to two times the memory reservation of the container.Note
This parameter isn't applicable to jobs that are running on Fargate resources and shouldn't be provided.
logConfiguration (dict) --
The log configuration specification for the container.
This parameter maps to LogConfig in the Create a container section of the Docker Remote API and the --log-driver option to docker run . By default, containers use the same logging driver that the Docker daemon uses. However the container might use a different logging driver than the Docker daemon by specifying a log driver with this parameter in the container definition. To use a different logging driver for a container, the log system must be configured properly on the container instance (or on a different log server for remote logging options). For more information on the options for different supported log drivers, see Configure logging drivers in the Docker documentation.
Note
Batch currently supports a subset of the logging drivers available to the Docker daemon (shown in the LogConfiguration data type).
This parameter requires version 1.18 of the Docker Remote API or greater on your container instance. To check the Docker Remote API version on your container instance, log into your container instance and run the following command: sudo docker version | grep "Server API version"
Note
The Amazon ECS container agent running on a container instance must register the logging drivers available on that instance with the ECS_AVAILABLE_LOGGING_DRIVERS environment variable before containers placed on that instance can use these log configuration options. For more information, see Amazon ECS container agent configuration in the Amazon Elastic Container Service Developer Guide .
logDriver (string) --
The log driver to use for the container. The valid values listed for this parameter are log drivers that the Amazon ECS container agent can communicate with by default.
The supported log drivers are awslogs , fluentd , gelf , json-file , journald , logentries , syslog , and splunk .
Note
Jobs that are running on Fargate resources are restricted to the awslogs and splunk log drivers.
awslogs
Specifies the Amazon CloudWatch Logs logging driver. For more information, see Using the awslogs log driver in the Batch User Guide and Amazon CloudWatch Logs logging driver in the Docker documentation.
fluentd
Specifies the Fluentd logging driver. For more information, including usage and options, see Fluentd logging driver in the Docker documentation.
gelf
Specifies the Graylog Extended Format (GELF) logging driver. For more information, including usage and options, see Graylog Extended Format logging driver in the Docker documentation.
journald
Specifies the journald logging driver. For more information, including usage and options, see Journald logging driver in the Docker documentation.
json-file
Specifies the JSON file logging driver. For more information, including usage and options, see JSON File logging driver in the Docker documentation.
splunk
Specifies the Splunk logging driver. For more information, including usage and options, see Splunk logging driver in the Docker documentation.
syslog
Specifies the syslog logging driver. For more information, including usage and options, see Syslog logging driver in the Docker documentation.
Note
If you have a custom driver that's not listed earlier that you want to work with the Amazon ECS container agent, you can fork the Amazon ECS container agent project that's available on GitHub and customize it to work with that driver. We encourage you to submit pull requests for changes that you want to have included. However, Amazon Web Services doesn't currently support running modified copies of this software.
This parameter requires version 1.18 of the Docker Remote API or greater on your container instance. To check the Docker Remote API version on your container instance, log into your container instance and run the following command: sudo docker version | grep "Server API version"
options (dict) --
The configuration options to send to the log driver. This parameter requires version 1.19 of the Docker Remote API or greater on your container instance. To check the Docker Remote API version on your container instance, log into your container instance and run the following command: sudo docker version | grep "Server API version"
secretOptions (list) --
The secrets to pass to the log configuration. For more information, see Specifying sensitive data in the Batch User Guide .
(dict) --
An object representing the secret to expose to your container. Secrets can be exposed to a container in the following ways:
secrets container definition parameter.secretOptions container definition parameter.For more information, see Specifying sensitive data in the Batch User Guide .
name (string) --
The name of the secret.
valueFrom (string) --
The secret to expose to the container. The supported values are either the full ARN of the Secrets Manager secret or the full ARN of the parameter in the Amazon Web Services Systems Manager Parameter Store.
Note
If the Amazon Web Services Systems Manager Parameter Store parameter exists in the same Region as the job you're launching, then you can use either the full ARN or name of the parameter. If the parameter exists in a different Region, then the full ARN must be specified.
secrets (list) --
The secrets for the container. For more information, see Specifying sensitive data in the Batch User Guide .
(dict) --
An object representing the secret to expose to your container. Secrets can be exposed to a container in the following ways:
secrets container definition parameter.secretOptions container definition parameter.For more information, see Specifying sensitive data in the Batch User Guide .
name (string) --
The name of the secret.
valueFrom (string) --
The secret to expose to the container. The supported values are either the full ARN of the Secrets Manager secret or the full ARN of the parameter in the Amazon Web Services Systems Manager Parameter Store.
Note
If the Amazon Web Services Systems Manager Parameter Store parameter exists in the same Region as the job you're launching, then you can use either the full ARN or name of the parameter. If the parameter exists in a different Region, then the full ARN must be specified.
networkConfiguration (dict) --
The network configuration for jobs that are running on Fargate resources. Jobs that are running on EC2 resources must not specify this parameter.
assignPublicIp (string) --
Indicates whether the job should have a public IP address. For a job that is running on Fargate resources in a private subnet to send outbound traffic to the internet (for example, to pull container images), the private subnet requires a NAT gateway be attached to route requests to the internet. For more information, see Amazon ECS task networking . The default value is "DISABLED".
fargatePlatformConfiguration (dict) --
The platform configuration for jobs that are running on Fargate resources. Jobs that are running on EC2 resources must not specify this parameter.
platformVersion (string) --
The Fargate platform version where the jobs are running. A platform version is specified only for jobs that are running on Fargate resources. If one isn't specified, the LATEST platform version is used by default. This uses a recent, approved version of the Fargate platform for compute resources. For more information, see Fargate platform versions in the Amazon Elastic Container Service Developer Guide .
tags (dict) --
The tags applied to the job definition.
propagateTags (boolean) --
Specifies whether to propagate the tags from the job or job definition to the corresponding Amazon ECS task. If no value is specified, the tags aren't propagated. Tags can only be propagated to the tasks during task creation. For tags with the same name, job tags are given priority over job definitions tags. If the total number of combined tags from the job and job definition is over 50, the job is moved to the FAILED state.
platformCapabilities (list) --
The platform capabilities required by the job definition. If no value is specified, it defaults to EC2 . Jobs run on Fargate resources specify FARGATE .
nextToken (string) --
The nextToken value to include in a future DescribeJobDefinitions request. When the results of a DescribeJobDefinitions request exceed maxResults , this value can be used to retrieve the next page of results. This value is null when there are no more results to return.
Exceptions
Batch.Client.exceptions.ClientExceptionBatch.Client.exceptions.ServerExceptionExamples
This example describes all of your active job definitions.
response = client.describe_job_definitions(
status='ACTIVE',
)
print(response)
Expected Output:
{
'jobDefinitions': [
{
'type': 'container',
'containerProperties': {
'command': [
'sleep',
'60',
],
'environment': [
],
'image': 'busybox',
'mountPoints': [
],
'resourceRequirements': [
{
'type': 'MEMORY',
'value': '128',
},
{
'type': 'VCPU',
'value': '1',
},
],
'ulimits': [
],
'volumes': [
],
},
'jobDefinitionArn': 'arn:aws:batch:us-east-1:012345678910:job-definition/sleep60:1',
'jobDefinitionName': 'sleep60',
'revision': 1,
'status': 'ACTIVE',
},
],
'ResponseMetadata': {
'...': '...',
},
}
describe_job_queues(**kwargs)¶Describes one or more of your job queues.
See also: AWS API Documentation
Request Syntax
response = client.describe_job_queues(
jobQueues=[
'string',
],
maxResults=123,
nextToken='string'
)
A list of up to 100 queue names or full queue Amazon Resource Name (ARN) entries.
DescribeJobQueues in paginated output. When this parameter is used, DescribeJobQueues only returns maxResults results in a single page and a nextToken response element. The remaining results of the initial request can be seen by sending another DescribeJobQueues request with the returned nextToken value. This value can be between 1 and 100. If this parameter isn't used, then DescribeJobQueues returns up to 100 results and a nextToken value if applicable.The nextToken value returned from a previous paginated DescribeJobQueues request where maxResults was used and the results exceeded the value of that parameter. Pagination continues from the end of the previous results that returned the nextToken value. This value is null when there are no more results to return.
Note
This token should be treated as an opaque identifier that's only used to retrieve the next items in a list and not for other programmatic purposes.
dict
Response Syntax
{
'jobQueues': [
{
'jobQueueName': 'string',
'jobQueueArn': 'string',
'state': 'ENABLED'|'DISABLED',
'schedulingPolicyArn': 'string',
'status': 'CREATING'|'UPDATING'|'DELETING'|'DELETED'|'VALID'|'INVALID',
'statusReason': 'string',
'priority': 123,
'computeEnvironmentOrder': [
{
'order': 123,
'computeEnvironment': 'string'
},
],
'tags': {
'string': 'string'
}
},
],
'nextToken': 'string'
}
Response Structure
(dict) --
jobQueues (list) --
The list of job queues.
(dict) --
An object representing the details of an Batch job queue.
jobQueueName (string) --
The name of the job queue.
jobQueueArn (string) --
The Amazon Resource Name (ARN) of the job queue.
state (string) --
Describes the ability of the queue to accept new jobs. If the job queue state is ENABLED , it's able to accept jobs. If the job queue state is DISABLED , new jobs can't be added to the queue, but jobs already in the queue can finish.
schedulingPolicyArn (string) --
The Amazon Resource Name (ARN) of the scheduling policy. The format is aws:*Partition* :batch:*Region* :*Account* :scheduling-policy/*Name* `` . For example, ``aws:aws:batch:us-west-2:012345678910:scheduling-policy/MySchedulingPolicy .
status (string) --
The status of the job queue (for example, CREATING or VALID ).
statusReason (string) --
A short, human-readable string to provide additional details about the current status of the job queue.
priority (integer) --
The priority of the job queue. Job queues with a higher priority (or a higher integer value for the priority parameter) are evaluated first when associated with the same compute environment. Priority is determined in descending order, for example, a job queue with a priority value of 10 is given scheduling preference over a job queue with a priority value of 1 . All of the compute environments must be either EC2 (EC2 or SPOT ) or Fargate (FARGATE or FARGATE_SPOT ); EC2 and Fargate compute environments can't be mixed.
computeEnvironmentOrder (list) --
The compute environments that are attached to the job queue and the order that job placement is preferred. Compute environments are selected for job placement in ascending order.
(dict) --
The order in which compute environments are tried for job placement within a queue. Compute environments are tried in ascending order. For example, if two compute environments are associated with a job queue, the compute environment with a lower order integer value is tried for job placement first. Compute environments must be in the VALID state before you can associate them with a job queue. All of the compute environments must be either EC2 (EC2 or SPOT ) or Fargate (FARGATE or FARGATE_SPOT ); EC2 and Fargate compute environments can't be mixed.
Note
All compute environments that are associated with a job queue must share the same architecture. Batch doesn't support mixing compute environment architecture types in a single job queue.
order (integer) --
The order of the compute environment. Compute environments are tried in ascending order. For example, if two compute environments are associated with a job queue, the compute environment with a lower order integer value is tried for job placement first.
computeEnvironment (string) --
The Amazon Resource Name (ARN) of the compute environment.
tags (dict) --
The tags applied to the job queue. For more information, see Tagging your Batch resources in Batch User Guide .
nextToken (string) --
The nextToken value to include in a future DescribeJobQueues request. When the results of a DescribeJobQueues request exceed maxResults , this value can be used to retrieve the next page of results. This value is null when there are no more results to return.
Exceptions
Batch.Client.exceptions.ClientExceptionBatch.Client.exceptions.ServerExceptionExamples
This example describes the HighPriority job queue.
response = client.describe_job_queues(
jobQueues=[
'HighPriority',
],
)
print(response)
Expected Output:
{
'jobQueues': [
{
'computeEnvironmentOrder': [
{
'computeEnvironment': 'arn:aws:batch:us-east-1:012345678910:compute-environment/C4OnDemand',
'order': 1,
},
],
'jobQueueArn': 'arn:aws:batch:us-east-1:012345678910:job-queue/HighPriority',
'jobQueueName': 'HighPriority',
'priority': 1,
'state': 'ENABLED',
'status': 'VALID',
'statusReason': 'JobQueue Healthy',
},
],
'ResponseMetadata': {
'...': '...',
},
}
describe_jobs(**kwargs)¶Describes a list of Batch jobs.
See also: AWS API Documentation
Request Syntax
response = client.describe_jobs(
jobs=[
'string',
]
)
[REQUIRED]
A list of up to 100 job IDs.
{
'jobs': [
{
'jobArn': 'string',
'jobName': 'string',
'jobId': 'string',
'jobQueue': 'string',
'status': 'SUBMITTED'|'PENDING'|'RUNNABLE'|'STARTING'|'RUNNING'|'SUCCEEDED'|'FAILED',
'shareIdentifier': 'string',
'schedulingPriority': 123,
'attempts': [
{
'container': {
'containerInstanceArn': 'string',
'taskArn': 'string',
'exitCode': 123,
'reason': 'string',
'logStreamName': 'string',
'networkInterfaces': [
{
'attachmentId': 'string',
'ipv6Address': 'string',
'privateIpv4Address': 'string'
},
]
},
'startedAt': 123,
'stoppedAt': 123,
'statusReason': 'string'
},
],
'statusReason': 'string',
'createdAt': 123,
'retryStrategy': {
'attempts': 123,
'evaluateOnExit': [
{
'onStatusReason': 'string',
'onReason': 'string',
'onExitCode': 'string',
'action': 'RETRY'|'EXIT'
},
]
},
'startedAt': 123,
'stoppedAt': 123,
'dependsOn': [
{
'jobId': 'string',
'type': 'N_TO_N'|'SEQUENTIAL'
},
],
'jobDefinition': 'string',
'parameters': {
'string': 'string'
},
'container': {
'image': 'string',
'vcpus': 123,
'memory': 123,
'command': [
'string',
],
'jobRoleArn': 'string',
'executionRoleArn': 'string',
'volumes': [
{
'host': {
'sourcePath': 'string'
},
'name': 'string',
'efsVolumeConfiguration': {
'fileSystemId': 'string',
'rootDirectory': 'string',
'transitEncryption': 'ENABLED'|'DISABLED',
'transitEncryptionPort': 123,
'authorizationConfig': {
'accessPointId': 'string',
'iam': 'ENABLED'|'DISABLED'
}
}
},
],
'environment': [
{
'name': 'string',
'value': 'string'
},
],
'mountPoints': [
{
'containerPath': 'string',
'readOnly': True|False,
'sourceVolume': 'string'
},
],
'readonlyRootFilesystem': True|False,
'ulimits': [
{
'hardLimit': 123,
'name': 'string',
'softLimit': 123
},
],
'privileged': True|False,
'user': 'string',
'exitCode': 123,
'reason': 'string',
'containerInstanceArn': 'string',
'taskArn': 'string',
'logStreamName': 'string',
'instanceType': 'string',
'networkInterfaces': [
{
'attachmentId': 'string',
'ipv6Address': 'string',
'privateIpv4Address': 'string'
},
],
'resourceRequirements': [
{
'value': 'string',
'type': 'GPU'|'VCPU'|'MEMORY'
},
],
'linuxParameters': {
'devices': [
{
'hostPath': 'string',
'containerPath': 'string',
'permissions': [
'READ'|'WRITE'|'MKNOD',
]
},
],
'initProcessEnabled': True|False,
'sharedMemorySize': 123,
'tmpfs': [
{
'containerPath': 'string',
'size': 123,
'mountOptions': [
'string',
]
},
],
'maxSwap': 123,
'swappiness': 123
},
'logConfiguration': {
'logDriver': 'json-file'|'syslog'|'journald'|'gelf'|'fluentd'|'awslogs'|'splunk',
'options': {
'string': 'string'
},
'secretOptions': [
{
'name': 'string',
'valueFrom': 'string'
},
]
},
'secrets': [
{
'name': 'string',
'valueFrom': 'string'
},
],
'networkConfiguration': {
'assignPublicIp': 'ENABLED'|'DISABLED'
},
'fargatePlatformConfiguration': {
'platformVersion': 'string'
}
},
'nodeDetails': {
'nodeIndex': 123,
'isMainNode': True|False
},
'nodeProperties': {
'numNodes': 123,
'mainNode': 123,
'nodeRangeProperties': [
{
'targetNodes': 'string',
'container': {
'image': 'string',
'vcpus': 123,
'memory': 123,
'command': [
'string',
],
'jobRoleArn': 'string',
'executionRoleArn': 'string',
'volumes': [
{
'host': {
'sourcePath': 'string'
},
'name': 'string',
'efsVolumeConfiguration': {
'fileSystemId': 'string',
'rootDirectory': 'string',
'transitEncryption': 'ENABLED'|'DISABLED',
'transitEncryptionPort': 123,
'authorizationConfig': {
'accessPointId': 'string',
'iam': 'ENABLED'|'DISABLED'
}
}
},
],
'environment': [
{
'name': 'string',
'value': 'string'
},
],
'mountPoints': [
{
'containerPath': 'string',
'readOnly': True|False,
'sourceVolume': 'string'
},
],
'readonlyRootFilesystem': True|False,
'privileged': True|False,
'ulimits': [
{
'hardLimit': 123,
'name': 'string',
'softLimit': 123
},
],
'user': 'string',
'instanceType': 'string',
'resourceRequirements': [
{
'value': 'string',
'type': 'GPU'|'VCPU'|'MEMORY'
},
],
'linuxParameters': {
'devices': [
{
'hostPath': 'string',
'containerPath': 'string',
'permissions': [
'READ'|'WRITE'|'MKNOD',
]
},
],
'initProcessEnabled': True|False,
'sharedMemorySize': 123,
'tmpfs': [
{
'containerPath': 'string',
'size': 123,
'mountOptions': [
'string',
]
},
],
'maxSwap': 123,
'swappiness': 123
},
'logConfiguration': {
'logDriver': 'json-file'|'syslog'|'journald'|'gelf'|'fluentd'|'awslogs'|'splunk',
'options': {
'string': 'string'
},
'secretOptions': [
{
'name': 'string',
'valueFrom': 'string'
},
]
},
'secrets': [
{
'name': 'string',
'valueFrom': 'string'
},
],
'networkConfiguration': {
'assignPublicIp': 'ENABLED'|'DISABLED'
},
'fargatePlatformConfiguration': {
'platformVersion': 'string'
}
}
},
]
},
'arrayProperties': {
'statusSummary': {
'string': 123
},
'size': 123,
'index': 123
},
'timeout': {
'attemptDurationSeconds': 123
},
'tags': {
'string': 'string'
},
'propagateTags': True|False,
'platformCapabilities': [
'EC2'|'FARGATE',
]
},
]
}
Response Structure
The list of jobs.
An object representing an Batch job.
The Amazon Resource Name (ARN) of the job.
The name of the job.
The ID for the job.
The Amazon Resource Name (ARN) of the job queue that the job is associated with.
The current status for the job.
Note
If your jobs don't progress to STARTING , see Jobs stuck in RUNNABLE status in the troubleshooting section of the Batch User Guide .
The share identifier for the job.
The scheduling policy of the job definition. This only affects jobs in job queues with a fair share policy. Jobs with a higher scheduling priority are scheduled before jobs with a lower scheduling priority.
A list of job attempts associated with this job.
An object representing a job attempt.
Details about the container in this job attempt.
The Amazon Resource Name (ARN) of the Amazon ECS container instance that hosts the job attempt.
The Amazon Resource Name (ARN) of the Amazon ECS task that's associated with the job attempt. Each container attempt receives a task ARN when they reach the STARTING status.
The exit code for the job attempt. A non-zero exit code is considered a failure.
A short (255 max characters) human-readable string to provide additional details about a running or stopped container.
The name of the CloudWatch Logs log stream associated with the container. The log group for Batch jobs is /aws/batch/job . Each container attempt receives a log stream name when they reach the RUNNING status.
The network interfaces associated with the job attempt.
An object representing the elastic network interface for a multi-node parallel job node.
The attachment ID for the network interface.
The private IPv6 address for the network interface.
The private IPv4 address for the network interface.
The Unix timestamp (in milliseconds) for when the attempt was started (when the attempt transitioned from the STARTING state to the RUNNING state).
The Unix timestamp (in milliseconds) for when the attempt was stopped (when the attempt transitioned from the RUNNING state to a terminal state, such as SUCCEEDED or FAILED ).
A short, human-readable string to provide additional details about the current status of the job attempt.
A short, human-readable string to provide additional details about the current status of the job.
The Unix timestamp (in milliseconds) for when the job was created. For non-array jobs and parent array jobs, this is when the job entered the SUBMITTED state (at the time SubmitJob was called). For array child jobs, this is when the child job was spawned by its parent and entered the PENDING state.
The retry strategy to use for this job if an attempt fails.
The number of times to move a job to the RUNNABLE status. You can specify between 1 and 10 attempts. If the value of attempts is greater than one, the job is retried on failure the same number of attempts as the value.
Array of up to 5 objects that specify conditions under which the job should be retried or failed. If this parameter is specified, then the attempts parameter must also be specified.
Specifies a set of conditions to be met, and an action to take (RETRY or EXIT ) if all conditions are met.
Contains a glob pattern to match against the StatusReason returned for a job. The pattern can be up to 512 characters in length. It can contain letters, numbers, periods (.), colons (:), and white space (including spaces or tabs). It can optionally end with an asterisk (*) so that only the start of the string needs to be an exact match.
The string can be between 1 and 512 characters in length.
Contains a glob pattern to match against the Reason returned for a job. The pattern can be up to 512 characters in length. It can contain letters, numbers, periods (.), colons (:), and white space (including spaces and tabs). It can optionally end with an asterisk (*) so that only the start of the string needs to be an exact match.
The string can be between 1 and 512 characters in length.
Contains a glob pattern to match against the decimal representation of the ExitCode returned for a job. The pattern can be up to 512 characters in length. It can contain only numbers, and can optionally end with an asterisk (*) so that only the start of the string needs to be an exact match.
The string can be between 1 and 512 characters in length.
Specifies the action to take if all of the specified conditions (onStatusReason , onReason , and onExitCode ) are met. The values aren't case sensitive.
The Unix timestamp (in milliseconds) for when the job was started (when the job transitioned from the STARTING state to the RUNNING state). This parameter isn't provided for child jobs of array jobs or multi-node parallel jobs.
The Unix timestamp (in milliseconds) for when the job was stopped (when the job transitioned from the RUNNING state to a terminal state, such as SUCCEEDED or FAILED ).
A list of job IDs that this job depends on.
An object representing an Batch job dependency.
The job ID of the Batch job associated with this dependency.
The type of the job dependency.
The Amazon Resource Name (ARN) of the job definition that's used by this job.
Additional parameters passed to the job that replace parameter substitution placeholders or override any corresponding parameter defaults from the job definition.
An object representing the details of the container that's associated with the job.
The image used to start the container.
The number of vCPUs reserved for the container. For jobs that run on EC2 resources, you can specify the vCPU requirement for the job using resourceRequirements , but you can't specify the vCPU requirements in both the vcpus and resourceRequirements object. This parameter maps to CpuShares in the Create a container section of the Docker Remote API and the --cpu-shares option to docker run . Each vCPU is equivalent to 1,024 CPU shares. You must specify at least one vCPU. This is required but can be specified in several places. It must be specified for each node at least once.
Note
This parameter isn't applicable to jobs that run on Fargate resources. For jobs that run on Fargate resources, you must specify the vCPU requirement for the job using resourceRequirements .
For jobs running on EC2 resources that didn't specify memory requirements using resourceRequirements , the number of MiB of memory reserved for the job. For other jobs, including all run on Fargate resources, see resourceRequirements .
The command that's passed to the container.
The Amazon Resource Name (ARN) associated with the job upon execution.
The Amazon Resource Name (ARN) of the execution role that Batch can assume. For more information, see Batch execution IAM role in the Batch User Guide .
A list of volumes associated with the job.
A data volume used in a job's container properties.
The contents of the host parameter determine whether your data volume persists on the host container instance and where it is stored. If the host parameter is empty, then the Docker daemon assigns a host path for your data volume. However, the data isn't guaranteed to persist after the containers associated with it stop running.
Note
This parameter isn't applicable to jobs that are running on Fargate resources and shouldn't be provided.
The path on the host container instance that's presented to the container. If this parameter is empty, then the Docker daemon has assigned a host path for you. If this parameter contains a file location, then the data volume persists at the specified location on the host container instance until you delete it manually. If the source path location doesn't exist on the host container instance, the Docker daemon creates it. If the location does exist, the contents of the source path folder are exported.
Note
This parameter isn't applicable to jobs that run on Fargate resources and shouldn't be provided.
The name of the volume. It can be up to 255 letters long. It can contain uppercase and lowercase letters, numbers, hyphens (-), and underscores (_). This name is referenced in the sourceVolume parameter of container definition mountPoints .
This parameter is specified when you are using an Amazon Elastic File System file system for job storage. Jobs that are running on Fargate resources must specify a platformVersion of at least 1.4.0 .
The Amazon EFS file system ID to use.
The directory within the Amazon EFS file system to mount as the root directory inside the host. If this parameter is omitted, the root of the Amazon EFS volume is used instead. Specifying / has the same effect as omitting this parameter. The maximum length is 4,096 characters.
Warning
If an EFS access point is specified in the authorizationConfig , the root directory parameter must either be omitted or set to / , which enforces the path set on the Amazon EFS access point.
Determines whether to enable encryption for Amazon EFS data in transit between the Amazon ECS host and the Amazon EFS server. Transit encryption must be enabled if Amazon EFS IAM authorization is used. If this parameter is omitted, the default value of DISABLED is used. For more information, see Encrypting data in transit in the Amazon Elastic File System User Guide .
The port to use when sending encrypted data between the Amazon ECS host and the Amazon EFS server. If you don't specify a transit encryption port, it uses the port selection strategy that the Amazon EFS mount helper uses. The value must be between 0 and 65,535. For more information, see EFS mount helper in the Amazon Elastic File System User Guide .
The authorization configuration details for the Amazon EFS file system.
The Amazon EFS access point ID to use. If an access point is specified, the root directory value specified in the EFSVolumeConfiguration must either be omitted or set to / which will enforce the path set on the EFS access point. If an access point is used, transit encryption must be enabled in the EFSVolumeConfiguration . For more information, see Working with Amazon EFS access points in the Amazon Elastic File System User Guide .
Whether or not to use the Batch job IAM role defined in a job definition when mounting the Amazon EFS file system. If enabled, transit encryption must be enabled in the EFSVolumeConfiguration . If this parameter is omitted, the default value of DISABLED is used. For more information, see Using Amazon EFS access points in the Batch User Guide . EFS IAM authorization requires that TransitEncryption be ENABLED and that a JobRoleArn is specified.
The environment variables to pass to a container.
Note
Environment variables must not start with AWS_BATCH ; this naming convention is reserved for variables that are set by the Batch service.
A key-value pair object.
The name of the key-value pair. For environment variables, this is the name of the environment variable.
The value of the key-value pair. For environment variables, this is the value of the environment variable.
The mount points for data volumes in your container.
Details on a Docker volume mount point that's used in a job's container properties. This parameter maps to Volumes in the Create a container section of the Docker Remote API and the --volume option to docker run.
The path on the container where the host volume is mounted.
If this value is true , the container has read-only access to the volume. Otherwise, the container can write to the volume. The default value is false .
The name of the volume to mount.
When this parameter is true, the container is given read-only access to its root file system. This parameter maps to ReadonlyRootfs in the Create a container section of the Docker Remote API and the --read-only option to ` docker run https://docs.docker.com/engine/reference/commandline/run/`__ .
A list of ulimit values to set in the container. This parameter maps to Ulimits in the Create a container section of the Docker Remote API and the --ulimit option to docker run .
Note
This parameter isn't applicable to jobs that are running on Fargate resources.
The ulimit settings to pass to the container.
Note
This object isn't applicable to jobs that are running on Fargate resources.
The hard limit for the ulimit type.
The type of the ulimit .
The soft limit for the ulimit type.
When this parameter is true, the container is given elevated permissions on the host container instance (similar to the root user). The default value is false.
Note
This parameter isn't applicable to jobs that are running on Fargate resources and shouldn't be provided, or specified as false.
The user name to use inside the container. This parameter maps to User in the Create a container section of the Docker Remote API and the --user option to docker run .
The exit code to return upon completion.
A short (255 max characters) human-readable string to provide additional details about a running or stopped container.
The Amazon Resource Name (ARN) of the container instance that the container is running on.
The Amazon Resource Name (ARN) of the Amazon ECS task that's associated with the container job. Each container attempt receives a task ARN when they reach the STARTING status.
The name of the CloudWatch Logs log stream associated with the container. The log group for Batch jobs is /aws/batch/job . Each container attempt receives a log stream name when they reach the RUNNING status.
The instance type of the underlying host infrastructure of a multi-node parallel job.
Note
This parameter isn't applicable to jobs that are running on Fargate resources.
The network interfaces associated with the job.
An object representing the elastic network interface for a multi-node parallel job node.
The attachment ID for the network interface.
The private IPv6 address for the network interface.
The private IPv4 address for the network interface.
The type and amount of resources to assign to a container. The supported resources include GPU , MEMORY , and VCPU .
The type and amount of a resource to assign to a container. The supported resources include GPU , MEMORY , and VCPU .
The quantity of the specified resource to reserve for the container. The values vary based on the type specified.
type="GPU"
The number of physical GPUs to reserve for the container. The number of GPUs reserved for all containers in a job shouldn't exceed the number of available GPUs on the compute resource that the job is launched on.
Note
GPUs are not available for jobs that are running on Fargate resources.
type="MEMORY"
The memory hard limit (in MiB) present to the container. This parameter is supported for jobs that are running on EC2 resources. If your container attempts to exceed the memory specified, the container is terminated. This parameter maps to Memory in the Create a container section of the Docker Remote API and the --memory option to docker run . You must specify at least 4 MiB of memory for a job. This is required but can be specified in several places for multi-node parallel (MNP) jobs. It must be specified for each node at least once. This parameter maps to Memory in the Create a container section of the Docker Remote API and the --memory option to docker run .
Note
If you're trying to maximize your resource utilization by providing your jobs as much memory as possible for a particular instance type, see Memory management in the Batch User Guide .
For jobs that are running on Fargate resources, then value is the hard limit (in MiB), and must match one of the supported values and the VCPU values must be one of the values supported for that memory value.
value = 512
VCPU= 0.25value = 1024
VCPU= 0.25 or 0.5value = 2048
VCPU= 0.25, 0.5, or 1value = 3072
VCPU= 0.5, or 1value = 4096
VCPU= 0.5, 1, or 2value = 5120, 6144, or 7168
VCPU= 1 or 2value = 8192
VCPU= 1, 2, or 4value = 9216, 10240, 11264, 12288, 13312, 14336, 15360, or 16384
VCPU= 2 or 4value = 17408, 18432, 19456, 20480, 21504, 22528, 23552, 24576, 25600, 26624, 27648, 28672, 29696, or 30720
VCPU= 4type="VCPU"
The number of vCPUs reserved for the container. This parameter maps to CpuShares in the Create a container section of the Docker Remote API and the --cpu-shares option to docker run . Each vCPU is equivalent to 1,024 CPU shares. For EC2 resources, you must specify at least one vCPU. This is required but can be specified in several places; it must be specified for each node at least once.
For jobs that are running on Fargate resources, then value must match one of the supported values and the MEMORY values must be one of the values supported for that VCPU value. The supported values are 0.25, 0.5, 1, 2, and 4
value = 0.25
MEMORY= 512, 1024, or 2048value = 0.5
MEMORY= 1024, 2048, 3072, or 4096value = 1
MEMORY= 2048, 3072, 4096, 5120, 6144, 7168, or 8192value = 2
MEMORY= 4096, 5120, 6144, 7168, 8192, 9216, 10240, 11264, 12288, 13312, 14336, 15360, or 16384value = 4
MEMORY= 8192, 9216, 10240, 11264, 12288, 13312, 14336, 15360, 16384, 17408, 18432, 19456, 20480, 21504, 22528, 23552, 24576, 25600, 26624, 27648, 28672, 29696, or 30720
The type of resource to assign to a container. The supported resources include GPU , MEMORY , and VCPU .
Linux-specific modifications that are applied to the container, such as details for device mappings.
Any host devices to expose to the container. This parameter maps to Devices in the Create a container section of the Docker Remote API and the --device option to docker run .
Note
This parameter isn't applicable to jobs that are running on Fargate resources and shouldn't be provided.
An object representing a container instance host device.
Note
This object isn't applicable to jobs that are running on Fargate resources and shouldn't be provided.
The path for the device on the host container instance.
The path inside the container that's used to expose the host device. By default, the hostPath value is used.
The explicit permissions to provide to the container for the device. By default, the container has permissions for read , write , and mknod for the device.
If true, run an init process inside the container that forwards signals and reaps processes. This parameter maps to the --init option to docker run . This parameter requires version 1.25 of the Docker Remote API or greater on your container instance. To check the Docker Remote API version on your container instance, log into your container instance and run the following command: sudo docker version | grep "Server API version"
The value for the size (in MiB) of the /dev/shm volume. This parameter maps to the --shm-size option to docker run .
Note
This parameter isn't applicable to jobs that are running on Fargate resources and shouldn't be provided.
The container path, mount options, and size (in MiB) of the tmpfs mount. This parameter maps to the --tmpfs option to docker run .
Note
This parameter isn't applicable to jobs that are running on Fargate resources and shouldn't be provided.
The container path, mount options, and size of the tmpfs mount.
Note
This object isn't applicable to jobs that are running on Fargate resources.
The absolute file path in the container where the tmpfs volume is mounted.
The size (in MiB) of the tmpfs volume.
The list of tmpfs volume mount options.
Valid values: "defaults " | "ro " | "rw " | "suid " | "nosuid " | "dev " | "nodev " | "exec " | "noexec " | "sync " | "async " | "dirsync " | "remount " | "mand " | "nomand " | "atime " | "noatime " | "diratime " | "nodiratime " | "bind " | "rbind" | "unbindable" | "runbindable" | "private" | "rprivate" | "shared" | "rshared" | "slave" | "rslave" | "relatime " | "norelatime " | "strictatime " | "nostrictatime " | "mode " | "uid " | "gid " | "nr_inodes " | "nr_blocks " | "mpol "
The total amount of swap memory (in MiB) a container can use. This parameter is translated to the --memory-swap option to docker run where the value is the sum of the container memory plus the maxSwap value. For more information, see ` --memory-swap details <https://docs.docker.com/config/containers/resource_constraints/#--memory-swap-details>`__ in the Docker documentation.
If a maxSwap value of 0 is specified, the container doesn't use swap. Accepted values are 0 or any positive integer. If the maxSwap parameter is omitted, the container doesn't use the swap configuration for the container instance it is running on. A maxSwap value must be set for the swappiness parameter to be used.
Note
This parameter isn't applicable to jobs that are running on Fargate resources and shouldn't be provided.
This allows you to tune a container's memory swappiness behavior. A swappiness value of 0 causes swapping not to happen unless absolutely necessary. A swappiness value of 100 causes pages to be swapped very aggressively. Accepted values are whole numbers between 0 and 100 . If the swappiness parameter isn't specified, a default value of 60 is used. If a value isn't specified for maxSwap , then this parameter is ignored. If maxSwap is set to 0, the container doesn't use swap. This parameter maps to the --memory-swappiness option to docker run .
Consider the following when you use a per-container swap configuration.
Note
The Amazon ECS optimized AMIs don't have swap enabled by default. You must enable swap on the instance to use this feature. For more information, see Instance store swap volumes in the Amazon EC2 User Guide for Linux Instances or How do I allocate memory to work as swap space in an Amazon EC2 instance by using a swap file?
maxSwap and swappiness parameters are omitted from a job definition, each container will have a default swappiness value of 60, and the total swap usage will be limited to two times the memory reservation of the container.Note
This parameter isn't applicable to jobs that are running on Fargate resources and shouldn't be provided.
The log configuration specification for the container.
This parameter maps to LogConfig in the Create a container section of the Docker Remote API and the --log-driver option to docker run . By default, containers use the same logging driver that the Docker daemon uses. However, the container might use a different logging driver than the Docker daemon by specifying a log driver with this parameter in the container definition. To use a different logging driver for a container, the log system must be configured properly on the container instance. Or, alternatively, it must be configured on a different log server for remote logging options. For more information on the options for different supported log drivers, see Configure logging drivers in the Docker documentation.
Note
Batch currently supports a subset of the logging drivers available to the Docker daemon (shown in the LogConfiguration data type). Additional log drivers might be available in future releases of the Amazon ECS container agent.
This parameter requires version 1.18 of the Docker Remote API or greater on your container instance. To check the Docker Remote API version on your container instance, log into your container instance and run the following command: sudo docker version | grep "Server API version"
Note
The Amazon ECS container agent running on a container instance must register the logging drivers available on that instance with the ECS_AVAILABLE_LOGGING_DRIVERS environment variable before containers placed on that instance can use these log configuration options. For more information, see Amazon ECS container agent configuration in the Amazon Elastic Container Service Developer Guide .
The log driver to use for the container. The valid values listed for this parameter are log drivers that the Amazon ECS container agent can communicate with by default.
The supported log drivers are awslogs , fluentd , gelf , json-file , journald , logentries , syslog , and splunk .
Note
Jobs that are running on Fargate resources are restricted to the awslogs and splunk log drivers.
awslogs
Specifies the Amazon CloudWatch Logs logging driver. For more information, see Using the awslogs log driver in the Batch User Guide and Amazon CloudWatch Logs logging driver in the Docker documentation.
fluentd
Specifies the Fluentd logging driver. For more information, including usage and options, see Fluentd logging driver in the Docker documentation.
gelf
Specifies the Graylog Extended Format (GELF) logging driver. For more information, including usage and options, see Graylog Extended Format logging driver in the Docker documentation.
journald
Specifies the journald logging driver. For more information, including usage and options, see Journald logging driver in the Docker documentation.
json-file
Specifies the JSON file logging driver. For more information, including usage and options, see JSON File logging driver in the Docker documentation.
splunk
Specifies the Splunk logging driver. For more information, including usage and options, see Splunk logging driver in the Docker documentation.
syslog
Specifies the syslog logging driver. For more information, including usage and options, see Syslog logging driver in the Docker documentation.
Note
If you have a custom driver that's not listed earlier that you want to work with the Amazon ECS container agent, you can fork the Amazon ECS container agent project that's available on GitHub and customize it to work with that driver. We encourage you to submit pull requests for changes that you want to have included. However, Amazon Web Services doesn't currently support running modified copies of this software.
This parameter requires version 1.18 of the Docker Remote API or greater on your container instance. To check the Docker Remote API version on your container instance, log into your container instance and run the following command: sudo docker version | grep "Server API version"
The configuration options to send to the log driver. This parameter requires version 1.19 of the Docker Remote API or greater on your container instance. To check the Docker Remote API version on your container instance, log into your container instance and run the following command: sudo docker version | grep "Server API version"
The secrets to pass to the log configuration. For more information, see Specifying sensitive data in the Batch User Guide .
An object representing the secret to expose to your container. Secrets can be exposed to a container in the following ways:
secrets container definition parameter.secretOptions container definition parameter.For more information, see Specifying sensitive data in the Batch User Guide .
The name of the secret.
The secret to expose to the container. The supported values are either the full ARN of the Secrets Manager secret or the full ARN of the parameter in the Amazon Web Services Systems Manager Parameter Store.
Note
If the Amazon Web Services Systems Manager Parameter Store parameter exists in the same Region as the job you're launching, then you can use either the full ARN or name of the parameter. If the parameter exists in a different Region, then the full ARN must be specified.
The secrets to pass to the container. For more information, see Specifying sensitive data in the Batch User Guide .
An object representing the secret to expose to your container. Secrets can be exposed to a container in the following ways:
secrets container definition parameter.secretOptions container definition parameter.For more information, see Specifying sensitive data in the Batch User Guide .
The name of the secret.
The secret to expose to the container. The supported values are either the full ARN of the Secrets Manager secret or the full ARN of the parameter in the Amazon Web Services Systems Manager Parameter Store.
Note
If the Amazon Web Services Systems Manager Parameter Store parameter exists in the same Region as the job you're launching, then you can use either the full ARN or name of the parameter. If the parameter exists in a different Region, then the full ARN must be specified.
The network configuration for jobs that are running on Fargate resources. Jobs that are running on EC2 resources must not specify this parameter.
Indicates whether the job should have a public IP address. For a job that is running on Fargate resources in a private subnet to send outbound traffic to the internet (for example, to pull container images), the private subnet requires a NAT gateway be attached to route requests to the internet. For more information, see Amazon ECS task networking . The default value is "DISABLED".
The platform configuration for jobs that are running on Fargate resources. Jobs that are running on EC2 resources must not specify this parameter.
The Fargate platform version where the jobs are running. A platform version is specified only for jobs that are running on Fargate resources. If one isn't specified, the LATEST platform version is used by default. This uses a recent, approved version of the Fargate platform for compute resources. For more information, see Fargate platform versions in the Amazon Elastic Container Service Developer Guide .
An object representing the details of a node that's associated with a multi-node parallel job.
The node index for the node. Node index numbering begins at zero. This index is also available on the node with the AWS_BATCH_JOB_NODE_INDEX environment variable.
Specifies whether the current node is the main node for a multi-node parallel job.
An object representing the node properties of a multi-node parallel job.
Note
This isn't applicable to jobs that are running on Fargate resources.
The number of nodes associated with a multi-node parallel job.
Specifies the node index for the main node of a multi-node parallel job. This node index value must be fewer than the number of nodes.
A list of node ranges and their properties associated with a multi-node parallel job.
An object representing the properties of the node range for a multi-node parallel job.
The range of nodes, using node index values. A range of 0:3 indicates nodes with index values of 0 through 3 . If the starting range value is omitted (:n ), then 0 is used to start the range. If the ending range value is omitted (n: ), then the highest possible node index is used to end the range. Your accumulative node ranges must account for all nodes (0:n ). You can nest node ranges, for example 0:10 and 4:5 , in which case the 4:5 range properties override the 0:10 properties.
The container details for the node range.
The image used to start a container. This string is passed directly to the Docker daemon. Images in the Docker Hub registry are available by default. Other repositories are specified with `` repository-url /image :tag `` . Up to 255 letters (uppercase and lowercase), numbers, hyphens, underscores, colons, periods, forward slashes, and number signs are allowed. This parameter maps to Image in the Create a container section of the Docker Remote API and the IMAGE parameter of docker run .
Note
Docker image architecture must match the processor architecture of the compute resources that they're scheduled on. For example, ARM-based Docker images can only run on ARM-based compute resources.
registry/repository[:tag] or registry/repository[@digest] naming conventions. For example, ``public.ecr.aws/registry_alias /my-web-app :latest `` .012345678910.dkr.ecr.<region-name>.amazonaws.com/<repository-name> ).ubuntu or mongo ).amazon/amazon-ecs-agent ).quay.io/assemblyline/ubuntu ).This parameter is deprecated, use resourceRequirements to specify the vCPU requirements for the job definition. It's not supported for jobs running on Fargate resources. For jobs running on EC2 resources, it specifies the number of vCPUs reserved for the job.
Each vCPU is equivalent to 1,024 CPU shares. This parameter maps to CpuShares in the Create a container section of the Docker Remote API and the --cpu-shares option to docker run . The number of vCPUs must be specified but can be specified in several places. You must specify it at least once for each node.
This parameter is deprecated, use resourceRequirements to specify the memory requirements for the job definition. It's not supported for jobs running on Fargate resources. For jobs running on EC2 resources, it specifies the memory hard limit (in MiB) for a container. If your container attempts to exceed the specified number, it's terminated. You must specify at least 4 MiB of memory for a job using this parameter. The memory hard limit can be specified in several places. It must be specified for each node at least once.
The command that's passed to the container. This parameter maps to Cmd in the Create a container section of the Docker Remote API and the COMMAND parameter to docker run . For more information, see https://docs.docker.com/engine/reference/builder/#cmd .
The Amazon Resource Name (ARN) of the IAM role that the container can assume for Amazon Web Services permissions. For more information, see IAM roles for tasks in the Amazon Elastic Container Service Developer Guide .
The Amazon Resource Name (ARN) of the execution role that Batch can assume. For jobs that run on Fargate resources, you must provide an execution role. For more information, see Batch execution IAM role in the Batch User Guide .
A list of data volumes used in a job.
A data volume used in a job's container properties.
The contents of the host parameter determine whether your data volume persists on the host container instance and where it is stored. If the host parameter is empty, then the Docker daemon assigns a host path for your data volume. However, the data isn't guaranteed to persist after the containers associated with it stop running.
Note
This parameter isn't applicable to jobs that are running on Fargate resources and shouldn't be provided.
The path on the host container instance that's presented to the container. If this parameter is empty, then the Docker daemon has assigned a host path for you. If this parameter contains a file location, then the data volume persists at the specified location on the host container instance until you delete it manually. If the source path location doesn't exist on the host container instance, the Docker daemon creates it. If the location does exist, the contents of the source path folder are exported.
Note
This parameter isn't applicable to jobs that run on Fargate resources and shouldn't be provided.
The name of the volume. It can be up to 255 letters long. It can contain uppercase and lowercase letters, numbers, hyphens (-), and underscores (_). This name is referenced in the sourceVolume parameter of container definition mountPoints .
This parameter is specified when you are using an Amazon Elastic File System file system for job storage. Jobs that are running on Fargate resources must specify a platformVersion of at least 1.4.0 .
The Amazon EFS file system ID to use.
The directory within the Amazon EFS file system to mount as the root directory inside the host. If this parameter is omitted, the root of the Amazon EFS volume is used instead. Specifying / has the same effect as omitting this parameter. The maximum length is 4,096 characters.
Warning
If an EFS access point is specified in the authorizationConfig , the root directory parameter must either be omitted or set to / , which enforces the path set on the Amazon EFS access point.
Determines whether to enable encryption for Amazon EFS data in transit between the Amazon ECS host and the Amazon EFS server. Transit encryption must be enabled if Amazon EFS IAM authorization is used. If this parameter is omitted, the default value of DISABLED is used. For more information, see Encrypting data in transit in the Amazon Elastic File System User Guide .
The port to use when sending encrypted data between the Amazon ECS host and the Amazon EFS server. If you don't specify a transit encryption port, it uses the port selection strategy that the Amazon EFS mount helper uses. The value must be between 0 and 65,535. For more information, see EFS mount helper in the Amazon Elastic File System User Guide .
The authorization configuration details for the Amazon EFS file system.
The Amazon EFS access point ID to use. If an access point is specified, the root directory value specified in the EFSVolumeConfiguration must either be omitted or set to / which will enforce the path set on the EFS access point. If an access point is used, transit encryption must be enabled in the EFSVolumeConfiguration . For more information, see Working with Amazon EFS access points in the Amazon Elastic File System User Guide .
Whether or not to use the Batch job IAM role defined in a job definition when mounting the Amazon EFS file system. If enabled, transit encryption must be enabled in the EFSVolumeConfiguration . If this parameter is omitted, the default value of DISABLED is used. For more information, see Using Amazon EFS access points in the Batch User Guide . EFS IAM authorization requires that TransitEncryption be ENABLED and that a JobRoleArn is specified.
The environment variables to pass to a container. This parameter maps to Env in the Create a container section of the Docker Remote API and the --env option to docker run .
Warning
We don't recommend using plaintext environment variables for sensitive information, such as credential data.
Note
Environment variables must not start with AWS_BATCH ; this naming convention is reserved for variables that are set by the Batch service.
A key-value pair object.
The name of the key-value pair. For environment variables, this is the name of the environment variable.
The value of the key-value pair. For environment variables, this is the value of the environment variable.
The mount points for data volumes in your container. This parameter maps to Volumes in the Create a container section of the Docker Remote API and the --volume option to docker run .
Details on a Docker volume mount point that's used in a job's container properties. This parameter maps to Volumes in the Create a container section of the Docker Remote API and the --volume option to docker run.
The path on the container where the host volume is mounted.
If this value is true , the container has read-only access to the volume. Otherwise, the container can write to the volume. The default value is false .
The name of the volume to mount.
When this parameter is true, the container is given read-only access to its root file system. This parameter maps to ReadonlyRootfs in the Create a container section of the Docker Remote API and the --read-only option to docker run .
When this parameter is true, the container is given elevated permissions on the host container instance (similar to the root user). This parameter maps to Privileged in the Create a container section of the Docker Remote API and the --privileged option to docker run . The default value is false.
Note
This parameter isn't applicable to jobs that are running on Fargate resources and shouldn't be provided, or specified as false.
A list of ulimits to set in the container. This parameter maps to Ulimits in the Create a container section of the Docker Remote API and the --ulimit option to docker run .
Note
This parameter isn't applicable to jobs that are running on Fargate resources and shouldn't be provided.
The ulimit settings to pass to the container.
Note
This object isn't applicable to jobs that are running on Fargate resources.
The hard limit for the ulimit type.
The type of the ulimit .
The soft limit for the ulimit type.
The user name to use inside the container. This parameter maps to User in the Create a container section of the Docker Remote API and the --user option to docker run .
The instance type to use for a multi-node parallel job. All node groups in a multi-node parallel job must use the same instance type.
Note
This parameter isn't applicable to single-node container jobs or jobs that run on Fargate resources, and shouldn't be provided.
The type and amount of resources to assign to a container. The supported resources include GPU , MEMORY , and VCPU .
The type and amount of a resource to assign to a container. The supported resources include GPU , MEMORY , and VCPU .
The quantity of the specified resource to reserve for the container. The values vary based on the type specified.
type="GPU"
The number of physical GPUs to reserve for the container. The number of GPUs reserved for all containers in a job shouldn't exceed the number of available GPUs on the compute resource that the job is launched on.
Note
GPUs are not available for jobs that are running on Fargate resources.
type="MEMORY"
The memory hard limit (in MiB) present to the container. This parameter is supported for jobs that are running on EC2 resources. If your container attempts to exceed the memory specified, the container is terminated. This parameter maps to Memory in the Create a container section of the Docker Remote API and the --memory option to docker run . You must specify at least 4 MiB of memory for a job. This is required but can be specified in several places for multi-node parallel (MNP) jobs. It must be specified for each node at least once. This parameter maps to Memory in the Create a container section of the Docker Remote API and the --memory option to docker run .
Note
If you're trying to maximize your resource utilization by providing your jobs as much memory as possible for a particular instance type, see Memory management in the Batch User Guide .
For jobs that are running on Fargate resources, then value is the hard limit (in MiB), and must match one of the supported values and the VCPU values must be one of the values supported for that memory value.
value = 512
VCPU= 0.25value = 1024
VCPU= 0.25 or 0.5value = 2048
VCPU= 0.25, 0.5, or 1value = 3072
VCPU= 0.5, or 1value = 4096
VCPU= 0.5, 1, or 2value = 5120, 6144, or 7168
VCPU= 1 or 2value = 8192
VCPU= 1, 2, or 4value = 9216, 10240, 11264, 12288, 13312, 14336, 15360, or 16384
VCPU= 2 or 4value = 17408, 18432, 19456, 20480, 21504, 22528, 23552, 24576, 25600, 26624, 27648, 28672, 29696, or 30720
VCPU= 4type="VCPU"
The number of vCPUs reserved for the container. This parameter maps to CpuShares in the Create a container section of the Docker Remote API and the --cpu-shares option to docker run . Each vCPU is equivalent to 1,024 CPU shares. For EC2 resources, you must specify at least one vCPU. This is required but can be specified in several places; it must be specified for each node at least once.
For jobs that are running on Fargate resources, then value must match one of the supported values and the MEMORY values must be one of the values supported for that VCPU value. The supported values are 0.25, 0.5, 1, 2, and 4
value = 0.25
MEMORY= 512, 1024, or 2048value = 0.5
MEMORY= 1024, 2048, 3072, or 4096value = 1
MEMORY= 2048, 3072, 4096, 5120, 6144, 7168, or 8192value = 2
MEMORY= 4096, 5120, 6144, 7168, 8192, 9216, 10240, 11264, 12288, 13312, 14336, 15360, or 16384value = 4
MEMORY= 8192, 9216, 10240, 11264, 12288, 13312, 14336, 15360, 16384, 17408, 18432, 19456, 20480, 21504, 22528, 23552, 24576, 25600, 26624, 27648, 28672, 29696, or 30720
The type of resource to assign to a container. The supported resources include GPU , MEMORY , and VCPU .
Linux-specific modifications that are applied to the container, such as details for device mappings.
Any host devices to expose to the container. This parameter maps to Devices in the Create a container section of the Docker Remote API and the --device option to docker run .
Note
This parameter isn't applicable to jobs that are running on Fargate resources and shouldn't be provided.
An object representing a container instance host device.
Note
This object isn't applicable to jobs that are running on Fargate resources and shouldn't be provided.
The path for the device on the host container instance.
The path inside the container that's used to expose the host device. By default, the hostPath value is used.
The explicit permissions to provide to the container for the device. By default, the container has permissions for read , write , and mknod for the device.
If true, run an init process inside the container that forwards signals and reaps processes. This parameter maps to the --init option to docker run . This parameter requires version 1.25 of the Docker Remote API or greater on your container instance. To check the Docker Remote API version on your container instance, log into your container instance and run the following command: sudo docker version | grep "Server API version"
The value for the size (in MiB) of the /dev/shm volume. This parameter maps to the --shm-size option to docker run .
Note
This parameter isn't applicable to jobs that are running on Fargate resources and shouldn't be provided.
The container path, mount options, and size (in MiB) of the tmpfs mount. This parameter maps to the --tmpfs option to docker run .
Note
This parameter isn't applicable to jobs that are running on Fargate resources and shouldn't be provided.
The container path, mount options, and size of the tmpfs mount.
Note
This object isn't applicable to jobs that are running on Fargate resources.
The absolute file path in the container where the tmpfs volume is mounted.
The size (in MiB) of the tmpfs volume.
The list of tmpfs volume mount options.
Valid values: "defaults " | "ro " | "rw " | "suid " | "nosuid " | "dev " | "nodev " | "exec " | "noexec " | "sync " | "async " | "dirsync " | "remount " | "mand " | "nomand " | "atime " | "noatime " | "diratime " | "nodiratime " | "bind " | "rbind" | "unbindable" | "runbindable" | "private" | "rprivate" | "shared" | "rshared" | "slave" | "rslave" | "relatime " | "norelatime " | "strictatime " | "nostrictatime " | "mode " | "uid " | "gid " | "nr_inodes " | "nr_blocks " | "mpol "
The total amount of swap memory (in MiB) a container can use. This parameter is translated to the --memory-swap option to docker run where the value is the sum of the container memory plus the maxSwap value. For more information, see ` --memory-swap details <https://docs.docker.com/config/containers/resource_constraints/#--memory-swap-details>`__ in the Docker documentation.
If a maxSwap value of 0 is specified, the container doesn't use swap. Accepted values are 0 or any positive integer. If the maxSwap parameter is omitted, the container doesn't use the swap configuration for the container instance it is running on. A maxSwap value must be set for the swappiness parameter to be used.
Note
This parameter isn't applicable to jobs that are running on Fargate resources and shouldn't be provided.
This allows you to tune a container's memory swappiness behavior. A swappiness value of 0 causes swapping not to happen unless absolutely necessary. A swappiness value of 100 causes pages to be swapped very aggressively. Accepted values are whole numbers between 0 and 100 . If the swappiness parameter isn't specified, a default value of 60 is used. If a value isn't specified for maxSwap , then this parameter is ignored. If maxSwap is set to 0, the container doesn't use swap. This parameter maps to the --memory-swappiness option to docker run .
Consider the following when you use a per-container swap configuration.
Note
The Amazon ECS optimized AMIs don't have swap enabled by default. You must enable swap on the instance to use this feature. For more information, see Instance store swap volumes in the Amazon EC2 User Guide for Linux Instances or How do I allocate memory to work as swap space in an Amazon EC2 instance by using a swap file?
maxSwap and swappiness parameters are omitted from a job definition, each container will have a default swappiness value of 60, and the total swap usage will be limited to two times the memory reservation of the container.Note
This parameter isn't applicable to jobs that are running on Fargate resources and shouldn't be provided.
The log configuration specification for the container.
This parameter maps to LogConfig in the Create a container section of the Docker Remote API and the --log-driver option to docker run . By default, containers use the same logging driver that the Docker daemon uses. However the container might use a different logging driver than the Docker daemon by specifying a log driver with this parameter in the container definition. To use a different logging driver for a container, the log system must be configured properly on the container instance (or on a different log server for remote logging options). For more information on the options for different supported log drivers, see Configure logging drivers in the Docker documentation.
Note
Batch currently supports a subset of the logging drivers available to the Docker daemon (shown in the LogConfiguration data type).
This parameter requires version 1.18 of the Docker Remote API or greater on your container instance. To check the Docker Remote API version on your container instance, log into your container instance and run the following command: sudo docker version | grep "Server API version"
Note
The Amazon ECS container agent running on a container instance must register the logging drivers available on that instance with the ECS_AVAILABLE_LOGGING_DRIVERS environment variable before containers placed on that instance can use these log configuration options. For more information, see Amazon ECS container agent configuration in the Amazon Elastic Container Service Developer Guide .
The log driver to use for the container. The valid values listed for this parameter are log drivers that the Amazon ECS container agent can communicate with by default.
The supported log drivers are awslogs , fluentd , gelf , json-file , journald , logentries , syslog , and splunk .
Note
Jobs that are running on Fargate resources are restricted to the awslogs and splunk log drivers.
awslogs
Specifies the Amazon CloudWatch Logs logging driver. For more information, see Using the awslogs log driver in the Batch User Guide and Amazon CloudWatch Logs logging driver in the Docker documentation.
fluentd
Specifies the Fluentd logging driver. For more information, including usage and options, see Fluentd logging driver in the Docker documentation.
gelf
Specifies the Graylog Extended Format (GELF) logging driver. For more information, including usage and options, see Graylog Extended Format logging driver in the Docker documentation.
journald
Specifies the journald logging driver. For more information, including usage and options, see Journald logging driver in the Docker documentation.
json-file
Specifies the JSON file logging driver. For more information, including usage and options, see JSON File logging driver in the Docker documentation.
splunk
Specifies the Splunk logging driver. For more information, including usage and options, see Splunk logging driver in the Docker documentation.
syslog
Specifies the syslog logging driver. For more information, including usage and options, see Syslog logging driver in the Docker documentation.
Note
If you have a custom driver that's not listed earlier that you want to work with the Amazon ECS container agent, you can fork the Amazon ECS container agent project that's available on GitHub and customize it to work with that driver. We encourage you to submit pull requests for changes that you want to have included. However, Amazon Web Services doesn't currently support running modified copies of this software.
This parameter requires version 1.18 of the Docker Remote API or greater on your container instance. To check the Docker Remote API version on your container instance, log into your container instance and run the following command: sudo docker version | grep "Server API version"
The configuration options to send to the log driver. This parameter requires version 1.19 of the Docker Remote API or greater on your container instance. To check the Docker Remote API version on your container instance, log into your container instance and run the following command: sudo docker version | grep "Server API version"
The secrets to pass to the log configuration. For more information, see Specifying sensitive data in the Batch User Guide .
An object representing the secret to expose to your container. Secrets can be exposed to a container in the following ways:
secrets container definition parameter.secretOptions container definition parameter.For more information, see Specifying sensitive data in the Batch User Guide .
The name of the secret.
The secret to expose to the container. The supported values are either the full ARN of the Secrets Manager secret or the full ARN of the parameter in the Amazon Web Services Systems Manager Parameter Store.
Note
If the Amazon Web Services Systems Manager Parameter Store parameter exists in the same Region as the job you're launching, then you can use either the full ARN or name of the parameter. If the parameter exists in a different Region, then the full ARN must be specified.
The secrets for the container. For more information, see Specifying sensitive data in the Batch User Guide .
An object representing the secret to expose to your container. Secrets can be exposed to a container in the following ways:
secrets container definition parameter.secretOptions container definition parameter.For more information, see Specifying sensitive data in the Batch User Guide .
The name of the secret.
The secret to expose to the container. The supported values are either the full ARN of the Secrets Manager secret or the full ARN of the parameter in the Amazon Web Services Systems Manager Parameter Store.
Note
If the Amazon Web Services Systems Manager Parameter Store parameter exists in the same Region as the job you're launching, then you can use either the full ARN or name of the parameter. If the parameter exists in a different Region, then the full ARN must be specified.
The network configuration for jobs that are running on Fargate resources. Jobs that are running on EC2 resources must not specify this parameter.
Indicates whether the job should have a public IP address. For a job that is running on Fargate resources in a private subnet to send outbound traffic to the internet (for example, to pull container images), the private subnet requires a NAT gateway be attached to route requests to the internet. For more information, see Amazon ECS task networking . The default value is "DISABLED".
The platform configuration for jobs that are running on Fargate resources. Jobs that are running on EC2 resources must not specify this parameter.
The Fargate platform version where the jobs are running. A platform version is specified only for jobs that are running on Fargate resources. If one isn't specified, the LATEST platform version is used by default. This uses a recent, approved version of the Fargate platform for compute resources. For more information, see Fargate platform versions in the Amazon Elastic Container Service Developer Guide .
The array properties of the job, if it is an array job.
A summary of the number of array job children in each available job status. This parameter is returned for parent array jobs.
The size of the array job. This parameter is returned for parent array jobs.
The job index within the array that's associated with this job. This parameter is returned for array job children.
The timeout configuration for the job.
The time duration in seconds (measured from the job attempt's startedAt timestamp) after which Batch terminates your jobs if they have not finished. The minimum value for the timeout is 60 seconds.
The tags applied to the job.
Specifies whether to propagate the tags from the job or job definition to the corresponding Amazon ECS task. If no value is specified, the tags aren't propagated. Tags can only be propagated to the tasks during task creation. For tags with the same name, job tags are given priority over job definitions tags. If the total number of combined tags from the job and job definition is over 50, the job is moved to the FAILED state.
The platform capabilities required by the job definition. If no value is specified, it defaults to EC2 . Jobs run on Fargate resources specify FARGATE .
Exceptions
Batch.Client.exceptions.ClientExceptionBatch.Client.exceptions.ServerExceptionExamples
This example describes a job with the specified job ID.
response = client.describe_jobs(
jobs=[
'24fa2d7a-64c4-49d2-8b47-f8da4fbde8e9',
],
)
print(response)
Expected Output:
{
'jobs': [
{
'container': {
'command': [
'sleep',
'60',
],
'containerInstanceArn': 'arn:aws:ecs:us-east-1:012345678910:container-instance/5406d7cd-58bd-4b8f-9936-48d7c6b1526c',
'environment': [
],
'exitCode': 0,
'image': 'busybox',
'memory': 128,
'mountPoints': [
],
'ulimits': [
],
'vcpus': 1,
'volumes': [
],
},
'createdAt': 1480460782010,
'dependsOn': [
],
'jobDefinition': 'sleep60',
'jobId': '24fa2d7a-64c4-49d2-8b47-f8da4fbde8e9',
'jobName': 'example',
'jobQueue': 'arn:aws:batch:us-east-1:012345678910:job-queue/HighPriority',
'parameters': {
},
'startedAt': 1480460816500,
'status': 'SUCCEEDED',
'stoppedAt': 1480460880699,
},
],
'ResponseMetadata': {
'...': '...',
},
}
describe_scheduling_policies(**kwargs)¶Describes one or more of your scheduling policies.
See also: AWS API Documentation
Request Syntax
response = client.describe_scheduling_policies(
arns=[
'string',
]
)
[REQUIRED]
A list of up to 100 scheduling policy Amazon Resource Name (ARN) entries.
{
'schedulingPolicies': [
{
'name': 'string',
'arn': 'string',
'fairsharePolicy': {
'shareDecaySeconds': 123,
'computeReservation': 123,
'shareDistribution': [
{
'shareIdentifier': 'string',
'weightFactor': ...
},
]
},
'tags': {
'string': 'string'
}
},
]
}
Response Structure
The list of scheduling policies.
An object that represents a scheduling policy.
The name of the scheduling policy.
The Amazon Resource Name (ARN) of the scheduling policy. An example is ``arn:aws :batch:us-east-1 :123456789012 :scheduling-policy/HighPriority `` .
The fair share policy for the scheduling policy.
The time period to use to calculate a fair share percentage for each fair share identifier in use, in seconds. A value of zero (0) indicates that only current usage should be measured. The decay allows for more recently run jobs to have more weight than jobs that ran earlier. The maximum supported value is 604800 (1 week).
A value used to reserve some of the available maximum vCPU for fair share identifiers that have not yet been used.
The reserved ratio is ``(computeReservation /100)^*ActiveFairShares* `` where `` ActiveFairShares `` is the number of active fair share identifiers.
For example, a computeReservation value of 50 indicates that Batch should reserve 50% of the maximum available vCPU if there is only one fair share identifier, 25% if there are two fair share identifiers, and 12.5% if there are three fair share identifiers. A computeReservation value of 25 indicates that Batch should reserve 25% of the maximum available vCPU if there is only one fair share identifier, 6.25% if there are two fair share identifiers, and 1.56% if there are three fair share identifiers.
The minimum value is 0 and the maximum value is 99.
An array of SharedIdentifier objects that contain the weights for the fair share identifiers for the fair share policy. Fair share identifiers that aren't included have a default weight of 1.0 .
Specifies the weights for the fair share identifiers for the fair share policy. Fair share identifiers that aren't included have a default weight of 1.0 .
A fair share identifier or fair share identifier prefix. If the string ends with an asterisk (*), this entry specifies the weight factor to use for fair share identifiers that start with that prefix. The list of fair share identifiers in a fair share policy cannot overlap. For example, you can't have one that specifies a shareIdentifier of UserA* and another that specifies a shareIdentifier of UserA-1 .
There can be no more than 500 fair share identifiers active in a job queue.
The string is limited to 255 alphanumeric characters, optionally followed by an asterisk (*).
The weight factor for the fair share identifier. The default value is 1.0. A lower value has a higher priority for compute resources. For example, jobs that use a share identifier with a weight factor of 0.125 (1/8) get 8 times the compute resources of jobs that use a share identifier with a weight factor of 1.
The smallest supported value is 0.0001, and the largest supported value is 999.9999.
The tags that you apply to the scheduling policy to categorize and organize your resources. Each tag consists of a key and an optional value. For more information, see Tagging Amazon Web Services resources in Amazon Web Services General Reference .
Exceptions
Batch.Client.exceptions.ClientExceptionBatch.Client.exceptions.ServerExceptionget_paginator(operation_name)¶Create a paginator for an operation.
create_foo, and you'd normally invoke the
operation as client.create_foo(**kwargs), if the
create_foo operation can be paginated, you can use the
call client.get_paginator("create_foo").client.can_paginate method to
check if an operation is pageable.get_waiter(waiter_name)¶Returns an object that can wait for some condition.
list_jobs(**kwargs)¶Returns a list of Batch jobs.
You must specify only one of the following items:
You can filter the results by job status with the jobStatus parameter. If you don't specify a status, only RUNNING jobs are returned.
See also: AWS API Documentation
Request Syntax
response = client.list_jobs(
jobQueue='string',
arrayJobId='string',
multiNodeJobId='string',
jobStatus='SUBMITTED'|'PENDING'|'RUNNABLE'|'STARTING'|'RUNNING'|'SUCCEEDED'|'FAILED',
maxResults=123,
nextToken='string',
filters=[
{
'name': 'string',
'values': [
'string',
]
},
]
)
filters parameter is specified, the jobStatus parameter is ignored and jobs with any status are returned. If you don't specify a status, only RUNNING jobs are returned.ListJobs in paginated output. When this parameter is used, ListJobs only returns maxResults results in a single page and a nextToken response element. The remaining results of the initial request can be seen by sending another ListJobs request with the returned nextToken value. This value can be between 1 and 100. If this parameter isn't used, then ListJobs returns up to 100 results and a nextToken value if applicable.The nextToken value returned from a previous paginated ListJobs request where maxResults was used and the results exceeded the value of that parameter. Pagination continues from the end of the previous results that returned the nextToken value. This value is null when there are no more results to return.
Note
This token should be treated as an opaque identifier that's only used to retrieve the next items in a list and not for other programmatic purposes.
The filter to apply to the query. Only one filter can be used at a time. When the filter is used, jobStatus is ignored. The filter doesn't apply to child jobs in an array or multi-node parallel (MNP) jobs. The results are sorted by the createdAt field, with the most recent jobs being first.
JOB_NAME
The value of the filter is a case-insensitive match for the job name. If the value ends with an asterisk (*), the filter will match any job name that begins with the string before the '*'. This corresponds to the jobName value. For example, test1 matches both Test1 and test1 , and test1* matches both test1 and Test10 . When the JOB_NAME filter is used, the results are grouped by the job name and version.
JOB_DEFINITION
The value for the filter is the name or Amazon Resource Name (ARN) of the job definition. This corresponds to the jobDefinition value. The value is case sensitive. When the value for the filter is the job definition name, the results include all the jobs that used any revision of that job definition name. If the value ends with an asterisk (*), the filter will match any job definition name that begins with the string before the '*'. For example, jd1 matches only jd1 , and jd1* matches both jd1 and jd1A . The version of the job definition that's used doesn't affect the sort order. When the JOB_DEFINITION filter is used and the ARN is used (which is in the form arn:${Partition}:batch:${Region}:${Account}:job-definition/${JobDefinitionName}:${Revision} ), the results include jobs that used the specified revision of the job definition. Asterisk (*) is not supported when the ARN is used.
BEFORE_CREATED_AT
The value for the filter is the time that's before the job was created. This corresponds to the createdAt value. The value is a string representation of the number of milliseconds since 00:00:00 UTC (midnight) on January 1, 1970.
AFTER_CREATED_AT
The value for the filter is the time that's after the job was created. This corresponds to the createdAt value. The value is a string representation of the number of milliseconds since 00:00:00 UTC (midnight) on January 1, 1970.
A filter name and value pair that's used to return a more specific list of results from a ListJobs API operation.
The name of the filter. Filter names are case sensitive.
The filter values.
dict
Response Syntax
{
'jobSummaryList': [
{
'jobArn': 'string',
'jobId': 'string',
'jobName': 'string',
'createdAt': 123,
'status': 'SUBMITTED'|'PENDING'|'RUNNABLE'|'STARTING'|'RUNNING'|'SUCCEEDED'|'FAILED',
'statusReason': 'string',
'startedAt': 123,
'stoppedAt': 123,
'container': {
'exitCode': 123,
'reason': 'string'
},
'arrayProperties': {
'size': 123,
'index': 123
},
'nodeProperties': {
'isMainNode': True|False,
'numNodes': 123,
'nodeIndex': 123
},
'jobDefinition': 'string'
},
],
'nextToken': 'string'
}
Response Structure
(dict) --
jobSummaryList (list) --
A list of job summaries that match the request.
(dict) --
An object representing summary details of a job.
jobArn (string) --
The Amazon Resource Name (ARN) of the job.
jobId (string) --
The ID of the job.
jobName (string) --
The name of the job.
createdAt (integer) --
The Unix timestamp (in milliseconds) for when the job was created. For non-array jobs and parent array jobs, this is when the job entered the SUBMITTED state (at the time SubmitJob was called). For array child jobs, this is when the child job was spawned by its parent and entered the PENDING state.
status (string) --
The current status for the job.
statusReason (string) --
A short, human-readable string to provide additional details about the current status of the job.
startedAt (integer) --
The Unix timestamp for when the job was started (when the job transitioned from the STARTING state to the RUNNING state).
stoppedAt (integer) --
The Unix timestamp for when the job was stopped (when the job transitioned from the RUNNING state to a terminal state, such as SUCCEEDED or FAILED ).
container (dict) --
An object representing the details of the container that's associated with the job.
exitCode (integer) --
The exit code to return upon completion.
reason (string) --
A short (255 max characters) human-readable string to provide additional details about a running or stopped container.
arrayProperties (dict) --
The array properties of the job, if it is an array job.
size (integer) --
The size of the array job. This parameter is returned for parent array jobs.
index (integer) --
The job index within the array that's associated with this job. This parameter is returned for children of array jobs.
nodeProperties (dict) --
The node properties for a single node in a job summary list.
Note
This isn't applicable to jobs that are running on Fargate resources.
isMainNode (boolean) --
Specifies whether the current node is the main node for a multi-node parallel job.
numNodes (integer) --
The number of nodes associated with a multi-node parallel job.
nodeIndex (integer) --
The node index for the node. Node index numbering begins at zero. This index is also available on the node with the AWS_BATCH_JOB_NODE_INDEX environment variable.
jobDefinition (string) --
The Amazon Resource Name (ARN) of the job definition.
nextToken (string) --
The nextToken value to include in a future ListJobs request. When the results of a ListJobs request exceed maxResults , this value can be used to retrieve the next page of results. This value is null when there are no more results to return.
Exceptions
Batch.Client.exceptions.ClientExceptionBatch.Client.exceptions.ServerExceptionExamples
This example lists the running jobs in the HighPriority job queue.
response = client.list_jobs(
jobQueue='HighPriority',
)
print(response)
Expected Output:
{
'jobSummaryList': [
{
'jobId': 'e66ff5fd-a1ff-4640-b1a2-0b0a142f49bb',
'jobName': 'example',
},
],
'ResponseMetadata': {
'...': '...',
},
}
This example lists jobs in the HighPriority job queue that are in the SUBMITTED job status.
response = client.list_jobs(
jobQueue='HighPriority',
jobStatus='SUBMITTED',
)
print(response)
Expected Output:
{
'jobSummaryList': [
{
'jobId': '68f0c163-fbd4-44e6-9fd1-25b14a434786',
'jobName': 'example',
},
],
'ResponseMetadata': {
'...': '...',
},
}
list_scheduling_policies(**kwargs)¶Returns a list of Batch scheduling policies.
See also: AWS API Documentation
Request Syntax
response = client.list_scheduling_policies(
maxResults=123,
nextToken='string'
)
ListSchedulingPolicies in paginated output. When this parameter is used, ListSchedulingPolicies only returns maxResults results in a single page and a nextToken response element. You can see the remaining results of the initial request by sending another ListSchedulingPolicies request with the returned nextToken value. This value can be between 1 and 100. If this parameter isn't used, ListSchedulingPolicies returns up to 100 results and a nextToken value if applicable.The nextToken value that's returned from a previous paginated ListSchedulingPolicies request where maxResults was used and the results exceeded the value of that parameter. Pagination continues from the end of the previous results that returned the nextToken value. This value is null when there are no more results to return.
Note
This token should be treated as an opaque identifier that's only used to retrieve the next items in a list and not for other programmatic purposes.
dict
Response Syntax
{
'schedulingPolicies': [
{
'arn': 'string'
},
],
'nextToken': 'string'
}
Response Structure
(dict) --
schedulingPolicies (list) --
A list of scheduling policies that match the request.
(dict) --
An object that contains the details of a scheduling policy that's returned in a ListSchedulingPolicy action.
arn (string) --
Amazon Resource Name (ARN) of the scheduling policy.
nextToken (string) --
The nextToken value to include in a future ListSchedulingPolicies request. When the results of a ListSchedulingPolicies request exceed maxResults , this value can be used to retrieve the next page of results. This value is null when there are no more results to return.
Exceptions
Batch.Client.exceptions.ClientExceptionBatch.Client.exceptions.ServerExceptionLists the tags for an Batch resource. Batch resources that support tags are compute environments, jobs, job definitions, job queues, and scheduling policies. ARNs for child jobs of array and multi-node parallel (MNP) jobs are not supported.
See also: AWS API Documentation
Request Syntax
response = client.list_tags_for_resource(
resourceArn='string'
)
[REQUIRED]
The Amazon Resource Name (ARN) that identifies the resource that tags are listed for. Batch resources that support tags are compute environments, jobs, job definitions, job queues, and scheduling policies. ARNs for child jobs of array and multi-node parallel (MNP) jobs are not supported.
{
'tags': {
'string': 'string'
}
}
Response Structure
The tags for the resource.
Exceptions
Batch.Client.exceptions.ClientExceptionBatch.Client.exceptions.ServerExceptionExamples
This demonstrates calling the ListTagsForResource action.
response = client.list_tags_for_resource(
resourceArn='arn:aws:batch:us-east-1:123456789012:job-definition/sleep30:1',
)
print(response)
Expected Output:
{
'tags': {
'Department': 'Engineering',
'Stage': 'Alpha',
'User': 'JaneDoe',
},
'ResponseMetadata': {
'...': '...',
},
}
register_job_definition(**kwargs)¶Registers an Batch job definition.
See also: AWS API Documentation
Request Syntax
response = client.register_job_definition(
jobDefinitionName='string',
type='container'|'multinode',
parameters={
'string': 'string'
},
schedulingPriority=123,
containerProperties={
'image': 'string',
'vcpus': 123,
'memory': 123,
'command': [
'string',
],
'jobRoleArn': 'string',
'executionRoleArn': 'string',
'volumes': [
{
'host': {
'sourcePath': 'string'
},
'name': 'string',
'efsVolumeConfiguration': {
'fileSystemId': 'string',
'rootDirectory': 'string',
'transitEncryption': 'ENABLED'|'DISABLED',
'transitEncryptionPort': 123,
'authorizationConfig': {
'accessPointId': 'string',
'iam': 'ENABLED'|'DISABLED'
}
}
},
],
'environment': [
{
'name': 'string',
'value': 'string'
},
],
'mountPoints': [
{
'containerPath': 'string',
'readOnly': True|False,
'sourceVolume': 'string'
},
],
'readonlyRootFilesystem': True|False,
'privileged': True|False,
'ulimits': [
{
'hardLimit': 123,
'name': 'string',
'softLimit': 123
},
],
'user': 'string',
'instanceType': 'string',
'resourceRequirements': [
{
'value': 'string',
'type': 'GPU'|'VCPU'|'MEMORY'
},
],
'linuxParameters': {
'devices': [
{
'hostPath': 'string',
'containerPath': 'string',
'permissions': [
'READ'|'WRITE'|'MKNOD',
]
},
],
'initProcessEnabled': True|False,
'sharedMemorySize': 123,
'tmpfs': [
{
'containerPath': 'string',
'size': 123,
'mountOptions': [
'string',
]
},
],
'maxSwap': 123,
'swappiness': 123
},
'logConfiguration': {
'logDriver': 'json-file'|'syslog'|'journald'|'gelf'|'fluentd'|'awslogs'|'splunk',
'options': {
'string': 'string'
},
'secretOptions': [
{
'name': 'string',
'valueFrom': 'string'
},
]
},
'secrets': [
{
'name': 'string',
'valueFrom': 'string'
},
],
'networkConfiguration': {
'assignPublicIp': 'ENABLED'|'DISABLED'
},
'fargatePlatformConfiguration': {
'platformVersion': 'string'
}
},
nodeProperties={
'numNodes': 123,
'mainNode': 123,
'nodeRangeProperties': [
{
'targetNodes': 'string',
'container': {
'image': 'string',
'vcpus': 123,
'memory': 123,
'command': [
'string',
],
'jobRoleArn': 'string',
'executionRoleArn': 'string',
'volumes': [
{
'host': {
'sourcePath': 'string'
},
'name': 'string',
'efsVolumeConfiguration': {
'fileSystemId': 'string',
'rootDirectory': 'string',
'transitEncryption': 'ENABLED'|'DISABLED',
'transitEncryptionPort': 123,
'authorizationConfig': {
'accessPointId': 'string',
'iam': 'ENABLED'|'DISABLED'
}
}
},
],
'environment': [
{
'name': 'string',
'value': 'string'
},
],
'mountPoints': [
{
'containerPath': 'string',
'readOnly': True|False,
'sourceVolume': 'string'
},
],
'readonlyRootFilesystem': True|False,
'privileged': True|False,
'ulimits': [
{
'hardLimit': 123,
'name': 'string',
'softLimit': 123
},
],
'user': 'string',
'instanceType': 'string',
'resourceRequirements': [
{
'value': 'string',
'type': 'GPU'|'VCPU'|'MEMORY'
},
],
'linuxParameters': {
'devices': [
{
'hostPath': 'string',
'containerPath': 'string',
'permissions': [
'READ'|'WRITE'|'MKNOD',
]
},
],
'initProcessEnabled': True|False,
'sharedMemorySize': 123,
'tmpfs': [
{
'containerPath': 'string',
'size': 123,
'mountOptions': [
'string',
]
},
],
'maxSwap': 123,
'swappiness': 123
},
'logConfiguration': {
'logDriver': 'json-file'|'syslog'|'journald'|'gelf'|'fluentd'|'awslogs'|'splunk',
'options': {
'string': 'string'
},
'secretOptions': [
{
'name': 'string',
'valueFrom': 'string'
},
]
},
'secrets': [
{
'name': 'string',
'valueFrom': 'string'
},
],
'networkConfiguration': {
'assignPublicIp': 'ENABLED'|'DISABLED'
},
'fargatePlatformConfiguration': {
'platformVersion': 'string'
}
}
},
]
},
retryStrategy={
'attempts': 123,
'evaluateOnExit': [
{
'onStatusReason': 'string',
'onReason': 'string',
'onExitCode': 'string',
'action': 'RETRY'|'EXIT'
},
]
},
propagateTags=True|False,
timeout={
'attemptDurationSeconds': 123
},
tags={
'string': 'string'
},
platformCapabilities=[
'EC2'|'FARGATE',
]
)
[REQUIRED]
The name of the job definition to register. It can be up to 128 letters long. It can contain uppercase and lowercase letters, numbers, hyphens (-), and underscores (_).
[REQUIRED]
The type of job definition. For more information about multi-node parallel jobs, see Creating a multi-node parallel job definition in the Batch User Guide .
Note
If the job is run on Fargate resources, then multinode isn't supported.
Default parameter substitution placeholders to set in the job definition. Parameters are specified as a key-value pair mapping. Parameters in a SubmitJob request override any corresponding parameter defaults from the job definition.
The scheduling priority for jobs that are submitted with this job definition. This will only affect jobs in job queues with a fair share policy. Jobs with a higher scheduling priority will be scheduled before jobs with a lower scheduling priority.
The minimum supported value is 0 and the maximum supported value is 9999.
An object with various properties specific to single-node container-based jobs. If the job definition's type parameter is container , then you must specify either containerProperties or nodeProperties .
Note
If the job runs on Fargate resources, then you must not specify nodeProperties ; use only containerProperties .
The image used to start a container. This string is passed directly to the Docker daemon. Images in the Docker Hub registry are available by default. Other repositories are specified with `` repository-url /image :tag `` . Up to 255 letters (uppercase and lowercase), numbers, hyphens, underscores, colons, periods, forward slashes, and number signs are allowed. This parameter maps to Image in the Create a container section of the Docker Remote API and the IMAGE parameter of docker run .
Note
Docker image architecture must match the processor architecture of the compute resources that they're scheduled on. For example, ARM-based Docker images can only run on ARM-based compute resources.
registry/repository[:tag] or registry/repository[@digest] naming conventions. For example, ``public.ecr.aws/registry_alias /my-web-app :latest `` .012345678910.dkr.ecr.<region-name>.amazonaws.com/<repository-name> ).ubuntu or mongo ).amazon/amazon-ecs-agent ).quay.io/assemblyline/ubuntu ).This parameter is deprecated, use resourceRequirements to specify the vCPU requirements for the job definition. It's not supported for jobs running on Fargate resources. For jobs running on EC2 resources, it specifies the number of vCPUs reserved for the job.
Each vCPU is equivalent to 1,024 CPU shares. This parameter maps to CpuShares in the Create a container section of the Docker Remote API and the --cpu-shares option to docker run . The number of vCPUs must be specified but can be specified in several places. You must specify it at least once for each node.
This parameter is deprecated, use resourceRequirements to specify the memory requirements for the job definition. It's not supported for jobs running on Fargate resources. For jobs running on EC2 resources, it specifies the memory hard limit (in MiB) for a container. If your container attempts to exceed the specified number, it's terminated. You must specify at least 4 MiB of memory for a job using this parameter. The memory hard limit can be specified in several places. It must be specified for each node at least once.
The command that's passed to the container. This parameter maps to Cmd in the Create a container section of the Docker Remote API and the COMMAND parameter to docker run . For more information, see https://docs.docker.com/engine/reference/builder/#cmd .
The Amazon Resource Name (ARN) of the IAM role that the container can assume for Amazon Web Services permissions. For more information, see IAM roles for tasks in the Amazon Elastic Container Service Developer Guide .
The Amazon Resource Name (ARN) of the execution role that Batch can assume. For jobs that run on Fargate resources, you must provide an execution role. For more information, see Batch execution IAM role in the Batch User Guide .
A list of data volumes used in a job.
A data volume used in a job's container properties.
The contents of the host parameter determine whether your data volume persists on the host container instance and where it is stored. If the host parameter is empty, then the Docker daemon assigns a host path for your data volume. However, the data isn't guaranteed to persist after the containers associated with it stop running.
Note
This parameter isn't applicable to jobs that are running on Fargate resources and shouldn't be provided.
The path on the host container instance that's presented to the container. If this parameter is empty, then the Docker daemon has assigned a host path for you. If this parameter contains a file location, then the data volume persists at the specified location on the host container instance until you delete it manually. If the source path location doesn't exist on the host container instance, the Docker daemon creates it. If the location does exist, the contents of the source path folder are exported.
Note
This parameter isn't applicable to jobs that run on Fargate resources and shouldn't be provided.
The name of the volume. It can be up to 255 letters long. It can contain uppercase and lowercase letters, numbers, hyphens (-), and underscores (_). This name is referenced in the sourceVolume parameter of container definition mountPoints .
This parameter is specified when you are using an Amazon Elastic File System file system for job storage. Jobs that are running on Fargate resources must specify a platformVersion of at least 1.4.0 .
The Amazon EFS file system ID to use.
The directory within the Amazon EFS file system to mount as the root directory inside the host. If this parameter is omitted, the root of the Amazon EFS volume is used instead. Specifying / has the same effect as omitting this parameter. The maximum length is 4,096 characters.
Warning
If an EFS access point is specified in the authorizationConfig , the root directory parameter must either be omitted or set to / , which enforces the path set on the Amazon EFS access point.
Determines whether to enable encryption for Amazon EFS data in transit between the Amazon ECS host and the Amazon EFS server. Transit encryption must be enabled if Amazon EFS IAM authorization is used. If this parameter is omitted, the default value of DISABLED is used. For more information, see Encrypting data in transit in the Amazon Elastic File System User Guide .
The port to use when sending encrypted data between the Amazon ECS host and the Amazon EFS server. If you don't specify a transit encryption port, it uses the port selection strategy that the Amazon EFS mount helper uses. The value must be between 0 and 65,535. For more information, see EFS mount helper in the Amazon Elastic File System User Guide .
The authorization configuration details for the Amazon EFS file system.
The Amazon EFS access point ID to use. If an access point is specified, the root directory value specified in the EFSVolumeConfiguration must either be omitted or set to / which will enforce the path set on the EFS access point. If an access point is used, transit encryption must be enabled in the EFSVolumeConfiguration . For more information, see Working with Amazon EFS access points in the Amazon Elastic File System User Guide .
Whether or not to use the Batch job IAM role defined in a job definition when mounting the Amazon EFS file system. If enabled, transit encryption must be enabled in the EFSVolumeConfiguration . If this parameter is omitted, the default value of DISABLED is used. For more information, see Using Amazon EFS access points in the Batch User Guide . EFS IAM authorization requires that TransitEncryption be ENABLED and that a JobRoleArn is specified.
The environment variables to pass to a container. This parameter maps to Env in the Create a container section of the Docker Remote API and the --env option to docker run .
Warning
We don't recommend using plaintext environment variables for sensitive information, such as credential data.
Note
Environment variables must not start with AWS_BATCH ; this naming convention is reserved for variables that are set by the Batch service.
A key-value pair object.
The name of the key-value pair. For environment variables, this is the name of the environment variable.
The value of the key-value pair. For environment variables, this is the value of the environment variable.
The mount points for data volumes in your container. This parameter maps to Volumes in the Create a container section of the Docker Remote API and the --volume option to docker run .
Details on a Docker volume mount point that's used in a job's container properties. This parameter maps to Volumes in the Create a container section of the Docker Remote API and the --volume option to docker run.
The path on the container where the host volume is mounted.
If this value is true , the container has read-only access to the volume. Otherwise, the container can write to the volume. The default value is false .
The name of the volume to mount.
When this parameter is true, the container is given read-only access to its root file system. This parameter maps to ReadonlyRootfs in the Create a container section of the Docker Remote API and the --read-only option to docker run .
When this parameter is true, the container is given elevated permissions on the host container instance (similar to the root user). This parameter maps to Privileged in the Create a container section of the Docker Remote API and the --privileged option to docker run . The default value is false.
Note
This parameter isn't applicable to jobs that are running on Fargate resources and shouldn't be provided, or specified as false.
A list of ulimits to set in the container. This parameter maps to Ulimits in the Create a container section of the Docker Remote API and the --ulimit option to docker run .
Note
This parameter isn't applicable to jobs that are running on Fargate resources and shouldn't be provided.
The ulimit settings to pass to the container.
Note
This object isn't applicable to jobs that are running on Fargate resources.
The hard limit for the ulimit type.
The type of the ulimit .
The soft limit for the ulimit type.
The user name to use inside the container. This parameter maps to User in the Create a container section of the Docker Remote API and the --user option to docker run .
The instance type to use for a multi-node parallel job. All node groups in a multi-node parallel job must use the same instance type.
Note
This parameter isn't applicable to single-node container jobs or jobs that run on Fargate resources, and shouldn't be provided.
The type and amount of resources to assign to a container. The supported resources include GPU , MEMORY , and VCPU .
The type and amount of a resource to assign to a container. The supported resources include GPU , MEMORY , and VCPU .
The quantity of the specified resource to reserve for the container. The values vary based on the type specified.
type="GPU"
The number of physical GPUs to reserve for the container. The number of GPUs reserved for all containers in a job shouldn't exceed the number of available GPUs on the compute resource that the job is launched on.
Note
GPUs are not available for jobs that are running on Fargate resources.
type="MEMORY"
The memory hard limit (in MiB) present to the container. This parameter is supported for jobs that are running on EC2 resources. If your container attempts to exceed the memory specified, the container is terminated. This parameter maps to Memory in the Create a container section of the Docker Remote API and the --memory option to docker run . You must specify at least 4 MiB of memory for a job. This is required but can be specified in several places for multi-node parallel (MNP) jobs. It must be specified for each node at least once. This parameter maps to Memory in the Create a container section of the Docker Remote API and the --memory option to docker run .
Note
If you're trying to maximize your resource utilization by providing your jobs as much memory as possible for a particular instance type, see Memory management in the Batch User Guide .
For jobs that are running on Fargate resources, then value is the hard limit (in MiB), and must match one of the supported values and the VCPU values must be one of the values supported for that memory value.
value = 512
VCPU= 0.25value = 1024
VCPU= 0.25 or 0.5value = 2048
VCPU= 0.25, 0.5, or 1value = 3072
VCPU= 0.5, or 1value = 4096
VCPU= 0.5, 1, or 2value = 5120, 6144, or 7168
VCPU= 1 or 2value = 8192
VCPU= 1, 2, or 4value = 9216, 10240, 11264, 12288, 13312, 14336, 15360, or 16384
VCPU= 2 or 4value = 17408, 18432, 19456, 20480, 21504, 22528, 23552, 24576, 25600, 26624, 27648, 28672, 29696, or 30720
VCPU= 4type="VCPU"
The number of vCPUs reserved for the container. This parameter maps to CpuShares in the Create a container section of the Docker Remote API and the --cpu-shares option to docker run . Each vCPU is equivalent to 1,024 CPU shares. For EC2 resources, you must specify at least one vCPU. This is required but can be specified in several places; it must be specified for each node at least once.
For jobs that are running on Fargate resources, then value must match one of the supported values and the MEMORY values must be one of the values supported for that VCPU value. The supported values are 0.25, 0.5, 1, 2, and 4
value = 0.25
MEMORY= 512, 1024, or 2048value = 0.5
MEMORY= 1024, 2048, 3072, or 4096value = 1
MEMORY= 2048, 3072, 4096, 5120, 6144, 7168, or 8192value = 2
MEMORY= 4096, 5120, 6144, 7168, 8192, 9216, 10240, 11264, 12288, 13312, 14336, 15360, or 16384value = 4
MEMORY= 8192, 9216, 10240, 11264, 12288, 13312, 14336, 15360, 16384, 17408, 18432, 19456, 20480, 21504, 22528, 23552, 24576, 25600, 26624, 27648, 28672, 29696, or 30720
The type of resource to assign to a container. The supported resources include GPU , MEMORY , and VCPU .
Linux-specific modifications that are applied to the container, such as details for device mappings.
Any host devices to expose to the container. This parameter maps to Devices in the Create a container section of the Docker Remote API and the --device option to docker run .
Note
This parameter isn't applicable to jobs that are running on Fargate resources and shouldn't be provided.
An object representing a container instance host device.
Note
This object isn't applicable to jobs that are running on Fargate resources and shouldn't be provided.
The path for the device on the host container instance.
The path inside the container that's used to expose the host device. By default, the hostPath value is used.
The explicit permissions to provide to the container for the device. By default, the container has permissions for read , write , and mknod for the device.
If true, run an init process inside the container that forwards signals and reaps processes. This parameter maps to the --init option to docker run . This parameter requires version 1.25 of the Docker Remote API or greater on your container instance. To check the Docker Remote API version on your container instance, log into your container instance and run the following command: sudo docker version | grep "Server API version"
The value for the size (in MiB) of the /dev/shm volume. This parameter maps to the --shm-size option to docker run .
Note
This parameter isn't applicable to jobs that are running on Fargate resources and shouldn't be provided.
The container path, mount options, and size (in MiB) of the tmpfs mount. This parameter maps to the --tmpfs option to docker run .
Note
This parameter isn't applicable to jobs that are running on Fargate resources and shouldn't be provided.
The container path, mount options, and size of the tmpfs mount.
Note
This object isn't applicable to jobs that are running on Fargate resources.
The absolute file path in the container where the tmpfs volume is mounted.
The size (in MiB) of the tmpfs volume.
The list of tmpfs volume mount options.
Valid values: "defaults " | "ro " | "rw " | "suid " | "nosuid " | "dev " | "nodev " | "exec " | "noexec " | "sync " | "async " | "dirsync " | "remount " | "mand " | "nomand " | "atime " | "noatime " | "diratime " | "nodiratime " | "bind " | "rbind" | "unbindable" | "runbindable" | "private" | "rprivate" | "shared" | "rshared" | "slave" | "rslave" | "relatime " | "norelatime " | "strictatime " | "nostrictatime " | "mode " | "uid " | "gid " | "nr_inodes " | "nr_blocks " | "mpol "
The total amount of swap memory (in MiB) a container can use. This parameter is translated to the --memory-swap option to docker run where the value is the sum of the container memory plus the maxSwap value. For more information, see ` --memory-swap details <https://docs.docker.com/config/containers/resource_constraints/#--memory-swap-details>`__ in the Docker documentation.
If a maxSwap value of 0 is specified, the container doesn't use swap. Accepted values are 0 or any positive integer. If the maxSwap parameter is omitted, the container doesn't use the swap configuration for the container instance it is running on. A maxSwap value must be set for the swappiness parameter to be used.
Note
This parameter isn't applicable to jobs that are running on Fargate resources and shouldn't be provided.
This allows you to tune a container's memory swappiness behavior. A swappiness value of 0 causes swapping not to happen unless absolutely necessary. A swappiness value of 100 causes pages to be swapped very aggressively. Accepted values are whole numbers between 0 and 100 . If the swappiness parameter isn't specified, a default value of 60 is used. If a value isn't specified for maxSwap , then this parameter is ignored. If maxSwap is set to 0, the container doesn't use swap. This parameter maps to the --memory-swappiness option to docker run .
Consider the following when you use a per-container swap configuration.
Note
The Amazon ECS optimized AMIs don't have swap enabled by default. You must enable swap on the instance to use this feature. For more information, see Instance store swap volumes in the Amazon EC2 User Guide for Linux Instances or How do I allocate memory to work as swap space in an Amazon EC2 instance by using a swap file?
maxSwap and swappiness parameters are omitted from a job definition, each container will have a default swappiness value of 60, and the total swap usage will be limited to two times the memory reservation of the container.Note
This parameter isn't applicable to jobs that are running on Fargate resources and shouldn't be provided.
The log configuration specification for the container.
This parameter maps to LogConfig in the Create a container section of the Docker Remote API and the --log-driver option to docker run . By default, containers use the same logging driver that the Docker daemon uses. However the container might use a different logging driver than the Docker daemon by specifying a log driver with this parameter in the container definition. To use a different logging driver for a container, the log system must be configured properly on the container instance (or on a different log server for remote logging options). For more information on the options for different supported log drivers, see Configure logging drivers in the Docker documentation.
Note
Batch currently supports a subset of the logging drivers available to the Docker daemon (shown in the LogConfiguration data type).
This parameter requires version 1.18 of the Docker Remote API or greater on your container instance. To check the Docker Remote API version on your container instance, log into your container instance and run the following command: sudo docker version | grep "Server API version"
Note
The Amazon ECS container agent running on a container instance must register the logging drivers available on that instance with the ECS_AVAILABLE_LOGGING_DRIVERS environment variable before containers placed on that instance can use these log configuration options. For more information, see Amazon ECS container agent configuration in the Amazon Elastic Container Service Developer Guide .
The log driver to use for the container. The valid values listed for this parameter are log drivers that the Amazon ECS container agent can communicate with by default.
The supported log drivers are awslogs , fluentd , gelf , json-file , journald , logentries , syslog , and splunk .
Note
Jobs that are running on Fargate resources are restricted to the awslogs and splunk log drivers.
awslogs
Specifies the Amazon CloudWatch Logs logging driver. For more information, see Using the awslogs log driver in the Batch User Guide and Amazon CloudWatch Logs logging driver in the Docker documentation.
fluentd
Specifies the Fluentd logging driver. For more information, including usage and options, see Fluentd logging driver in the Docker documentation.
gelf
Specifies the Graylog Extended Format (GELF) logging driver. For more information, including usage and options, see Graylog Extended Format logging driver in the Docker documentation.
journald
Specifies the journald logging driver. For more information, including usage and options, see Journald logging driver in the Docker documentation.
json-file
Specifies the JSON file logging driver. For more information, including usage and options, see JSON File logging driver in the Docker documentation.
splunk
Specifies the Splunk logging driver. For more information, including usage and options, see Splunk logging driver in the Docker documentation.
syslog
Specifies the syslog logging driver. For more information, including usage and options, see Syslog logging driver in the Docker documentation.
Note
If you have a custom driver that's not listed earlier that you want to work with the Amazon ECS container agent, you can fork the Amazon ECS container agent project that's available on GitHub and customize it to work with that driver. We encourage you to submit pull requests for changes that you want to have included. However, Amazon Web Services doesn't currently support running modified copies of this software.
This parameter requires version 1.18 of the Docker Remote API or greater on your container instance. To check the Docker Remote API version on your container instance, log into your container instance and run the following command: sudo docker version | grep "Server API version"
The configuration options to send to the log driver. This parameter requires version 1.19 of the Docker Remote API or greater on your container instance. To check the Docker Remote API version on your container instance, log into your container instance and run the following command: sudo docker version | grep "Server API version"
The secrets to pass to the log configuration. For more information, see Specifying sensitive data in the Batch User Guide .
An object representing the secret to expose to your container. Secrets can be exposed to a container in the following ways:
secrets container definition parameter.secretOptions container definition parameter.For more information, see Specifying sensitive data in the Batch User Guide .
The name of the secret.
The secret to expose to the container. The supported values are either the full ARN of the Secrets Manager secret or the full ARN of the parameter in the Amazon Web Services Systems Manager Parameter Store.
Note
If the Amazon Web Services Systems Manager Parameter Store parameter exists in the same Region as the job you're launching, then you can use either the full ARN or name of the parameter. If the parameter exists in a different Region, then the full ARN must be specified.
The secrets for the container. For more information, see Specifying sensitive data in the Batch User Guide .
An object representing the secret to expose to your container. Secrets can be exposed to a container in the following ways:
secrets container definition parameter.secretOptions container definition parameter.For more information, see Specifying sensitive data in the Batch User Guide .
The name of the secret.
The secret to expose to the container. The supported values are either the full ARN of the Secrets Manager secret or the full ARN of the parameter in the Amazon Web Services Systems Manager Parameter Store.
Note
If the Amazon Web Services Systems Manager Parameter Store parameter exists in the same Region as the job you're launching, then you can use either the full ARN or name of the parameter. If the parameter exists in a different Region, then the full ARN must be specified.
The network configuration for jobs that are running on Fargate resources. Jobs that are running on EC2 resources must not specify this parameter.
Indicates whether the job should have a public IP address. For a job that is running on Fargate resources in a private subnet to send outbound traffic to the internet (for example, to pull container images), the private subnet requires a NAT gateway be attached to route requests to the internet. For more information, see Amazon ECS task networking . The default value is "DISABLED".
The platform configuration for jobs that are running on Fargate resources. Jobs that are running on EC2 resources must not specify this parameter.
The Fargate platform version where the jobs are running. A platform version is specified only for jobs that are running on Fargate resources. If one isn't specified, the LATEST platform version is used by default. This uses a recent, approved version of the Fargate platform for compute resources. For more information, see Fargate platform versions in the Amazon Elastic Container Service Developer Guide .
An object with various properties specific to multi-node parallel jobs. If you specify node properties for a job, it becomes a multi-node parallel job. For more information, see Multi-node Parallel Jobs in the Batch User Guide . If the job definition's type parameter is container , then you must specify either containerProperties or nodeProperties .
Note
If the job runs on Fargate resources, then you must not specify nodeProperties ; use containerProperties instead.
The number of nodes associated with a multi-node parallel job.
Specifies the node index for the main node of a multi-node parallel job. This node index value must be fewer than the number of nodes.
A list of node ranges and their properties associated with a multi-node parallel job.
An object representing the properties of the node range for a multi-node parallel job.
The range of nodes, using node index values. A range of 0:3 indicates nodes with index values of 0 through 3 . If the starting range value is omitted (:n ), then 0 is used to start the range. If the ending range value is omitted (n: ), then the highest possible node index is used to end the range. Your accumulative node ranges must account for all nodes (0:n ). You can nest node ranges, for example 0:10 and 4:5 , in which case the 4:5 range properties override the 0:10 properties.
The container details for the node range.
The image used to start a container. This string is passed directly to the Docker daemon. Images in the Docker Hub registry are available by default. Other repositories are specified with `` repository-url /image :tag `` . Up to 255 letters (uppercase and lowercase), numbers, hyphens, underscores, colons, periods, forward slashes, and number signs are allowed. This parameter maps to Image in the Create a container section of the Docker Remote API and the IMAGE parameter of docker run .
Note
Docker image architecture must match the processor architecture of the compute resources that they're scheduled on. For example, ARM-based Docker images can only run on ARM-based compute resources.
registry/repository[:tag] or registry/repository[@digest] naming conventions. For example, ``public.ecr.aws/registry_alias /my-web-app :latest `` .012345678910.dkr.ecr.<region-name>.amazonaws.com/<repository-name> ).ubuntu or mongo ).amazon/amazon-ecs-agent ).quay.io/assemblyline/ubuntu ).This parameter is deprecated, use resourceRequirements to specify the vCPU requirements for the job definition. It's not supported for jobs running on Fargate resources. For jobs running on EC2 resources, it specifies the number of vCPUs reserved for the job.
Each vCPU is equivalent to 1,024 CPU shares. This parameter maps to CpuShares in the Create a container section of the Docker Remote API and the --cpu-shares option to docker run . The number of vCPUs must be specified but can be specified in several places. You must specify it at least once for each node.
This parameter is deprecated, use resourceRequirements to specify the memory requirements for the job definition. It's not supported for jobs running on Fargate resources. For jobs running on EC2 resources, it specifies the memory hard limit (in MiB) for a container. If your container attempts to exceed the specified number, it's terminated. You must specify at least 4 MiB of memory for a job using this parameter. The memory hard limit can be specified in several places. It must be specified for each node at least once.
The command that's passed to the container. This parameter maps to Cmd in the Create a container section of the Docker Remote API and the COMMAND parameter to docker run . For more information, see https://docs.docker.com/engine/reference/builder/#cmd .
The Amazon Resource Name (ARN) of the IAM role that the container can assume for Amazon Web Services permissions. For more information, see IAM roles for tasks in the Amazon Elastic Container Service Developer Guide .
The Amazon Resource Name (ARN) of the execution role that Batch can assume. For jobs that run on Fargate resources, you must provide an execution role. For more information, see Batch execution IAM role in the Batch User Guide .
A list of data volumes used in a job.
A data volume used in a job's container properties.
The contents of the host parameter determine whether your data volume persists on the host container instance and where it is stored. If the host parameter is empty, then the Docker daemon assigns a host path for your data volume. However, the data isn't guaranteed to persist after the containers associated with it stop running.
Note
This parameter isn't applicable to jobs that are running on Fargate resources and shouldn't be provided.
The path on the host container instance that's presented to the container. If this parameter is empty, then the Docker daemon has assigned a host path for you. If this parameter contains a file location, then the data volume persists at the specified location on the host container instance until you delete it manually. If the source path location doesn't exist on the host container instance, the Docker daemon creates it. If the location does exist, the contents of the source path folder are exported.
Note
This parameter isn't applicable to jobs that run on Fargate resources and shouldn't be provided.
The name of the volume. It can be up to 255 letters long. It can contain uppercase and lowercase letters, numbers, hyphens (-), and underscores (_). This name is referenced in the sourceVolume parameter of container definition mountPoints .
This parameter is specified when you are using an Amazon Elastic File System file system for job storage. Jobs that are running on Fargate resources must specify a platformVersion of at least 1.4.0 .
The Amazon EFS file system ID to use.
The directory within the Amazon EFS file system to mount as the root directory inside the host. If this parameter is omitted, the root of the Amazon EFS volume is used instead. Specifying / has the same effect as omitting this parameter. The maximum length is 4,096 characters.
Warning
If an EFS access point is specified in the authorizationConfig , the root directory parameter must either be omitted or set to / , which enforces the path set on the Amazon EFS access point.
Determines whether to enable encryption for Amazon EFS data in transit between the Amazon ECS host and the Amazon EFS server. Transit encryption must be enabled if Amazon EFS IAM authorization is used. If this parameter is omitted, the default value of DISABLED is used. For more information, see Encrypting data in transit in the Amazon Elastic File System User Guide .
The port to use when sending encrypted data between the Amazon ECS host and the Amazon EFS server. If you don't specify a transit encryption port, it uses the port selection strategy that the Amazon EFS mount helper uses. The value must be between 0 and 65,535. For more information, see EFS mount helper in the Amazon Elastic File System User Guide .
The authorization configuration details for the Amazon EFS file system.
The Amazon EFS access point ID to use. If an access point is specified, the root directory value specified in the EFSVolumeConfiguration must either be omitted or set to / which will enforce the path set on the EFS access point. If an access point is used, transit encryption must be enabled in the EFSVolumeConfiguration . For more information, see Working with Amazon EFS access points in the Amazon Elastic File System User Guide .
Whether or not to use the Batch job IAM role defined in a job definition when mounting the Amazon EFS file system. If enabled, transit encryption must be enabled in the EFSVolumeConfiguration . If this parameter is omitted, the default value of DISABLED is used. For more information, see Using Amazon EFS access points in the Batch User Guide . EFS IAM authorization requires that TransitEncryption be ENABLED and that a JobRoleArn is specified.
The environment variables to pass to a container. This parameter maps to Env in the Create a container section of the Docker Remote API and the --env option to docker run .
Warning
We don't recommend using plaintext environment variables for sensitive information, such as credential data.
Note
Environment variables must not start with AWS_BATCH ; this naming convention is reserved for variables that are set by the Batch service.
A key-value pair object.
The name of the key-value pair. For environment variables, this is the name of the environment variable.
The value of the key-value pair. For environment variables, this is the value of the environment variable.
The mount points for data volumes in your container. This parameter maps to Volumes in the Create a container section of the Docker Remote API and the --volume option to docker run .
Details on a Docker volume mount point that's used in a job's container properties. This parameter maps to Volumes in the Create a container section of the Docker Remote API and the --volume option to docker run.
The path on the container where the host volume is mounted.
If this value is true , the container has read-only access to the volume. Otherwise, the container can write to the volume. The default value is false .
The name of the volume to mount.
When this parameter is true, the container is given read-only access to its root file system. This parameter maps to ReadonlyRootfs in the Create a container section of the Docker Remote API and the --read-only option to docker run .
When this parameter is true, the container is given elevated permissions on the host container instance (similar to the root user). This parameter maps to Privileged in the Create a container section of the Docker Remote API and the --privileged option to docker run . The default value is false.
Note
This parameter isn't applicable to jobs that are running on Fargate resources and shouldn't be provided, or specified as false.
A list of ulimits to set in the container. This parameter maps to Ulimits in the Create a container section of the Docker Remote API and the --ulimit option to docker run .
Note
This parameter isn't applicable to jobs that are running on Fargate resources and shouldn't be provided.
The ulimit settings to pass to the container.
Note
This object isn't applicable to jobs that are running on Fargate resources.
The hard limit for the ulimit type.
The type of the ulimit .
The soft limit for the ulimit type.
The user name to use inside the container. This parameter maps to User in the Create a container section of the Docker Remote API and the --user option to docker run .
The instance type to use for a multi-node parallel job. All node groups in a multi-node parallel job must use the same instance type.
Note
This parameter isn't applicable to single-node container jobs or jobs that run on Fargate resources, and shouldn't be provided.
The type and amount of resources to assign to a container. The supported resources include GPU , MEMORY , and VCPU .
The type and amount of a resource to assign to a container. The supported resources include GPU , MEMORY , and VCPU .
The quantity of the specified resource to reserve for the container. The values vary based on the type specified.
type="GPU"
The number of physical GPUs to reserve for the container. The number of GPUs reserved for all containers in a job shouldn't exceed the number of available GPUs on the compute resource that the job is launched on.
Note
GPUs are not available for jobs that are running on Fargate resources.
type="MEMORY"
The memory hard limit (in MiB) present to the container. This parameter is supported for jobs that are running on EC2 resources. If your container attempts to exceed the memory specified, the container is terminated. This parameter maps to Memory in the Create a container section of the Docker Remote API and the --memory option to docker run . You must specify at least 4 MiB of memory for a job. This is required but can be specified in several places for multi-node parallel (MNP) jobs. It must be specified for each node at least once. This parameter maps to Memory in the Create a container section of the Docker Remote API and the --memory option to docker run .
Note
If you're trying to maximize your resource utilization by providing your jobs as much memory as possible for a particular instance type, see Memory management in the Batch User Guide .
For jobs that are running on Fargate resources, then value is the hard limit (in MiB), and must match one of the supported values and the VCPU values must be one of the values supported for that memory value.
value = 512
VCPU= 0.25value = 1024
VCPU= 0.25 or 0.5value = 2048
VCPU= 0.25, 0.5, or 1value = 3072
VCPU= 0.5, or 1value = 4096
VCPU= 0.5, 1, or 2value = 5120, 6144, or 7168
VCPU= 1 or 2value = 8192
VCPU= 1, 2, or 4value = 9216, 10240, 11264, 12288, 13312, 14336, 15360, or 16384
VCPU= 2 or 4value = 17408, 18432, 19456, 20480, 21504, 22528, 23552, 24576, 25600, 26624, 27648, 28672, 29696, or 30720
VCPU= 4type="VCPU"
The number of vCPUs reserved for the container. This parameter maps to CpuShares in the Create a container section of the Docker Remote API and the --cpu-shares option to docker run . Each vCPU is equivalent to 1,024 CPU shares. For EC2 resources, you must specify at least one vCPU. This is required but can be specified in several places; it must be specified for each node at least once.
For jobs that are running on Fargate resources, then value must match one of the supported values and the MEMORY values must be one of the values supported for that VCPU value. The supported values are 0.25, 0.5, 1, 2, and 4
value = 0.25
MEMORY= 512, 1024, or 2048value = 0.5
MEMORY= 1024, 2048, 3072, or 4096value = 1
MEMORY= 2048, 3072, 4096, 5120, 6144, 7168, or 8192value = 2
MEMORY= 4096, 5120, 6144, 7168, 8192, 9216, 10240, 11264, 12288, 13312, 14336, 15360, or 16384value = 4
MEMORY= 8192, 9216, 10240, 11264, 12288, 13312, 14336, 15360, 16384, 17408, 18432, 19456, 20480, 21504, 22528, 23552, 24576, 25600, 26624, 27648, 28672, 29696, or 30720
The type of resource to assign to a container. The supported resources include GPU , MEMORY , and VCPU .
Linux-specific modifications that are applied to the container, such as details for device mappings.
Any host devices to expose to the container. This parameter maps to Devices in the Create a container section of the Docker Remote API and the --device option to docker run .
Note
This parameter isn't applicable to jobs that are running on Fargate resources and shouldn't be provided.
An object representing a container instance host device.
Note
This object isn't applicable to jobs that are running on Fargate resources and shouldn't be provided.
The path for the device on the host container instance.
The path inside the container that's used to expose the host device. By default, the hostPath value is used.
The explicit permissions to provide to the container for the device. By default, the container has permissions for read , write , and mknod for the device.
If true, run an init process inside the container that forwards signals and reaps processes. This parameter maps to the --init option to docker run . This parameter requires version 1.25 of the Docker Remote API or greater on your container instance. To check the Docker Remote API version on your container instance, log into your container instance and run the following command: sudo docker version | grep "Server API version"
The value for the size (in MiB) of the /dev/shm volume. This parameter maps to the --shm-size option to docker run .
Note
This parameter isn't applicable to jobs that are running on Fargate resources and shouldn't be provided.
The container path, mount options, and size (in MiB) of the tmpfs mount. This parameter maps to the --tmpfs option to docker run .
Note
This parameter isn't applicable to jobs that are running on Fargate resources and shouldn't be provided.
The container path, mount options, and size of the tmpfs mount.
Note
This object isn't applicable to jobs that are running on Fargate resources.
The absolute file path in the container where the tmpfs volume is mounted.
The size (in MiB) of the tmpfs volume.
The list of tmpfs volume mount options.
Valid values: "defaults " | "ro " | "rw " | "suid " | "nosuid " | "dev " | "nodev " | "exec " | "noexec " | "sync " | "async " | "dirsync " | "remount " | "mand " | "nomand " | "atime " | "noatime " | "diratime " | "nodiratime " | "bind " | "rbind" | "unbindable" | "runbindable" | "private" | "rprivate" | "shared" | "rshared" | "slave" | "rslave" | "relatime " | "norelatime " | "strictatime " | "nostrictatime " | "mode " | "uid " | "gid " | "nr_inodes " | "nr_blocks " | "mpol "
The total amount of swap memory (in MiB) a container can use. This parameter is translated to the --memory-swap option to docker run where the value is the sum of the container memory plus the maxSwap value. For more information, see ` --memory-swap details <https://docs.docker.com/config/containers/resource_constraints/#--memory-swap-details>`__ in the Docker documentation.
If a maxSwap value of 0 is specified, the container doesn't use swap. Accepted values are 0 or any positive integer. If the maxSwap parameter is omitted, the container doesn't use the swap configuration for the container instance it is running on. A maxSwap value must be set for the swappiness parameter to be used.
Note
This parameter isn't applicable to jobs that are running on Fargate resources and shouldn't be provided.
This allows you to tune a container's memory swappiness behavior. A swappiness value of 0 causes swapping not to happen unless absolutely necessary. A swappiness value of 100 causes pages to be swapped very aggressively. Accepted values are whole numbers between 0 and 100 . If the swappiness parameter isn't specified, a default value of 60 is used. If a value isn't specified for maxSwap , then this parameter is ignored. If maxSwap is set to 0, the container doesn't use swap. This parameter maps to the --memory-swappiness option to docker run .
Consider the following when you use a per-container swap configuration.
Note
The Amazon ECS optimized AMIs don't have swap enabled by default. You must enable swap on the instance to use this feature. For more information, see Instance store swap volumes in the Amazon EC2 User Guide for Linux Instances or How do I allocate memory to work as swap space in an Amazon EC2 instance by using a swap file?
maxSwap and swappiness parameters are omitted from a job definition, each container will have a default swappiness value of 60, and the total swap usage will be limited to two times the memory reservation of the container.Note
This parameter isn't applicable to jobs that are running on Fargate resources and shouldn't be provided.
The log configuration specification for the container.
This parameter maps to LogConfig in the Create a container section of the Docker Remote API and the --log-driver option to docker run . By default, containers use the same logging driver that the Docker daemon uses. However the container might use a different logging driver than the Docker daemon by specifying a log driver with this parameter in the container definition. To use a different logging driver for a container, the log system must be configured properly on the container instance (or on a different log server for remote logging options). For more information on the options for different supported log drivers, see Configure logging drivers in the Docker documentation.
Note
Batch currently supports a subset of the logging drivers available to the Docker daemon (shown in the LogConfiguration data type).
This parameter requires version 1.18 of the Docker Remote API or greater on your container instance. To check the Docker Remote API version on your container instance, log into your container instance and run the following command: sudo docker version | grep "Server API version"
Note
The Amazon ECS container agent running on a container instance must register the logging drivers available on that instance with the ECS_AVAILABLE_LOGGING_DRIVERS environment variable before containers placed on that instance can use these log configuration options. For more information, see Amazon ECS container agent configuration in the Amazon Elastic Container Service Developer Guide .
The log driver to use for the container. The valid values listed for this parameter are log drivers that the Amazon ECS container agent can communicate with by default.
The supported log drivers are awslogs , fluentd , gelf , json-file , journald , logentries , syslog , and splunk .
Note
Jobs that are running on Fargate resources are restricted to the awslogs and splunk log drivers.
awslogs
Specifies the Amazon CloudWatch Logs logging driver. For more information, see Using the awslogs log driver in the Batch User Guide and Amazon CloudWatch Logs logging driver in the Docker documentation.
fluentd
Specifies the Fluentd logging driver. For more information, including usage and options, see Fluentd logging driver in the Docker documentation.
gelf
Specifies the Graylog Extended Format (GELF) logging driver. For more information, including usage and options, see Graylog Extended Format logging driver in the Docker documentation.
journald
Specifies the journald logging driver. For more information, including usage and options, see Journald logging driver in the Docker documentation.
json-file
Specifies the JSON file logging driver. For more information, including usage and options, see JSON File logging driver in the Docker documentation.
splunk
Specifies the Splunk logging driver. For more information, including usage and options, see Splunk logging driver in the Docker documentation.
syslog
Specifies the syslog logging driver. For more information, including usage and options, see Syslog logging driver in the Docker documentation.
Note
If you have a custom driver that's not listed earlier that you want to work with the Amazon ECS container agent, you can fork the Amazon ECS container agent project that's available on GitHub and customize it to work with that driver. We encourage you to submit pull requests for changes that you want to have included. However, Amazon Web Services doesn't currently support running modified copies of this software.
This parameter requires version 1.18 of the Docker Remote API or greater on your container instance. To check the Docker Remote API version on your container instance, log into your container instance and run the following command: sudo docker version | grep "Server API version"
The configuration options to send to the log driver. This parameter requires version 1.19 of the Docker Remote API or greater on your container instance. To check the Docker Remote API version on your container instance, log into your container instance and run the following command: sudo docker version | grep "Server API version"
The secrets to pass to the log configuration. For more information, see Specifying sensitive data in the Batch User Guide .
An object representing the secret to expose to your container. Secrets can be exposed to a container in the following ways:
secrets container definition parameter.secretOptions container definition parameter.For more information, see Specifying sensitive data in the Batch User Guide .
The name of the secret.
The secret to expose to the container. The supported values are either the full ARN of the Secrets Manager secret or the full ARN of the parameter in the Amazon Web Services Systems Manager Parameter Store.
Note
If the Amazon Web Services Systems Manager Parameter Store parameter exists in the same Region as the job you're launching, then you can use either the full ARN or name of the parameter. If the parameter exists in a different Region, then the full ARN must be specified.
The secrets for the container. For more information, see Specifying sensitive data in the Batch User Guide .
An object representing the secret to expose to your container. Secrets can be exposed to a container in the following ways:
secrets container definition parameter.secretOptions container definition parameter.For more information, see Specifying sensitive data in the Batch User Guide .
The name of the secret.
The secret to expose to the container. The supported values are either the full ARN of the Secrets Manager secret or the full ARN of the parameter in the Amazon Web Services Systems Manager Parameter Store.
Note
If the Amazon Web Services Systems Manager Parameter Store parameter exists in the same Region as the job you're launching, then you can use either the full ARN or name of the parameter. If the parameter exists in a different Region, then the full ARN must be specified.
The network configuration for jobs that are running on Fargate resources. Jobs that are running on EC2 resources must not specify this parameter.
Indicates whether the job should have a public IP address. For a job that is running on Fargate resources in a private subnet to send outbound traffic to the internet (for example, to pull container images), the private subnet requires a NAT gateway be attached to route requests to the internet. For more information, see Amazon ECS task networking . The default value is "DISABLED".
The platform configuration for jobs that are running on Fargate resources. Jobs that are running on EC2 resources must not specify this parameter.
The Fargate platform version where the jobs are running. A platform version is specified only for jobs that are running on Fargate resources. If one isn't specified, the LATEST platform version is used by default. This uses a recent, approved version of the Fargate platform for compute resources. For more information, see Fargate platform versions in the Amazon Elastic Container Service Developer Guide .
The retry strategy to use for failed jobs that are submitted with this job definition. Any retry strategy that's specified during a SubmitJob operation overrides the retry strategy defined here. If a job is terminated due to a timeout, it isn't retried.
The number of times to move a job to the RUNNABLE status. You can specify between 1 and 10 attempts. If the value of attempts is greater than one, the job is retried on failure the same number of attempts as the value.
Array of up to 5 objects that specify conditions under which the job should be retried or failed. If this parameter is specified, then the attempts parameter must also be specified.
Specifies a set of conditions to be met, and an action to take (RETRY or EXIT ) if all conditions are met.
Contains a glob pattern to match against the StatusReason returned for a job. The pattern can be up to 512 characters in length. It can contain letters, numbers, periods (.), colons (:), and white space (including spaces or tabs). It can optionally end with an asterisk (*) so that only the start of the string needs to be an exact match.
The string can be between 1 and 512 characters in length.
Contains a glob pattern to match against the Reason returned for a job. The pattern can be up to 512 characters in length. It can contain letters, numbers, periods (.), colons (:), and white space (including spaces and tabs). It can optionally end with an asterisk (*) so that only the start of the string needs to be an exact match.
The string can be between 1 and 512 characters in length.
Contains a glob pattern to match against the decimal representation of the ExitCode returned for a job. The pattern can be up to 512 characters in length. It can contain only numbers, and can optionally end with an asterisk (*) so that only the start of the string needs to be an exact match.
The string can be between 1 and 512 characters in length.
Specifies the action to take if all of the specified conditions (onStatusReason , onReason , and onExitCode ) are met. The values aren't case sensitive.
FAILED state.The timeout configuration for jobs that are submitted with this job definition, after which Batch terminates your jobs if they have not finished. If a job is terminated due to a timeout, it isn't retried. The minimum value for the timeout is 60 seconds. Any timeout configuration that's specified during a SubmitJob operation overrides the timeout configuration defined here. For more information, see Job Timeouts in the Batch User Guide .
The time duration in seconds (measured from the job attempt's startedAt timestamp) after which Batch terminates your jobs if they have not finished. The minimum value for the timeout is 60 seconds.
The tags that you apply to the job definition to help you categorize and organize your resources. Each tag consists of a key and an optional value. For more information, see Tagging Amazon Web Services Resources in Batch User Guide .
The platform capabilities required by the job definition. If no value is specified, it defaults to EC2 . To run the job on Fargate resources, specify FARGATE .
dict
Response Syntax
{
'jobDefinitionName': 'string',
'jobDefinitionArn': 'string',
'revision': 123
}
Response Structure
(dict) --
jobDefinitionName (string) --
The name of the job definition.
jobDefinitionArn (string) --
The Amazon Resource Name (ARN) of the job definition.
revision (integer) --
The revision of the job definition.
Exceptions
Batch.Client.exceptions.ClientExceptionBatch.Client.exceptions.ServerExceptionExamples
This example registers a job definition for a simple container job.
response = client.register_job_definition(
type='container',
containerProperties={
'command': [
'sleep',
'10',
],
'image': 'busybox',
'resourceRequirements': [
{
'type': 'MEMORY',
'value': '128',
},
{
'type': 'VCPU',
'value': '1',
},
],
},
jobDefinitionName='sleep10',
)
print(response)
Expected Output:
{
'jobDefinitionArn': 'arn:aws:batch:us-east-1:012345678910:job-definition/sleep10:1',
'jobDefinitionName': 'sleep10',
'revision': 1,
'ResponseMetadata': {
'...': '...',
},
}
This demonstrates calling the RegisterJobDefinition action, including tags.
response = client.register_job_definition(
type='container',
containerProperties={
'command': [
'sleep',
'30',
],
'image': 'busybox',
'resourceRequirements': [
{
'type': 'MEMORY',
'value': '128',
},
{
'type': 'VCPU',
'value': '1',
},
],
},
jobDefinitionName='sleep30',
tags={
'Department': 'Engineering',
'User': 'JaneDoe',
},
)
print(response)
Expected Output:
{
'jobDefinitionArn': 'arn:aws:batch:us-east-1:012345678910:job-definition/sleep30:1',
'jobDefinitionName': 'sleep30',
'revision': 1,
'ResponseMetadata': {
'...': '...',
},
}
submit_job(**kwargs)¶Submits an Batch job from a job definition. Parameters that are specified during SubmitJob override parameters defined in the job definition. vCPU and memory requirements that are specified in the resourceRequirements objects in the job definition are the exception. They can't be overridden this way using the memory and vcpus parameters. Rather, you must specify updates to job definition parameters in a resourceRequirements object that's included in the containerOverrides parameter.
Note
Job queues with a scheduling policy are limited to 500 active fair share identifiers at a time.
Warning
Jobs that run on Fargate resources can't be guaranteed to run for more than 14 days. This is because, after 14 days, Fargate resources might become unavailable and job might be terminated.
See also: AWS API Documentation
Request Syntax
response = client.submit_job(
jobName='string',
jobQueue='string',
shareIdentifier='string',
schedulingPriorityOverride=123,
arrayProperties={
'size': 123
},
dependsOn=[
{
'jobId': 'string',
'type': 'N_TO_N'|'SEQUENTIAL'
},
],
jobDefinition='string',
parameters={
'string': 'string'
},
containerOverrides={
'vcpus': 123,
'memory': 123,
'command': [
'string',
],
'instanceType': 'string',
'environment': [
{
'name': 'string',
'value': 'string'
},
],
'resourceRequirements': [
{
'value': 'string',
'type': 'GPU'|'VCPU'|'MEMORY'
},
]
},
nodeOverrides={
'numNodes': 123,
'nodePropertyOverrides': [
{
'targetNodes': 'string',
'containerOverrides': {
'vcpus': 123,
'memory': 123,
'command': [
'string',
],
'instanceType': 'string',
'environment': [
{
'name': 'string',
'value': 'string'
},
],
'resourceRequirements': [
{
'value': 'string',
'type': 'GPU'|'VCPU'|'MEMORY'
},
]
}
},
]
},
retryStrategy={
'attempts': 123,
'evaluateOnExit': [
{
'onStatusReason': 'string',
'onReason': 'string',
'onExitCode': 'string',
'action': 'RETRY'|'EXIT'
},
]
},
propagateTags=True|False,
timeout={
'attemptDurationSeconds': 123
},
tags={
'string': 'string'
}
)
[REQUIRED]
The name of the job. It can be up to 128 letters long. The first character must be alphanumeric, can contain uppercase and lowercase letters, numbers, hyphens (-), and underscores (_).
[REQUIRED]
The job queue where the job is submitted. You can specify either the name or the Amazon Resource Name (ARN) of the queue.
The scheduling priority for the job. This will only affect jobs in job queues with a fair share policy. Jobs with a higher scheduling priority will be scheduled before jobs with a lower scheduling priority. This will override any scheduling priority in the job definition.
The minimum supported value is 0 and the maximum supported value is 9999.
The array properties for the submitted job, such as the size of the array. The array size can be between 2 and 10,000. If you specify array properties for a job, it becomes an array job. For more information, see Array Jobs in the Batch User Guide .
The size of the array job.
A list of dependencies for the job. A job can depend upon a maximum of 20 jobs. You can specify a SEQUENTIAL type dependency without specifying a job ID for array jobs so that each child array job completes sequentially, starting at index 0. You can also specify an N_TO_N type dependency with a job ID for array jobs. In that case, each index child of this job must wait for the corresponding index child of each dependency to complete before it can begin.
An object representing an Batch job dependency.
The job ID of the Batch job associated with this dependency.
The type of the job dependency.
[REQUIRED]
The job definition used by this job. This value can be one of name , name:revision , or the Amazon Resource Name (ARN) for the job definition. If name is specified without a revision then the latest active revision is used.
Additional parameters passed to the job that replace parameter substitution placeholders that are set in the job definition. Parameters are specified as a key and value pair mapping. Parameters in a SubmitJob request override any corresponding parameter defaults from the job definition.
A list of container overrides in the JSON format that specify the name of a container in the specified job definition and the overrides it receives. You can override the default command for a container, which is specified in the job definition or the Docker image, with a command override. You can also override existing environment variables on a container or add new environment variables to it with an environment override.
This parameter is deprecated, use resourceRequirements to override the vcpus parameter that's set in the job definition. It's not supported for jobs running on Fargate resources. For jobs running on EC2 resources, it overrides the vcpus parameter set in the job definition, but doesn't override any vCPU requirement specified in the resourceRequirements structure in the job definition. To override vCPU requirements that are specified in the resourceRequirements structure in the job definition, resourceRequirements must be specified in the SubmitJob request, with type set to VCPU and value set to the new value. For more information, see Can't override job definition resource requirements in the Batch User Guide .
This parameter is deprecated, use resourceRequirements to override the memory requirements specified in the job definition. It's not supported for jobs running on Fargate resources. For jobs running on EC2 resources, it overrides the memory parameter set in the job definition, but doesn't override any memory requirement specified in the resourceRequirements structure in the job definition. To override memory requirements that are specified in the resourceRequirements structure in the job definition, resourceRequirements must be specified in the SubmitJob request, with type set to MEMORY and value set to the new value. For more information, see Can't override job definition resource requirements in the Batch User Guide .
The command to send to the container that overrides the default command from the Docker image or the job definition.
The instance type to use for a multi-node parallel job.
Note
This parameter isn't applicable to single-node container jobs or jobs that run on Fargate resources, and shouldn't be provided.
The environment variables to send to the container. You can add new environment variables, which are added to the container at launch, or you can override the existing environment variables from the Docker image or the job definition.
Note
Environment variables must not start with AWS_BATCH ; this naming convention is reserved for variables that are set by the Batch service.
A key-value pair object.
The name of the key-value pair. For environment variables, this is the name of the environment variable.
The value of the key-value pair. For environment variables, this is the value of the environment variable.
The type and amount of resources to assign to a container. This overrides the settings in the job definition. The supported resources include GPU , MEMORY , and VCPU .
The type and amount of a resource to assign to a container. The supported resources include GPU , MEMORY , and VCPU .
The quantity of the specified resource to reserve for the container. The values vary based on the type specified.
type="GPU"
The number of physical GPUs to reserve for the container. The number of GPUs reserved for all containers in a job shouldn't exceed the number of available GPUs on the compute resource that the job is launched on.
Note
GPUs are not available for jobs that are running on Fargate resources.
type="MEMORY"
The memory hard limit (in MiB) present to the container. This parameter is supported for jobs that are running on EC2 resources. If your container attempts to exceed the memory specified, the container is terminated. This parameter maps to Memory in the Create a container section of the Docker Remote API and the --memory option to docker run . You must specify at least 4 MiB of memory for a job. This is required but can be specified in several places for multi-node parallel (MNP) jobs. It must be specified for each node at least once. This parameter maps to Memory in the Create a container section of the Docker Remote API and the --memory option to docker run .
Note
If you're trying to maximize your resource utilization by providing your jobs as much memory as possible for a particular instance type, see Memory management in the Batch User Guide .
For jobs that are running on Fargate resources, then value is the hard limit (in MiB), and must match one of the supported values and the VCPU values must be one of the values supported for that memory value.
value = 512
VCPU= 0.25value = 1024
VCPU= 0.25 or 0.5value = 2048
VCPU= 0.25, 0.5, or 1value = 3072
VCPU= 0.5, or 1value = 4096
VCPU= 0.5, 1, or 2value = 5120, 6144, or 7168
VCPU= 1 or 2value = 8192
VCPU= 1, 2, or 4value = 9216, 10240, 11264, 12288, 13312, 14336, 15360, or 16384
VCPU= 2 or 4value = 17408, 18432, 19456, 20480, 21504, 22528, 23552, 24576, 25600, 26624, 27648, 28672, 29696, or 30720
VCPU= 4type="VCPU"
The number of vCPUs reserved for the container. This parameter maps to CpuShares in the Create a container section of the Docker Remote API and the --cpu-shares option to docker run . Each vCPU is equivalent to 1,024 CPU shares. For EC2 resources, you must specify at least one vCPU. This is required but can be specified in several places; it must be specified for each node at least once.
For jobs that are running on Fargate resources, then value must match one of the supported values and the MEMORY values must be one of the values supported for that VCPU value. The supported values are 0.25, 0.5, 1, 2, and 4
value = 0.25
MEMORY= 512, 1024, or 2048value = 0.5
MEMORY= 1024, 2048, 3072, or 4096value = 1
MEMORY= 2048, 3072, 4096, 5120, 6144, 7168, or 8192value = 2
MEMORY= 4096, 5120, 6144, 7168, 8192, 9216, 10240, 11264, 12288, 13312, 14336, 15360, or 16384value = 4
MEMORY= 8192, 9216, 10240, 11264, 12288, 13312, 14336, 15360, 16384, 17408, 18432, 19456, 20480, 21504, 22528, 23552, 24576, 25600, 26624, 27648, 28672, 29696, or 30720
The type of resource to assign to a container. The supported resources include GPU , MEMORY , and VCPU .
A list of node overrides in JSON format that specify the node range to target and the container overrides for that node range.
Note
This parameter isn't applicable to jobs that are running on Fargate resources; use containerOverrides instead.
The number of nodes to use with a multi-node parallel job. This value overrides the number of nodes that are specified in the job definition. To use this override:
: or n: ).The node property overrides for the job.
Object representing any node overrides to a job definition that's used in a SubmitJob API operation.
The range of nodes, using node index values, that's used to override. A range of 0:3 indicates nodes with index values of 0 through 3 . If the starting range value is omitted (:n ), then 0 is used to start the range. If the ending range value is omitted (n: ), then the highest possible node index is used to end the range.
The overrides that should be sent to a node range.
This parameter is deprecated, use resourceRequirements to override the vcpus parameter that's set in the job definition. It's not supported for jobs running on Fargate resources. For jobs running on EC2 resources, it overrides the vcpus parameter set in the job definition, but doesn't override any vCPU requirement specified in the resourceRequirements structure in the job definition. To override vCPU requirements that are specified in the resourceRequirements structure in the job definition, resourceRequirements must be specified in the SubmitJob request, with type set to VCPU and value set to the new value. For more information, see Can't override job definition resource requirements in the Batch User Guide .
This parameter is deprecated, use resourceRequirements to override the memory requirements specified in the job definition. It's not supported for jobs running on Fargate resources. For jobs running on EC2 resources, it overrides the memory parameter set in the job definition, but doesn't override any memory requirement specified in the resourceRequirements structure in the job definition. To override memory requirements that are specified in the resourceRequirements structure in the job definition, resourceRequirements must be specified in the SubmitJob request, with type set to MEMORY and value set to the new value. For more information, see Can't override job definition resource requirements in the Batch User Guide .
The command to send to the container that overrides the default command from the Docker image or the job definition.
The instance type to use for a multi-node parallel job.
Note
This parameter isn't applicable to single-node container jobs or jobs that run on Fargate resources, and shouldn't be provided.
The environment variables to send to the container. You can add new environment variables, which are added to the container at launch, or you can override the existing environment variables from the Docker image or the job definition.
Note
Environment variables must not start with AWS_BATCH ; this naming convention is reserved for variables that are set by the Batch service.
A key-value pair object.
The name of the key-value pair. For environment variables, this is the name of the environment variable.
The value of the key-value pair. For environment variables, this is the value of the environment variable.
The type and amount of resources to assign to a container. This overrides the settings in the job definition. The supported resources include GPU , MEMORY , and VCPU .
The type and amount of a resource to assign to a container. The supported resources include GPU , MEMORY , and VCPU .
The quantity of the specified resource to reserve for the container. The values vary based on the type specified.
type="GPU"
The number of physical GPUs to reserve for the container. The number of GPUs reserved for all containers in a job shouldn't exceed the number of available GPUs on the compute resource that the job is launched on.
Note
GPUs are not available for jobs that are running on Fargate resources.
type="MEMORY"
The memory hard limit (in MiB) present to the container. This parameter is supported for jobs that are running on EC2 resources. If your container attempts to exceed the memory specified, the container is terminated. This parameter maps to Memory in the Create a container section of the Docker Remote API and the --memory option to docker run . You must specify at least 4 MiB of memory for a job. This is required but can be specified in several places for multi-node parallel (MNP) jobs. It must be specified for each node at least once. This parameter maps to Memory in the Create a container section of the Docker Remote API and the --memory option to docker run .
Note
If you're trying to maximize your resource utilization by providing your jobs as much memory as possible for a particular instance type, see Memory management in the Batch User Guide .
For jobs that are running on Fargate resources, then value is the hard limit (in MiB), and must match one of the supported values and the VCPU values must be one of the values supported for that memory value.
value = 512
VCPU= 0.25value = 1024
VCPU= 0.25 or 0.5value = 2048
VCPU= 0.25, 0.5, or 1value = 3072
VCPU= 0.5, or 1value = 4096
VCPU= 0.5, 1, or 2value = 5120, 6144, or 7168
VCPU= 1 or 2value = 8192
VCPU= 1, 2, or 4value = 9216, 10240, 11264, 12288, 13312, 14336, 15360, or 16384
VCPU= 2 or 4value = 17408, 18432, 19456, 20480, 21504, 22528, 23552, 24576, 25600, 26624, 27648, 28672, 29696, or 30720
VCPU= 4type="VCPU"
The number of vCPUs reserved for the container. This parameter maps to CpuShares in the Create a container section of the Docker Remote API and the --cpu-shares option to docker run . Each vCPU is equivalent to 1,024 CPU shares. For EC2 resources, you must specify at least one vCPU. This is required but can be specified in several places; it must be specified for each node at least once.
For jobs that are running on Fargate resources, then value must match one of the supported values and the MEMORY values must be one of the values supported for that VCPU value. The supported values are 0.25, 0.5, 1, 2, and 4
value = 0.25
MEMORY= 512, 1024, or 2048value = 0.5
MEMORY= 1024, 2048, 3072, or 4096value = 1
MEMORY= 2048, 3072, 4096, 5120, 6144, 7168, or 8192value = 2
MEMORY= 4096, 5120, 6144, 7168, 8192, 9216, 10240, 11264, 12288, 13312, 14336, 15360, or 16384value = 4
MEMORY= 8192, 9216, 10240, 11264, 12288, 13312, 14336, 15360, 16384, 17408, 18432, 19456, 20480, 21504, 22528, 23552, 24576, 25600, 26624, 27648, 28672, 29696, or 30720
The type of resource to assign to a container. The supported resources include GPU , MEMORY , and VCPU .
The retry strategy to use for failed jobs from this SubmitJob operation. When a retry strategy is specified here, it overrides the retry strategy defined in the job definition.
The number of times to move a job to the RUNNABLE status. You can specify between 1 and 10 attempts. If the value of attempts is greater than one, the job is retried on failure the same number of attempts as the value.
Array of up to 5 objects that specify conditions under which the job should be retried or failed. If this parameter is specified, then the attempts parameter must also be specified.
Specifies a set of conditions to be met, and an action to take (RETRY or EXIT ) if all conditions are met.
Contains a glob pattern to match against the StatusReason returned for a job. The pattern can be up to 512 characters in length. It can contain letters, numbers, periods (.), colons (:), and white space (including spaces or tabs). It can optionally end with an asterisk (*) so that only the start of the string needs to be an exact match.
The string can be between 1 and 512 characters in length.
Contains a glob pattern to match against the Reason returned for a job. The pattern can be up to 512 characters in length. It can contain letters, numbers, periods (.), colons (:), and white space (including spaces and tabs). It can optionally end with an asterisk (*) so that only the start of the string needs to be an exact match.
The string can be between 1 and 512 characters in length.
Contains a glob pattern to match against the decimal representation of the ExitCode returned for a job. The pattern can be up to 512 characters in length. It can contain only numbers, and can optionally end with an asterisk (*) so that only the start of the string needs to be an exact match.
The string can be between 1 and 512 characters in length.
Specifies the action to take if all of the specified conditions (onStatusReason , onReason , and onExitCode ) are met. The values aren't case sensitive.
FAILED state. When specified, this overrides the tag propagation setting in the job definition.The timeout configuration for this SubmitJob operation. You can specify a timeout duration after which Batch terminates your jobs if they haven't finished. If a job is terminated due to a timeout, it isn't retried. The minimum value for the timeout is 60 seconds. This configuration overrides any timeout configuration specified in the job definition. For array jobs, child jobs have the same timeout configuration as the parent job. For more information, see Job Timeouts in the Amazon Elastic Container Service Developer Guide .
The time duration in seconds (measured from the job attempt's startedAt timestamp) after which Batch terminates your jobs if they have not finished. The minimum value for the timeout is 60 seconds.
The tags that you apply to the job request to help you categorize and organize your resources. Each tag consists of a key and an optional value. For more information, see Tagging Amazon Web Services Resources in Amazon Web Services General Reference .
dict
Response Syntax
{
'jobArn': 'string',
'jobName': 'string',
'jobId': 'string'
}
Response Structure
(dict) --
jobArn (string) --
The Amazon Resource Name (ARN) for the job.
jobName (string) --
The name of the job.
jobId (string) --
The unique identifier for the job.
Exceptions
Batch.Client.exceptions.ClientExceptionBatch.Client.exceptions.ServerExceptionExamples
This example submits a simple container job called example to the HighPriority job queue.
response = client.submit_job(
jobDefinition='sleep60',
jobName='example',
jobQueue='HighPriority',
)
print(response)
Expected Output:
{
'jobId': '876da822-4198-45f2-a252-6cea32512ea8',
'jobName': 'example',
'ResponseMetadata': {
'...': '...',
},
}
tag_resource(**kwargs)¶Associates the specified tags to a resource with the specified resourceArn . If existing tags on a resource aren't specified in the request parameters, they aren't changed. When a resource is deleted, the tags that are associated with that resource are deleted as well. Batch resources that support tags are compute environments, jobs, job definitions, job queues, and scheduling policies. ARNs for child jobs of array and multi-node parallel (MNP) jobs are not supported.
See also: AWS API Documentation
Request Syntax
response = client.tag_resource(
resourceArn='string',
tags={
'string': 'string'
}
)
[REQUIRED]
The Amazon Resource Name (ARN) of the resource that tags are added to. Batch resources that support tags are compute environments, jobs, job definitions, job queues, and scheduling policies. ARNs for child jobs of array and multi-node parallel (MNP) jobs are not supported.
[REQUIRED]
The tags that you apply to the resource to help you categorize and organize your resources. Each tag consists of a key and an optional value. For more information, see Tagging Amazon Web Services Resources in Amazon Web Services General Reference .
dict
Response Syntax
{}
Response Structure
Exceptions
Batch.Client.exceptions.ClientExceptionBatch.Client.exceptions.ServerExceptionExamples
This demonstrates calling the TagResource action.
response = client.tag_resource(
resourceArn='arn:aws:batch:us-east-1:123456789012:job-definition/sleep30:1',
tags={
'Stage': 'Alpha',
},
)
print(response)
Expected Output:
{
'ResponseMetadata': {
'...': '...',
},
}
terminate_job(**kwargs)¶Terminates a job in a job queue. Jobs that are in the STARTING or RUNNING state are terminated, which causes them to transition to FAILED . Jobs that have not progressed to the STARTING state are cancelled.
See also: AWS API Documentation
Request Syntax
response = client.terminate_job(
jobId='string',
reason='string'
)
[REQUIRED]
The Batch job ID of the job to terminate.
[REQUIRED]
A message to attach to the job that explains the reason for canceling it. This message is returned by future DescribeJobs operations on the job. This message is also recorded in the Batch activity logs.
dict
Response Syntax
{}
Response Structure
Exceptions
Batch.Client.exceptions.ClientExceptionBatch.Client.exceptions.ServerExceptionExamples
This example terminates a job with the specified job ID.
response = client.terminate_job(
jobId='61e743ed-35e4-48da-b2de-5c8333821c84',
reason='Terminating job.',
)
print(response)
Expected Output:
{
'ResponseMetadata': {
'...': '...',
},
}
untag_resource(**kwargs)¶Deletes specified tags from an Batch resource.
See also: AWS API Documentation
Request Syntax
response = client.untag_resource(
resourceArn='string',
tagKeys=[
'string',
]
)
[REQUIRED]
The Amazon Resource Name (ARN) of the resource from which to delete tags. Batch resources that support tags are compute environments, jobs, job definitions, job queues, and scheduling policies. ARNs for child jobs of array and multi-node parallel (MNP) jobs are not supported.
[REQUIRED]
The keys of the tags to be removed.
dict
Response Syntax
{}
Response Structure
Exceptions
Batch.Client.exceptions.ClientExceptionBatch.Client.exceptions.ServerExceptionExamples
This demonstrates calling the UntagResource action.
response = client.untag_resource(
resourceArn='arn:aws:batch:us-east-1:123456789012:job-definition/sleep30:1',
tagKeys=[
'Stage',
],
)
print(response)
Expected Output:
{
'ResponseMetadata': {
'...': '...',
},
}
update_compute_environment(**kwargs)¶Updates an Batch compute environment.
See also: AWS API Documentation
Request Syntax
response = client.update_compute_environment(
computeEnvironment='string',
state='ENABLED'|'DISABLED',
unmanagedvCpus=123,
computeResources={
'minvCpus': 123,
'maxvCpus': 123,
'desiredvCpus': 123,
'subnets': [
'string',
],
'securityGroupIds': [
'string',
],
'allocationStrategy': 'BEST_FIT_PROGRESSIVE'|'SPOT_CAPACITY_OPTIMIZED',
'instanceTypes': [
'string',
],
'ec2KeyPair': 'string',
'instanceRole': 'string',
'tags': {
'string': 'string'
},
'placementGroup': 'string',
'bidPercentage': 123,
'launchTemplate': {
'launchTemplateId': 'string',
'launchTemplateName': 'string',
'version': 'string'
},
'ec2Configuration': [
{
'imageType': 'string',
'imageIdOverride': 'string'
},
],
'updateToLatestImageVersion': True|False,
'type': 'EC2'|'SPOT'|'FARGATE'|'FARGATE_SPOT',
'imageId': 'string'
},
serviceRole='string',
updatePolicy={
'terminateJobsOnUpdate': True|False,
'jobExecutionTimeoutMinutes': 123
}
)
[REQUIRED]
The name or full Amazon Resource Name (ARN) of the compute environment to update.
The state of the compute environment. Compute environments in the ENABLED state can accept jobs from a queue and scale in or out automatically based on the workload demand of its associated queues.
If the state is ENABLED , then the Batch scheduler can attempt to place jobs from an associated job queue on the compute resources within the environment. If the compute environment is managed, then it can scale its instances out or in automatically, based on the job queue demand.
If the state is DISABLED , then the Batch scheduler doesn't attempt to place jobs within the environment. Jobs in a STARTING or RUNNING state continue to progress normally. Managed compute environments in the DISABLED state don't scale out. However, they scale in to minvCpus value after instances become idle.
Details of the compute resources managed by the compute environment. Required for a managed compute environment. For more information, see Compute Environments in the Batch User Guide .
The minimum number of Amazon EC2 vCPUs that an environment should maintain (even if the compute environment is DISABLED ).
Note
This parameter isn't applicable to jobs that are running on Fargate resources, and shouldn't be specified.
The maximum number of Amazon EC2 vCPUs that an environment can reach.
Note
With both BEST_FIT_PROGRESSIVE and SPOT_CAPACITY_OPTIMIZED allocation strategies, Batch might need to exceed maxvCpus to meet your capacity requirements. In this event, Batch never exceeds maxvCpus by more than a single instance. That is, no more than a single instance from among those specified in your compute environment.
The desired number of Amazon EC2 vCPUS in the compute environment. Batch modifies this value between the minimum and maximum values based on job queue demand.
Note
This parameter isn't applicable to jobs that are running on Fargate resources, and shouldn't be specified.
The VPC subnets where the compute resources are launched. Fargate compute resources can contain up to 16 subnets. For Fargate compute resources, providing an empty list will be handled as if this parameter wasn't specified and no change is made. For EC2 compute resources, providing an empty list removes the VPC subnets from the compute resource. For more information, see VPCs and subnets in the Amazon VPC User Guide .
When updating a compute environment, changing the VPC subnets requires an infrastructure update of the compute environment. For more information, see Updating compute environments in the Batch User Guide .
The Amazon EC2 security groups associated with instances launched in the compute environment. This parameter is required for Fargate compute resources, where it can contain up to 5 security groups. For Fargate compute resources, providing an empty list is handled as if this parameter wasn't specified and no change is made. For EC2 compute resources, providing an empty list removes the security groups from the compute resource.
When updating a compute environment, changing the EC2 security groups requires an infrastructure update of the compute environment. For more information, see Updating compute environments in the Batch User Guide .
The allocation strategy to use for the compute resource if not enough instances of the best fitting instance type can be allocated. This might be because of availability of the instance type in the Region or Amazon EC2 service limits . For more information, see Allocation strategies in the Batch User Guide .
When updating a compute environment, changing the allocation strategy requires an infrastructure update of the compute environment. For more information, see Updating compute environments in the Batch User Guide . BEST_FIT isn't supported when updating a compute environment.
Note
This parameter isn't applicable to jobs that are running on Fargate resources, and shouldn't be specified.
BEST_FIT_PROGRESSIVE
Batch will select additional instance types that are large enough to meet the requirements of the jobs in the queue, with a preference for instance types with a lower cost per unit vCPU. If additional instances of the previously selected instance types aren't available, Batch will select new instance types.
SPOT_CAPACITY_OPTIMIZED
Batch will select one or more instance types that are large enough to meet the requirements of the jobs in the queue, with a preference for instance types that are less likely to be interrupted. This allocation strategy is only available for Spot Instance compute resources.
With both BEST_FIT_PROGRESSIVE and SPOT_CAPACITY_OPTIMIZED strategies, Batch might need to go above maxvCpus to meet your capacity requirements. In this event, Batch never exceeds maxvCpus by more than a single instance.
The instances types that can be launched. You can specify instance families to launch any instance type within those families (for example, c5 or p3 ), or you can specify specific sizes within a family (such as c5.8xlarge ). You can also choose optimal to select instance types (from the C4, M4, and R4 instance families) that match the demand of your job queues.
When updating a compute environment, changing this setting requires an infrastructure update of the compute environment. For more information, see Updating compute environments in the Batch User Guide .
Note
This parameter isn't applicable to jobs that are running on Fargate resources, and shouldn't be specified.
Note
When you create a compute environment, the instance types that you select for the compute environment must share the same architecture. For example, you can't mix x86 and ARM instances in the same compute environment.
Note
Currently, optimal uses instance types from the C4, M4, and R4 instance families. In Regions that don't have instance types from those instance families, instance types from the C5, M5. and R5 instance families are used.
The Amazon EC2 key pair that's used for instances launched in the compute environment. You can use this key pair to log in to your instances with SSH. To remove the Amazon EC2 key pair, set this value to an empty string.
When updating a compute environment, changing the EC2 key pair requires an infrastructure update of the compute environment. For more information, see Updating compute environments in the Batch User Guide .
Note
This parameter isn't applicable to jobs that are running on Fargate resources, and shouldn't be specified.
The Amazon ECS instance profile applied to Amazon EC2 instances in a compute environment. You can specify the short name or full Amazon Resource Name (ARN) of an instance profile. For example, `` ecsInstanceRole `` or ``arn:aws:iam::<aws_account_id> :instance-profile/ecsInstanceRole `` . For more information, see Amazon ECS instance role in the Batch User Guide .
When updating a compute environment, changing this setting requires an infrastructure update of the compute environment. For more information, see Updating compute environments in the Batch User Guide .
Note
This parameter isn't applicable to jobs that are running on Fargate resources, and shouldn't be specified.
Key-value pair tags to be applied to EC2 resources that are launched in the compute environment. For Batch, these take the form of "String1": "String2", where String1 is the tag key and String2 is the tag value−for example, { "Name": "Batch Instance - C4OnDemand" } . This is helpful for recognizing your Batch instances in the Amazon EC2 console. These tags aren't seen when using the Batch ListTagsForResource API operation.
When updating a compute environment, changing this setting requires an infrastructure update of the compute environment. For more information, see Updating compute environments in the Batch User Guide .
Note
This parameter isn't applicable to jobs that are running on Fargate resources, and shouldn't be specified.
The Amazon EC2 placement group to associate with your compute resources. If you intend to submit multi-node parallel jobs to your compute environment, you should consider creating a cluster placement group and associate it with your compute resources. This keeps your multi-node parallel job on a logical grouping of instances within a single Availability Zone with high network flow potential. For more information, see Placement groups in the Amazon EC2 User Guide for Linux Instances .
When updating a compute environment, changing the placement group requires an infrastructure update of the compute environment. For more information, see Updating compute environments in the Batch User Guide .
Note
This parameter isn't applicable to jobs that are running on Fargate resources, and shouldn't be specified.
The maximum percentage that a Spot Instance price can be when compared with the On-Demand price for that instance type before instances are launched. For example, if your maximum percentage is 20%, then the Spot price must be less than 20% of the current On-Demand price for that Amazon EC2 instance. You always pay the lowest (market) price and never more than your maximum percentage.
When updating a compute environment, changing the bid percentage requires an infrastructure update of the compute environment. For more information, see Updating compute environments in the Batch User Guide .
Note
This parameter isn't applicable to jobs that are running on Fargate resources, and shouldn't be specified.
The updated launch template to use for your compute resources. You must specify either the launch template ID or launch template name in the request, but not both. For more information, see Launch template support in the Batch User Guide . To remove the custom launch template and use the default launch template, set launchTemplateId or launchTemplateName member of the launch template specification to an empty string. Removing the launch template from a compute environment will not remove the AMI specified in the launch template. In order to update the AMI specified in a launch template, the updateToLatestImageVersion parameter must be set to true .
When updating a compute environment, changing the launch template requires an infrastructure update of the compute environment. For more information, see Updating compute environments in the Batch User Guide .
Note
This parameter isn't applicable to jobs that are running on Fargate resources, and shouldn't be specified.
The ID of the launch template.
The name of the launch template.
The version number of the launch template, $Latest , or $Default .
If the value is $Latest , the latest version of the launch template is used. If the value is $Default , the default version of the launch template is used.
Warning
If the AMI ID that's used in a compute environment is from the launch template, the AMI isn't changed when the compute environment is updated. It's only changed if the updateToLatestImageVersion parameter for the compute environment is set to true . During an infrastructure update, if either $Latest or $Default is specified, Batch re-evaluates the launch template version, and it might use a different version of the launch template. This is the case even if the launch template isn't specified in the update. When updating a compute environment, changing the launch template requires an infrastructure update of the compute environment. For more information, see Updating compute environments in the Batch User Guide .
Default: $Default .
Provides information used to select Amazon Machine Images (AMIs) for EC2 instances in the compute environment. If Ec2Configuration isn't specified, the default is ECS_AL2 .
When updating a compute environment, changing this setting requires an infrastructure update of the compute environment. For more information, see Updating compute environments in the Batch User Guide . To remove the EC2 configuration and any custom AMI ID specified in imageIdOverride , set this value to an empty string.
One or two values can be provided.
Note
This parameter isn't applicable to jobs that are running on Fargate resources, and shouldn't be specified.
Provides information used to select Amazon Machine Images (AMIs) for instances in the compute environment. If Ec2Configuration isn't specified, the default is ECS_AL2 (Amazon Linux 2 ).
Note
This object isn't applicable to jobs that are running on Fargate resources.
The image type to match with the instance type to select an AMI. If the imageIdOverride parameter isn't specified, then a recent Amazon ECS-optimized Amazon Linux 2 AMI (ECS_AL2 ) is used. If a new image type is specified in an update, but neither an imageId nor a imageIdOverride parameter is specified, then the latest Amazon ECS optimized AMI for that image type that's supported by Batch is used.
ECS_AL2Amazon Linux 2 − Default for all non-GPU instance families.
ECS_AL2_NVIDIAAmazon Linux 2 (GPU) −Default for all GPU instance families (for example
P4andG4) and can be used for all non Amazon Web Services Graviton-based instance types.ECS_AL1Amazon Linux . Amazon Linux is reaching the end-of-life of standard support. For more information, see Amazon Linux AMI .
The AMI ID used for instances launched in the compute environment that match the image type. This setting overrides the imageId set in the computeResource object.
Note
The AMI that you choose for a compute environment must match the architecture of the instance types that you intend to use for that compute environment. For example, if your compute environment uses A1 instance types, the compute resource AMI that you choose must support ARM instances. Amazon ECS vends both x86 and ARM versions of the Amazon ECS-optimized Amazon Linux 2 AMI. For more information, see Amazon ECS-optimized Amazon Linux 2 AMI in the Amazon Elastic Container Service Developer Guide .
Specifies whether the AMI ID is updated to the latest one that's supported by Batch when the compute environment has an infrastructure update. The default value is false .
Note
If an AMI ID is specified in the imageId or imageIdOverride parameters or by the launch template specified in the launchTemplate parameter, this parameter is ignored. For more information on updating AMI IDs during an infrastructure update, see Updating the AMI ID in the Batch User Guide .
When updating a compute environment, changing this setting requires an infrastructure update of the compute environment. For more information, see Updating compute environments in the Batch User Guide .
The type of compute environment: EC2 , SPOT , FARGATE , or FARGATE_SPOT . For more information, see Compute environments in the Batch User Guide .
If you choose SPOT , you must also specify an Amazon EC2 Spot Fleet role with the spotIamFleetRole parameter. For more information, see Amazon EC2 spot fleet role in the Batch User Guide .
When updating a compute environment, changing the type of a compute environment requires an infrastructure update of the compute environment. For more information, see Updating compute environments in the Batch User Guide .
The Amazon Machine Image (AMI) ID used for instances launched in the compute environment. This parameter is overridden by the imageIdOverride member of the Ec2Configuration structure. To remove the custom AMI ID and use the default AMI ID, set this value to an empty string.
When updating a compute environment, changing the AMI ID requires an infrastructure update of the compute environment. For more information, see Updating compute environments in the Batch User Guide .
Note
This parameter isn't applicable to jobs that are running on Fargate resources, and shouldn't be specified.
Note
The AMI that you choose for a compute environment must match the architecture of the instance types that you intend to use for that compute environment. For example, if your compute environment uses A1 instance types, the compute resource AMI that you choose must support ARM instances. Amazon ECS vends both x86 and ARM versions of the Amazon ECS-optimized Amazon Linux 2 AMI. For more information, see Amazon ECS-optimized Amazon Linux 2 AMI in the Amazon Elastic Container Service Developer Guide .
The full Amazon Resource Name (ARN) of the IAM role that allows Batch to make calls to other Amazon Web Services services on your behalf. For more information, see Batch service IAM role in the Batch User Guide .
Warning
If the compute environment has a service-linked role, it can't be changed to use a regular IAM role. Likewise, if the compute environment has a regular IAM role, it can't be changed to use a service-linked role. To update the parameters for the compute environment that require an infrastructure update to change, the AWSServiceRoleForBatch service-linked role must be used. For more information, see Updating compute environments in the Batch User Guide .
If your specified role has a path other than / , then you must either specify the full role ARN (recommended) or prefix the role name with the path.
Note
Depending on how you created your Batch service role, its ARN might contain the service-role path prefix. When you only specify the name of the service role, Batch assumes that your ARN doesn't use the service-role path prefix. Because of this, we recommend that you specify the full ARN of your service role when you create compute environments.
Specifies the updated infrastructure update policy for the compute environment. For more information about infrastructure updates, see Updating compute environments in the Batch User Guide .
Specifies whether jobs are automatically terminated when the computer environment infrastructure is updated. The default value is false .
Specifies the job timeout, in minutes, when the compute environment infrastructure is updated. The default value is 30.
dict
Response Syntax
{
'computeEnvironmentName': 'string',
'computeEnvironmentArn': 'string'
}
Response Structure
(dict) --
computeEnvironmentName (string) --
The name of the compute environment. It can be up to 128 letters long. It can contain uppercase and lowercase letters, numbers, hyphens (-), and underscores (_).
computeEnvironmentArn (string) --
The Amazon Resource Name (ARN) of the compute environment.
Exceptions
Batch.Client.exceptions.ClientExceptionBatch.Client.exceptions.ServerExceptionExamples
This example disables the P2OnDemand compute environment so it can be deleted.
response = client.update_compute_environment(
computeEnvironment='P2OnDemand',
state='DISABLED',
)
print(response)
Expected Output:
{
'computeEnvironmentArn': 'arn:aws:batch:us-east-1:012345678910:compute-environment/P2OnDemand',
'computeEnvironmentName': 'P2OnDemand',
'ResponseMetadata': {
'...': '...',
},
}
update_job_queue(**kwargs)¶Updates a job queue.
See also: AWS API Documentation
Request Syntax
response = client.update_job_queue(
jobQueue='string',
state='ENABLED'|'DISABLED',
schedulingPolicyArn='string',
priority=123,
computeEnvironmentOrder=[
{
'order': 123,
'computeEnvironment': 'string'
},
]
)
[REQUIRED]
The name or the Amazon Resource Name (ARN) of the job queue.
ENABLED , it can accept jobs. If the job queue state is DISABLED , new jobs can't be added to the queue, but jobs already in the queue can finish.aws:*Partition* :batch:*Region* :*Account* :scheduling-policy/*Name* `` . For example, ``aws:aws:batch:us-west-2:012345678910:scheduling-policy/MySchedulingPolicy .priority parameter) are evaluated first when associated with the same compute environment. Priority is determined in descending order. For example, a job queue with a priority value of 10 is given scheduling preference over a job queue with a priority value of 1 . All of the compute environments must be either EC2 (EC2 or SPOT ) or Fargate (FARGATE or FARGATE_SPOT ). EC2 and Fargate compute environments can't be mixed.Details the set of compute environments mapped to a job queue and their order relative to each other. This is one of the parameters used by the job scheduler to determine which compute environment runs a given job. Compute environments must be in the VALID state before you can associate them with a job queue. All of the compute environments must be either EC2 (EC2 or SPOT ) or Fargate (FARGATE or FARGATE_SPOT ). EC2 and Fargate compute environments can't be mixed.
Note
All compute environments that are associated with a job queue must share the same architecture. Batch doesn't support mixing compute environment architecture types in a single job queue.
The order in which compute environments are tried for job placement within a queue. Compute environments are tried in ascending order. For example, if two compute environments are associated with a job queue, the compute environment with a lower order integer value is tried for job placement first. Compute environments must be in the VALID state before you can associate them with a job queue. All of the compute environments must be either EC2 (EC2 or SPOT ) or Fargate (FARGATE or FARGATE_SPOT ); EC2 and Fargate compute environments can't be mixed.
Note
All compute environments that are associated with a job queue must share the same architecture. Batch doesn't support mixing compute environment architecture types in a single job queue.
The order of the compute environment. Compute environments are tried in ascending order. For example, if two compute environments are associated with a job queue, the compute environment with a lower order integer value is tried for job placement first.
The Amazon Resource Name (ARN) of the compute environment.
dict
Response Syntax
{
'jobQueueName': 'string',
'jobQueueArn': 'string'
}
Response Structure
(dict) --
jobQueueName (string) --
The name of the job queue.
jobQueueArn (string) --
The Amazon Resource Name (ARN) of the job queue.
Exceptions
Batch.Client.exceptions.ClientExceptionBatch.Client.exceptions.ServerExceptionExamples
This example disables a job queue so that it can be deleted.
response = client.update_job_queue(
jobQueue='GPGPU',
state='DISABLED',
)
print(response)
Expected Output:
{
'jobQueueArn': 'arn:aws:batch:us-east-1:012345678910:job-queue/GPGPU',
'jobQueueName': 'GPGPU',
'ResponseMetadata': {
'...': '...',
},
}
update_scheduling_policy(**kwargs)¶Updates a scheduling policy.
See also: AWS API Documentation
Request Syntax
response = client.update_scheduling_policy(
arn='string',
fairsharePolicy={
'shareDecaySeconds': 123,
'computeReservation': 123,
'shareDistribution': [
{
'shareIdentifier': 'string',
'weightFactor': ...
},
]
}
)
[REQUIRED]
The Amazon Resource Name (ARN) of the scheduling policy to update.
The fair share policy.
The time period to use to calculate a fair share percentage for each fair share identifier in use, in seconds. A value of zero (0) indicates that only current usage should be measured. The decay allows for more recently run jobs to have more weight than jobs that ran earlier. The maximum supported value is 604800 (1 week).
A value used to reserve some of the available maximum vCPU for fair share identifiers that have not yet been used.
The reserved ratio is ``(computeReservation /100)^*ActiveFairShares* `` where `` ActiveFairShares `` is the number of active fair share identifiers.
For example, a computeReservation value of 50 indicates that Batch should reserve 50% of the maximum available vCPU if there is only one fair share identifier, 25% if there are two fair share identifiers, and 12.5% if there are three fair share identifiers. A computeReservation value of 25 indicates that Batch should reserve 25% of the maximum available vCPU if there is only one fair share identifier, 6.25% if there are two fair share identifiers, and 1.56% if there are three fair share identifiers.
The minimum value is 0 and the maximum value is 99.
An array of SharedIdentifier objects that contain the weights for the fair share identifiers for the fair share policy. Fair share identifiers that aren't included have a default weight of 1.0 .
Specifies the weights for the fair share identifiers for the fair share policy. Fair share identifiers that aren't included have a default weight of 1.0 .
A fair share identifier or fair share identifier prefix. If the string ends with an asterisk (*), this entry specifies the weight factor to use for fair share identifiers that start with that prefix. The list of fair share identifiers in a fair share policy cannot overlap. For example, you can't have one that specifies a shareIdentifier of UserA* and another that specifies a shareIdentifier of UserA-1 .
There can be no more than 500 fair share identifiers active in a job queue.
The string is limited to 255 alphanumeric characters, optionally followed by an asterisk (*).
The weight factor for the fair share identifier. The default value is 1.0. A lower value has a higher priority for compute resources. For example, jobs that use a share identifier with a weight factor of 0.125 (1/8) get 8 times the compute resources of jobs that use a share identifier with a weight factor of 1.
The smallest supported value is 0.0001, and the largest supported value is 999.9999.
dict
Response Syntax
{}
Response Structure
Exceptions
Batch.Client.exceptions.ClientExceptionBatch.Client.exceptions.ServerExceptionThe available paginators are:
Batch.Paginator.DescribeComputeEnvironmentsBatch.Paginator.DescribeJobDefinitionsBatch.Paginator.DescribeJobQueuesBatch.Paginator.ListJobsBatch.Paginator.ListSchedulingPoliciesBatch.Paginator.DescribeComputeEnvironments¶paginator = client.get_paginator('describe_compute_environments')
paginate(**kwargs)¶Creates an iterator that will paginate through responses from Batch.Client.describe_compute_environments().
See also: AWS API Documentation
Request Syntax
response_iterator = paginator.paginate(
computeEnvironments=[
'string',
],
PaginationConfig={
'MaxItems': 123,
'PageSize': 123,
'StartingToken': 'string'
}
)
A list of up to 100 compute environment names or full Amazon Resource Name (ARN) entries.
A dictionary that provides parameters to control pagination.
The total number of items to return. If the total number of items available is more than the value specified in max-items then a NextToken will be provided in the output that you can use to resume pagination.
The size of each page.
A token to specify where to start paginating. This is the NextToken from a previous response.
dict
Response Syntax
{
'computeEnvironments': [
{
'computeEnvironmentName': 'string',
'computeEnvironmentArn': 'string',
'unmanagedvCpus': 123,
'ecsClusterArn': 'string',
'tags': {
'string': 'string'
},
'type': 'MANAGED'|'UNMANAGED',
'state': 'ENABLED'|'DISABLED',
'status': 'CREATING'|'UPDATING'|'DELETING'|'DELETED'|'VALID'|'INVALID',
'statusReason': 'string',
'computeResources': {
'type': 'EC2'|'SPOT'|'FARGATE'|'FARGATE_SPOT',
'allocationStrategy': 'BEST_FIT'|'BEST_FIT_PROGRESSIVE'|'SPOT_CAPACITY_OPTIMIZED',
'minvCpus': 123,
'maxvCpus': 123,
'desiredvCpus': 123,
'instanceTypes': [
'string',
],
'imageId': 'string',
'subnets': [
'string',
],
'securityGroupIds': [
'string',
],
'ec2KeyPair': 'string',
'instanceRole': 'string',
'tags': {
'string': 'string'
},
'placementGroup': 'string',
'bidPercentage': 123,
'spotIamFleetRole': 'string',
'launchTemplate': {
'launchTemplateId': 'string',
'launchTemplateName': 'string',
'version': 'string'
},
'ec2Configuration': [
{
'imageType': 'string',
'imageIdOverride': 'string'
},
]
},
'serviceRole': 'string',
'updatePolicy': {
'terminateJobsOnUpdate': True|False,
'jobExecutionTimeoutMinutes': 123
}
},
],
'NextToken': 'string'
}
Response Structure
(dict) --
computeEnvironments (list) --
The list of compute environments.
(dict) --
An object representing an Batch compute environment.
computeEnvironmentName (string) --
The name of the compute environment. It can be up to 128 letters long. It can contain uppercase and lowercase letters, numbers, hyphens (-), and underscores (_).
computeEnvironmentArn (string) --
The Amazon Resource Name (ARN) of the compute environment.
unmanagedvCpus (integer) --
The maximum number of VCPUs expected to be used for an unmanaged compute environment.
ecsClusterArn (string) --
The Amazon Resource Name (ARN) of the underlying Amazon ECS cluster used by the compute environment.
tags (dict) --
The tags applied to the compute environment.
type (string) --
The type of the compute environment: MANAGED or UNMANAGED . For more information, see Compute environments in the Batch User Guide .
state (string) --
The state of the compute environment. The valid values are ENABLED or DISABLED .
If the state is ENABLED , then the Batch scheduler can attempt to place jobs from an associated job queue on the compute resources within the environment. If the compute environment is managed, then it can scale its instances out or in automatically, based on the job queue demand.
If the state is DISABLED , then the Batch scheduler doesn't attempt to place jobs within the environment. Jobs in a STARTING or RUNNING state continue to progress normally. Managed compute environments in the DISABLED state don't scale out. However, they scale in to minvCpus value after instances become idle.
status (string) --
The current status of the compute environment (for example, CREATING or VALID ).
statusReason (string) --
A short, human-readable string to provide additional details about the current status of the compute environment.
computeResources (dict) --
The compute resources defined for the compute environment. For more information, see Compute environments in the Batch User Guide .
type (string) --
The type of compute environment: EC2 , SPOT , FARGATE , or FARGATE_SPOT . For more information, see Compute environments in the Batch User Guide .
If you choose SPOT , you must also specify an Amazon EC2 Spot Fleet role with the spotIamFleetRole parameter. For more information, see Amazon EC2 spot fleet role in the Batch User Guide .
allocationStrategy (string) --
The allocation strategy to use for the compute resource if not enough instances of the best fitting instance type can be allocated. This might be because of availability of the instance type in the Region or Amazon EC2 service limits . For more information, see Allocation strategies in the Batch User Guide .
Note
This parameter isn't applicable to jobs that are running on Fargate resources, and shouldn't be specified.
BEST_FIT (default)
Batch selects an instance type that best fits the needs of the jobs with a preference for the lowest-cost instance type. If additional instances of the selected instance type aren't available, Batch waits for the additional instances to be available. If there aren't enough instances available, or if the user is reaching Amazon EC2 service limits then additional jobs aren't run until the currently running jobs have completed. This allocation strategy keeps costs lower but can limit scaling. If you are using Spot Fleets with BEST_FIT then the Spot Fleet IAM Role must be specified. Compute resources that use a BEST_FIT allocation strategy don't support infrastructure updates and can't update some parameters. For more information, see Updating compute environments in the Batch User Guide .
BEST_FIT_PROGRESSIVE
Batch will select additional instance types that are large enough to meet the requirements of the jobs in the queue, with a preference for instance types with a lower cost per unit vCPU. If additional instances of the previously selected instance types aren't available, Batch will select new instance types.
SPOT_CAPACITY_OPTIMIZED
Batch will select one or more instance types that are large enough to meet the requirements of the jobs in the queue, with a preference for instance types that are less likely to be interrupted. This allocation strategy is only available for Spot Instance compute resources.
With both BEST_FIT_PROGRESSIVE and SPOT_CAPACITY_OPTIMIZED strategies, Batch might need to go above maxvCpus to meet your capacity requirements. In this event, Batch never exceeds maxvCpus by more than a single instance.
minvCpus (integer) --
The minimum number of Amazon EC2 vCPUs that an environment should maintain (even if the compute environment is DISABLED ).
Note
This parameter isn't applicable to jobs that are running on Fargate resources, and shouldn't be specified.
maxvCpus (integer) --
The maximum number of Amazon EC2 vCPUs that a compute environment can reach.
Note
With both BEST_FIT_PROGRESSIVE and SPOT_CAPACITY_OPTIMIZED allocation strategies, Batch might need to exceed maxvCpus to meet your capacity requirements. In this event, Batch never exceeds maxvCpus by more than a single instance. For example, no more than a single instance from among those specified in your compute environment is allocated.
desiredvCpus (integer) --
The desired number of Amazon EC2 vCPUS in the compute environment. Batch modifies this value between the minimum and maximum values, based on job queue demand.
Note
This parameter isn't applicable to jobs that are running on Fargate resources, and shouldn't be specified.
instanceTypes (list) --
The instances types that can be launched. You can specify instance families to launch any instance type within those families (for example, c5 or p3 ), or you can specify specific sizes within a family (such as c5.8xlarge ). You can also choose optimal to select instance types (from the C4, M4, and R4 instance families) that match the demand of your job queues.
Note
This parameter isn't applicable to jobs that are running on Fargate resources, and shouldn't be specified.
Note
When you create a compute environment, the instance types that you select for the compute environment must share the same architecture. For example, you can't mix x86 and ARM instances in the same compute environment.
Note
Currently, optimal uses instance types from the C4, M4, and R4 instance families. In Regions that don't have instance types from those instance families, instance types from the C5, M5. and R5 instance families are used.
imageId (string) --
The Amazon Machine Image (AMI) ID used for instances launched in the compute environment. This parameter is overridden by the imageIdOverride member of the Ec2Configuration structure.
Note
This parameter isn't applicable to jobs that are running on Fargate resources, and shouldn't be specified.
Note
The AMI that you choose for a compute environment must match the architecture of the instance types that you intend to use for that compute environment. For example, if your compute environment uses A1 instance types, the compute resource AMI that you choose must support ARM instances. Amazon ECS vends both x86 and ARM versions of the Amazon ECS-optimized Amazon Linux 2 AMI. For more information, see Amazon ECS-optimized Amazon Linux 2 AMI in the Amazon Elastic Container Service Developer Guide .
subnets (list) --
The VPC subnets where the compute resources are launched. These subnets must be within the same VPC. Fargate compute resources can contain up to 16 subnets. For more information, see VPCs and subnets in the Amazon VPC User Guide .
securityGroupIds (list) --
The Amazon EC2 security groups associated with instances launched in the compute environment. One or more security groups must be specified, either in securityGroupIds or using a launch template referenced in launchTemplate . This parameter is required for jobs that are running on Fargate resources and must contain at least one security group. Fargate doesn't support launch templates. If security groups are specified using both securityGroupIds and launchTemplate , the values in securityGroupIds are used.
ec2KeyPair (string) --
The Amazon EC2 key pair that's used for instances launched in the compute environment. You can use this key pair to log in to your instances with SSH.
Note
This parameter isn't applicable to jobs that are running on Fargate resources, and shouldn't be specified.
instanceRole (string) --
The Amazon ECS instance profile applied to Amazon EC2 instances in a compute environment. You can specify the short name or full Amazon Resource Name (ARN) of an instance profile. For example, `` ecsInstanceRole `` or ``arn:aws:iam::<aws_account_id> :instance-profile/ecsInstanceRole `` . For more information, see Amazon ECS instance role in the Batch User Guide .
Note
This parameter isn't applicable to jobs that are running on Fargate resources, and shouldn't be specified.
tags (dict) --
Key-value pair tags to be applied to EC2 resources that are launched in the compute environment. For Batch, these take the form of "String1": "String2", where String1 is the tag key and String2 is the tag value−for example, { "Name": "Batch Instance - C4OnDemand" } . This is helpful for recognizing your Batch instances in the Amazon EC2 console. Updating these tags requires an infrastructure update to the compute environment. For more information, see Updating compute environments in the Batch User Guide . These tags aren't seen when using the Batch ListTagsForResource API operation.
Note
This parameter isn't applicable to jobs that are running on Fargate resources, and shouldn't be specified.
placementGroup (string) --
The Amazon EC2 placement group to associate with your compute resources. If you intend to submit multi-node parallel jobs to your compute environment, you should consider creating a cluster placement group and associate it with your compute resources. This keeps your multi-node parallel job on a logical grouping of instances within a single Availability Zone with high network flow potential. For more information, see Placement groups in the Amazon EC2 User Guide for Linux Instances .
Note
This parameter isn't applicable to jobs that are running on Fargate resources, and shouldn't be specified.
bidPercentage (integer) --
The maximum percentage that a Spot Instance price can be when compared with the On-Demand price for that instance type before instances are launched. For example, if your maximum percentage is 20%, then the Spot price must be less than 20% of the current On-Demand price for that Amazon EC2 instance. You always pay the lowest (market) price and never more than your maximum percentage. If you leave this field empty, the default value is 100% of the On-Demand price.
Note
This parameter isn't applicable to jobs that are running on Fargate resources, and shouldn't be specified.
spotIamFleetRole (string) --
The Amazon Resource Name (ARN) of the Amazon EC2 Spot Fleet IAM role applied to a SPOT compute environment. This role is required if the allocation strategy set to BEST_FIT or if the allocation strategy isn't specified. For more information, see Amazon EC2 spot fleet role in the Batch User Guide .
Note
This parameter isn't applicable to jobs that are running on Fargate resources, and shouldn't be specified.
Warning
To tag your Spot Instances on creation, the Spot Fleet IAM role specified here must use the newer AmazonEC2SpotFleetTaggingRole managed policy. The previously recommended AmazonEC2SpotFleetRole managed policy doesn't have the required permissions to tag Spot Instances. For more information, see Spot instances not tagged on creation in the Batch User Guide .
launchTemplate (dict) --
The launch template to use for your compute resources. Any other compute resource parameters that you specify in a CreateComputeEnvironment API operation override the same parameters in the launch template. You must specify either the launch template ID or launch template name in the request, but not both. For more information, see Launch template support in the Batch User Guide .
Note
This parameter isn't applicable to jobs that are running on Fargate resources, and shouldn't be specified.
launchTemplateId (string) --
The ID of the launch template.
launchTemplateName (string) --
The name of the launch template.
version (string) --
The version number of the launch template, $Latest , or $Default .
If the value is $Latest , the latest version of the launch template is used. If the value is $Default , the default version of the launch template is used.
Warning
If the AMI ID that's used in a compute environment is from the launch template, the AMI isn't changed when the compute environment is updated. It's only changed if the updateToLatestImageVersion parameter for the compute environment is set to true . During an infrastructure update, if either $Latest or $Default is specified, Batch re-evaluates the launch template version, and it might use a different version of the launch template. This is the case even if the launch template isn't specified in the update. When updating a compute environment, changing the launch template requires an infrastructure update of the compute environment. For more information, see Updating compute environments in the Batch User Guide .
Default: $Default .
ec2Configuration (list) --
Provides information used to select Amazon Machine Images (AMIs) for EC2 instances in the compute environment. If Ec2Configuration isn't specified, the default is ECS_AL2 .
One or two values can be provided.
Note
This parameter isn't applicable to jobs that are running on Fargate resources, and shouldn't be specified.
(dict) --
Provides information used to select Amazon Machine Images (AMIs) for instances in the compute environment. If Ec2Configuration isn't specified, the default is ECS_AL2 (Amazon Linux 2 ).
Note
This object isn't applicable to jobs that are running on Fargate resources.
imageType (string) --
The image type to match with the instance type to select an AMI. If the imageIdOverride parameter isn't specified, then a recent Amazon ECS-optimized Amazon Linux 2 AMI (ECS_AL2 ) is used. If a new image type is specified in an update, but neither an imageId nor a imageIdOverride parameter is specified, then the latest Amazon ECS optimized AMI for that image type that's supported by Batch is used.
ECS_AL2
Amazon Linux 2 − Default for all non-GPU instance families.
ECS_AL2_NVIDIA
Amazon Linux 2 (GPU) −Default for all GPU instance families (for example
P4andG4) and can be used for all non Amazon Web Services Graviton-based instance types.ECS_AL1
Amazon Linux . Amazon Linux is reaching the end-of-life of standard support. For more information, see Amazon Linux AMI .
imageIdOverride (string) --
The AMI ID used for instances launched in the compute environment that match the image type. This setting overrides the imageId set in the computeResource object.
Note
The AMI that you choose for a compute environment must match the architecture of the instance types that you intend to use for that compute environment. For example, if your compute environment uses A1 instance types, the compute resource AMI that you choose must support ARM instances. Amazon ECS vends both x86 and ARM versions of the Amazon ECS-optimized Amazon Linux 2 AMI. For more information, see Amazon ECS-optimized Amazon Linux 2 AMI in the Amazon Elastic Container Service Developer Guide .
serviceRole (string) --
The service role associated with the compute environment that allows Batch to make calls to Amazon Web Services API operations on your behalf. For more information, see Batch service IAM role in the Batch User Guide .
updatePolicy (dict) --
Specifies the infrastructure update policy for the compute environment. For more information about infrastructure updates, see Updating compute environments in the Batch User Guide .
terminateJobsOnUpdate (boolean) --
Specifies whether jobs are automatically terminated when the computer environment infrastructure is updated. The default value is false .
jobExecutionTimeoutMinutes (integer) --
Specifies the job timeout, in minutes, when the compute environment infrastructure is updated. The default value is 30.
NextToken (string) --
A token to resume pagination.
Batch.Paginator.DescribeJobDefinitions¶paginator = client.get_paginator('describe_job_definitions')
paginate(**kwargs)¶Creates an iterator that will paginate through responses from Batch.Client.describe_job_definitions().
See also: AWS API Documentation
Request Syntax
response_iterator = paginator.paginate(
jobDefinitions=[
'string',
],
jobDefinitionName='string',
status='string',
PaginationConfig={
'MaxItems': 123,
'PageSize': 123,
'StartingToken': 'string'
}
)
A list of up to 100 job definitions. Each entry in the list can either be an ARN in the format arn:aws:batch:${Region}:${Account}:job-definition/${JobDefinitionName}:${Revision} or a short version using the form ${JobDefinitionName}:${Revision} .
A dictionary that provides parameters to control pagination.
The total number of items to return. If the total number of items available is more than the value specified in max-items then a NextToken will be provided in the output that you can use to resume pagination.
The size of each page.
A token to specify where to start paginating. This is the NextToken from a previous response.
dict
Response Syntax
{
'jobDefinitions': [
{
'jobDefinitionName': 'string',
'jobDefinitionArn': 'string',
'revision': 123,
'status': 'string',
'type': 'string',
'schedulingPriority': 123,
'parameters': {
'string': 'string'
},
'retryStrategy': {
'attempts': 123,
'evaluateOnExit': [
{
'onStatusReason': 'string',
'onReason': 'string',
'onExitCode': 'string',
'action': 'RETRY'|'EXIT'
},
]
},
'containerProperties': {
'image': 'string',
'vcpus': 123,
'memory': 123,
'command': [
'string',
],
'jobRoleArn': 'string',
'executionRoleArn': 'string',
'volumes': [
{
'host': {
'sourcePath': 'string'
},
'name': 'string',
'efsVolumeConfiguration': {
'fileSystemId': 'string',
'rootDirectory': 'string',
'transitEncryption': 'ENABLED'|'DISABLED',
'transitEncryptionPort': 123,
'authorizationConfig': {
'accessPointId': 'string',
'iam': 'ENABLED'|'DISABLED'
}
}
},
],
'environment': [
{
'name': 'string',
'value': 'string'
},
],
'mountPoints': [
{
'containerPath': 'string',
'readOnly': True|False,
'sourceVolume': 'string'
},
],
'readonlyRootFilesystem': True|False,
'privileged': True|False,
'ulimits': [
{
'hardLimit': 123,
'name': 'string',
'softLimit': 123
},
],
'user': 'string',
'instanceType': 'string',
'resourceRequirements': [
{
'value': 'string',
'type': 'GPU'|'VCPU'|'MEMORY'
},
],
'linuxParameters': {
'devices': [
{
'hostPath': 'string',
'containerPath': 'string',
'permissions': [
'READ'|'WRITE'|'MKNOD',
]
},
],
'initProcessEnabled': True|False,
'sharedMemorySize': 123,
'tmpfs': [
{
'containerPath': 'string',
'size': 123,
'mountOptions': [
'string',
]
},
],
'maxSwap': 123,
'swappiness': 123
},
'logConfiguration': {
'logDriver': 'json-file'|'syslog'|'journald'|'gelf'|'fluentd'|'awslogs'|'splunk',
'options': {
'string': 'string'
},
'secretOptions': [
{
'name': 'string',
'valueFrom': 'string'
},
]
},
'secrets': [
{
'name': 'string',
'valueFrom': 'string'
},
],
'networkConfiguration': {
'assignPublicIp': 'ENABLED'|'DISABLED'
},
'fargatePlatformConfiguration': {
'platformVersion': 'string'
}
},
'timeout': {
'attemptDurationSeconds': 123
},
'nodeProperties': {
'numNodes': 123,
'mainNode': 123,
'nodeRangeProperties': [
{
'targetNodes': 'string',
'container': {
'image': 'string',
'vcpus': 123,
'memory': 123,
'command': [
'string',
],
'jobRoleArn': 'string',
'executionRoleArn': 'string',
'volumes': [
{
'host': {
'sourcePath': 'string'
},
'name': 'string',
'efsVolumeConfiguration': {
'fileSystemId': 'string',
'rootDirectory': 'string',
'transitEncryption': 'ENABLED'|'DISABLED',
'transitEncryptionPort': 123,
'authorizationConfig': {
'accessPointId': 'string',
'iam': 'ENABLED'|'DISABLED'
}
}
},
],
'environment': [
{
'name': 'string',
'value': 'string'
},
],
'mountPoints': [
{
'containerPath': 'string',
'readOnly': True|False,
'sourceVolume': 'string'
},
],
'readonlyRootFilesystem': True|False,
'privileged': True|False,
'ulimits': [
{
'hardLimit': 123,
'name': 'string',
'softLimit': 123
},
],
'user': 'string',
'instanceType': 'string',
'resourceRequirements': [
{
'value': 'string',
'type': 'GPU'|'VCPU'|'MEMORY'
},
],
'linuxParameters': {
'devices': [
{
'hostPath': 'string',
'containerPath': 'string',
'permissions': [
'READ'|'WRITE'|'MKNOD',
]
},
],
'initProcessEnabled': True|False,
'sharedMemorySize': 123,
'tmpfs': [
{
'containerPath': 'string',
'size': 123,
'mountOptions': [
'string',
]
},
],
'maxSwap': 123,
'swappiness': 123
},
'logConfiguration': {
'logDriver': 'json-file'|'syslog'|'journald'|'gelf'|'fluentd'|'awslogs'|'splunk',
'options': {
'string': 'string'
},
'secretOptions': [
{
'name': 'string',
'valueFrom': 'string'
},
]
},
'secrets': [
{
'name': 'string',
'valueFrom': 'string'
},
],
'networkConfiguration': {
'assignPublicIp': 'ENABLED'|'DISABLED'
},
'fargatePlatformConfiguration': {
'platformVersion': 'string'
}
}
},
]
},
'tags': {
'string': 'string'
},
'propagateTags': True|False,
'platformCapabilities': [
'EC2'|'FARGATE',
]
},
],
'NextToken': 'string'
}
Response Structure
(dict) --
jobDefinitions (list) --
The list of job definitions.
(dict) --
An object representing an Batch job definition.
jobDefinitionName (string) --
The name of the job definition.
jobDefinitionArn (string) --
The Amazon Resource Name (ARN) for the job definition.
revision (integer) --
The revision of the job definition.
status (string) --
The status of the job definition.
type (string) --
The type of job definition, either container or multinode . If the job is run on Fargate resources, then multinode isn't supported. For more information about multi-node parallel jobs, see Creating a multi-node parallel job definition in the Batch User Guide .
schedulingPriority (integer) --
The scheduling priority of the job definition. This only affects jobs in job queues with a fair share policy. Jobs with a higher scheduling priority are scheduled before jobs with a lower scheduling priority.
parameters (dict) --
Default parameters or parameter substitution placeholders that are set in the job definition. Parameters are specified as a key-value pair mapping. Parameters in a SubmitJob request override any corresponding parameter defaults from the job definition. For more information about specifying parameters, see Job definition parameters in the Batch User Guide .
retryStrategy (dict) --
The retry strategy to use for failed jobs that are submitted with this job definition.
attempts (integer) --
The number of times to move a job to the RUNNABLE status. You can specify between 1 and 10 attempts. If the value of attempts is greater than one, the job is retried on failure the same number of attempts as the value.
evaluateOnExit (list) --
Array of up to 5 objects that specify conditions under which the job should be retried or failed. If this parameter is specified, then the attempts parameter must also be specified.
(dict) --
Specifies a set of conditions to be met, and an action to take (RETRY or EXIT ) if all conditions are met.
onStatusReason (string) --
Contains a glob pattern to match against the StatusReason returned for a job. The pattern can be up to 512 characters in length. It can contain letters, numbers, periods (.), colons (:), and white space (including spaces or tabs). It can optionally end with an asterisk (*) so that only the start of the string needs to be an exact match.
The string can be between 1 and 512 characters in length.
onReason (string) --
Contains a glob pattern to match against the Reason returned for a job. The pattern can be up to 512 characters in length. It can contain letters, numbers, periods (.), colons (:), and white space (including spaces and tabs). It can optionally end with an asterisk (*) so that only the start of the string needs to be an exact match.
The string can be between 1 and 512 characters in length.
onExitCode (string) --
Contains a glob pattern to match against the decimal representation of the ExitCode returned for a job. The pattern can be up to 512 characters in length. It can contain only numbers, and can optionally end with an asterisk (*) so that only the start of the string needs to be an exact match.
The string can be between 1 and 512 characters in length.
action (string) --
Specifies the action to take if all of the specified conditions (onStatusReason , onReason , and onExitCode ) are met. The values aren't case sensitive.
containerProperties (dict) --
An object with various properties specific to container-based jobs.
image (string) --
The image used to start a container. This string is passed directly to the Docker daemon. Images in the Docker Hub registry are available by default. Other repositories are specified with `` repository-url /image :tag `` . Up to 255 letters (uppercase and lowercase), numbers, hyphens, underscores, colons, periods, forward slashes, and number signs are allowed. This parameter maps to Image in the Create a container section of the Docker Remote API and the IMAGE parameter of docker run .
Note
Docker image architecture must match the processor architecture of the compute resources that they're scheduled on. For example, ARM-based Docker images can only run on ARM-based compute resources.
registry/repository[:tag] or registry/repository[@digest] naming conventions. For example, ``public.ecr.aws/registry_alias /my-web-app :latest `` .012345678910.dkr.ecr.<region-name>.amazonaws.com/<repository-name> ).ubuntu or mongo ).amazon/amazon-ecs-agent ).quay.io/assemblyline/ubuntu ).vcpus (integer) --
This parameter is deprecated, use resourceRequirements to specify the vCPU requirements for the job definition. It's not supported for jobs running on Fargate resources. For jobs running on EC2 resources, it specifies the number of vCPUs reserved for the job.
Each vCPU is equivalent to 1,024 CPU shares. This parameter maps to CpuShares in the Create a container section of the Docker Remote API and the --cpu-shares option to docker run . The number of vCPUs must be specified but can be specified in several places. You must specify it at least once for each node.
memory (integer) --
This parameter is deprecated, use resourceRequirements to specify the memory requirements for the job definition. It's not supported for jobs running on Fargate resources. For jobs running on EC2 resources, it specifies the memory hard limit (in MiB) for a container. If your container attempts to exceed the specified number, it's terminated. You must specify at least 4 MiB of memory for a job using this parameter. The memory hard limit can be specified in several places. It must be specified for each node at least once.
command (list) --
The command that's passed to the container. This parameter maps to Cmd in the Create a container section of the Docker Remote API and the COMMAND parameter to docker run . For more information, see https://docs.docker.com/engine/reference/builder/#cmd .
jobRoleArn (string) --
The Amazon Resource Name (ARN) of the IAM role that the container can assume for Amazon Web Services permissions. For more information, see IAM roles for tasks in the Amazon Elastic Container Service Developer Guide .
executionRoleArn (string) --
The Amazon Resource Name (ARN) of the execution role that Batch can assume. For jobs that run on Fargate resources, you must provide an execution role. For more information, see Batch execution IAM role in the Batch User Guide .
volumes (list) --
A list of data volumes used in a job.
(dict) --
A data volume used in a job's container properties.
host (dict) --
The contents of the host parameter determine whether your data volume persists on the host container instance and where it is stored. If the host parameter is empty, then the Docker daemon assigns a host path for your data volume. However, the data isn't guaranteed to persist after the containers associated with it stop running.
Note
This parameter isn't applicable to jobs that are running on Fargate resources and shouldn't be provided.
sourcePath (string) --
The path on the host container instance that's presented to the container. If this parameter is empty, then the Docker daemon has assigned a host path for you. If this parameter contains a file location, then the data volume persists at the specified location on the host container instance until you delete it manually. If the source path location doesn't exist on the host container instance, the Docker daemon creates it. If the location does exist, the contents of the source path folder are exported.
Note
This parameter isn't applicable to jobs that run on Fargate resources and shouldn't be provided.
name (string) --
The name of the volume. It can be up to 255 letters long. It can contain uppercase and lowercase letters, numbers, hyphens (-), and underscores (_). This name is referenced in the sourceVolume parameter of container definition mountPoints .
efsVolumeConfiguration (dict) --
This parameter is specified when you are using an Amazon Elastic File System file system for job storage. Jobs that are running on Fargate resources must specify a platformVersion of at least 1.4.0 .
fileSystemId (string) --
The Amazon EFS file system ID to use.
rootDirectory (string) --
The directory within the Amazon EFS file system to mount as the root directory inside the host. If this parameter is omitted, the root of the Amazon EFS volume is used instead. Specifying / has the same effect as omitting this parameter. The maximum length is 4,096 characters.
Warning
If an EFS access point is specified in the authorizationConfig , the root directory parameter must either be omitted or set to / , which enforces the path set on the Amazon EFS access point.
transitEncryption (string) --
Determines whether to enable encryption for Amazon EFS data in transit between the Amazon ECS host and the Amazon EFS server. Transit encryption must be enabled if Amazon EFS IAM authorization is used. If this parameter is omitted, the default value of DISABLED is used. For more information, see Encrypting data in transit in the Amazon Elastic File System User Guide .
transitEncryptionPort (integer) --
The port to use when sending encrypted data between the Amazon ECS host and the Amazon EFS server. If you don't specify a transit encryption port, it uses the port selection strategy that the Amazon EFS mount helper uses. The value must be between 0 and 65,535. For more information, see EFS mount helper in the Amazon Elastic File System User Guide .
authorizationConfig (dict) --
The authorization configuration details for the Amazon EFS file system.
accessPointId (string) --
The Amazon EFS access point ID to use. If an access point is specified, the root directory value specified in the EFSVolumeConfiguration must either be omitted or set to / which will enforce the path set on the EFS access point. If an access point is used, transit encryption must be enabled in the EFSVolumeConfiguration . For more information, see Working with Amazon EFS access points in the Amazon Elastic File System User Guide .
iam (string) --
Whether or not to use the Batch job IAM role defined in a job definition when mounting the Amazon EFS file system. If enabled, transit encryption must be enabled in the EFSVolumeConfiguration . If this parameter is omitted, the default value of DISABLED is used. For more information, see Using Amazon EFS access points in the Batch User Guide . EFS IAM authorization requires that TransitEncryption be ENABLED and that a JobRoleArn is specified.
environment (list) --
The environment variables to pass to a container. This parameter maps to Env in the Create a container section of the Docker Remote API and the --env option to docker run .
Warning
We don't recommend using plaintext environment variables for sensitive information, such as credential data.
Note
Environment variables must not start with AWS_BATCH ; this naming convention is reserved for variables that are set by the Batch service.
(dict) --
A key-value pair object.
name (string) --
The name of the key-value pair. For environment variables, this is the name of the environment variable.
value (string) --
The value of the key-value pair. For environment variables, this is the value of the environment variable.
mountPoints (list) --
The mount points for data volumes in your container. This parameter maps to Volumes in the Create a container section of the Docker Remote API and the --volume option to docker run .
(dict) --
Details on a Docker volume mount point that's used in a job's container properties. This parameter maps to Volumes in the Create a container section of the Docker Remote API and the --volume option to docker run.
containerPath (string) --
The path on the container where the host volume is mounted.
readOnly (boolean) --
If this value is true , the container has read-only access to the volume. Otherwise, the container can write to the volume. The default value is false .
sourceVolume (string) --
The name of the volume to mount.
readonlyRootFilesystem (boolean) --
When this parameter is true, the container is given read-only access to its root file system. This parameter maps to ReadonlyRootfs in the Create a container section of the Docker Remote API and the --read-only option to docker run .
privileged (boolean) --
When this parameter is true, the container is given elevated permissions on the host container instance (similar to the root user). This parameter maps to Privileged in the Create a container section of the Docker Remote API and the --privileged option to docker run . The default value is false.
Note
This parameter isn't applicable to jobs that are running on Fargate resources and shouldn't be provided, or specified as false.
ulimits (list) --
A list of ulimits to set in the container. This parameter maps to Ulimits in the Create a container section of the Docker Remote API and the --ulimit option to docker run .
Note
This parameter isn't applicable to jobs that are running on Fargate resources and shouldn't be provided.
(dict) --
The ulimit settings to pass to the container.
Note
This object isn't applicable to jobs that are running on Fargate resources.
hardLimit (integer) --
The hard limit for the ulimit type.
name (string) --
The type of the ulimit .
softLimit (integer) --
The soft limit for the ulimit type.
user (string) --
The user name to use inside the container. This parameter maps to User in the Create a container section of the Docker Remote API and the --user option to docker run .
instanceType (string) --
The instance type to use for a multi-node parallel job. All node groups in a multi-node parallel job must use the same instance type.
Note
This parameter isn't applicable to single-node container jobs or jobs that run on Fargate resources, and shouldn't be provided.
resourceRequirements (list) --
The type and amount of resources to assign to a container. The supported resources include GPU , MEMORY , and VCPU .
(dict) --
The type and amount of a resource to assign to a container. The supported resources include GPU , MEMORY , and VCPU .
value (string) --
The quantity of the specified resource to reserve for the container. The values vary based on the type specified.
type="GPU"
The number of physical GPUs to reserve for the container. The number of GPUs reserved for all containers in a job shouldn't exceed the number of available GPUs on the compute resource that the job is launched on.
Note
GPUs are not available for jobs that are running on Fargate resources.
type="MEMORY"
The memory hard limit (in MiB) present to the container. This parameter is supported for jobs that are running on EC2 resources. If your container attempts to exceed the memory specified, the container is terminated. This parameter maps to Memory in the Create a container section of the Docker Remote API and the --memory option to docker run . You must specify at least 4 MiB of memory for a job. This is required but can be specified in several places for multi-node parallel (MNP) jobs. It must be specified for each node at least once. This parameter maps to Memory in the Create a container section of the Docker Remote API and the --memory option to docker run .
Note
If you're trying to maximize your resource utilization by providing your jobs as much memory as possible for a particular instance type, see Memory management in the Batch User Guide .
For jobs that are running on Fargate resources, then value is the hard limit (in MiB), and must match one of the supported values and the VCPU values must be one of the values supported for that memory value.
value = 512
VCPU= 0.25value = 1024
VCPU= 0.25 or 0.5value = 2048
VCPU= 0.25, 0.5, or 1value = 3072
VCPU= 0.5, or 1value = 4096
VCPU= 0.5, 1, or 2value = 5120, 6144, or 7168
VCPU= 1 or 2value = 8192
VCPU= 1, 2, or 4value = 9216, 10240, 11264, 12288, 13312, 14336, 15360, or 16384
VCPU= 2 or 4value = 17408, 18432, 19456, 20480, 21504, 22528, 23552, 24576, 25600, 26624, 27648, 28672, 29696, or 30720
VCPU= 4type="VCPU"
The number of vCPUs reserved for the container. This parameter maps to CpuShares in the Create a container section of the Docker Remote API and the --cpu-shares option to docker run . Each vCPU is equivalent to 1,024 CPU shares. For EC2 resources, you must specify at least one vCPU. This is required but can be specified in several places; it must be specified for each node at least once.
For jobs that are running on Fargate resources, then value must match one of the supported values and the MEMORY values must be one of the values supported for that VCPU value. The supported values are 0.25, 0.5, 1, 2, and 4
value = 0.25
MEMORY= 512, 1024, or 2048value = 0.5
MEMORY= 1024, 2048, 3072, or 4096value = 1
MEMORY= 2048, 3072, 4096, 5120, 6144, 7168, or 8192value = 2
MEMORY= 4096, 5120, 6144, 7168, 8192, 9216, 10240, 11264, 12288, 13312, 14336, 15360, or 16384value = 4
MEMORY= 8192, 9216, 10240, 11264, 12288, 13312, 14336, 15360, 16384, 17408, 18432, 19456, 20480, 21504, 22528, 23552, 24576, 25600, 26624, 27648, 28672, 29696, or 30720
type (string) --
The type of resource to assign to a container. The supported resources include GPU , MEMORY , and VCPU .
linuxParameters (dict) --
Linux-specific modifications that are applied to the container, such as details for device mappings.
devices (list) --
Any host devices to expose to the container. This parameter maps to Devices in the Create a container section of the Docker Remote API and the --device option to docker run .
Note
This parameter isn't applicable to jobs that are running on Fargate resources and shouldn't be provided.
(dict) --
An object representing a container instance host device.
Note
This object isn't applicable to jobs that are running on Fargate resources and shouldn't be provided.
hostPath (string) --
The path for the device on the host container instance.
containerPath (string) --
The path inside the container that's used to expose the host device. By default, the hostPath value is used.
permissions (list) --
The explicit permissions to provide to the container for the device. By default, the container has permissions for read , write , and mknod for the device.
initProcessEnabled (boolean) --
If true, run an init process inside the container that forwards signals and reaps processes. This parameter maps to the --init option to docker run . This parameter requires version 1.25 of the Docker Remote API or greater on your container instance. To check the Docker Remote API version on your container instance, log into your container instance and run the following command: sudo docker version | grep "Server API version"
sharedMemorySize (integer) --
The value for the size (in MiB) of the /dev/shm volume. This parameter maps to the --shm-size option to docker run .
Note
This parameter isn't applicable to jobs that are running on Fargate resources and shouldn't be provided.
tmpfs (list) --
The container path, mount options, and size (in MiB) of the tmpfs mount. This parameter maps to the --tmpfs option to docker run .
Note
This parameter isn't applicable to jobs that are running on Fargate resources and shouldn't be provided.
(dict) --
The container path, mount options, and size of the tmpfs mount.
Note
This object isn't applicable to jobs that are running on Fargate resources.
containerPath (string) --
The absolute file path in the container where the tmpfs volume is mounted.
size (integer) --
The size (in MiB) of the tmpfs volume.
mountOptions (list) --
The list of tmpfs volume mount options.
Valid values: "defaults " | "ro " | "rw " | "suid " | "nosuid " | "dev " | "nodev " | "exec " | "noexec " | "sync " | "async " | "dirsync " | "remount " | "mand " | "nomand " | "atime " | "noatime " | "diratime " | "nodiratime " | "bind " | "rbind" | "unbindable" | "runbindable" | "private" | "rprivate" | "shared" | "rshared" | "slave" | "rslave" | "relatime " | "norelatime " | "strictatime " | "nostrictatime " | "mode " | "uid " | "gid " | "nr_inodes " | "nr_blocks " | "mpol "
maxSwap (integer) --
The total amount of swap memory (in MiB) a container can use. This parameter is translated to the --memory-swap option to docker run where the value is the sum of the container memory plus the maxSwap value. For more information, see ` --memory-swap details <https://docs.docker.com/config/containers/resource_constraints/#--memory-swap-details>`__ in the Docker documentation.
If a maxSwap value of 0 is specified, the container doesn't use swap. Accepted values are 0 or any positive integer. If the maxSwap parameter is omitted, the container doesn't use the swap configuration for the container instance it is running on. A maxSwap value must be set for the swappiness parameter to be used.
Note
This parameter isn't applicable to jobs that are running on Fargate resources and shouldn't be provided.
swappiness (integer) --
This allows you to tune a container's memory swappiness behavior. A swappiness value of 0 causes swapping not to happen unless absolutely necessary. A swappiness value of 100 causes pages to be swapped very aggressively. Accepted values are whole numbers between 0 and 100 . If the swappiness parameter isn't specified, a default value of 60 is used. If a value isn't specified for maxSwap , then this parameter is ignored. If maxSwap is set to 0, the container doesn't use swap. This parameter maps to the --memory-swappiness option to docker run .
Consider the following when you use a per-container swap configuration.
Note
The Amazon ECS optimized AMIs don't have swap enabled by default. You must enable swap on the instance to use this feature. For more information, see Instance store swap volumes in the Amazon EC2 User Guide for Linux Instances or How do I allocate memory to work as swap space in an Amazon EC2 instance by using a swap file?
maxSwap and swappiness parameters are omitted from a job definition, each container will have a default swappiness value of 60, and the total swap usage will be limited to two times the memory reservation of the container.Note
This parameter isn't applicable to jobs that are running on Fargate resources and shouldn't be provided.
logConfiguration (dict) --
The log configuration specification for the container.
This parameter maps to LogConfig in the Create a container section of the Docker Remote API and the --log-driver option to docker run . By default, containers use the same logging driver that the Docker daemon uses. However the container might use a different logging driver than the Docker daemon by specifying a log driver with this parameter in the container definition. To use a different logging driver for a container, the log system must be configured properly on the container instance (or on a different log server for remote logging options). For more information on the options for different supported log drivers, see Configure logging drivers in the Docker documentation.
Note
Batch currently supports a subset of the logging drivers available to the Docker daemon (shown in the LogConfiguration data type).
This parameter requires version 1.18 of the Docker Remote API or greater on your container instance. To check the Docker Remote API version on your container instance, log into your container instance and run the following command: sudo docker version | grep "Server API version"
Note
The Amazon ECS container agent running on a container instance must register the logging drivers available on that instance with the ECS_AVAILABLE_LOGGING_DRIVERS environment variable before containers placed on that instance can use these log configuration options. For more information, see Amazon ECS container agent configuration in the Amazon Elastic Container Service Developer Guide .
logDriver (string) --
The log driver to use for the container. The valid values listed for this parameter are log drivers that the Amazon ECS container agent can communicate with by default.
The supported log drivers are awslogs , fluentd , gelf , json-file , journald , logentries , syslog , and splunk .
Note
Jobs that are running on Fargate resources are restricted to the awslogs and splunk log drivers.
awslogs
Specifies the Amazon CloudWatch Logs logging driver. For more information, see Using the awslogs log driver in the Batch User Guide and Amazon CloudWatch Logs logging driver in the Docker documentation.
fluentd
Specifies the Fluentd logging driver. For more information, including usage and options, see Fluentd logging driver in the Docker documentation.
gelf
Specifies the Graylog Extended Format (GELF) logging driver. For more information, including usage and options, see Graylog Extended Format logging driver in the Docker documentation.
journald
Specifies the journald logging driver. For more information, including usage and options, see Journald logging driver in the Docker documentation.
json-file
Specifies the JSON file logging driver. For more information, including usage and options, see JSON File logging driver in the Docker documentation.
splunk
Specifies the Splunk logging driver. For more information, including usage and options, see Splunk logging driver in the Docker documentation.
syslog
Specifies the syslog logging driver. For more information, including usage and options, see Syslog logging driver in the Docker documentation.
Note
If you have a custom driver that's not listed earlier that you want to work with the Amazon ECS container agent, you can fork the Amazon ECS container agent project that's available on GitHub and customize it to work with that driver. We encourage you to submit pull requests for changes that you want to have included. However, Amazon Web Services doesn't currently support running modified copies of this software.
This parameter requires version 1.18 of the Docker Remote API or greater on your container instance. To check the Docker Remote API version on your container instance, log into your container instance and run the following command: sudo docker version | grep "Server API version"
options (dict) --
The configuration options to send to the log driver. This parameter requires version 1.19 of the Docker Remote API or greater on your container instance. To check the Docker Remote API version on your container instance, log into your container instance and run the following command: sudo docker version | grep "Server API version"
secretOptions (list) --
The secrets to pass to the log configuration. For more information, see Specifying sensitive data in the Batch User Guide .
(dict) --
An object representing the secret to expose to your container. Secrets can be exposed to a container in the following ways:
secrets container definition parameter.secretOptions container definition parameter.For more information, see Specifying sensitive data in the Batch User Guide .
name (string) --
The name of the secret.
valueFrom (string) --
The secret to expose to the container. The supported values are either the full ARN of the Secrets Manager secret or the full ARN of the parameter in the Amazon Web Services Systems Manager Parameter Store.
Note
If the Amazon Web Services Systems Manager Parameter Store parameter exists in the same Region as the job you're launching, then you can use either the full ARN or name of the parameter. If the parameter exists in a different Region, then the full ARN must be specified.
secrets (list) --
The secrets for the container. For more information, see Specifying sensitive data in the Batch User Guide .
(dict) --
An object representing the secret to expose to your container. Secrets can be exposed to a container in the following ways:
secrets container definition parameter.secretOptions container definition parameter.For more information, see Specifying sensitive data in the Batch User Guide .
name (string) --
The name of the secret.
valueFrom (string) --
The secret to expose to the container. The supported values are either the full ARN of the Secrets Manager secret or the full ARN of the parameter in the Amazon Web Services Systems Manager Parameter Store.
Note
If the Amazon Web Services Systems Manager Parameter Store parameter exists in the same Region as the job you're launching, then you can use either the full ARN or name of the parameter. If the parameter exists in a different Region, then the full ARN must be specified.
networkConfiguration (dict) --
The network configuration for jobs that are running on Fargate resources. Jobs that are running on EC2 resources must not specify this parameter.
assignPublicIp (string) --
Indicates whether the job should have a public IP address. For a job that is running on Fargate resources in a private subnet to send outbound traffic to the internet (for example, to pull container images), the private subnet requires a NAT gateway be attached to route requests to the internet. For more information, see Amazon ECS task networking . The default value is "DISABLED".
fargatePlatformConfiguration (dict) --
The platform configuration for jobs that are running on Fargate resources. Jobs that are running on EC2 resources must not specify this parameter.
platformVersion (string) --
The Fargate platform version where the jobs are running. A platform version is specified only for jobs that are running on Fargate resources. If one isn't specified, the LATEST platform version is used by default. This uses a recent, approved version of the Fargate platform for compute resources. For more information, see Fargate platform versions in the Amazon Elastic Container Service Developer Guide .
timeout (dict) --
The timeout configuration for jobs that are submitted with this job definition. You can specify a timeout duration after which Batch terminates your jobs if they haven't finished.
attemptDurationSeconds (integer) --
The time duration in seconds (measured from the job attempt's startedAt timestamp) after which Batch terminates your jobs if they have not finished. The minimum value for the timeout is 60 seconds.
nodeProperties (dict) --
An object with various properties specific to multi-node parallel jobs.
Note
If the job runs on Fargate resources, then you must not specify nodeProperties ; use containerProperties instead.
numNodes (integer) --
The number of nodes associated with a multi-node parallel job.
mainNode (integer) --
Specifies the node index for the main node of a multi-node parallel job. This node index value must be fewer than the number of nodes.
nodeRangeProperties (list) --
A list of node ranges and their properties associated with a multi-node parallel job.
(dict) --
An object representing the properties of the node range for a multi-node parallel job.
targetNodes (string) --
The range of nodes, using node index values. A range of 0:3 indicates nodes with index values of 0 through 3 . If the starting range value is omitted (:n ), then 0 is used to start the range. If the ending range value is omitted (n: ), then the highest possible node index is used to end the range. Your accumulative node ranges must account for all nodes (0:n ). You can nest node ranges, for example 0:10 and 4:5 , in which case the 4:5 range properties override the 0:10 properties.
container (dict) --
The container details for the node range.
image (string) --
The image used to start a container. This string is passed directly to the Docker daemon. Images in the Docker Hub registry are available by default. Other repositories are specified with `` repository-url /image :tag `` . Up to 255 letters (uppercase and lowercase), numbers, hyphens, underscores, colons, periods, forward slashes, and number signs are allowed. This parameter maps to Image in the Create a container section of the Docker Remote API and the IMAGE parameter of docker run .
Note
Docker image architecture must match the processor architecture of the compute resources that they're scheduled on. For example, ARM-based Docker images can only run on ARM-based compute resources.
registry/repository[:tag] or registry/repository[@digest] naming conventions. For example, ``public.ecr.aws/registry_alias /my-web-app :latest `` .012345678910.dkr.ecr.<region-name>.amazonaws.com/<repository-name> ).ubuntu or mongo ).amazon/amazon-ecs-agent ).quay.io/assemblyline/ubuntu ).vcpus (integer) --
This parameter is deprecated, use resourceRequirements to specify the vCPU requirements for the job definition. It's not supported for jobs running on Fargate resources. For jobs running on EC2 resources, it specifies the number of vCPUs reserved for the job.
Each vCPU is equivalent to 1,024 CPU shares. This parameter maps to CpuShares in the Create a container section of the Docker Remote API and the --cpu-shares option to docker run . The number of vCPUs must be specified but can be specified in several places. You must specify it at least once for each node.
memory (integer) --
This parameter is deprecated, use resourceRequirements to specify the memory requirements for the job definition. It's not supported for jobs running on Fargate resources. For jobs running on EC2 resources, it specifies the memory hard limit (in MiB) for a container. If your container attempts to exceed the specified number, it's terminated. You must specify at least 4 MiB of memory for a job using this parameter. The memory hard limit can be specified in several places. It must be specified for each node at least once.
command (list) --
The command that's passed to the container. This parameter maps to Cmd in the Create a container section of the Docker Remote API and the COMMAND parameter to docker run . For more information, see https://docs.docker.com/engine/reference/builder/#cmd .
jobRoleArn (string) --
The Amazon Resource Name (ARN) of the IAM role that the container can assume for Amazon Web Services permissions. For more information, see IAM roles for tasks in the Amazon Elastic Container Service Developer Guide .
executionRoleArn (string) --
The Amazon Resource Name (ARN) of the execution role that Batch can assume. For jobs that run on Fargate resources, you must provide an execution role. For more information, see Batch execution IAM role in the Batch User Guide .
volumes (list) --
A list of data volumes used in a job.
(dict) --
A data volume used in a job's container properties.
host (dict) --
The contents of the host parameter determine whether your data volume persists on the host container instance and where it is stored. If the host parameter is empty, then the Docker daemon assigns a host path for your data volume. However, the data isn't guaranteed to persist after the containers associated with it stop running.
Note
This parameter isn't applicable to jobs that are running on Fargate resources and shouldn't be provided.
sourcePath (string) --
The path on the host container instance that's presented to the container. If this parameter is empty, then the Docker daemon has assigned a host path for you. If this parameter contains a file location, then the data volume persists at the specified location on the host container instance until you delete it manually. If the source path location doesn't exist on the host container instance, the Docker daemon creates it. If the location does exist, the contents of the source path folder are exported.
Note
This parameter isn't applicable to jobs that run on Fargate resources and shouldn't be provided.
name (string) --
The name of the volume. It can be up to 255 letters long. It can contain uppercase and lowercase letters, numbers, hyphens (-), and underscores (_). This name is referenced in the sourceVolume parameter of container definition mountPoints .
efsVolumeConfiguration (dict) --
This parameter is specified when you are using an Amazon Elastic File System file system for job storage. Jobs that are running on Fargate resources must specify a platformVersion of at least 1.4.0 .
fileSystemId (string) --
The Amazon EFS file system ID to use.
rootDirectory (string) --
The directory within the Amazon EFS file system to mount as the root directory inside the host. If this parameter is omitted, the root of the Amazon EFS volume is used instead. Specifying / has the same effect as omitting this parameter. The maximum length is 4,096 characters.
Warning
If an EFS access point is specified in the authorizationConfig , the root directory parameter must either be omitted or set to / , which enforces the path set on the Amazon EFS access point.
transitEncryption (string) --
Determines whether to enable encryption for Amazon EFS data in transit between the Amazon ECS host and the Amazon EFS server. Transit encryption must be enabled if Amazon EFS IAM authorization is used. If this parameter is omitted, the default value of DISABLED is used. For more information, see Encrypting data in transit in the Amazon Elastic File System User Guide .
transitEncryptionPort (integer) --
The port to use when sending encrypted data between the Amazon ECS host and the Amazon EFS server. If you don't specify a transit encryption port, it uses the port selection strategy that the Amazon EFS mount helper uses. The value must be between 0 and 65,535. For more information, see EFS mount helper in the Amazon Elastic File System User Guide .
authorizationConfig (dict) --
The authorization configuration details for the Amazon EFS file system.
accessPointId (string) --
The Amazon EFS access point ID to use. If an access point is specified, the root directory value specified in the EFSVolumeConfiguration must either be omitted or set to / which will enforce the path set on the EFS access point. If an access point is used, transit encryption must be enabled in the EFSVolumeConfiguration . For more information, see Working with Amazon EFS access points in the Amazon Elastic File System User Guide .
iam (string) --
Whether or not to use the Batch job IAM role defined in a job definition when mounting the Amazon EFS file system. If enabled, transit encryption must be enabled in the EFSVolumeConfiguration . If this parameter is omitted, the default value of DISABLED is used. For more information, see Using Amazon EFS access points in the Batch User Guide . EFS IAM authorization requires that TransitEncryption be ENABLED and that a JobRoleArn is specified.
environment (list) --
The environment variables to pass to a container. This parameter maps to Env in the Create a container section of the Docker Remote API and the --env option to docker run .
Warning
We don't recommend using plaintext environment variables for sensitive information, such as credential data.
Note
Environment variables must not start with AWS_BATCH ; this naming convention is reserved for variables that are set by the Batch service.
(dict) --
A key-value pair object.
name (string) --
The name of the key-value pair. For environment variables, this is the name of the environment variable.
value (string) --
The value of the key-value pair. For environment variables, this is the value of the environment variable.
mountPoints (list) --
The mount points for data volumes in your container. This parameter maps to Volumes in the Create a container section of the Docker Remote API and the --volume option to docker run .
(dict) --
Details on a Docker volume mount point that's used in a job's container properties. This parameter maps to Volumes in the Create a container section of the Docker Remote API and the --volume option to docker run.
containerPath (string) --
The path on the container where the host volume is mounted.
readOnly (boolean) --
If this value is true , the container has read-only access to the volume. Otherwise, the container can write to the volume. The default value is false .
sourceVolume (string) --
The name of the volume to mount.
readonlyRootFilesystem (boolean) --
When this parameter is true, the container is given read-only access to its root file system. This parameter maps to ReadonlyRootfs in the Create a container section of the Docker Remote API and the --read-only option to docker run .
privileged (boolean) --
When this parameter is true, the container is given elevated permissions on the host container instance (similar to the root user). This parameter maps to Privileged in the Create a container section of the Docker Remote API and the --privileged option to docker run . The default value is false.
Note
This parameter isn't applicable to jobs that are running on Fargate resources and shouldn't be provided, or specified as false.
ulimits (list) --
A list of ulimits to set in the container. This parameter maps to Ulimits in the Create a container section of the Docker Remote API and the --ulimit option to docker run .
Note
This parameter isn't applicable to jobs that are running on Fargate resources and shouldn't be provided.
(dict) --
The ulimit settings to pass to the container.
Note
This object isn't applicable to jobs that are running on Fargate resources.
hardLimit (integer) --
The hard limit for the ulimit type.
name (string) --
The type of the ulimit .
softLimit (integer) --
The soft limit for the ulimit type.
user (string) --
The user name to use inside the container. This parameter maps to User in the Create a container section of the Docker Remote API and the --user option to docker run .
instanceType (string) --
The instance type to use for a multi-node parallel job. All node groups in a multi-node parallel job must use the same instance type.
Note
This parameter isn't applicable to single-node container jobs or jobs that run on Fargate resources, and shouldn't be provided.
resourceRequirements (list) --
The type and amount of resources to assign to a container. The supported resources include GPU , MEMORY , and VCPU .
(dict) --
The type and amount of a resource to assign to a container. The supported resources include GPU , MEMORY , and VCPU .
value (string) --
The quantity of the specified resource to reserve for the container. The values vary based on the type specified.
type="GPU"
The number of physical GPUs to reserve for the container. The number of GPUs reserved for all containers in a job shouldn't exceed the number of available GPUs on the compute resource that the job is launched on.
Note
GPUs are not available for jobs that are running on Fargate resources.
type="MEMORY"
The memory hard limit (in MiB) present to the container. This parameter is supported for jobs that are running on EC2 resources. If your container attempts to exceed the memory specified, the container is terminated. This parameter maps to Memory in the Create a container section of the Docker Remote API and the --memory option to docker run . You must specify at least 4 MiB of memory for a job. This is required but can be specified in several places for multi-node parallel (MNP) jobs. It must be specified for each node at least once. This parameter maps to Memory in the Create a container section of the Docker Remote API and the --memory option to docker run .
Note
If you're trying to maximize your resource utilization by providing your jobs as much memory as possible for a particular instance type, see Memory management in the Batch User Guide .
For jobs that are running on Fargate resources, then value is the hard limit (in MiB), and must match one of the supported values and the VCPU values must be one of the values supported for that memory value.
value = 512
VCPU= 0.25value = 1024
VCPU= 0.25 or 0.5value = 2048
VCPU= 0.25, 0.5, or 1value = 3072
VCPU= 0.5, or 1value = 4096
VCPU= 0.5, 1, or 2value = 5120, 6144, or 7168
VCPU= 1 or 2value = 8192
VCPU= 1, 2, or 4value = 9216, 10240, 11264, 12288, 13312, 14336, 15360, or 16384
VCPU= 2 or 4value = 17408, 18432, 19456, 20480, 21504, 22528, 23552, 24576, 25600, 26624, 27648, 28672, 29696, or 30720
VCPU= 4type="VCPU"
The number of vCPUs reserved for the container. This parameter maps to CpuShares in the Create a container section of the Docker Remote API and the --cpu-shares option to docker run . Each vCPU is equivalent to 1,024 CPU shares. For EC2 resources, you must specify at least one vCPU. This is required but can be specified in several places; it must be specified for each node at least once.
For jobs that are running on Fargate resources, then value must match one of the supported values and the MEMORY values must be one of the values supported for that VCPU value. The supported values are 0.25, 0.5, 1, 2, and 4
value = 0.25
MEMORY= 512, 1024, or 2048value = 0.5
MEMORY= 1024, 2048, 3072, or 4096value = 1
MEMORY= 2048, 3072, 4096, 5120, 6144, 7168, or 8192value = 2
MEMORY= 4096, 5120, 6144, 7168, 8192, 9216, 10240, 11264, 12288, 13312, 14336, 15360, or 16384value = 4
MEMORY= 8192, 9216, 10240, 11264, 12288, 13312, 14336, 15360, 16384, 17408, 18432, 19456, 20480, 21504, 22528, 23552, 24576, 25600, 26624, 27648, 28672, 29696, or 30720
type (string) --
The type of resource to assign to a container. The supported resources include GPU , MEMORY , and VCPU .
linuxParameters (dict) --
Linux-specific modifications that are applied to the container, such as details for device mappings.
devices (list) --
Any host devices to expose to the container. This parameter maps to Devices in the Create a container section of the Docker Remote API and the --device option to docker run .
Note
This parameter isn't applicable to jobs that are running on Fargate resources and shouldn't be provided.
(dict) --
An object representing a container instance host device.
Note
This object isn't applicable to jobs that are running on Fargate resources and shouldn't be provided.
hostPath (string) --
The path for the device on the host container instance.
containerPath (string) --
The path inside the container that's used to expose the host device. By default, the hostPath value is used.
permissions (list) --
The explicit permissions to provide to the container for the device. By default, the container has permissions for read , write , and mknod for the device.
initProcessEnabled (boolean) --
If true, run an init process inside the container that forwards signals and reaps processes. This parameter maps to the --init option to docker run . This parameter requires version 1.25 of the Docker Remote API or greater on your container instance. To check the Docker Remote API version on your container instance, log into your container instance and run the following command: sudo docker version | grep "Server API version"
sharedMemorySize (integer) --
The value for the size (in MiB) of the /dev/shm volume. This parameter maps to the --shm-size option to docker run .
Note
This parameter isn't applicable to jobs that are running on Fargate resources and shouldn't be provided.
tmpfs (list) --
The container path, mount options, and size (in MiB) of the tmpfs mount. This parameter maps to the --tmpfs option to docker run .
Note
This parameter isn't applicable to jobs that are running on Fargate resources and shouldn't be provided.
(dict) --
The container path, mount options, and size of the tmpfs mount.
Note
This object isn't applicable to jobs that are running on Fargate resources.
containerPath (string) --
The absolute file path in the container where the tmpfs volume is mounted.
size (integer) --
The size (in MiB) of the tmpfs volume.
mountOptions (list) --
The list of tmpfs volume mount options.
Valid values: "defaults " | "ro " | "rw " | "suid " | "nosuid " | "dev " | "nodev " | "exec " | "noexec " | "sync " | "async " | "dirsync " | "remount " | "mand " | "nomand " | "atime " | "noatime " | "diratime " | "nodiratime " | "bind " | "rbind" | "unbindable" | "runbindable" | "private" | "rprivate" | "shared" | "rshared" | "slave" | "rslave" | "relatime " | "norelatime " | "strictatime " | "nostrictatime " | "mode " | "uid " | "gid " | "nr_inodes " | "nr_blocks " | "mpol "
maxSwap (integer) --
The total amount of swap memory (in MiB) a container can use. This parameter is translated to the --memory-swap option to docker run where the value is the sum of the container memory plus the maxSwap value. For more information, see ` --memory-swap details <https://docs.docker.com/config/containers/resource_constraints/#--memory-swap-details>`__ in the Docker documentation.
If a maxSwap value of 0 is specified, the container doesn't use swap. Accepted values are 0 or any positive integer. If the maxSwap parameter is omitted, the container doesn't use the swap configuration for the container instance it is running on. A maxSwap value must be set for the swappiness parameter to be used.
Note
This parameter isn't applicable to jobs that are running on Fargate resources and shouldn't be provided.
swappiness (integer) --
This allows you to tune a container's memory swappiness behavior. A swappiness value of 0 causes swapping not to happen unless absolutely necessary. A swappiness value of 100 causes pages to be swapped very aggressively. Accepted values are whole numbers between 0 and 100 . If the swappiness parameter isn't specified, a default value of 60 is used. If a value isn't specified for maxSwap , then this parameter is ignored. If maxSwap is set to 0, the container doesn't use swap. This parameter maps to the --memory-swappiness option to docker run .
Consider the following when you use a per-container swap configuration.
Note
The Amazon ECS optimized AMIs don't have swap enabled by default. You must enable swap on the instance to use this feature. For more information, see Instance store swap volumes in the Amazon EC2 User Guide for Linux Instances or How do I allocate memory to work as swap space in an Amazon EC2 instance by using a swap file?
maxSwap and swappiness parameters are omitted from a job definition, each container will have a default swappiness value of 60, and the total swap usage will be limited to two times the memory reservation of the container.Note
This parameter isn't applicable to jobs that are running on Fargate resources and shouldn't be provided.
logConfiguration (dict) --
The log configuration specification for the container.
This parameter maps to LogConfig in the Create a container section of the Docker Remote API and the --log-driver option to docker run . By default, containers use the same logging driver that the Docker daemon uses. However the container might use a different logging driver than the Docker daemon by specifying a log driver with this parameter in the container definition. To use a different logging driver for a container, the log system must be configured properly on the container instance (or on a different log server for remote logging options). For more information on the options for different supported log drivers, see Configure logging drivers in the Docker documentation.
Note
Batch currently supports a subset of the logging drivers available to the Docker daemon (shown in the LogConfiguration data type).
This parameter requires version 1.18 of the Docker Remote API or greater on your container instance. To check the Docker Remote API version on your container instance, log into your container instance and run the following command: sudo docker version | grep "Server API version"
Note
The Amazon ECS container agent running on a container instance must register the logging drivers available on that instance with the ECS_AVAILABLE_LOGGING_DRIVERS environment variable before containers placed on that instance can use these log configuration options. For more information, see Amazon ECS container agent configuration in the Amazon Elastic Container Service Developer Guide .
logDriver (string) --
The log driver to use for the container. The valid values listed for this parameter are log drivers that the Amazon ECS container agent can communicate with by default.
The supported log drivers are awslogs , fluentd , gelf , json-file , journald , logentries , syslog , and splunk .
Note
Jobs that are running on Fargate resources are restricted to the awslogs and splunk log drivers.
awslogs
Specifies the Amazon CloudWatch Logs logging driver. For more information, see Using the awslogs log driver in the Batch User Guide and Amazon CloudWatch Logs logging driver in the Docker documentation.
fluentd
Specifies the Fluentd logging driver. For more information, including usage and options, see Fluentd logging driver in the Docker documentation.
gelf
Specifies the Graylog Extended Format (GELF) logging driver. For more information, including usage and options, see Graylog Extended Format logging driver in the Docker documentation.
journald
Specifies the journald logging driver. For more information, including usage and options, see Journald logging driver in the Docker documentation.
json-file
Specifies the JSON file logging driver. For more information, including usage and options, see JSON File logging driver in the Docker documentation.
splunk
Specifies the Splunk logging driver. For more information, including usage and options, see Splunk logging driver in the Docker documentation.
syslog
Specifies the syslog logging driver. For more information, including usage and options, see Syslog logging driver in the Docker documentation.
Note
If you have a custom driver that's not listed earlier that you want to work with the Amazon ECS container agent, you can fork the Amazon ECS container agent project that's available on GitHub and customize it to work with that driver. We encourage you to submit pull requests for changes that you want to have included. However, Amazon Web Services doesn't currently support running modified copies of this software.
This parameter requires version 1.18 of the Docker Remote API or greater on your container instance. To check the Docker Remote API version on your container instance, log into your container instance and run the following command: sudo docker version | grep "Server API version"
options (dict) --
The configuration options to send to the log driver. This parameter requires version 1.19 of the Docker Remote API or greater on your container instance. To check the Docker Remote API version on your container instance, log into your container instance and run the following command: sudo docker version | grep "Server API version"
secretOptions (list) --
The secrets to pass to the log configuration. For more information, see Specifying sensitive data in the Batch User Guide .
(dict) --
An object representing the secret to expose to your container. Secrets can be exposed to a container in the following ways:
secrets container definition parameter.secretOptions container definition parameter.For more information, see Specifying sensitive data in the Batch User Guide .
name (string) --
The name of the secret.
valueFrom (string) --
The secret to expose to the container. The supported values are either the full ARN of the Secrets Manager secret or the full ARN of the parameter in the Amazon Web Services Systems Manager Parameter Store.
Note
If the Amazon Web Services Systems Manager Parameter Store parameter exists in the same Region as the job you're launching, then you can use either the full ARN or name of the parameter. If the parameter exists in a different Region, then the full ARN must be specified.
secrets (list) --
The secrets for the container. For more information, see Specifying sensitive data in the Batch User Guide .
(dict) --
An object representing the secret to expose to your container. Secrets can be exposed to a container in the following ways:
secrets container definition parameter.secretOptions container definition parameter.For more information, see Specifying sensitive data in the Batch User Guide .
name (string) --
The name of the secret.
valueFrom (string) --
The secret to expose to the container. The supported values are either the full ARN of the Secrets Manager secret or the full ARN of the parameter in the Amazon Web Services Systems Manager Parameter Store.
Note
If the Amazon Web Services Systems Manager Parameter Store parameter exists in the same Region as the job you're launching, then you can use either the full ARN or name of the parameter. If the parameter exists in a different Region, then the full ARN must be specified.
networkConfiguration (dict) --
The network configuration for jobs that are running on Fargate resources. Jobs that are running on EC2 resources must not specify this parameter.
assignPublicIp (string) --
Indicates whether the job should have a public IP address. For a job that is running on Fargate resources in a private subnet to send outbound traffic to the internet (for example, to pull container images), the private subnet requires a NAT gateway be attached to route requests to the internet. For more information, see Amazon ECS task networking . The default value is "DISABLED".
fargatePlatformConfiguration (dict) --
The platform configuration for jobs that are running on Fargate resources. Jobs that are running on EC2 resources must not specify this parameter.
platformVersion (string) --
The Fargate platform version where the jobs are running. A platform version is specified only for jobs that are running on Fargate resources. If one isn't specified, the LATEST platform version is used by default. This uses a recent, approved version of the Fargate platform for compute resources. For more information, see Fargate platform versions in the Amazon Elastic Container Service Developer Guide .
tags (dict) --
The tags applied to the job definition.
propagateTags (boolean) --
Specifies whether to propagate the tags from the job or job definition to the corresponding Amazon ECS task. If no value is specified, the tags aren't propagated. Tags can only be propagated to the tasks during task creation. For tags with the same name, job tags are given priority over job definitions tags. If the total number of combined tags from the job and job definition is over 50, the job is moved to the FAILED state.
platformCapabilities (list) --
The platform capabilities required by the job definition. If no value is specified, it defaults to EC2 . Jobs run on Fargate resources specify FARGATE .
NextToken (string) --
A token to resume pagination.
Batch.Paginator.DescribeJobQueues¶paginator = client.get_paginator('describe_job_queues')
paginate(**kwargs)¶Creates an iterator that will paginate through responses from Batch.Client.describe_job_queues().
See also: AWS API Documentation
Request Syntax
response_iterator = paginator.paginate(
jobQueues=[
'string',
],
PaginationConfig={
'MaxItems': 123,
'PageSize': 123,
'StartingToken': 'string'
}
)
A list of up to 100 queue names or full queue Amazon Resource Name (ARN) entries.
A dictionary that provides parameters to control pagination.
The total number of items to return. If the total number of items available is more than the value specified in max-items then a NextToken will be provided in the output that you can use to resume pagination.
The size of each page.
A token to specify where to start paginating. This is the NextToken from a previous response.
dict
Response Syntax
{
'jobQueues': [
{
'jobQueueName': 'string',
'jobQueueArn': 'string',
'state': 'ENABLED'|'DISABLED',
'schedulingPolicyArn': 'string',
'status': 'CREATING'|'UPDATING'|'DELETING'|'DELETED'|'VALID'|'INVALID',
'statusReason': 'string',
'priority': 123,
'computeEnvironmentOrder': [
{
'order': 123,
'computeEnvironment': 'string'
},
],
'tags': {
'string': 'string'
}
},
],
'NextToken': 'string'
}
Response Structure
(dict) --
jobQueues (list) --
The list of job queues.
(dict) --
An object representing the details of an Batch job queue.
jobQueueName (string) --
The name of the job queue.
jobQueueArn (string) --
The Amazon Resource Name (ARN) of the job queue.
state (string) --
Describes the ability of the queue to accept new jobs. If the job queue state is ENABLED , it's able to accept jobs. If the job queue state is DISABLED , new jobs can't be added to the queue, but jobs already in the queue can finish.
schedulingPolicyArn (string) --
The Amazon Resource Name (ARN) of the scheduling policy. The format is aws:*Partition* :batch:*Region* :*Account* :scheduling-policy/*Name* `` . For example, ``aws:aws:batch:us-west-2:012345678910:scheduling-policy/MySchedulingPolicy .
status (string) --
The status of the job queue (for example, CREATING or VALID ).
statusReason (string) --
A short, human-readable string to provide additional details about the current status of the job queue.
priority (integer) --
The priority of the job queue. Job queues with a higher priority (or a higher integer value for the priority parameter) are evaluated first when associated with the same compute environment. Priority is determined in descending order, for example, a job queue with a priority value of 10 is given scheduling preference over a job queue with a priority value of 1 . All of the compute environments must be either EC2 (EC2 or SPOT ) or Fargate (FARGATE or FARGATE_SPOT ); EC2 and Fargate compute environments can't be mixed.
computeEnvironmentOrder (list) --
The compute environments that are attached to the job queue and the order that job placement is preferred. Compute environments are selected for job placement in ascending order.
(dict) --
The order in which compute environments are tried for job placement within a queue. Compute environments are tried in ascending order. For example, if two compute environments are associated with a job queue, the compute environment with a lower order integer value is tried for job placement first. Compute environments must be in the VALID state before you can associate them with a job queue. All of the compute environments must be either EC2 (EC2 or SPOT ) or Fargate (FARGATE or FARGATE_SPOT ); EC2 and Fargate compute environments can't be mixed.
Note
All compute environments that are associated with a job queue must share the same architecture. Batch doesn't support mixing compute environment architecture types in a single job queue.
order (integer) --
The order of the compute environment. Compute environments are tried in ascending order. For example, if two compute environments are associated with a job queue, the compute environment with a lower order integer value is tried for job placement first.
computeEnvironment (string) --
The Amazon Resource Name (ARN) of the compute environment.
tags (dict) --
The tags applied to the job queue. For more information, see Tagging your Batch resources in Batch User Guide .
NextToken (string) --
A token to resume pagination.
Batch.Paginator.ListJobs¶paginator = client.get_paginator('list_jobs')
paginate(**kwargs)¶Creates an iterator that will paginate through responses from Batch.Client.list_jobs().
See also: AWS API Documentation
Request Syntax
response_iterator = paginator.paginate(
jobQueue='string',
arrayJobId='string',
multiNodeJobId='string',
jobStatus='SUBMITTED'|'PENDING'|'RUNNABLE'|'STARTING'|'RUNNING'|'SUCCEEDED'|'FAILED',
filters=[
{
'name': 'string',
'values': [
'string',
]
},
],
PaginationConfig={
'MaxItems': 123,
'PageSize': 123,
'StartingToken': 'string'
}
)
filters parameter is specified, the jobStatus parameter is ignored and jobs with any status are returned. If you don't specify a status, only RUNNING jobs are returned.The filter to apply to the query. Only one filter can be used at a time. When the filter is used, jobStatus is ignored. The filter doesn't apply to child jobs in an array or multi-node parallel (MNP) jobs. The results are sorted by the createdAt field, with the most recent jobs being first.
JOB_NAME
The value of the filter is a case-insensitive match for the job name. If the value ends with an asterisk (*), the filter will match any job name that begins with the string before the '*'. This corresponds to the jobName value. For example, test1 matches both Test1 and test1 , and test1* matches both test1 and Test10 . When the JOB_NAME filter is used, the results are grouped by the job name and version.
JOB_DEFINITION
The value for the filter is the name or Amazon Resource Name (ARN) of the job definition. This corresponds to the jobDefinition value. The value is case sensitive. When the value for the filter is the job definition name, the results include all the jobs that used any revision of that job definition name. If the value ends with an asterisk (*), the filter will match any job definition name that begins with the string before the '*'. For example, jd1 matches only jd1 , and jd1* matches both jd1 and jd1A . The version of the job definition that's used doesn't affect the sort order. When the JOB_DEFINITION filter is used and the ARN is used (which is in the form arn:${Partition}:batch:${Region}:${Account}:job-definition/${JobDefinitionName}:${Revision} ), the results include jobs that used the specified revision of the job definition. Asterisk (*) is not supported when the ARN is used.
BEFORE_CREATED_AT
The value for the filter is the time that's before the job was created. This corresponds to the createdAt value. The value is a string representation of the number of milliseconds since 00:00:00 UTC (midnight) on January 1, 1970.
AFTER_CREATED_AT
The value for the filter is the time that's after the job was created. This corresponds to the createdAt value. The value is a string representation of the number of milliseconds since 00:00:00 UTC (midnight) on January 1, 1970.
A filter name and value pair that's used to return a more specific list of results from a ListJobs API operation.
The name of the filter. Filter names are case sensitive.
The filter values.
A dictionary that provides parameters to control pagination.
The total number of items to return. If the total number of items available is more than the value specified in max-items then a NextToken will be provided in the output that you can use to resume pagination.
The size of each page.
A token to specify where to start paginating. This is the NextToken from a previous response.
dict
Response Syntax
{
'jobSummaryList': [
{
'jobArn': 'string',
'jobId': 'string',
'jobName': 'string',
'createdAt': 123,
'status': 'SUBMITTED'|'PENDING'|'RUNNABLE'|'STARTING'|'RUNNING'|'SUCCEEDED'|'FAILED',
'statusReason': 'string',
'startedAt': 123,
'stoppedAt': 123,
'container': {
'exitCode': 123,
'reason': 'string'
},
'arrayProperties': {
'size': 123,
'index': 123
},
'nodeProperties': {
'isMainNode': True|False,
'numNodes': 123,
'nodeIndex': 123
},
'jobDefinition': 'string'
},
],
'NextToken': 'string'
}
Response Structure
(dict) --
jobSummaryList (list) --
A list of job summaries that match the request.
(dict) --
An object representing summary details of a job.
jobArn (string) --
The Amazon Resource Name (ARN) of the job.
jobId (string) --
The ID of the job.
jobName (string) --
The name of the job.
createdAt (integer) --
The Unix timestamp (in milliseconds) for when the job was created. For non-array jobs and parent array jobs, this is when the job entered the SUBMITTED state (at the time SubmitJob was called). For array child jobs, this is when the child job was spawned by its parent and entered the PENDING state.
status (string) --
The current status for the job.
statusReason (string) --
A short, human-readable string to provide additional details about the current status of the job.
startedAt (integer) --
The Unix timestamp for when the job was started (when the job transitioned from the STARTING state to the RUNNING state).
stoppedAt (integer) --
The Unix timestamp for when the job was stopped (when the job transitioned from the RUNNING state to a terminal state, such as SUCCEEDED or FAILED ).
container (dict) --
An object representing the details of the container that's associated with the job.
exitCode (integer) --
The exit code to return upon completion.
reason (string) --
A short (255 max characters) human-readable string to provide additional details about a running or stopped container.
arrayProperties (dict) --
The array properties of the job, if it is an array job.
size (integer) --
The size of the array job. This parameter is returned for parent array jobs.
index (integer) --
The job index within the array that's associated with this job. This parameter is returned for children of array jobs.
nodeProperties (dict) --
The node properties for a single node in a job summary list.
Note
This isn't applicable to jobs that are running on Fargate resources.
isMainNode (boolean) --
Specifies whether the current node is the main node for a multi-node parallel job.
numNodes (integer) --
The number of nodes associated with a multi-node parallel job.
nodeIndex (integer) --
The node index for the node. Node index numbering begins at zero. This index is also available on the node with the AWS_BATCH_JOB_NODE_INDEX environment variable.
jobDefinition (string) --
The Amazon Resource Name (ARN) of the job definition.
NextToken (string) --
A token to resume pagination.
Batch.Paginator.ListSchedulingPolicies¶paginator = client.get_paginator('list_scheduling_policies')
paginate(**kwargs)¶Creates an iterator that will paginate through responses from Batch.Client.list_scheduling_policies().
See also: AWS API Documentation
Request Syntax
response_iterator = paginator.paginate(
PaginationConfig={
'MaxItems': 123,
'PageSize': 123,
'StartingToken': 'string'
}
)
A dictionary that provides parameters to control pagination.
The total number of items to return. If the total number of items available is more than the value specified in max-items then a NextToken will be provided in the output that you can use to resume pagination.
The size of each page.
A token to specify where to start paginating. This is the NextToken from a previous response.
{
'schedulingPolicies': [
{
'arn': 'string'
},
],
'NextToken': 'string'
}
Response Structure
A list of scheduling policies that match the request.
An object that contains the details of a scheduling policy that's returned in a ListSchedulingPolicy action.
Amazon Resource Name (ARN) of the scheduling policy.
A token to resume pagination.