Table of Contents
A low-level client representing Amazon SageMaker Service:
import boto3
client = boto3.client('sagemaker')
These are the available methods:
Adds or overwrites one or more tags for the specified Amazon SageMaker resource. You can add tags to notebook instances, training jobs, hyperparameter tuning jobs, models, endpoint configurations, and endpoints.
Each tag consists of a key and an optional value. Tag keys must be unique per resource. For more information about tags, see For more information, see AWS Tagging Strategies .
Note
Tags that you add to a hyperparameter tuning job by calling this API are also added to any training jobs that the hyperparameter tuning job launches after you call this API, but not to training jobs that the hyperparameter tuning job launched before you called this API. To make sure that the tags associated with a hyperparameter tuning job are also added to all training jobs that the hyperparameter tuning job launches, add the tags when you first create the tuning job by specifying them in the Tags parameter of CreateHyperParameterTuningJob
See also: AWS API Documentation
Request Syntax
response = client.add_tags(
ResourceArn='string',
Tags=[
{
'Key': 'string',
'Value': 'string'
},
]
)
[REQUIRED]
The Amazon Resource Name (ARN) of the resource that you want to tag.
[REQUIRED]
An array of Tag objects. Each tag is a key-value pair. Only the key parameter is required. If you don't specify a value, Amazon SageMaker sets the value to an empty string.
Describes a tag.
The tag key.
The tag value.
dict
Response Syntax
{
'Tags': [
{
'Key': 'string',
'Value': 'string'
},
]
}
Response Structure
(dict) --
Tags (list) --
A list of tags associated with the Amazon SageMaker resource.
(dict) --
Describes a tag.
Key (string) --
The tag key.
Value (string) --
The tag value.
Check if an operation can be paginated.
Create a machine learning algorithm that you can use in Amazon SageMaker and list in the AWS Marketplace.
See also: AWS API Documentation
Request Syntax
response = client.create_algorithm(
AlgorithmName='string',
AlgorithmDescription='string',
TrainingSpecification={
'TrainingImage': 'string',
'TrainingImageDigest': 'string',
'SupportedHyperParameters': [
{
'Name': 'string',
'Description': 'string',
'Type': 'Integer'|'Continuous'|'Categorical'|'FreeText',
'Range': {
'IntegerParameterRangeSpecification': {
'MinValue': 'string',
'MaxValue': 'string'
},
'ContinuousParameterRangeSpecification': {
'MinValue': 'string',
'MaxValue': 'string'
},
'CategoricalParameterRangeSpecification': {
'Values': [
'string',
]
}
},
'IsTunable': True|False,
'IsRequired': True|False,
'DefaultValue': 'string'
},
],
'SupportedTrainingInstanceTypes': [
'ml.m4.xlarge'|'ml.m4.2xlarge'|'ml.m4.4xlarge'|'ml.m4.10xlarge'|'ml.m4.16xlarge'|'ml.m5.large'|'ml.m5.xlarge'|'ml.m5.2xlarge'|'ml.m5.4xlarge'|'ml.m5.12xlarge'|'ml.m5.24xlarge'|'ml.c4.xlarge'|'ml.c4.2xlarge'|'ml.c4.4xlarge'|'ml.c4.8xlarge'|'ml.p2.xlarge'|'ml.p2.8xlarge'|'ml.p2.16xlarge'|'ml.p3.2xlarge'|'ml.p3.8xlarge'|'ml.p3.16xlarge'|'ml.c5.xlarge'|'ml.c5.2xlarge'|'ml.c5.4xlarge'|'ml.c5.9xlarge'|'ml.c5.18xlarge',
],
'SupportsDistributedTraining': True|False,
'MetricDefinitions': [
{
'Name': 'string',
'Regex': 'string'
},
],
'TrainingChannels': [
{
'Name': 'string',
'Description': 'string',
'IsRequired': True|False,
'SupportedContentTypes': [
'string',
],
'SupportedCompressionTypes': [
'None'|'Gzip',
],
'SupportedInputModes': [
'Pipe'|'File',
]
},
],
'SupportedTuningJobObjectiveMetrics': [
{
'Type': 'Maximize'|'Minimize',
'MetricName': 'string'
},
]
},
InferenceSpecification={
'Containers': [
{
'ContainerHostname': 'string',
'Image': 'string',
'ImageDigest': 'string',
'ModelDataUrl': 'string',
'ProductId': 'string'
},
],
'SupportedTransformInstanceTypes': [
'ml.m4.xlarge'|'ml.m4.2xlarge'|'ml.m4.4xlarge'|'ml.m4.10xlarge'|'ml.m4.16xlarge'|'ml.c4.xlarge'|'ml.c4.2xlarge'|'ml.c4.4xlarge'|'ml.c4.8xlarge'|'ml.p2.xlarge'|'ml.p2.8xlarge'|'ml.p2.16xlarge'|'ml.p3.2xlarge'|'ml.p3.8xlarge'|'ml.p3.16xlarge'|'ml.c5.xlarge'|'ml.c5.2xlarge'|'ml.c5.4xlarge'|'ml.c5.9xlarge'|'ml.c5.18xlarge'|'ml.m5.large'|'ml.m5.xlarge'|'ml.m5.2xlarge'|'ml.m5.4xlarge'|'ml.m5.12xlarge'|'ml.m5.24xlarge',
],
'SupportedRealtimeInferenceInstanceTypes': [
'ml.t2.medium'|'ml.t2.large'|'ml.t2.xlarge'|'ml.t2.2xlarge'|'ml.m4.xlarge'|'ml.m4.2xlarge'|'ml.m4.4xlarge'|'ml.m4.10xlarge'|'ml.m4.16xlarge'|'ml.m5.large'|'ml.m5.xlarge'|'ml.m5.2xlarge'|'ml.m5.4xlarge'|'ml.m5.12xlarge'|'ml.m5.24xlarge'|'ml.c4.large'|'ml.c4.xlarge'|'ml.c4.2xlarge'|'ml.c4.4xlarge'|'ml.c4.8xlarge'|'ml.p2.xlarge'|'ml.p2.8xlarge'|'ml.p2.16xlarge'|'ml.p3.2xlarge'|'ml.p3.8xlarge'|'ml.p3.16xlarge'|'ml.c5.large'|'ml.c5.xlarge'|'ml.c5.2xlarge'|'ml.c5.4xlarge'|'ml.c5.9xlarge'|'ml.c5.18xlarge',
],
'SupportedContentTypes': [
'string',
],
'SupportedResponseMIMETypes': [
'string',
]
},
ValidationSpecification={
'ValidationRole': 'string',
'ValidationProfiles': [
{
'ProfileName': 'string',
'TrainingJobDefinition': {
'TrainingInputMode': 'Pipe'|'File',
'HyperParameters': {
'string': 'string'
},
'InputDataConfig': [
{
'ChannelName': 'string',
'DataSource': {
'S3DataSource': {
'S3DataType': 'ManifestFile'|'S3Prefix'|'AugmentedManifestFile',
'S3Uri': 'string',
'S3DataDistributionType': 'FullyReplicated'|'ShardedByS3Key',
'AttributeNames': [
'string',
]
}
},
'ContentType': 'string',
'CompressionType': 'None'|'Gzip',
'RecordWrapperType': 'None'|'RecordIO',
'InputMode': 'Pipe'|'File',
'ShuffleConfig': {
'Seed': 123
}
},
],
'OutputDataConfig': {
'KmsKeyId': 'string',
'S3OutputPath': 'string'
},
'ResourceConfig': {
'InstanceType': 'ml.m4.xlarge'|'ml.m4.2xlarge'|'ml.m4.4xlarge'|'ml.m4.10xlarge'|'ml.m4.16xlarge'|'ml.m5.large'|'ml.m5.xlarge'|'ml.m5.2xlarge'|'ml.m5.4xlarge'|'ml.m5.12xlarge'|'ml.m5.24xlarge'|'ml.c4.xlarge'|'ml.c4.2xlarge'|'ml.c4.4xlarge'|'ml.c4.8xlarge'|'ml.p2.xlarge'|'ml.p2.8xlarge'|'ml.p2.16xlarge'|'ml.p3.2xlarge'|'ml.p3.8xlarge'|'ml.p3.16xlarge'|'ml.c5.xlarge'|'ml.c5.2xlarge'|'ml.c5.4xlarge'|'ml.c5.9xlarge'|'ml.c5.18xlarge',
'InstanceCount': 123,
'VolumeSizeInGB': 123,
'VolumeKmsKeyId': 'string'
},
'StoppingCondition': {
'MaxRuntimeInSeconds': 123
}
},
'TransformJobDefinition': {
'MaxConcurrentTransforms': 123,
'MaxPayloadInMB': 123,
'BatchStrategy': 'MultiRecord'|'SingleRecord',
'Environment': {
'string': 'string'
},
'TransformInput': {
'DataSource': {
'S3DataSource': {
'S3DataType': 'ManifestFile'|'S3Prefix'|'AugmentedManifestFile',
'S3Uri': 'string'
}
},
'ContentType': 'string',
'CompressionType': 'None'|'Gzip',
'SplitType': 'None'|'Line'|'RecordIO'
},
'TransformOutput': {
'S3OutputPath': 'string',
'Accept': 'string',
'AssembleWith': 'None'|'Line',
'KmsKeyId': 'string'
},
'TransformResources': {
'InstanceType': 'ml.m4.xlarge'|'ml.m4.2xlarge'|'ml.m4.4xlarge'|'ml.m4.10xlarge'|'ml.m4.16xlarge'|'ml.c4.xlarge'|'ml.c4.2xlarge'|'ml.c4.4xlarge'|'ml.c4.8xlarge'|'ml.p2.xlarge'|'ml.p2.8xlarge'|'ml.p2.16xlarge'|'ml.p3.2xlarge'|'ml.p3.8xlarge'|'ml.p3.16xlarge'|'ml.c5.xlarge'|'ml.c5.2xlarge'|'ml.c5.4xlarge'|'ml.c5.9xlarge'|'ml.c5.18xlarge'|'ml.m5.large'|'ml.m5.xlarge'|'ml.m5.2xlarge'|'ml.m5.4xlarge'|'ml.m5.12xlarge'|'ml.m5.24xlarge',
'InstanceCount': 123,
'VolumeKmsKeyId': 'string'
}
}
},
]
},
CertifyForMarketplace=True|False
)
[REQUIRED]
The name of the algorithm. The name must have 1 to 63 characters. Valid characters are a-z, A-Z, 0-9, and - (hyphen).
[REQUIRED]
Specifies details about training jobs run by this algorithm, including the following:
The Amazon Amazon ECR registry path of the Docker image that contains the training algorithm.
An MD5 hash of the training algorithm that identifies the Docker image used for training.
A list of the HyperParameterSpecification objects, that define the supported hyperparameters. This is required if the algorithm supports automatic model tuning.>
Defines a hyperparameter to be used by an algorithm.
The name of this hyperparameter. The name must be unique.
A brief description of the hyperparameter.
The type of this hyperparameter. The valid types are Integer , Continuous , Categorical , and FreeText .
The allowed range for this hyperparameter.
A IntegerParameterRangeSpecification object that defines the possible values for an integer hyperparameter.
The minimum integer value allowed.
The maximum integer value allowed.
A ContinuousParameterRangeSpecification object that defines the possible values for a continuous hyperparameter.
The minimum floating-point value allowed.
The maximum floating-point value allowed.
A CategoricalParameterRangeSpecification object that defines the possible values for a categorical hyperparameter.
The allowed categories for the hyperparameter.
Indicates whether this hyperparameter is tunable in a hyperparameter tuning job.
Indicates whether this hyperparameter is required.
The default value for this hyperparameter. If a default value is specified, a hyperparameter cannot be required.
A list of the instance types that this algorithm can use for training.
Indicates whether the algorithm supports distributed training. If set to false, buyers can’t request more than one instance during training.
A list of MetricDefinition objects, which are used for parsing metrics generated by the algorithm.
Specifies a metric that the training algorithm writes to stderr or stdout . Amazon SageMakerhyperparameter tuning captures all defined metrics. You specify one metric that a hyperparameter tuning job uses as its objective metric to choose the best training job.
The name of the metric.
A regular expression that searches the output of a training job and gets the value of the metric. For more information about using regular expressions to define metrics, see Defining Objective Metrics .
A list of ChannelSpecification objects, which specify the input sources to be used by the algorithm.
Defines a named input source, called a channel, to be used by an algorithm.
The name of the channel./sagemaker/eia
A brief description of the channel.
Indicates whether the channel is required by the algorithm.
The supported MIME types for the data.
The allowed compression types, if data compression is used.
The allowed input mode, either FILE or PIPE.
In FILE mode, Amazon SageMaker copies the data from the input source onto the local Amazon Elastic Block Store (Amazon EBS) volumes before starting your training algorithm. This is the most commonly used input mode.
In PIPE mode, Amazon SageMaker streams input data from the source directly to your algorithm without using the EBS volume.
A list of the metrics that the alogorithm emits that can be used as the objective metric in a hyperparameter tuning job.
Defines the objective metric for a hyperparameter tuning job. Hyperparameter tuning uses the value of this metric to evaluate the training jobs it launches, and returns the training job that results in either the highest or lowest value for this metric, depending on the value you specify for the Type parameter.
Whether to minimize or maximize the objective metric.
The name of the metric to use for the objective metric.
Specifies details about inference jobs that the algorithm runs, including the following:
The Amazon ECR registry path of the Docker image that contains the inference code.
Describes the Docker container for the model package.
The DNS host name for the Docker container.
The Amazon EC2 Container Registry path where inference code is stored.
If you are using your own custom algorithm instead of an algorithm provided by Amazon SageMaker, the inference code must meet Amazon SageMaker requirements. Amazon SageMaker supports both registry/repository[:tag] and registry/repository[@digest] image path formats. For more information, see Using Your Own Algorithms with Amazon SageMaker .
An MD5 hash of the training algorithm that identifies the Docker image used for training.
The Amazon S3 path where the model artifacts, which result from model training, are stored. This path must point to a single gzip compressed tar archive (.tar.gz suffix).
The ID of the model package.
A list of the instance types on which a transformation job can be run or on which an endpoint can be deployed.
A list of the instance types that are used to generate inferences in real-time.
The supported MIME types for the input data.
The supported MIME types for the output data.
Specifies configurations for one or more training jobs and that Amazon SageMaker runs to test the algorithm's training code and, optionally, one or more batch transform jobs that Amazon SageMaker runs to test the algorithm's inference code.
The IAM roles that Amazon SageMaker uses to run the training jobs.
An array of AlgorithmValidationProfile objects, each of which specifies a training job and batch transform job that Amazon SageMaker runs to validate your algorithm.
Defines a training job and a batch transform job that Amazon SageMaker runs to validate your algorithm.
The data provided in the validation profile is made available to your buyers on AWS Marketplace.
The name of the profile for the algorithm. The name must have 1 to 63 characters. Valid characters are a-z, A-Z, 0-9, and - (hyphen).
The TrainingJobDefinition object that describes the training job that Amazon SageMaker runs to validate your algorithm.
The input mode used by the algorithm for the training job. For the input modes that Amazon SageMaker algorithms support, see Algorithms .
If an algorithm supports the File input mode, Amazon SageMaker downloads the training data from S3 to the provisioned ML storage Volume, and mounts the directory to docker volume for training container. If an algorithm supports the Pipe input mode, Amazon SageMaker streams data directly from S3 to the container.
The hyperparameters used for the training job.
An array of Channel objects, each of which specifies an input source.
A channel is a named input source that training algorithms can consume.
The name of the channel.
The location of the channel data.
The S3 location of the data source that is associated with a channel.
If you choose S3Prefix , S3Uri identifies a key name prefix. Amazon SageMaker uses all objects that match the specified key name prefix for model training.
If you choose ManifestFile , S3Uri identifies an object that is a manifest file containing a list of object keys that you want Amazon SageMaker to use for model training.
If you choose AugmentedManifestFile , S3Uri identifies an object that is an augmented manifest file in JSON lines format. This file contains the data you want to use for model training. AugmentedManifestFile can only be used if the Channel's input mode is Pipe .
Depending on the value specified for the S3DataType , identifies either a key name prefix or a manifest. For example:
If you want Amazon SageMaker to replicate the entire dataset on each ML compute instance that is launched for model training, specify FullyReplicated .
If you want Amazon SageMaker to replicate a subset of data on each ML compute instance that is launched for model training, specify ShardedByS3Key . If there are n ML compute instances launched for a training job, each instance gets approximately 1/n of the number of S3 objects. In this case, model training on each machine uses only the subset of training data.
Don't choose more ML compute instances for training than available S3 objects. If you do, some nodes won't get any data and you will pay for nodes that aren't getting any training data. This applies in both File and Pipemodes. Keep this in mind when developing algorithms.
In distributed training, where you use multiple ML compute EC2 instances, you might choose ShardedByS3Key . If the algorithm requires copying training data to the ML storage volume (when TrainingInputMode is set to File ), this copies 1/n of the number of objects.
A list of one or more attribute names to use that are found in a specified augmented manifest file.
The MIME type of the data.
If training data is compressed, the compression type. The default value is None . CompressionType is used only in Pipe input mode. In File mode, leave this field unset or set it to None.
Specify RecordIO as the value when input data is in raw format but the training algorithm requires the RecordIO format. In this case, Amazon SageMaker wraps each individual S3 object in a RecordIO record. If the input data is already in RecordIO format, you don't need to set this attribute. For more information, see Create a Dataset Using RecordIO .
In File mode, leave this field unset or set it to None.
(Optional) The input mode to use for the data channel in a training job. If you don't set a value for InputMode , Amazon SageMaker uses the value set for TrainingInputMode . Use this parameter to override the TrainingInputMode setting in a AlgorithmSpecification request when you have a channel that needs a different input mode from the training job's general setting. To download the data from Amazon Simple Storage Service (Amazon S3) to the provisioned ML storage volume, and mount the directory to a Docker volume, use File input mode. To stream data directly from Amazon S3 to the container, choose Pipe input mode.
To use a model for incremental training, choose File input model.
A configuration for a shuffle option for input data in a channel. If you use S3Prefix for S3DataType , this shuffles the results of the S3 key prefix matches. If you use ManifestFile , the order of the S3 object references in the ManifestFile is shuffled. If you use AugmentedManifestFile , the order of the JSON lines in the AugmentedManifestFile is shuffled. The shuffling order is determined using the Seed value.
For Pipe input mode, shuffling is done at the start of every epoch. With large datasets this ensures that the order of the training data is different for each epoch, it helps reduce bias and possible overfitting. In a multi-node training job when ShuffleConfig is combined with S3DataDistributionType of ShardedByS3Key , the data is shuffled across nodes so that the content sent to a particular node on the first epoch might be sent to a different node on the second epoch.
Determines the shuffling order in ShuffleConfig . value.
the path to the S3 bucket where you want to store model artifacts. Amazon SageMaker creates subfolders for the artifacts.
The AWS Key Management Service (AWS KMS) key that Amazon SageMaker uses to encrypt the model artifacts at rest using Amazon S3 server-side encryption. The KmsKeyId can be any of the following formats:
If you don't provide the KMS key ID, Amazon SageMaker uses the default KMS key for Amazon S3 for your role's account. For more information, see KMS-Managed Encryption Keys in Amazon Simple Storage Service Developer Guide.
Note
The KMS key policy must grant permission to the IAM role that you specify in your CreateTrainingJob request. Using Key Policies in AWS KMS in the AWS Key Management Service Developer Guide .
Identifies the S3 path where you want Amazon SageMaker to store the model artifacts. For example, s3://bucket-name/key-name-prefix .
The resources, including the ML compute instances and ML storage volumes, to use for model training.
The ML compute instance type.
The number of ML compute instances to use. For distributed training, provide a value greater than 1.
The size of the ML storage volume that you want to provision.
ML storage volumes store model artifacts and incremental states. Training algorithms might also use the ML storage volume for scratch space. If you want to store the training data in the ML storage volume, choose File as the TrainingInputMode in the algorithm specification.
You must specify sufficient ML storage for your scenario.
Note
Amazon SageMaker supports only the General Purpose SSD (gp2) ML storage volume type.
The AWS Key Management Service (AWS KMS) key that Amazon SageMaker uses to encrypt data on the storage volume attached to the ML compute instance(s) that run the training job. The VolumeKmsKeyId can be any of the following formats:
Sets a duration for training. Use this parameter to cap model training costs.
To stop a job, Amazon SageMaker sends the algorithm the SIGTERM signal, which delays job termination for 120 seconds. Algorithms might use this 120-second window to save the model artifacts.
The maximum length of time, in seconds, that the training or compilation job can run. If the job does not complete during this time, Amazon SageMaker ends the job. If value is not specified, default value is 1 day. Maximum value is 5 days.
The TransformJobDefinition object that describes the transform job that Amazon SageMaker runs to validate your algorithm.
The maximum number of parallel requests that can be sent to each instance in a transform job. The default value is 1.
The maximum payload size allowed, in MB. A payload is the data portion of a record (without metadata).
A string that determines the number of records included in a single mini-batch.
SingleRecord means only one record is used per mini-batch. MultiRecord means a mini-batch is set to contain as many records that can fit within the MaxPayloadInMB limit.
The environment variables to set in the Docker container. We support up to 16 key and values entries in the map.
A description of the input source and the way the transform job consumes it.
Describes the location of the channel data, meaning the S3 location of the input data that the model can consume.
The S3 location of the data source that is associated with a channel.
If you choose S3Prefix , S3Uri identifies a key name prefix. Amazon SageMaker uses all objects with the specified key name prefix for batch transform.
If you choose ManifestFile , S3Uri identifies an object that is a manifest file containing a list of object keys that you want Amazon SageMaker to use for batch transform.
Depending on the value specified for the S3DataType , identifies either a key name prefix or a manifest. For example:
The multipurpose internet mail extension (MIME) type of the data. Amazon SageMaker uses the MIME type with each http call to transfer data to the transform job.
Compressing data helps save on storage space. If your transform data is compressed, specify the compression type. Amazon SageMaker automatically decompresses the data for the transform job accordingly. The default value is None .
The method to use to split the transform job's data into smaller batches. The default value is None . If you don't want to split the data, specify None . If you want to split records on a newline character boundary, specify Line . To split records according to the RecordIO format, specify RecordIO .
Amazon SageMaker will send maximum number of records per batch in each request up to the MaxPayloadInMB limit. For more information, see RecordIO data format .
Note
For information about the RecordIO format, see Data Format .
Identifies the Amazon S3 location where you want Amazon SageMaker to save the results from the transform job.
The Amazon S3 path where you want Amazon SageMaker to store the results of the transform job. For example, s3://bucket-name/key-name-prefix .
For every S3 object used as input for the transform job, the transformed data is stored in a corresponding subfolder in the location under the output prefix. For example, the input data s3://bucket-name/input-name-prefix/dataset01/data.csv will have the transformed data stored at s3://bucket-name/key-name-prefix/dataset01/ , based on the original name, as a series of .part files (.part0001, part0002, etc).
The MIME type used to specify the output data. Amazon SageMaker uses the MIME type with each http call to transfer data from the transform job.
Defines how to assemble the results of the transform job as a single S3 object. You should select a format that is most convenient to you. To concatenate the results in binary format, specify None . To add a newline character at the end of every transformed record, specify Line .
The AWS Key Management Service (AWS KMS) key that Amazon SageMaker uses to encrypt the model artifacts at rest using Amazon S3 server-side encryption. The KmsKeyId can be any of the following formats:
If you don't provide a KMS key ID, Amazon SageMaker uses the default KMS key for Amazon S3 for your role's account. For more information, see KMS-Managed Encryption Keys in the Amazon Simple Storage Service Developer Guide.
The KMS key policy must grant permission to the IAM role that you specify in your CreateTramsformJob request. For more information, see Using Key Policies in AWS KMS in the AWS Key Management Service Developer Guide .
Identifies the ML compute instances for the transform job.
The ML compute instance type for the transform job. For using built-in algorithms to transform moderately sized datasets, ml.m4.xlarge or ml.m5.large should suffice. There is no default value for InstanceType .
The number of ML compute instances to use in the transform job. For distributed transform, provide a value greater than 1. The default value is 1 .
The AWS Key Management Service (AWS KMS) key that Amazon SageMaker uses to encrypt data on the storage volume attached to the ML compute instance(s) that run the batch transform job. The VolumeKmsKeyId can be any of the following formats:
dict
Response Syntax
{
'AlgorithmArn': 'string'
}
Response Structure
(dict) --
AlgorithmArn (string) --
The Amazon Resource Name (ARN) of the new algorithm.
Create a git repository as a resource in your Amazon SageMaker account. You can associate the repository with notebook instances so that you can use git source control for the notebooks you create. The git repository is a resource in your Amazon SageMaker account, so it can be associated with more than one notebook instance, and it persists independently from the lifecycle of any notebook instances it is associated with.
The repository can be hosted either in AWS CodeCommit or in any other git repository.
See also: AWS API Documentation
Request Syntax
response = client.create_code_repository(
CodeRepositoryName='string',
GitConfig={
'RepositoryUrl': 'string',
'Branch': 'string',
'SecretArn': 'string'
}
)
[REQUIRED]
The name of the git repository. The name must have 1 to 63 characters. Valid characters are a-z, A-Z, 0-9, and - (hyphen).
[REQUIRED]
Specifies details about the repository, including the URL where the repository is located, the default branch, and credentials to use to access the repository.
The URL where the git repository is located.
The default brach for the git repository.
The Amazon Resource Name (ARN) of the AWS Secrets Manager secret that contains the credentials used to access the git repository. The secret must have a staging label of AWSCURRENT and must be in the following format:
{"username": *UserName* , "password": *Password* }
dict
Response Syntax
{
'CodeRepositoryArn': 'string'
}
Response Structure
(dict) --
CodeRepositoryArn (string) --
The Amazon Resource Name (ARN) of the new repository.
Starts a model compilation job. After the model has been compiled, Amazon SageMaker saves the resulting model artifacts to an Amazon Simple Storage Service (Amazon S3) bucket that you specify.
If you choose to host your model using Amazon SageMaker hosting services, you can use the resulting model artifacts as part of the model. You can also use the artifacts with AWS IoT Greengrass. In that case, deploy them as an ML resource.
In the request body, you provide the following:
You can also provide a Tag to track the model compilation job's resource use and costs. The response body contains the CompilationJobArn for the compiled job.
To stop a model compilation job, use StopCompilationJob . To get information about a particular model compilation job, use DescribeCompilationJob . To get information about multiple model compilation jobs, use ListCompilationJobs .
See also: AWS API Documentation
Request Syntax
response = client.create_compilation_job(
CompilationJobName='string',
RoleArn='string',
InputConfig={
'S3Uri': 'string',
'DataInputConfig': 'string',
'Framework': 'TENSORFLOW'|'MXNET'|'ONNX'|'PYTORCH'|'XGBOOST'
},
OutputConfig={
'S3OutputLocation': 'string',
'TargetDevice': 'ml_m4'|'ml_m5'|'ml_c4'|'ml_c5'|'ml_p2'|'ml_p3'|'jetson_tx1'|'jetson_tx2'|'rasp3b'|'deeplens'
},
StoppingCondition={
'MaxRuntimeInSeconds': 123
}
)
[REQUIRED]
A name for the model compilation job. The name must be unique within the AWS Region and within your AWS account.
[REQUIRED]
The Amazon Resource Name (ARN) of an IIAMAM role that enables Amazon SageMaker to perform tasks on your behalf.
During model compilation, Amazon SageMaker needs your permission to:
You grant permissions for all of these tasks to an IAM role. To pass this role to Amazon SageMaker, the caller of this API must have the iam:PassRole permission. For more information, see Amazon SageMaker Roles.
[REQUIRED]
Provides information about the location of input model artifacts, the name and shape of the expected data inputs, and the framework in which the model was trained.
The S3 path where the model artifacts, which result from model training, are stored. This path must point to a single gzip compressed tar archive (.tar.gz suffix).
Specifies the name and shape of the expected data inputs for your trained model with a JSON dictionary form. The data inputs are InputConfig$Framework specific.
Identifies the framework in which the model was trained. For example: TENSORFLOW.
[REQUIRED]
Provides information about the output location for the compiled model and the target device the model runs on.
Identifies the S3 path where you want Amazon SageMaker to store the model artifacts. For example, s3://bucket-name/key-name-prefix.
Identifies the device that you want to run your model on after it has been compiled. For example: ml_c5.
[REQUIRED]
The duration allowed for model compilation.
The maximum length of time, in seconds, that the training or compilation job can run. If the job does not complete during this time, Amazon SageMaker ends the job. If value is not specified, default value is 1 day. Maximum value is 5 days.
dict
Response Syntax
{
'CompilationJobArn': 'string'
}
Response Structure
(dict) --
CompilationJobArn (string) --
If the action is successful, the service sends back an HTTP 200 response. Amazon SageMaker returns the following data in JSON format:
Creates an endpoint using the endpoint configuration specified in the request. Amazon SageMaker uses the endpoint to provision resources and deploy models. You create the endpoint configuration with the CreateEndpointConfig API.
Note
Use this API only for hosting models using Amazon SageMaker hosting services.
The endpoint name must be unique within an AWS Region in your AWS account.
When it receives the request, Amazon SageMaker creates the endpoint, launches the resources (ML compute instances), and deploys the model(s) on them.
When Amazon SageMaker receives the request, it sets the endpoint status to Creating . After it creates the endpoint, it sets the status to InService . Amazon SageMaker can then process incoming requests for inferences. To check the status of an endpoint, use the DescribeEndpoint API.
For an example, see Exercise 1: Using the K-Means Algorithm Provided by Amazon SageMaker .
If any of the models hosted at this endpoint get model data from an Amazon S3 location, Amazon SageMaker uses AWS Security Token Service to download model artifacts from the S3 path you provided. AWS STS is activated in your IAM user account by default. If you previously deactivated AWS STS for a region, you need to reactivate AWS STS for that region. For more information, see Activating and Deactivating AWS STS i an AWS Region in the AWS Identity and Access Management User Guide .
See also: AWS API Documentation
Request Syntax
response = client.create_endpoint(
EndpointName='string',
EndpointConfigName='string',
Tags=[
{
'Key': 'string',
'Value': 'string'
},
]
)
[REQUIRED]
The name of the endpoint. The name must be unique within an AWS Region in your AWS account.
[REQUIRED]
The name of an endpoint configuration. For more information, see CreateEndpointConfig .
An array of key-value pairs. For more information, see Using Cost Allocation Tags in the AWS Billing and Cost Management User Guide .
Describes a tag.
The tag key.
The tag value.
dict
Response Syntax
{
'EndpointArn': 'string'
}
Response Structure
(dict) --
EndpointArn (string) --
The Amazon Resource Name (ARN) of the endpoint.
Creates an endpoint configuration that Amazon SageMaker hosting services uses to deploy models. In the configuration, you identify one or more models, created using the CreateModel API, to deploy and the resources that you want Amazon SageMaker to provision. Then you call the CreateEndpoint API.
Note
Use this API only if you want to use Amazon SageMaker hosting services to deploy models into production.
In the request, you define one or more ProductionVariant s, each of which identifies a model. Each ProductionVariant parameter also describes the resources that you want Amazon SageMaker to provision. This includes the number and type of ML compute instances to deploy.
If you are hosting multiple models, you also assign a VariantWeight to specify how much traffic you want to allocate to each model. For example, suppose that you want to host two models, A and B, and you assign traffic weight 2 for model A and 1 for model B. Amazon SageMaker distributes two-thirds of the traffic to Model A, and one-third to model B.
See also: AWS API Documentation
Request Syntax
response = client.create_endpoint_config(
EndpointConfigName='string',
ProductionVariants=[
{
'VariantName': 'string',
'ModelName': 'string',
'InitialInstanceCount': 123,
'InstanceType': 'ml.t2.medium'|'ml.t2.large'|'ml.t2.xlarge'|'ml.t2.2xlarge'|'ml.m4.xlarge'|'ml.m4.2xlarge'|'ml.m4.4xlarge'|'ml.m4.10xlarge'|'ml.m4.16xlarge'|'ml.m5.large'|'ml.m5.xlarge'|'ml.m5.2xlarge'|'ml.m5.4xlarge'|'ml.m5.12xlarge'|'ml.m5.24xlarge'|'ml.c4.large'|'ml.c4.xlarge'|'ml.c4.2xlarge'|'ml.c4.4xlarge'|'ml.c4.8xlarge'|'ml.p2.xlarge'|'ml.p2.8xlarge'|'ml.p2.16xlarge'|'ml.p3.2xlarge'|'ml.p3.8xlarge'|'ml.p3.16xlarge'|'ml.c5.large'|'ml.c5.xlarge'|'ml.c5.2xlarge'|'ml.c5.4xlarge'|'ml.c5.9xlarge'|'ml.c5.18xlarge',
'InitialVariantWeight': ...,
'AcceleratorType': 'ml.eia1.medium'|'ml.eia1.large'|'ml.eia1.xlarge'
},
],
Tags=[
{
'Key': 'string',
'Value': 'string'
},
],
KmsKeyId='string'
)
[REQUIRED]
The name of the endpoint configuration. You specify this name in a CreateEndpoint request.
[REQUIRED]
An array of ProductionVariant objects, one for each model that you want to host at this endpoint.
Identifies a model that you want to host and the resources to deploy for hosting it. If you are deploying multiple models, tell Amazon SageMaker how to distribute traffic among the models by specifying variant weights.
The name of the production variant.
The name of the model that you want to host. This is the name that you specified when creating the model.
Number of instances to launch initially.
The ML compute instance type.
Determines initial traffic distribution among all of the models that you specify in the endpoint configuration. The traffic to a production variant is determined by the ratio of the VariantWeight to the sum of all VariantWeight values across all ProductionVariants. If unspecified, it defaults to 1.0.
The size of the Elastic Inference (EI) instance to use for the production variant. EI instances provide on-demand GPU computing for inference. For more information, see Using Elastic Inference in Amazon SageMaker . For more information, see Using Elastic Inference in Amazon SageMaker .
An array of key-value pairs. For more information, see Using Cost Allocation Tags in the AWS Billing and Cost Management User Guide .
Describes a tag.
The tag key.
The tag value.
dict
Response Syntax
{
'EndpointConfigArn': 'string'
}
Response Structure
(dict) --
EndpointConfigArn (string) --
The Amazon Resource Name (ARN) of the endpoint configuration.
Starts a hyperparameter tuning job. A hyperparameter tuning job finds the best version of a model by running many training jobs on your dataset using the algorithm you choose and values for hyperparameters within ranges that you specify. It then chooses the hyperparameter values that result in a model that performs the best, as measured by an objective metric that you choose.
See also: AWS API Documentation
Request Syntax
response = client.create_hyper_parameter_tuning_job(
HyperParameterTuningJobName='string',
HyperParameterTuningJobConfig={
'Strategy': 'Bayesian',
'HyperParameterTuningJobObjective': {
'Type': 'Maximize'|'Minimize',
'MetricName': 'string'
},
'ResourceLimits': {
'MaxNumberOfTrainingJobs': 123,
'MaxParallelTrainingJobs': 123
},
'ParameterRanges': {
'IntegerParameterRanges': [
{
'Name': 'string',
'MinValue': 'string',
'MaxValue': 'string'
},
],
'ContinuousParameterRanges': [
{
'Name': 'string',
'MinValue': 'string',
'MaxValue': 'string'
},
],
'CategoricalParameterRanges': [
{
'Name': 'string',
'Values': [
'string',
]
},
]
}
},
TrainingJobDefinition={
'StaticHyperParameters': {
'string': 'string'
},
'AlgorithmSpecification': {
'TrainingImage': 'string',
'TrainingInputMode': 'Pipe'|'File',
'AlgorithmName': 'string',
'MetricDefinitions': [
{
'Name': 'string',
'Regex': 'string'
},
]
},
'RoleArn': 'string',
'InputDataConfig': [
{
'ChannelName': 'string',
'DataSource': {
'S3DataSource': {
'S3DataType': 'ManifestFile'|'S3Prefix'|'AugmentedManifestFile',
'S3Uri': 'string',
'S3DataDistributionType': 'FullyReplicated'|'ShardedByS3Key',
'AttributeNames': [
'string',
]
}
},
'ContentType': 'string',
'CompressionType': 'None'|'Gzip',
'RecordWrapperType': 'None'|'RecordIO',
'InputMode': 'Pipe'|'File',
'ShuffleConfig': {
'Seed': 123
}
},
],
'VpcConfig': {
'SecurityGroupIds': [
'string',
],
'Subnets': [
'string',
]
},
'OutputDataConfig': {
'KmsKeyId': 'string',
'S3OutputPath': 'string'
},
'ResourceConfig': {
'InstanceType': 'ml.m4.xlarge'|'ml.m4.2xlarge'|'ml.m4.4xlarge'|'ml.m4.10xlarge'|'ml.m4.16xlarge'|'ml.m5.large'|'ml.m5.xlarge'|'ml.m5.2xlarge'|'ml.m5.4xlarge'|'ml.m5.12xlarge'|'ml.m5.24xlarge'|'ml.c4.xlarge'|'ml.c4.2xlarge'|'ml.c4.4xlarge'|'ml.c4.8xlarge'|'ml.p2.xlarge'|'ml.p2.8xlarge'|'ml.p2.16xlarge'|'ml.p3.2xlarge'|'ml.p3.8xlarge'|'ml.p3.16xlarge'|'ml.c5.xlarge'|'ml.c5.2xlarge'|'ml.c5.4xlarge'|'ml.c5.9xlarge'|'ml.c5.18xlarge',
'InstanceCount': 123,
'VolumeSizeInGB': 123,
'VolumeKmsKeyId': 'string'
},
'StoppingCondition': {
'MaxRuntimeInSeconds': 123
},
'EnableNetworkIsolation': True|False
},
WarmStartConfig={
'ParentHyperParameterTuningJobs': [
{
'HyperParameterTuningJobName': 'string'
},
],
'WarmStartType': 'IdenticalDataAndAlgorithm'|'TransferLearning'
},
Tags=[
{
'Key': 'string',
'Value': 'string'
},
]
)
[REQUIRED]
The name of the tuning job. This name is the prefix for the names of all training jobs that this tuning job launches. The name must be unique within the same AWS account and AWS Region. The name must have { } to { } characters. Valid characters are a-z, A-Z, 0-9, and : + = @ _ % - (hyphen). The name is not case sensitive.
[REQUIRED]
The HyperParameterTuningJobConfig object that describes the tuning job, including the search strategy, the objective metric used to evaluate training jobs, ranges of parameters to search, and resource limits for the tuning job. For more information, see automatic-model-tuning
Specifies the search strategy for hyperparameters. Currently, the only valid value is Bayesian .
The HyperParameterTuningJobObjective object that specifies the objective metric for this tuning job.
Whether to minimize or maximize the objective metric.
The name of the metric to use for the objective metric.
The ResourceLimits object that specifies the maximum number of training jobs and parallel training jobs for this tuning job.
The maximum number of training jobs that a hyperparameter tuning job can launch.
The maximum number of concurrent training jobs that a hyperparameter tuning job can launch.
The ParameterRanges object that specifies the ranges of hyperparameters that this tuning job searches.
The array of IntegerParameterRange objects that specify ranges of integer hyperparameters that a hyperparameter tuning job searches.
For a hyperparameter of the integer type, specifies the range that a hyperparameter tuning job searches.
The name of the hyperparameter to search.
The minimum value of the hyperparameter to search.
The maximum value of the hyperparameter to search.
The array of ContinuousParameterRange objects that specify ranges of continuous hyperparameters that a hyperparameter tuning job searches.
A list of continuous hyperparameters to tune.
The name of the continuous hyperparameter to tune.
The minimum value for the hyperparameter. The tuning job uses floating-point values between this value and MaxValue for tuning.
The maximum value for the hyperparameter. The tuning job uses floating-point values between MinValue value and this value for tuning.
The array of CategoricalParameterRange objects that specify ranges of categorical hyperparameters that a hyperparameter tuning job searches.
A list of categorical hyperparameters to tune.
The name of the categorical hyperparameter to tune.
A list of the categories for the hyperparameter.
[REQUIRED]
The HyperParameterTrainingJobDefinition object that describes the training jobs that this tuning job launches, including static hyperparameters, input data configuration, output data configuration, resource configuration, and stopping condition.
Specifies the values of hyperparameters that do not change for the tuning job.
The HyperParameterAlgorithmSpecification object that specifies the algorithm to use for the training jobs that the tuning job launches.
The registry path of the Docker image that contains the training algorithm. For information about Docker registry paths for built-in algorithms, see Algorithms Provided by Amazon SageMaker: Common Parameters .
The input mode that the algorithm supports: File or Pipe. In File input mode, Amazon SageMaker downloads the training data from Amazon S3 to the storage volume that is attached to the training instance and mounts the directory to the Docker volume for the training container. In Pipe input mode, Amazon SageMaker streams data directly from Amazon S3 to the container.
If you specify File mode, make sure that you provision the storage volume that is attached to the training instance with enough capacity to accommodate the training data downloaded from Amazon S3, the model artifacts, and intermediate information.
For more information about input modes, see Algorithms .
The name of the resource algorithm to use for the hyperparameter tuning job. If you specify a value for this parameter, do not specify a value for TrainingImage .
An array of MetricDefinition objects that specify the metrics that the algorithm emits.
Specifies a metric that the training algorithm writes to stderr or stdout . Amazon SageMakerhyperparameter tuning captures all defined metrics. You specify one metric that a hyperparameter tuning job uses as its objective metric to choose the best training job.
The name of the metric.
A regular expression that searches the output of a training job and gets the value of the metric. For more information about using regular expressions to define metrics, see Defining Objective Metrics .
The Amazon Resource Name (ARN) of the IAM role associated with the training jobs that the tuning job launches.
An array of Channel objects that specify the input for the training jobs that the tuning job launches.
A channel is a named input source that training algorithms can consume.
The name of the channel.
The location of the channel data.
The S3 location of the data source that is associated with a channel.
If you choose S3Prefix , S3Uri identifies a key name prefix. Amazon SageMaker uses all objects that match the specified key name prefix for model training.
If you choose ManifestFile , S3Uri identifies an object that is a manifest file containing a list of object keys that you want Amazon SageMaker to use for model training.
If you choose AugmentedManifestFile , S3Uri identifies an object that is an augmented manifest file in JSON lines format. This file contains the data you want to use for model training. AugmentedManifestFile can only be used if the Channel's input mode is Pipe .
Depending on the value specified for the S3DataType , identifies either a key name prefix or a manifest. For example:
If you want Amazon SageMaker to replicate the entire dataset on each ML compute instance that is launched for model training, specify FullyReplicated .
If you want Amazon SageMaker to replicate a subset of data on each ML compute instance that is launched for model training, specify ShardedByS3Key . If there are n ML compute instances launched for a training job, each instance gets approximately 1/n of the number of S3 objects. In this case, model training on each machine uses only the subset of training data.
Don't choose more ML compute instances for training than available S3 objects. If you do, some nodes won't get any data and you will pay for nodes that aren't getting any training data. This applies in both File and Pipemodes. Keep this in mind when developing algorithms.
In distributed training, where you use multiple ML compute EC2 instances, you might choose ShardedByS3Key . If the algorithm requires copying training data to the ML storage volume (when TrainingInputMode is set to File ), this copies 1/n of the number of objects.
A list of one or more attribute names to use that are found in a specified augmented manifest file.
The MIME type of the data.
If training data is compressed, the compression type. The default value is None . CompressionType is used only in Pipe input mode. In File mode, leave this field unset or set it to None.
Specify RecordIO as the value when input data is in raw format but the training algorithm requires the RecordIO format. In this case, Amazon SageMaker wraps each individual S3 object in a RecordIO record. If the input data is already in RecordIO format, you don't need to set this attribute. For more information, see Create a Dataset Using RecordIO .
In File mode, leave this field unset or set it to None.
(Optional) The input mode to use for the data channel in a training job. If you don't set a value for InputMode , Amazon SageMaker uses the value set for TrainingInputMode . Use this parameter to override the TrainingInputMode setting in a AlgorithmSpecification request when you have a channel that needs a different input mode from the training job's general setting. To download the data from Amazon Simple Storage Service (Amazon S3) to the provisioned ML storage volume, and mount the directory to a Docker volume, use File input mode. To stream data directly from Amazon S3 to the container, choose Pipe input mode.
To use a model for incremental training, choose File input model.
A configuration for a shuffle option for input data in a channel. If you use S3Prefix for S3DataType , this shuffles the results of the S3 key prefix matches. If you use ManifestFile , the order of the S3 object references in the ManifestFile is shuffled. If you use AugmentedManifestFile , the order of the JSON lines in the AugmentedManifestFile is shuffled. The shuffling order is determined using the Seed value.
For Pipe input mode, shuffling is done at the start of every epoch. With large datasets this ensures that the order of the training data is different for each epoch, it helps reduce bias and possible overfitting. In a multi-node training job when ShuffleConfig is combined with S3DataDistributionType of ShardedByS3Key , the data is shuffled across nodes so that the content sent to a particular node on the first epoch might be sent to a different node on the second epoch.
Determines the shuffling order in ShuffleConfig . value.
The VpcConfig object that specifies the VPC that you want the training jobs that this hyperparameter tuning job launches to connect to. Control access to and from your training container by configuring the VPC. For more information, see Protect Training Jobs by Using an Amazon Virtual Private Cloud .
The VPC security group IDs, in the form sg-xxxxxxxx. Specify the security groups for the VPC that is specified in the Subnets field.
The ID of the subnets in the VPC to which you want to connect your training job or model.
Specifies the path to the Amazon S3 bucket where you store model artifacts from the training jobs that the tuning job launches.
The AWS Key Management Service (AWS KMS) key that Amazon SageMaker uses to encrypt the model artifacts at rest using Amazon S3 server-side encryption. The KmsKeyId can be any of the following formats:
If you don't provide the KMS key ID, Amazon SageMaker uses the default KMS key for Amazon S3 for your role's account. For more information, see KMS-Managed Encryption Keys in Amazon Simple Storage Service Developer Guide.
Note
The KMS key policy must grant permission to the IAM role that you specify in your CreateTrainingJob request. Using Key Policies in AWS KMS in the AWS Key Management Service Developer Guide .
Identifies the S3 path where you want Amazon SageMaker to store the model artifacts. For example, s3://bucket-name/key-name-prefix .
The resources, including the compute instances and storage volumes, to use for the training jobs that the tuning job launches.
Storage volumes store model artifacts and incremental states. Training algorithms might also use storage volumes for scratch space. If you want Amazon SageMaker to use the storage volume to store the training data, choose File as the TrainingInputMode in the algorithm specification. For distributed training algorithms, specify an instance count greater than 1.
The ML compute instance type.
The number of ML compute instances to use. For distributed training, provide a value greater than 1.
The size of the ML storage volume that you want to provision.
ML storage volumes store model artifacts and incremental states. Training algorithms might also use the ML storage volume for scratch space. If you want to store the training data in the ML storage volume, choose File as the TrainingInputMode in the algorithm specification.
You must specify sufficient ML storage for your scenario.
Note
Amazon SageMaker supports only the General Purpose SSD (gp2) ML storage volume type.
The AWS Key Management Service (AWS KMS) key that Amazon SageMaker uses to encrypt data on the storage volume attached to the ML compute instance(s) that run the training job. The VolumeKmsKeyId can be any of the following formats:
Sets a maximum duration for the training jobs that the tuning job launches. Use this parameter to limit model training costs.
To stop a job, Amazon SageMaker sends the algorithm the SIGTERM signal. This delays job termination for 120 seconds. Algorithms might use this 120-second window to save the model artifacts.
When Amazon SageMaker terminates a job because the stopping condition has been met, training algorithms provided by Amazon SageMaker save the intermediate results of the job.
The maximum length of time, in seconds, that the training or compilation job can run. If the job does not complete during this time, Amazon SageMaker ends the job. If value is not specified, default value is 1 day. Maximum value is 5 days.
Isolates the training container. No inbound or outbound network calls can be made, except for calls between peers within a training cluster for distributed training. If network isolation is used for training jobs that are configured to use a VPC, Amazon SageMaker downloads and uploads customer data and model artifacts through the specifed VPC, but the training container does not have network access.
Note
The Semantic Segmentation built-in algorithm does not support network isolation.
Specifies configuration for starting the hyperparameter tuning job using one or more previous tuning jobs as a starting point. The results of previous tuning jobs are used to inform which combinations of hyperparameters to search over in the new tuning job.
All training jobs launched by the new hyperparameter tuning job are evaluated by using the objective metric. If you specify IDENTICAL_DATA_AND_ALGORITHM as the WarmStartType for the warm start configuration, the training job that performs the best in the new tuning job is compared to the best training jobs from the parent tuning jobs. From these, the training job that performs the best as measured by the objective metric is returned as the overall best training job.
Note
All training jobs launched by parent hyperparameter tuning jobs and the new hyperparameter tuning jobs count against the limit of training jobs for the tuning job.
An array of hyperparameter tuning jobs that are used as the starting point for the new hyperparameter tuning job. For more information about warm starting a hyperparameter tuning job, see Using a Previous Hyperparameter Tuning Job as a Starting Point .
Hyperparameter tuning jobs created before October 1, 2018 cannot be used as parent jobs for warm start tuning jobs.
A previously completed or stopped hyperparameter tuning job to be used as a starting point for a new hyperparameter tuning job.
The name of the hyperparameter tuning job to be used as a starting point for a new hyperparameter tuning job.
Specifies one of the following:
IDENTICAL_DATA_AND_ALGORITHM
The new hyperparameter tuning job uses the same input data and training image as the parent tuning jobs. You can change the hyperparameter ranges to search and the maximum number of training jobs that the hyperparameter tuning job launches. You cannot use a new version of the training algorithm, unless the changes in the new version do not affect the algorithm itself. For example, changes that improve logging or adding support for a different data format are allowed. You can also change hyperparameters from tunable to static, and from static to tunable, but the total number of static plus tunable hyperparameters must remain the same as it is in all parent jobs. The objective metric for the new tuning job must be the same as for all parent jobs.
TRANSFER_LEARNING
The new hyperparameter tuning job can include input data, hyperparameter ranges, maximum number of concurrent training jobs, and maximum number of training jobs that are different than those of its parent hyperparameter tuning jobs. The training image can also be a different version from the version used in the parent hyperparameter tuning job. You can also change hyperparameters from tunable to static, and from static to tunable, but the total number of static plus tunable hyperparameters must remain the same as it is in all parent jobs. The objective metric for the new tuning job must be the same as for all parent jobs.
An array of key-value pairs. You can use tags to categorize your AWS resources in different ways, for example, by purpose, owner, or environment. For more information, see AWS Tagging Strategies .
Tags that you specify for the tuning job are also added to all training jobs that the tuning job launches.
Describes a tag.
The tag key.
The tag value.
dict
Response Syntax
{
'HyperParameterTuningJobArn': 'string'
}
Response Structure
(dict) --
HyperParameterTuningJobArn (string) --
The Amazon Resource Name (ARN) of the tuning job. Amazon SageMaker assigns an ARN to a hyperparameter tuning job when you create it.
Creates a job that uses human workers to label the data objects in your input dataset. You can use the labeled data to train machine learning models
You can select your workforce from one of three providers:
You can also use automated data labeling to reduce the number of data objects that need to be labeled by a human. Automated data labeling uses active learning to determine if a data object can be labeled by machine or if it needs to be sent to a human worker.
The data objects to be labeled are contained in an Amazon S3 bucket. You create a manifest file that describes the location of each object. For more information, see Using Input and Output Data .
The output can be used as the manifest file for another labeling job or as training data for your machine learning models.
See also: AWS API Documentation
Request Syntax
response = client.create_labeling_job(
LabelingJobName='string',
LabelAttributeName='string',
InputConfig={
'DataSource': {
'S3DataSource': {
'ManifestS3Uri': 'string'
}
},
'DataAttributes': {
'ContentClassifiers': [
'FreeOfPersonallyIdentifiableInformation'|'FreeOfAdultContent',
]
}
},
OutputConfig={
'S3OutputPath': 'string',
'KmsKeyId': 'string'
},
RoleArn='string',
LabelCategoryConfigS3Uri='string',
StoppingConditions={
'MaxHumanLabeledObjectCount': 123,
'MaxPercentageOfInputDatasetLabeled': 123
},
LabelingJobAlgorithmsConfig={
'LabelingJobAlgorithmSpecificationArn': 'string',
'InitialActiveLearningModelArn': 'string',
'LabelingJobResourceConfig': {
'VolumeKmsKeyId': 'string'
}
},
HumanTaskConfig={
'WorkteamArn': 'string',
'UiConfig': {
'UiTemplateS3Uri': 'string'
},
'PreHumanTaskLambdaArn': 'string',
'TaskKeywords': [
'string',
],
'TaskTitle': 'string',
'TaskDescription': 'string',
'NumberOfHumanWorkersPerDataObject': 123,
'TaskTimeLimitInSeconds': 123,
'TaskAvailabilityLifetimeInSeconds': 123,
'MaxConcurrentTaskCount': 123,
'AnnotationConsolidationConfig': {
'AnnotationConsolidationLambdaArn': 'string'
},
'PublicWorkforceTaskPrice': {
'AmountInUsd': {
'Dollars': 123,
'Cents': 123,
'TenthFractionsOfACent': 123
}
}
},
Tags=[
{
'Key': 'string',
'Value': 'string'
},
]
)
[REQUIRED]
The name of the labeling job. This name is used to identify the job in a list of labeling jobs.
[REQUIRED]
The attribute name to use for the label in the output manifest file. This is the key for the key/value pair formed with the label that a worker assigns to the object. The name can't end with "-metadata" or "-ref".
[REQUIRED]
Input data for the labeling job, such as the Amazon S3 location of the data objects and the location of the manifest file that describes the data objects.
The location of the input data.
The Amazon S3 location of the input data objects.
The Amazon S3 location of the manifest file that describes the input data objects.
Attributes of the data specified by the customer.
Declares that your content is free of personally identifiable information or adult content. Amazon SageMaker may restrict the Amazon Mechanical Turk workers that can view your task based on this information.
[REQUIRED]
The location of the output data and the AWS Key Management Service key ID for the key used to encrypt the output data, if any.
The Amazon S3 location to write output data.
The AWS Key Management Service ID of the key used to encrypt the output data, if any.
[REQUIRED]
The Amazon Resource Number (ARN) that Amazon SageMaker assumes to perform tasks on your behalf during data labeling. You must grant this role the necessary permissions so that Amazon SageMaker can successfully complete data labeling.
A set of conditions for stopping the labeling job. If any of the conditions are met, the job is automatically stopped. You can use these conditions to control the cost of data labeling.
The maximum number of objects that can be labeled by human workers.
The maximum number of input data objects that should be labeled.
Configures the information required to perform automated data labeling.
Specifies the Amazon Resource Name (ARN) of the algorithm used for auto-labeling. You must select one of the following ARNs:
At the end of an auto-label job Amazon SageMaker Ground Truth sends the Amazon Resource Nam (ARN) of the final model used for auto-labeling. You can use this model as the starting point for subsequent similar jobs by providing the ARN of the model here.
Provides configuration information for a labeling job.
The AWS Key Management Service key ID for the key used to encrypt the output data, if any.
[REQUIRED]
Configures the information required for human workers to complete a labeling task.
The Amazon Resource Name (ARN) of the work team assigned to complete the tasks.
Information about the user interface that workers use to complete the labeling task.
The Amazon S3 bucket location of the UI template.
The Amazon Resource Name (ARN) of a Lambda function that is run before a data object is sent to a human worker. Use this function to provide input to a custom labeling job.
Keywords used to describe the task so that workers on Amazon Mechanical Turk can discover the task.
A title for the task for your human workers.
A description of the task for your human workers.
The number of human workers that will label an object.
The amount of time that a worker has to complete a task.
The length of time that a task remains available for labelling by human workers.
Defines the maximum number of data objects that can be labeled by human workers at the same time. Each object may have more than one worker at one time.
Configures how labels are consolidated across human workers.
The Amazon Resource Name (ARN) of a Lambda function implements the logic for annotation consolidation.
Amazon SageMaker Ground Truth provides three annotation consolidation functions that you can choose to use. They are:
For more information, see Annotation Consolidation .
The price that you pay for each task performed by a public worker.
Defines the amount of money paid to a worker in United States dollars.
The whole number of dollars in the amount.
The fractional portion, in cents, of the amount.
Fractions of a cent, in tenths.
An array of key/value pairs. For more information, see Using Cost Allocation Tags in the AWS Billing and Cost Management User Guide .
Describes a tag.
The tag key.
The tag value.
dict
Response Syntax
{
'LabelingJobArn': 'string'
}
Response Structure
(dict) --
LabelingJobArn (string) --
The Amazon Resource Name (ARN) of the labeling job. You use this ARN to identify the labeling job.
Creates a model in Amazon SageMaker. In the request, you name the model and describe a primary container. For the primary container, you specify the docker image containing inference code, artifacts (from prior training), and custom environment map that the inference code uses when you deploy the model for predictions.
Use this API to create a model if you want to use Amazon SageMaker hosting services or run a batch transform job.
To host your model, you create an endpoint configuration with the CreateEndpointConfig API, and then create an endpoint with the CreateEndpoint API. Amazon SageMaker then deploys all of the containers that you defined for the model in the hosting environment.
To run a batch transform using your model, you start a job with the CreateTransformJob API. Amazon SageMaker uses your model and your dataset to get inferences which are then saved to a specified S3 location.
In the CreateModel request, you must define a container with the PrimaryContainer parameter.
In the request, you also provide an IAM role that Amazon SageMaker can assume to access model artifacts and docker image for deployment on ML compute hosting instances or for batch transform jobs. In addition, you also use the IAM role to manage permissions the inference code needs. For example, if the inference code access any other AWS resources, you grant necessary permissions via this role.
See also: AWS API Documentation
Request Syntax
response = client.create_model(
ModelName='string',
PrimaryContainer={
'ContainerHostname': 'string',
'Image': 'string',
'ModelDataUrl': 'string',
'Environment': {
'string': 'string'
},
'ModelPackageName': 'string'
},
Containers=[
{
'ContainerHostname': 'string',
'Image': 'string',
'ModelDataUrl': 'string',
'Environment': {
'string': 'string'
},
'ModelPackageName': 'string'
},
],
ExecutionRoleArn='string',
Tags=[
{
'Key': 'string',
'Value': 'string'
},
],
VpcConfig={
'SecurityGroupIds': [
'string',
],
'Subnets': [
'string',
]
},
EnableNetworkIsolation=True|False
)
[REQUIRED]
The name of the new model.
The location of the primary docker image containing inference code, associated artifacts, and custom environment map that the inference code uses when the model is deployed for predictions.
The DNS host name for the container after Amazon SageMaker deploys it.
The Amazon EC2 Container Registry (Amazon ECR) path where inference code is stored. If you are using your own custom algorithm instead of an algorithm provided by Amazon SageMaker, the inference code must meet Amazon SageMaker requirements. Amazon SageMaker supports both registry/repository[:tag] and registry/repository[@digest] image path formats. For more information, see Using Your Own Algorithms with Amazon SageMaker
The S3 path where the model artifacts, which result from model training, are stored. This path must point to a single gzip compressed tar archive (.tar.gz suffix).
If you provide a value for this parameter, Amazon SageMaker uses AWS Security Token Service to download model artifacts from the S3 path you provide. AWS STS is activated in your IAM user account by default. If you previously deactivated AWS STS for a region, you need to reactivate AWS STS for that region. For more information, see Activating and Deactivating AWS STS i an AWS Region in the AWS Identity and Access Management User Guide .
The environment variables to set in the Docker container. Each key and value in the Environment string to string map can have length of up to 1024. We support up to 16 entries in the map.
The name of the model package in this container.
Specifies the containers in the inference pipeline.
Describes the container, as part of model definition.
The DNS host name for the container after Amazon SageMaker deploys it.
The Amazon EC2 Container Registry (Amazon ECR) path where inference code is stored. If you are using your own custom algorithm instead of an algorithm provided by Amazon SageMaker, the inference code must meet Amazon SageMaker requirements. Amazon SageMaker supports both registry/repository[:tag] and registry/repository[@digest] image path formats. For more information, see Using Your Own Algorithms with Amazon SageMaker
The S3 path where the model artifacts, which result from model training, are stored. This path must point to a single gzip compressed tar archive (.tar.gz suffix).
If you provide a value for this parameter, Amazon SageMaker uses AWS Security Token Service to download model artifacts from the S3 path you provide. AWS STS is activated in your IAM user account by default. If you previously deactivated AWS STS for a region, you need to reactivate AWS STS for that region. For more information, see Activating and Deactivating AWS STS i an AWS Region in the AWS Identity and Access Management User Guide .
The environment variables to set in the Docker container. Each key and value in the Environment string to string map can have length of up to 1024. We support up to 16 entries in the map.
The name of the model package in this container.
[REQUIRED]
The Amazon Resource Name (ARN) of the IAM role that Amazon SageMaker can assume to access model artifacts and docker image for deployment on ML compute instances or for batch transform jobs. Deploying on ML compute instances is part of model hosting. For more information, see Amazon SageMaker Roles .
Note
To be able to pass this role to Amazon SageMaker, the caller of this API must have the iam:PassRole permission.
An array of key-value pairs. For more information, see Using Cost Allocation Tags in the AWS Billing and Cost Management User Guide .
Describes a tag.
The tag key.
The tag value.
A VpcConfig object that specifies the VPC that you want your model to connect to. Control access to and from your model container by configuring the VPC. VpcConfig is used in hosting services and in batch transform. For more information, see Protect Endpoints by Using an Amazon Virtual Private Cloud and Protect Data in Batch Transform Jobs by Using an Amazon Virtual Private Cloud .
The VPC security group IDs, in the form sg-xxxxxxxx. Specify the security groups for the VPC that is specified in the Subnets field.
The ID of the subnets in the VPC to which you want to connect your training job or model.
Isolates the model container. No inbound or outbound network calls can be made to or from the model container.
Note
The Semantic Segmentation built-in algorithm does not support network isolation.
dict
Response Syntax
{
'ModelArn': 'string'
}
Response Structure
(dict) --
ModelArn (string) --
The ARN of the model created in Amazon SageMaker.
Creates a model package that you can use to create Amazon SageMaker models or list on AWS Marketplace. Buyers can subscribe to model packages listed on AWS Marketplace to create models in Amazon SageMaker.
See also: AWS API Documentation
Request Syntax
response = client.create_model_package(
ModelPackageName='string',
ModelPackageDescription='string',
InferenceSpecification={
'Containers': [
{
'ContainerHostname': 'string',
'Image': 'string',
'ImageDigest': 'string',
'ModelDataUrl': 'string',
'ProductId': 'string'
},
],
'SupportedTransformInstanceTypes': [
'ml.m4.xlarge'|'ml.m4.2xlarge'|'ml.m4.4xlarge'|'ml.m4.10xlarge'|'ml.m4.16xlarge'|'ml.c4.xlarge'|'ml.c4.2xlarge'|'ml.c4.4xlarge'|'ml.c4.8xlarge'|'ml.p2.xlarge'|'ml.p2.8xlarge'|'ml.p2.16xlarge'|'ml.p3.2xlarge'|'ml.p3.8xlarge'|'ml.p3.16xlarge'|'ml.c5.xlarge'|'ml.c5.2xlarge'|'ml.c5.4xlarge'|'ml.c5.9xlarge'|'ml.c5.18xlarge'|'ml.m5.large'|'ml.m5.xlarge'|'ml.m5.2xlarge'|'ml.m5.4xlarge'|'ml.m5.12xlarge'|'ml.m5.24xlarge',
],
'SupportedRealtimeInferenceInstanceTypes': [
'ml.t2.medium'|'ml.t2.large'|'ml.t2.xlarge'|'ml.t2.2xlarge'|'ml.m4.xlarge'|'ml.m4.2xlarge'|'ml.m4.4xlarge'|'ml.m4.10xlarge'|'ml.m4.16xlarge'|'ml.m5.large'|'ml.m5.xlarge'|'ml.m5.2xlarge'|'ml.m5.4xlarge'|'ml.m5.12xlarge'|'ml.m5.24xlarge'|'ml.c4.large'|'ml.c4.xlarge'|'ml.c4.2xlarge'|'ml.c4.4xlarge'|'ml.c4.8xlarge'|'ml.p2.xlarge'|'ml.p2.8xlarge'|'ml.p2.16xlarge'|'ml.p3.2xlarge'|'ml.p3.8xlarge'|'ml.p3.16xlarge'|'ml.c5.large'|'ml.c5.xlarge'|'ml.c5.2xlarge'|'ml.c5.4xlarge'|'ml.c5.9xlarge'|'ml.c5.18xlarge',
],
'SupportedContentTypes': [
'string',
],
'SupportedResponseMIMETypes': [
'string',
]
},
ValidationSpecification={
'ValidationRole': 'string',
'ValidationProfiles': [
{
'ProfileName': 'string',
'TransformJobDefinition': {
'MaxConcurrentTransforms': 123,
'MaxPayloadInMB': 123,
'BatchStrategy': 'MultiRecord'|'SingleRecord',
'Environment': {
'string': 'string'
},
'TransformInput': {
'DataSource': {
'S3DataSource': {
'S3DataType': 'ManifestFile'|'S3Prefix'|'AugmentedManifestFile',
'S3Uri': 'string'
}
},
'ContentType': 'string',
'CompressionType': 'None'|'Gzip',
'SplitType': 'None'|'Line'|'RecordIO'
},
'TransformOutput': {
'S3OutputPath': 'string',
'Accept': 'string',
'AssembleWith': 'None'|'Line',
'KmsKeyId': 'string'
},
'TransformResources': {
'InstanceType': 'ml.m4.xlarge'|'ml.m4.2xlarge'|'ml.m4.4xlarge'|'ml.m4.10xlarge'|'ml.m4.16xlarge'|'ml.c4.xlarge'|'ml.c4.2xlarge'|'ml.c4.4xlarge'|'ml.c4.8xlarge'|'ml.p2.xlarge'|'ml.p2.8xlarge'|'ml.p2.16xlarge'|'ml.p3.2xlarge'|'ml.p3.8xlarge'|'ml.p3.16xlarge'|'ml.c5.xlarge'|'ml.c5.2xlarge'|'ml.c5.4xlarge'|'ml.c5.9xlarge'|'ml.c5.18xlarge'|'ml.m5.large'|'ml.m5.xlarge'|'ml.m5.2xlarge'|'ml.m5.4xlarge'|'ml.m5.12xlarge'|'ml.m5.24xlarge',
'InstanceCount': 123,
'VolumeKmsKeyId': 'string'
}
}
},
]
},
SourceAlgorithmSpecification={
'SourceAlgorithms': [
{
'ModelDataUrl': 'string',
'AlgorithmName': 'string'
},
]
},
CertifyForMarketplace=True|False
)
[REQUIRED]
The name of the model package. The name must have 1 to 63 characters. Valid characters are a-z, A-Z, 0-9, and - (hyphen).
Specifies details about inference jobs that can be run with models based on this model package, including the following:
The Amazon ECR registry path of the Docker image that contains the inference code.
Describes the Docker container for the model package.
The DNS host name for the Docker container.
The Amazon EC2 Container Registry path where inference code is stored.
If you are using your own custom algorithm instead of an algorithm provided by Amazon SageMaker, the inference code must meet Amazon SageMaker requirements. Amazon SageMaker supports both registry/repository[:tag] and registry/repository[@digest] image path formats. For more information, see Using Your Own Algorithms with Amazon SageMaker .
An MD5 hash of the training algorithm that identifies the Docker image used for training.
The Amazon S3 path where the model artifacts, which result from model training, are stored. This path must point to a single gzip compressed tar archive (.tar.gz suffix).
The ID of the model package.
A list of the instance types on which a transformation job can be run or on which an endpoint can be deployed.
A list of the instance types that are used to generate inferences in real-time.
The supported MIME types for the input data.
The supported MIME types for the output data.
Specifies configurations for one or more transform jobs that Amazon SageMaker runs to test the model package.
The IAM roles to be used for the validation of a model package.
An array of ModelPackageValidationProfile objects, each of which specifies a batch transform job that Amazon SageMaker runs to validate your model package.
Contains data such as the inputs and targeted instance types that are used in the process of validating the model package.
The data provided in the validation profile is made available to your buyers on AWS Marketplace.
The name of the profile for the model package.
The TransformJobDefinition object that describes the transform job used for the validation of the model package.
The maximum number of parallel requests that can be sent to each instance in a transform job. The default value is 1.
The maximum payload size allowed, in MB. A payload is the data portion of a record (without metadata).
A string that determines the number of records included in a single mini-batch.
SingleRecord means only one record is used per mini-batch. MultiRecord means a mini-batch is set to contain as many records that can fit within the MaxPayloadInMB limit.
The environment variables to set in the Docker container. We support up to 16 key and values entries in the map.
A description of the input source and the way the transform job consumes it.
Describes the location of the channel data, meaning the S3 location of the input data that the model can consume.
The S3 location of the data source that is associated with a channel.
If you choose S3Prefix , S3Uri identifies a key name prefix. Amazon SageMaker uses all objects with the specified key name prefix for batch transform.
If you choose ManifestFile , S3Uri identifies an object that is a manifest file containing a list of object keys that you want Amazon SageMaker to use for batch transform.
Depending on the value specified for the S3DataType , identifies either a key name prefix or a manifest. For example:
The multipurpose internet mail extension (MIME) type of the data. Amazon SageMaker uses the MIME type with each http call to transfer data to the transform job.
Compressing data helps save on storage space. If your transform data is compressed, specify the compression type. Amazon SageMaker automatically decompresses the data for the transform job accordingly. The default value is None .
The method to use to split the transform job's data into smaller batches. The default value is None . If you don't want to split the data, specify None . If you want to split records on a newline character boundary, specify Line . To split records according to the RecordIO format, specify RecordIO .
Amazon SageMaker will send maximum number of records per batch in each request up to the MaxPayloadInMB limit. For more information, see RecordIO data format .
Note
For information about the RecordIO format, see Data Format .
Identifies the Amazon S3 location where you want Amazon SageMaker to save the results from the transform job.
The Amazon S3 path where you want Amazon SageMaker to store the results of the transform job. For example, s3://bucket-name/key-name-prefix .
For every S3 object used as input for the transform job, the transformed data is stored in a corresponding subfolder in the location under the output prefix. For example, the input data s3://bucket-name/input-name-prefix/dataset01/data.csv will have the transformed data stored at s3://bucket-name/key-name-prefix/dataset01/ , based on the original name, as a series of .part files (.part0001, part0002, etc).
The MIME type used to specify the output data. Amazon SageMaker uses the MIME type with each http call to transfer data from the transform job.
Defines how to assemble the results of the transform job as a single S3 object. You should select a format that is most convenient to you. To concatenate the results in binary format, specify None . To add a newline character at the end of every transformed record, specify Line .
The AWS Key Management Service (AWS KMS) key that Amazon SageMaker uses to encrypt the model artifacts at rest using Amazon S3 server-side encryption. The KmsKeyId can be any of the following formats:
If you don't provide a KMS key ID, Amazon SageMaker uses the default KMS key for Amazon S3 for your role's account. For more information, see KMS-Managed Encryption Keys in the Amazon Simple Storage Service Developer Guide.
The KMS key policy must grant permission to the IAM role that you specify in your CreateTramsformJob request. For more information, see Using Key Policies in AWS KMS in the AWS Key Management Service Developer Guide .
Identifies the ML compute instances for the transform job.
The ML compute instance type for the transform job. For using built-in algorithms to transform moderately sized datasets, ml.m4.xlarge or ml.m5.large should suffice. There is no default value for InstanceType .
The number of ML compute instances to use in the transform job. For distributed transform, provide a value greater than 1. The default value is 1 .
The AWS Key Management Service (AWS KMS) key that Amazon SageMaker uses to encrypt data on the storage volume attached to the ML compute instance(s) that run the batch transform job. The VolumeKmsKeyId can be any of the following formats:
Details about the algorithm that was used to create the model package.
A list of the algorithms that were used to create a model package.
Specifies an algorithm that was used to create the model package. The algorithm must be either an algorithm resource in your Amazon SageMaker account or an algorithm in AWS Marketplace that you are subscribed to.
The Amazon S3 path where the model artifacts, which result from model training, are stored. This path must point to a single gzip compressed tar archive (.tar.gz suffix).
The name of an algorithm that was used to create the model package. The algorithm must be either an algorithm resource in your Amazon SageMaker account or an algorithm in AWS Marketplace that you are subscribed to.
dict
Response Syntax
{
'ModelPackageArn': 'string'
}
Response Structure
(dict) --
ModelPackageArn (string) --
The Amazon Resource Name (ARN) of the new model package.
Creates an Amazon SageMaker notebook instance. A notebook instance is a machine learning (ML) compute instance running on a Jupyter notebook.
In a CreateNotebookInstance request, specify the type of ML compute instance that you want to run. Amazon SageMaker launches the instance, installs common libraries that you can use to explore datasets for model training, and attaches an ML storage volume to the notebook instance.
Amazon SageMaker also provides a set of example notebooks. Each notebook demonstrates how to use Amazon SageMaker with a specific algorithm or with a machine learning framework.
After receiving the request, Amazon SageMaker does the following:
After creating the notebook instance, Amazon SageMaker returns its Amazon Resource Name (ARN).
After Amazon SageMaker creates the notebook instance, you can connect to the Jupyter server and work in Jupyter notebooks. For example, you can write code to explore a dataset that you can use for model training, train a model, host models by creating Amazon SageMaker endpoints, and validate hosted models.
For more information, see How It Works .
See also: AWS API Documentation
Request Syntax
response = client.create_notebook_instance(
NotebookInstanceName='string',
InstanceType='ml.t2.medium'|'ml.t2.large'|'ml.t2.xlarge'|'ml.t2.2xlarge'|'ml.t3.medium'|'ml.t3.large'|'ml.t3.xlarge'|'ml.t3.2xlarge'|'ml.m4.xlarge'|'ml.m4.2xlarge'|'ml.m4.4xlarge'|'ml.m4.10xlarge'|'ml.m4.16xlarge'|'ml.m5.xlarge'|'ml.m5.2xlarge'|'ml.m5.4xlarge'|'ml.m5.12xlarge'|'ml.m5.24xlarge'|'ml.c4.xlarge'|'ml.c4.2xlarge'|'ml.c4.4xlarge'|'ml.c4.8xlarge'|'ml.c5.xlarge'|'ml.c5.2xlarge'|'ml.c5.4xlarge'|'ml.c5.9xlarge'|'ml.c5.18xlarge'|'ml.c5d.xlarge'|'ml.c5d.2xlarge'|'ml.c5d.4xlarge'|'ml.c5d.9xlarge'|'ml.c5d.18xlarge'|'ml.p2.xlarge'|'ml.p2.8xlarge'|'ml.p2.16xlarge'|'ml.p3.2xlarge'|'ml.p3.8xlarge'|'ml.p3.16xlarge',
SubnetId='string',
SecurityGroupIds=[
'string',
],
RoleArn='string',
KmsKeyId='string',
Tags=[
{
'Key': 'string',
'Value': 'string'
},
],
LifecycleConfigName='string',
DirectInternetAccess='Enabled'|'Disabled',
VolumeSizeInGB=123,
AcceleratorTypes=[
'ml.eia1.medium'|'ml.eia1.large'|'ml.eia1.xlarge',
],
DefaultCodeRepository='string',
AdditionalCodeRepositories=[
'string',
]
)
[REQUIRED]
The name of the new notebook instance.
[REQUIRED]
The type of ML compute instance to launch for the notebook instance.
The VPC security group IDs, in the form sg-xxxxxxxx. The security groups must be for the same VPC as specified in the subnet.
[REQUIRED]
When you send any requests to AWS resources from the notebook instance, Amazon SageMaker assumes this role to perform tasks on your behalf. You must grant this role necessary permissions so Amazon SageMaker can perform these tasks. The policy must allow the Amazon SageMaker service principal (sagemaker.amazonaws.com) permissions to assume this role. For more information, see Amazon SageMaker Roles .
Note
To be able to pass this role to Amazon SageMaker, the caller of this API must have the iam:PassRole permission.
A list of tags to associate with the notebook instance. You can add tags later by using the CreateTags API.
Describes a tag.
The tag key.
The tag value.
Sets whether Amazon SageMaker provides internet access to the notebook instance. If you set this to Disabled this notebook instance will be able to access resources only in your VPC, and will not be able to connect to Amazon SageMaker training and endpoint services unless your configure a NAT Gateway in your VPC.
For more information, see Notebook Instances Are Internet-Enabled by Default . You can set the value of this parameter to Disabled only if you set a value for the SubnetId parameter.
A list of Elastic Inference (EI) instance types to associate with this notebook instance. Currently, only one instance type can be associated with a notebook intance. For more information, see Using Elastic Inference in Amazon SageMaker .
An array of up to 3 git repositories to associate with the notebook instance. These can be either the names of git repositories stored as resources in your account, or the URL of git repositories in AWS CodeCommit or in any other git repository. These repositories are cloned at the same level as the default repository of your notebook instance. For more information, see Associating Git Repositories with Amazon SageMaker Notebook Instances .
dict
Response Syntax
{
'NotebookInstanceArn': 'string'
}
Response Structure
(dict) --
NotebookInstanceArn (string) --
The Amazon Resource Name (ARN) of the notebook instance.
Creates a lifecycle configuration that you can associate with a notebook instance. A lifecycle configuration is a collection of shell scripts that run when you create or start a notebook instance.
Each lifecycle configuration script has a limit of 16384 characters.
The value of the $PATH environment variable that is available to both scripts is /sbin:bin:/usr/sbin:/usr/bin .
View CloudWatch Logs for notebook instance lifecycle configurations in log group /aws/sagemaker/NotebookInstances in log stream [notebook-instance-name]/[LifecycleConfigHook] .
Lifecycle configuration scripts cannot run for longer than 5 minutes. If a script runs for longer than 5 minutes, it fails and the notebook instance is not created or started.
For information about notebook instance lifestyle configurations, see Step 2.1: (Optional) Customize a Notebook Instance .
See also: AWS API Documentation
Request Syntax
response = client.create_notebook_instance_lifecycle_config(
NotebookInstanceLifecycleConfigName='string',
OnCreate=[
{
'Content': 'string'
},
],
OnStart=[
{
'Content': 'string'
},
]
)
[REQUIRED]
The name of the lifecycle configuration.
A shell script that runs only once, when you create a notebook instance. The shell script must be a base64-encoded string.
Contains the notebook instance lifecycle configuration script.
Each lifecycle configuration script has a limit of 16384 characters.
The value of the $PATH environment variable that is available to both scripts is /sbin:bin:/usr/sbin:/usr/bin .
View CloudWatch Logs for notebook instance lifecycle configurations in log group /aws/sagemaker/NotebookInstances in log stream [notebook-instance-name]/[LifecycleConfigHook] .
Lifecycle configuration scripts cannot run for longer than 5 minutes. If a script runs for longer than 5 minutes, it fails and the notebook instance is not created or started.
For information about notebook instance lifestyle configurations, see Step 2.1: (Optional) Customize a Notebook Instance .
A base64-encoded string that contains a shell script for a notebook instance lifecycle configuration.
A shell script that runs every time you start a notebook instance, including when you create the notebook instance. The shell script must be a base64-encoded string.
Contains the notebook instance lifecycle configuration script.
Each lifecycle configuration script has a limit of 16384 characters.
The value of the $PATH environment variable that is available to both scripts is /sbin:bin:/usr/sbin:/usr/bin .
View CloudWatch Logs for notebook instance lifecycle configurations in log group /aws/sagemaker/NotebookInstances in log stream [notebook-instance-name]/[LifecycleConfigHook] .
Lifecycle configuration scripts cannot run for longer than 5 minutes. If a script runs for longer than 5 minutes, it fails and the notebook instance is not created or started.
For information about notebook instance lifestyle configurations, see Step 2.1: (Optional) Customize a Notebook Instance .
A base64-encoded string that contains a shell script for a notebook instance lifecycle configuration.
dict
Response Syntax
{
'NotebookInstanceLifecycleConfigArn': 'string'
}
Response Structure
(dict) --
NotebookInstanceLifecycleConfigArn (string) --
The Amazon Resource Name (ARN) of the lifecycle configuration.
Returns a URL that you can use to connect to the Jupyter server from a notebook instance. In the Amazon SageMaker console, when you choose Open next to a notebook instance, Amazon SageMaker opens a new tab showing the Jupyter server home page from the notebook instance. The console uses this API to get the URL and show the page.
You can restrict access to this API and to the URL that it returns to a list of IP addresses that you specify. To restrict access, attach an IAM policy that denies access to this API unless the call comes from an IP address in the specified list to every AWS Identity and Access Management user, group, or role used to access the notebook instance. Use the NotIpAddress condition operator and the aws:SourceIP condition context key to specify the list of IP addresses that you want to have access to the notebook instance. For more information, see Limit Access to a Notebook Instance by IP Address .
See also: AWS API Documentation
Request Syntax
response = client.create_presigned_notebook_instance_url(
NotebookInstanceName='string',
SessionExpirationDurationInSeconds=123
)
[REQUIRED]
The name of the notebook instance.
dict
Response Syntax
{
'AuthorizedUrl': 'string'
}
Response Structure
(dict) --
AuthorizedUrl (string) --
A JSON object that contains the URL string.
Starts a model training job. After training completes, Amazon SageMaker saves the resulting model artifacts to an Amazon S3 location that you specify.
If you choose to host your model using Amazon SageMaker hosting services, you can use the resulting model artifacts as part of the model. You can also use the artifacts in a deep learning service other than Amazon SageMaker, provided that you know how to use them for inferences.
In the request body, you provide the following:
For more information about Amazon SageMaker, see How It Works .
See also: AWS API Documentation
Request Syntax
response = client.create_training_job(
TrainingJobName='string',
HyperParameters={
'string': 'string'
},
AlgorithmSpecification={
'TrainingImage': 'string',
'AlgorithmName': 'string',
'TrainingInputMode': 'Pipe'|'File',
'MetricDefinitions': [
{
'Name': 'string',
'Regex': 'string'
},
]
},
RoleArn='string',
InputDataConfig=[
{
'ChannelName': 'string',
'DataSource': {
'S3DataSource': {
'S3DataType': 'ManifestFile'|'S3Prefix'|'AugmentedManifestFile',
'S3Uri': 'string',
'S3DataDistributionType': 'FullyReplicated'|'ShardedByS3Key',
'AttributeNames': [
'string',
]
}
},
'ContentType': 'string',
'CompressionType': 'None'|'Gzip',
'RecordWrapperType': 'None'|'RecordIO',
'InputMode': 'Pipe'|'File',
'ShuffleConfig': {
'Seed': 123
}
},
],
OutputDataConfig={
'KmsKeyId': 'string',
'S3OutputPath': 'string'
},
ResourceConfig={
'InstanceType': 'ml.m4.xlarge'|'ml.m4.2xlarge'|'ml.m4.4xlarge'|'ml.m4.10xlarge'|'ml.m4.16xlarge'|'ml.m5.large'|'ml.m5.xlarge'|'ml.m5.2xlarge'|'ml.m5.4xlarge'|'ml.m5.12xlarge'|'ml.m5.24xlarge'|'ml.c4.xlarge'|'ml.c4.2xlarge'|'ml.c4.4xlarge'|'ml.c4.8xlarge'|'ml.p2.xlarge'|'ml.p2.8xlarge'|'ml.p2.16xlarge'|'ml.p3.2xlarge'|'ml.p3.8xlarge'|'ml.p3.16xlarge'|'ml.c5.xlarge'|'ml.c5.2xlarge'|'ml.c5.4xlarge'|'ml.c5.9xlarge'|'ml.c5.18xlarge',
'InstanceCount': 123,
'VolumeSizeInGB': 123,
'VolumeKmsKeyId': 'string'
},
VpcConfig={
'SecurityGroupIds': [
'string',
],
'Subnets': [
'string',
]
},
StoppingCondition={
'MaxRuntimeInSeconds': 123
},
Tags=[
{
'Key': 'string',
'Value': 'string'
},
],
EnableNetworkIsolation=True|False
)
[REQUIRED]
The name of the training job. The name must be unique within an AWS Region in an AWS account.
Algorithm-specific parameters that influence the quality of the model. You set hyperparameters before you start the learning process. For a list of hyperparameters for each training algorithm provided by Amazon SageMaker, see Algorithms .
You can specify a maximum of 100 hyperparameters. Each hyperparameter is a key-value pair. Each key and value is limited to 256 characters, as specified by the Length Constraint .
[REQUIRED]
The registry path of the Docker image that contains the training algorithm and algorithm-specific metadata, including the input mode. For more information about algorithms provided by Amazon SageMaker, see Algorithms . For information about providing your own algorithms, see Using Your Own Algorithms with Amazon SageMaker .
The registry path of the Docker image that contains the training algorithm. For information about docker registry paths for built-in algorithms, see Algorithms Provided by Amazon SageMaker: Common Parameters .
The name of the algorithm resource to use for the training job. This must be an algorithm resource that you created or subscribe to on AWS Marketplace. If you specify a value for this parameter, you can't specify a value for TrainingImage .
The input mode that the algorithm supports. For the input modes that Amazon SageMaker algorithms support, see Algorithms . If an algorithm supports the File input mode, Amazon SageMaker downloads the training data from S3 to the provisioned ML storage Volume, and mounts the directory to docker volume for training container. If an algorithm supports the Pipe input mode, Amazon SageMaker streams data directly from S3 to the container.
In File mode, make sure you provision ML storage volume with sufficient capacity to accommodate the data download from S3. In addition to the training data, the ML storage volume also stores the output model. The algorithm container use ML storage volume to also store intermediate information, if any.
For distributed algorithms using File mode, training data is distributed uniformly, and your training duration is predictable if the input data objects size is approximately same. Amazon SageMaker does not split the files any further for model training. If the object sizes are skewed, training won't be optimal as the data distribution is also skewed where one host in a training cluster is overloaded, thus becoming bottleneck in training.
A list of metric definition objects. Each object specifies the metric name and regular expressions used to parse algorithm logs. Amazon SageMaker publishes each metric to Amazon CloudWatch.
Specifies a metric that the training algorithm writes to stderr or stdout . Amazon SageMakerhyperparameter tuning captures all defined metrics. You specify one metric that a hyperparameter tuning job uses as its objective metric to choose the best training job.
The name of the metric.
A regular expression that searches the output of a training job and gets the value of the metric. For more information about using regular expressions to define metrics, see Defining Objective Metrics .
[REQUIRED]
The Amazon Resource Name (ARN) of an IAM role that Amazon SageMaker assumes to perform tasks on your behalf.
During model training, Amazon SageMaker needs your permission to read input data from an S3 bucket, download a Docker image that contains training code, write model artifacts to an S3 bucket, write logs to Amazon CloudWatch Logs, and publish metrics to Amazon CloudWatch. You grant permissions for all of these tasks to an IAM role. For more information, see Amazon SageMaker Roles .
Note
To be able to pass this role to Amazon SageMaker, the caller of this API must have the iam:PassRole permission.
An array of Channel objects. Each channel is a named input source. InputDataConfig describes the input data and its location.
Algorithms can accept input data from one or more channels. For example, an algorithm might have two channels of input data, training_data and validation_data . The configuration for each channel provides the S3 location where the input data is stored. It also provides information about the stored data: the MIME type, compression method, and whether the data is wrapped in RecordIO format.
Depending on the input mode that the algorithm supports, Amazon SageMaker either copies input data files from an S3 bucket to a local directory in the Docker container, or makes it available as input streams.
A channel is a named input source that training algorithms can consume.
The name of the channel.
The location of the channel data.
The S3 location of the data source that is associated with a channel.
If you choose S3Prefix , S3Uri identifies a key name prefix. Amazon SageMaker uses all objects that match the specified key name prefix for model training.
If you choose ManifestFile , S3Uri identifies an object that is a manifest file containing a list of object keys that you want Amazon SageMaker to use for model training.
If you choose AugmentedManifestFile , S3Uri identifies an object that is an augmented manifest file in JSON lines format. This file contains the data you want to use for model training. AugmentedManifestFile can only be used if the Channel's input mode is Pipe .
Depending on the value specified for the S3DataType , identifies either a key name prefix or a manifest. For example:
If you want Amazon SageMaker to replicate the entire dataset on each ML compute instance that is launched for model training, specify FullyReplicated .
If you want Amazon SageMaker to replicate a subset of data on each ML compute instance that is launched for model training, specify ShardedByS3Key . If there are n ML compute instances launched for a training job, each instance gets approximately 1/n of the number of S3 objects. In this case, model training on each machine uses only the subset of training data.
Don't choose more ML compute instances for training than available S3 objects. If you do, some nodes won't get any data and you will pay for nodes that aren't getting any training data. This applies in both File and Pipemodes. Keep this in mind when developing algorithms.
In distributed training, where you use multiple ML compute EC2 instances, you might choose ShardedByS3Key . If the algorithm requires copying training data to the ML storage volume (when TrainingInputMode is set to File ), this copies 1/n of the number of objects.
A list of one or more attribute names to use that are found in a specified augmented manifest file.
The MIME type of the data.
If training data is compressed, the compression type. The default value is None . CompressionType is used only in Pipe input mode. In File mode, leave this field unset or set it to None.
Specify RecordIO as the value when input data is in raw format but the training algorithm requires the RecordIO format. In this case, Amazon SageMaker wraps each individual S3 object in a RecordIO record. If the input data is already in RecordIO format, you don't need to set this attribute. For more information, see Create a Dataset Using RecordIO .
In File mode, leave this field unset or set it to None.
(Optional) The input mode to use for the data channel in a training job. If you don't set a value for InputMode , Amazon SageMaker uses the value set for TrainingInputMode . Use this parameter to override the TrainingInputMode setting in a AlgorithmSpecification request when you have a channel that needs a different input mode from the training job's general setting. To download the data from Amazon Simple Storage Service (Amazon S3) to the provisioned ML storage volume, and mount the directory to a Docker volume, use File input mode. To stream data directly from Amazon S3 to the container, choose Pipe input mode.
To use a model for incremental training, choose File input model.
A configuration for a shuffle option for input data in a channel. If you use S3Prefix for S3DataType , this shuffles the results of the S3 key prefix matches. If you use ManifestFile , the order of the S3 object references in the ManifestFile is shuffled. If you use AugmentedManifestFile , the order of the JSON lines in the AugmentedManifestFile is shuffled. The shuffling order is determined using the Seed value.
For Pipe input mode, shuffling is done at the start of every epoch. With large datasets this ensures that the order of the training data is different for each epoch, it helps reduce bias and possible overfitting. In a multi-node training job when ShuffleConfig is combined with S3DataDistributionType of ShardedByS3Key , the data is shuffled across nodes so that the content sent to a particular node on the first epoch might be sent to a different node on the second epoch.
Determines the shuffling order in ShuffleConfig . value.
[REQUIRED]
Specifies the path to the S3 bucket where you want to store model artifacts. Amazon SageMaker creates subfolders for the artifacts.
The AWS Key Management Service (AWS KMS) key that Amazon SageMaker uses to encrypt the model artifacts at rest using Amazon S3 server-side encryption. The KmsKeyId can be any of the following formats:
If you don't provide the KMS key ID, Amazon SageMaker uses the default KMS key for Amazon S3 for your role's account. For more information, see KMS-Managed Encryption Keys in Amazon Simple Storage Service Developer Guide.
Note
The KMS key policy must grant permission to the IAM role that you specify in your CreateTrainingJob request. Using Key Policies in AWS KMS in the AWS Key Management Service Developer Guide .
Identifies the S3 path where you want Amazon SageMaker to store the model artifacts. For example, s3://bucket-name/key-name-prefix .
[REQUIRED]
The resources, including the ML compute instances and ML storage volumes, to use for model training.
ML storage volumes store model artifacts and incremental states. Training algorithms might also use ML storage volumes for scratch space. If you want Amazon SageMaker to use the ML storage volume to store the training data, choose File as the TrainingInputMode in the algorithm specification. For distributed training algorithms, specify an instance count greater than 1.
The ML compute instance type.
The number of ML compute instances to use. For distributed training, provide a value greater than 1.
The size of the ML storage volume that you want to provision.
ML storage volumes store model artifacts and incremental states. Training algorithms might also use the ML storage volume for scratch space. If you want to store the training data in the ML storage volume, choose File as the TrainingInputMode in the algorithm specification.
You must specify sufficient ML storage for your scenario.
Note
Amazon SageMaker supports only the General Purpose SSD (gp2) ML storage volume type.
The AWS Key Management Service (AWS KMS) key that Amazon SageMaker uses to encrypt data on the storage volume attached to the ML compute instance(s) that run the training job. The VolumeKmsKeyId can be any of the following formats:
A VpcConfig object that specifies the VPC that you want your training job to connect to. Control access to and from your training container by configuring the VPC. For more information, see Protect Training Jobs by Using an Amazon Virtual Private Cloud .
The VPC security group IDs, in the form sg-xxxxxxxx. Specify the security groups for the VPC that is specified in the Subnets field.
The ID of the subnets in the VPC to which you want to connect your training job or model.
[REQUIRED]
Sets a duration for training. Use this parameter to cap model training costs. To stop a job, Amazon SageMaker sends the algorithm the SIGTERM signal, which delays job termination for 120 seconds. Algorithms might use this 120-second window to save the model artifacts.
When Amazon SageMaker terminates a job because the stopping condition has been met, training algorithms provided by Amazon SageMaker save the intermediate results of the job. This intermediate data is a valid model artifact. You can use it to create a model using the CreateModel API.
The maximum length of time, in seconds, that the training or compilation job can run. If the job does not complete during this time, Amazon SageMaker ends the job. If value is not specified, default value is 1 day. Maximum value is 5 days.
An array of key-value pairs. For more information, see Using Cost Allocation Tags in the AWS Billing and Cost Management User Guide .
Describes a tag.
The tag key.
The tag value.
Isolates the training container. No inbound or outbound network calls can be made, except for calls between peers within a training cluster for distributed training. If network isolation is used for training jobs that are configured to use a VPC, Amazon SageMaker downloads and uploads customer data and model artifacts through the specifed VPC, but the training container does not have network access.
Note
The Semantic Segmentation built-in algorithm does not support network isolation.
dict
Response Syntax
{
'TrainingJobArn': 'string'
}
Response Structure
(dict) --
TrainingJobArn (string) --
The Amazon Resource Name (ARN) of the training job.
Starts a transform job. A transform job uses a trained model to get inferences on a dataset and saves these results to an Amazon S3 location that you specify.
To perform batch transformations, you create a transform job and use the data that you have readily available.
In the request body, you provide the following:
For more information about how batch transformation works Amazon SageMaker, see How It Works .
See also: AWS API Documentation
Request Syntax
response = client.create_transform_job(
TransformJobName='string',
ModelName='string',
MaxConcurrentTransforms=123,
MaxPayloadInMB=123,
BatchStrategy='MultiRecord'|'SingleRecord',
Environment={
'string': 'string'
},
TransformInput={
'DataSource': {
'S3DataSource': {
'S3DataType': 'ManifestFile'|'S3Prefix'|'AugmentedManifestFile',
'S3Uri': 'string'
}
},
'ContentType': 'string',
'CompressionType': 'None'|'Gzip',
'SplitType': 'None'|'Line'|'RecordIO'
},
TransformOutput={
'S3OutputPath': 'string',
'Accept': 'string',
'AssembleWith': 'None'|'Line',
'KmsKeyId': 'string'
},
TransformResources={
'InstanceType': 'ml.m4.xlarge'|'ml.m4.2xlarge'|'ml.m4.4xlarge'|'ml.m4.10xlarge'|'ml.m4.16xlarge'|'ml.c4.xlarge'|'ml.c4.2xlarge'|'ml.c4.4xlarge'|'ml.c4.8xlarge'|'ml.p2.xlarge'|'ml.p2.8xlarge'|'ml.p2.16xlarge'|'ml.p3.2xlarge'|'ml.p3.8xlarge'|'ml.p3.16xlarge'|'ml.c5.xlarge'|'ml.c5.2xlarge'|'ml.c5.4xlarge'|'ml.c5.9xlarge'|'ml.c5.18xlarge'|'ml.m5.large'|'ml.m5.xlarge'|'ml.m5.2xlarge'|'ml.m5.4xlarge'|'ml.m5.12xlarge'|'ml.m5.24xlarge',
'InstanceCount': 123,
'VolumeKmsKeyId': 'string'
},
Tags=[
{
'Key': 'string',
'Value': 'string'
},
]
)
[REQUIRED]
The name of the transform job. The name must be unique within an AWS Region in an AWS account.
[REQUIRED]
The name of the model that you want to use for the transform job. ModelName must be the name of an existing Amazon SageMaker model within an AWS Region in an AWS account.
The maximum payload size allowed, in MB. A payload is the data portion of a record (without metadata). The value in MaxPayloadInMB must be greater or equal to the size of a single record. You can approximate the size of a record by dividing the size of your dataset by the number of records. Then multiply this value by the number of records you want in a mini-batch. We recommend to enter a slightly larger value than this to ensure the records fit within the maximum payload size. The default value is 6 MB.
For cases where the payload might be arbitrarily large and is transmitted using HTTP chunked encoding, set the value to 0 . This feature only works in supported algorithms. Currently, Amazon SageMaker built-in algorithms do not support this feature.
Determines the number of records included in a single mini-batch. SingleRecord means only one record is used per mini-batch. MultiRecord means a mini-batch is set to contain as many records that can fit within the MaxPayloadInMB limit.
Batch transform will automatically split your input data into whatever payload size is specified if you set SplitType to Line and BatchStrategy to MultiRecord . There's no need to split the dataset into smaller files or to use larger payload sizes unless the records in your dataset are very large.
The environment variables to set in the Docker container. We support up to 16 key and values entries in the map.
[REQUIRED]
Describes the input source and the way the transform job consumes it.
Describes the location of the channel data, meaning the S3 location of the input data that the model can consume.
The S3 location of the data source that is associated with a channel.
If you choose S3Prefix , S3Uri identifies a key name prefix. Amazon SageMaker uses all objects with the specified key name prefix for batch transform.
If you choose ManifestFile , S3Uri identifies an object that is a manifest file containing a list of object keys that you want Amazon SageMaker to use for batch transform.
Depending on the value specified for the S3DataType , identifies either a key name prefix or a manifest. For example:
The multipurpose internet mail extension (MIME) type of the data. Amazon SageMaker uses the MIME type with each http call to transfer data to the transform job.
Compressing data helps save on storage space. If your transform data is compressed, specify the compression type. Amazon SageMaker automatically decompresses the data for the transform job accordingly. The default value is None .
The method to use to split the transform job's data into smaller batches. The default value is None . If you don't want to split the data, specify None . If you want to split records on a newline character boundary, specify Line . To split records according to the RecordIO format, specify RecordIO .
Amazon SageMaker will send maximum number of records per batch in each request up to the MaxPayloadInMB limit. For more information, see RecordIO data format .
Note
For information about the RecordIO format, see Data Format .
[REQUIRED]
Describes the results of the transform job.
The Amazon S3 path where you want Amazon SageMaker to store the results of the transform job. For example, s3://bucket-name/key-name-prefix .
For every S3 object used as input for the transform job, the transformed data is stored in a corresponding subfolder in the location under the output prefix. For example, the input data s3://bucket-name/input-name-prefix/dataset01/data.csv will have the transformed data stored at s3://bucket-name/key-name-prefix/dataset01/ , based on the original name, as a series of .part files (.part0001, part0002, etc).
The MIME type used to specify the output data. Amazon SageMaker uses the MIME type with each http call to transfer data from the transform job.
Defines how to assemble the results of the transform job as a single S3 object. You should select a format that is most convenient to you. To concatenate the results in binary format, specify None . To add a newline character at the end of every transformed record, specify Line .
The AWS Key Management Service (AWS KMS) key that Amazon SageMaker uses to encrypt the model artifacts at rest using Amazon S3 server-side encryption. The KmsKeyId can be any of the following formats:
If you don't provide a KMS key ID, Amazon SageMaker uses the default KMS key for Amazon S3 for your role's account. For more information, see KMS-Managed Encryption Keys in the Amazon Simple Storage Service Developer Guide.
The KMS key policy must grant permission to the IAM role that you specify in your CreateTramsformJob request. For more information, see Using Key Policies in AWS KMS in the AWS Key Management Service Developer Guide .
[REQUIRED]
Describes the resources, including ML instance types and ML instance count, to use for the transform job.
The ML compute instance type for the transform job. For using built-in algorithms to transform moderately sized datasets, ml.m4.xlarge or ml.m5.large should suffice. There is no default value for InstanceType .
The number of ML compute instances to use in the transform job. For distributed transform, provide a value greater than 1. The default value is 1 .
The AWS Key Management Service (AWS KMS) key that Amazon SageMaker uses to encrypt data on the storage volume attached to the ML compute instance(s) that run the batch transform job. The VolumeKmsKeyId can be any of the following formats:
An array of key-value pairs. Adding tags is optional. For more information, see Using Cost Allocation Tags in the AWS Billing and Cost Management User Guide .
Describes a tag.
The tag key.
The tag value.
dict
Response Syntax
{
'TransformJobArn': 'string'
}
Response Structure
(dict) --
TransformJobArn (string) --
The Amazon Resource Name (ARN) of the transform job.
Creates a new work team for labeling your data. A work team is defined by one or more Amazon Cognito user pools. You must first create the user pools before you can create a work team.
You cannot create more than 25 work teams in an account and region.
See also: AWS API Documentation
Request Syntax
response = client.create_workteam(
WorkteamName='string',
MemberDefinitions=[
{
'CognitoMemberDefinition': {
'UserPool': 'string',
'UserGroup': 'string',
'ClientId': 'string'
}
},
],
Description='string',
Tags=[
{
'Key': 'string',
'Value': 'string'
},
]
)
[REQUIRED]
The name of the work team. Use this name to identify the work team.
[REQUIRED]
A list of MemberDefinition objects that contains objects that identify the Amazon Cognito user pool that makes up the work team. For more information, see Amazon Cognito User Pools .
All of the CognitoMemberDefinition objects that make up the member definition must have the same ClientId and UserPool values.
Defines the Amazon Cognito user group that is part of a work team.
The Amazon Cognito user group that is part of the work team.
An identifier for a user pool. The user pool must be in the same region as the service that you are calling.
An identifier for a user group.
An identifier for an application client. You must create the app client ID using Amazon Cognito.
[REQUIRED]
A description of the work team.
Describes a tag.
The tag key.
The tag value.
dict
Response Syntax
{
'WorkteamArn': 'string'
}
Response Structure
(dict) --
WorkteamArn (string) --
The Amazon Resource Name (ARN) of the work team. You can use this ARN to identify the work team.
Removes the specified algorithm from your account.
See also: AWS API Documentation
Request Syntax
response = client.delete_algorithm(
AlgorithmName='string'
)
[REQUIRED]
The name of the algorithm to delete.
Deletes the specified git repository from your account.
See also: AWS API Documentation
Request Syntax
response = client.delete_code_repository(
CodeRepositoryName='string'
)
[REQUIRED]
The name of the git repository to delete.
Deletes an endpoint. Amazon SageMaker frees up all of the resources that were deployed when the endpoint was created.
Amazon SageMaker retires any custom KMS key grants associated with the endpoint, meaning you don't need to use the RevokeGrant API call.
See also: AWS API Documentation
Request Syntax
response = client.delete_endpoint(
EndpointName='string'
)
[REQUIRED]
The name of the endpoint that you want to delete.
Deletes an endpoint configuration. The DeleteEndpointConfig API deletes only the specified configuration. It does not delete endpoints created using the configuration.
See also: AWS API Documentation
Request Syntax
response = client.delete_endpoint_config(
EndpointConfigName='string'
)
[REQUIRED]
The name of the endpoint configuration that you want to delete.
Deletes a model. The DeleteModel API deletes only the model entry that was created in Amazon SageMaker when you called the CreateModel API. It does not delete model artifacts, inference code, or the IAM role that you specified when creating the model.
See also: AWS API Documentation
Request Syntax
response = client.delete_model(
ModelName='string'
)
[REQUIRED]
The name of the model to delete.
Deletes a model package.
A model package is used to create Amazon SageMaker models or list on AWS Marketplace. Buyers can subscribe to model packages listed on AWS Marketplace to create models in Amazon SageMaker.
See also: AWS API Documentation
Request Syntax
response = client.delete_model_package(
ModelPackageName='string'
)
[REQUIRED]
The name of the model package. The name must have 1 to 63 characters. Valid characters are a-z, A-Z, 0-9, and - (hyphen).
Deletes an Amazon SageMaker notebook instance. Before you can delete a notebook instance, you must call the StopNotebookInstance API.
Warning
When you delete a notebook instance, you lose all of your data. Amazon SageMaker removes the ML compute instance, and deletes the ML storage volume and the network interface associated with the notebook instance.
See also: AWS API Documentation
Request Syntax
response = client.delete_notebook_instance(
NotebookInstanceName='string'
)
[REQUIRED]
The name of the Amazon SageMaker notebook instance to delete.
Deletes a notebook instance lifecycle configuration.
See also: AWS API Documentation
Request Syntax
response = client.delete_notebook_instance_lifecycle_config(
NotebookInstanceLifecycleConfigName='string'
)
[REQUIRED]
The name of the lifecycle configuration to delete.
Deletes the specified tags from an Amazon SageMaker resource.
To list a resource's tags, use the ListTags API.
Note
When you call this API to delete tags from a hyperparameter tuning job, the deleted tags are not removed from training jobs that the hyperparameter tuning job launched before you called this API.
See also: AWS API Documentation
Request Syntax
response = client.delete_tags(
ResourceArn='string',
TagKeys=[
'string',
]
)
[REQUIRED]
The Amazon Resource Name (ARN) of the resource whose tags you want to delete.
[REQUIRED]
An array or one or more tag keys to delete.
dict
Response Syntax
{}
Response Structure
Deletes an existing work team. This operation can't be undone.
See also: AWS API Documentation
Request Syntax
response = client.delete_workteam(
WorkteamName='string'
)
[REQUIRED]
The name of the work team to delete.
{
'Success': True|False
}
Response Structure
Returns true if the work team was successfully deleted; otherwise, returns false .
Returns a description of the specified algorithm that is in your account.
See also: AWS API Documentation
Request Syntax
response = client.describe_algorithm(
AlgorithmName='string'
)
[REQUIRED]
The name of the algorithm to describe.
{
'AlgorithmName': 'string',
'AlgorithmArn': 'string',
'AlgorithmDescription': 'string',
'CreationTime': datetime(2015, 1, 1),
'TrainingSpecification': {
'TrainingImage': 'string',
'TrainingImageDigest': 'string',
'SupportedHyperParameters': [
{
'Name': 'string',
'Description': 'string',
'Type': 'Integer'|'Continuous'|'Categorical'|'FreeText',
'Range': {
'IntegerParameterRangeSpecification': {
'MinValue': 'string',
'MaxValue': 'string'
},
'ContinuousParameterRangeSpecification': {
'MinValue': 'string',
'MaxValue': 'string'
},
'CategoricalParameterRangeSpecification': {
'Values': [
'string',
]
}
},
'IsTunable': True|False,
'IsRequired': True|False,
'DefaultValue': 'string'
},
],
'SupportedTrainingInstanceTypes': [
'ml.m4.xlarge'|'ml.m4.2xlarge'|'ml.m4.4xlarge'|'ml.m4.10xlarge'|'ml.m4.16xlarge'|'ml.m5.large'|'ml.m5.xlarge'|'ml.m5.2xlarge'|'ml.m5.4xlarge'|'ml.m5.12xlarge'|'ml.m5.24xlarge'|'ml.c4.xlarge'|'ml.c4.2xlarge'|'ml.c4.4xlarge'|'ml.c4.8xlarge'|'ml.p2.xlarge'|'ml.p2.8xlarge'|'ml.p2.16xlarge'|'ml.p3.2xlarge'|'ml.p3.8xlarge'|'ml.p3.16xlarge'|'ml.c5.xlarge'|'ml.c5.2xlarge'|'ml.c5.4xlarge'|'ml.c5.9xlarge'|'ml.c5.18xlarge',
],
'SupportsDistributedTraining': True|False,
'MetricDefinitions': [
{
'Name': 'string',
'Regex': 'string'
},
],
'TrainingChannels': [
{
'Name': 'string',
'Description': 'string',
'IsRequired': True|False,
'SupportedContentTypes': [
'string',
],
'SupportedCompressionTypes': [
'None'|'Gzip',
],
'SupportedInputModes': [
'Pipe'|'File',
]
},
],
'SupportedTuningJobObjectiveMetrics': [
{
'Type': 'Maximize'|'Minimize',
'MetricName': 'string'
},
]
},
'InferenceSpecification': {
'Containers': [
{
'ContainerHostname': 'string',
'Image': 'string',
'ImageDigest': 'string',
'ModelDataUrl': 'string',
'ProductId': 'string'
},
],
'SupportedTransformInstanceTypes': [
'ml.m4.xlarge'|'ml.m4.2xlarge'|'ml.m4.4xlarge'|'ml.m4.10xlarge'|'ml.m4.16xlarge'|'ml.c4.xlarge'|'ml.c4.2xlarge'|'ml.c4.4xlarge'|'ml.c4.8xlarge'|'ml.p2.xlarge'|'ml.p2.8xlarge'|'ml.p2.16xlarge'|'ml.p3.2xlarge'|'ml.p3.8xlarge'|'ml.p3.16xlarge'|'ml.c5.xlarge'|'ml.c5.2xlarge'|'ml.c5.4xlarge'|'ml.c5.9xlarge'|'ml.c5.18xlarge'|'ml.m5.large'|'ml.m5.xlarge'|'ml.m5.2xlarge'|'ml.m5.4xlarge'|'ml.m5.12xlarge'|'ml.m5.24xlarge',
],
'SupportedRealtimeInferenceInstanceTypes': [
'ml.t2.medium'|'ml.t2.large'|'ml.t2.xlarge'|'ml.t2.2xlarge'|'ml.m4.xlarge'|'ml.m4.2xlarge'|'ml.m4.4xlarge'|'ml.m4.10xlarge'|'ml.m4.16xlarge'|'ml.m5.large'|'ml.m5.xlarge'|'ml.m5.2xlarge'|'ml.m5.4xlarge'|'ml.m5.12xlarge'|'ml.m5.24xlarge'|'ml.c4.large'|'ml.c4.xlarge'|'ml.c4.2xlarge'|'ml.c4.4xlarge'|'ml.c4.8xlarge'|'ml.p2.xlarge'|'ml.p2.8xlarge'|'ml.p2.16xlarge'|'ml.p3.2xlarge'|'ml.p3.8xlarge'|'ml.p3.16xlarge'|'ml.c5.large'|'ml.c5.xlarge'|'ml.c5.2xlarge'|'ml.c5.4xlarge'|'ml.c5.9xlarge'|'ml.c5.18xlarge',
],
'SupportedContentTypes': [
'string',
],
'SupportedResponseMIMETypes': [
'string',
]
},
'ValidationSpecification': {
'ValidationRole': 'string',
'ValidationProfiles': [
{
'ProfileName': 'string',
'TrainingJobDefinition': {
'TrainingInputMode': 'Pipe'|'File',
'HyperParameters': {
'string': 'string'
},
'InputDataConfig': [
{
'ChannelName': 'string',
'DataSource': {
'S3DataSource': {
'S3DataType': 'ManifestFile'|'S3Prefix'|'AugmentedManifestFile',
'S3Uri': 'string',
'S3DataDistributionType': 'FullyReplicated'|'ShardedByS3Key',
'AttributeNames': [
'string',
]
}
},
'ContentType': 'string',
'CompressionType': 'None'|'Gzip',
'RecordWrapperType': 'None'|'RecordIO',
'InputMode': 'Pipe'|'File',
'ShuffleConfig': {
'Seed': 123
}
},
],
'OutputDataConfig': {
'KmsKeyId': 'string',
'S3OutputPath': 'string'
},
'ResourceConfig': {
'InstanceType': 'ml.m4.xlarge'|'ml.m4.2xlarge'|'ml.m4.4xlarge'|'ml.m4.10xlarge'|'ml.m4.16xlarge'|'ml.m5.large'|'ml.m5.xlarge'|'ml.m5.2xlarge'|'ml.m5.4xlarge'|'ml.m5.12xlarge'|'ml.m5.24xlarge'|'ml.c4.xlarge'|'ml.c4.2xlarge'|'ml.c4.4xlarge'|'ml.c4.8xlarge'|'ml.p2.xlarge'|'ml.p2.8xlarge'|'ml.p2.16xlarge'|'ml.p3.2xlarge'|'ml.p3.8xlarge'|'ml.p3.16xlarge'|'ml.c5.xlarge'|'ml.c5.2xlarge'|'ml.c5.4xlarge'|'ml.c5.9xlarge'|'ml.c5.18xlarge',
'InstanceCount': 123,
'VolumeSizeInGB': 123,
'VolumeKmsKeyId': 'string'
},
'StoppingCondition': {
'MaxRuntimeInSeconds': 123
}
},
'TransformJobDefinition': {
'MaxConcurrentTransforms': 123,
'MaxPayloadInMB': 123,
'BatchStrategy': 'MultiRecord'|'SingleRecord',
'Environment': {
'string': 'string'
},
'TransformInput': {
'DataSource': {
'S3DataSource': {
'S3DataType': 'ManifestFile'|'S3Prefix'|'AugmentedManifestFile',
'S3Uri': 'string'
}
},
'ContentType': 'string',
'CompressionType': 'None'|'Gzip',
'SplitType': 'None'|'Line'|'RecordIO'
},
'TransformOutput': {
'S3OutputPath': 'string',
'Accept': 'string',
'AssembleWith': 'None'|'Line',
'KmsKeyId': 'string'
},
'TransformResources': {
'InstanceType': 'ml.m4.xlarge'|'ml.m4.2xlarge'|'ml.m4.4xlarge'|'ml.m4.10xlarge'|'ml.m4.16xlarge'|'ml.c4.xlarge'|'ml.c4.2xlarge'|'ml.c4.4xlarge'|'ml.c4.8xlarge'|'ml.p2.xlarge'|'ml.p2.8xlarge'|'ml.p2.16xlarge'|'ml.p3.2xlarge'|'ml.p3.8xlarge'|'ml.p3.16xlarge'|'ml.c5.xlarge'|'ml.c5.2xlarge'|'ml.c5.4xlarge'|'ml.c5.9xlarge'|'ml.c5.18xlarge'|'ml.m5.large'|'ml.m5.xlarge'|'ml.m5.2xlarge'|'ml.m5.4xlarge'|'ml.m5.12xlarge'|'ml.m5.24xlarge',
'InstanceCount': 123,
'VolumeKmsKeyId': 'string'
}
}
},
]
},
'AlgorithmStatus': 'Pending'|'InProgress'|'Completed'|'Failed'|'Deleting',
'AlgorithmStatusDetails': {
'ValidationStatuses': [
{
'Name': 'string',
'Status': 'NotStarted'|'InProgress'|'Completed'|'Failed',
'FailureReason': 'string'
},
],
'ImageScanStatuses': [
{
'Name': 'string',
'Status': 'NotStarted'|'InProgress'|'Completed'|'Failed',
'FailureReason': 'string'
},
]
},
'ProductId': 'string',
'CertifyForMarketplace': True|False
}
Response Structure
The name of the algorithm being described.
The Amazon Resource Name (ARN) of the algorithm.>
A brief summary about the algorithm.
A timestamp specifying when the algorithm was created.
Details about training jobs run by this algorithm.
The Amazon Amazon ECR registry path of the Docker image that contains the training algorithm.
An MD5 hash of the training algorithm that identifies the Docker image used for training.
A list of the HyperParameterSpecification objects, that define the supported hyperparameters. This is required if the algorithm supports automatic model tuning.>
Defines a hyperparameter to be used by an algorithm.
The name of this hyperparameter. The name must be unique.
A brief description of the hyperparameter.
The type of this hyperparameter. The valid types are Integer , Continuous , Categorical , and FreeText .
The allowed range for this hyperparameter.
A IntegerParameterRangeSpecification object that defines the possible values for an integer hyperparameter.
The minimum integer value allowed.
The maximum integer value allowed.
A ContinuousParameterRangeSpecification object that defines the possible values for a continuous hyperparameter.
The minimum floating-point value allowed.
The maximum floating-point value allowed.
A CategoricalParameterRangeSpecification object that defines the possible values for a categorical hyperparameter.
The allowed categories for the hyperparameter.
Indicates whether this hyperparameter is tunable in a hyperparameter tuning job.
Indicates whether this hyperparameter is required.
The default value for this hyperparameter. If a default value is specified, a hyperparameter cannot be required.
A list of the instance types that this algorithm can use for training.
Indicates whether the algorithm supports distributed training. If set to false, buyers can’t request more than one instance during training.
A list of MetricDefinition objects, which are used for parsing metrics generated by the algorithm.
Specifies a metric that the training algorithm writes to stderr or stdout . Amazon SageMakerhyperparameter tuning captures all defined metrics. You specify one metric that a hyperparameter tuning job uses as its objective metric to choose the best training job.
The name of the metric.
A regular expression that searches the output of a training job and gets the value of the metric. For more information about using regular expressions to define metrics, see Defining Objective Metrics .
A list of ChannelSpecification objects, which specify the input sources to be used by the algorithm.
Defines a named input source, called a channel, to be used by an algorithm.
The name of the channel./sagemaker/eia
A brief description of the channel.
Indicates whether the channel is required by the algorithm.
The supported MIME types for the data.
The allowed compression types, if data compression is used.
The allowed input mode, either FILE or PIPE.
In FILE mode, Amazon SageMaker copies the data from the input source onto the local Amazon Elastic Block Store (Amazon EBS) volumes before starting your training algorithm. This is the most commonly used input mode.
In PIPE mode, Amazon SageMaker streams input data from the source directly to your algorithm without using the EBS volume.
A list of the metrics that the alogorithm emits that can be used as the objective metric in a hyperparameter tuning job.
Defines the objective metric for a hyperparameter tuning job. Hyperparameter tuning uses the value of this metric to evaluate the training jobs it launches, and returns the training job that results in either the highest or lowest value for this metric, depending on the value you specify for the Type parameter.
Whether to minimize or maximize the objective metric.
The name of the metric to use for the objective metric.
Details about inference jobs that the algorithm runs.
The Amazon ECR registry path of the Docker image that contains the inference code.
Describes the Docker container for the model package.
The DNS host name for the Docker container.
The Amazon EC2 Container Registry path where inference code is stored.
If you are using your own custom algorithm instead of an algorithm provided by Amazon SageMaker, the inference code must meet Amazon SageMaker requirements. Amazon SageMaker supports both registry/repository[:tag] and registry/repository[@digest] image path formats. For more information, see Using Your Own Algorithms with Amazon SageMaker .
An MD5 hash of the training algorithm that identifies the Docker image used for training.
The Amazon S3 path where the model artifacts, which result from model training, are stored. This path must point to a single gzip compressed tar archive (.tar.gz suffix).
The ID of the model package.
A list of the instance types on which a transformation job can be run or on which an endpoint can be deployed.
A list of the instance types that are used to generate inferences in real-time.
The supported MIME types for the input data.
The supported MIME types for the output data.
Details about configurations for one or more training jobs that Amazon SageMaker runs to test the algorithm.
The IAM roles that Amazon SageMaker uses to run the training jobs.
An array of AlgorithmValidationProfile objects, each of which specifies a training job and batch transform job that Amazon SageMaker runs to validate your algorithm.
Defines a training job and a batch transform job that Amazon SageMaker runs to validate your algorithm.
The data provided in the validation profile is made available to your buyers on AWS Marketplace.
The name of the profile for the algorithm. The name must have 1 to 63 characters. Valid characters are a-z, A-Z, 0-9, and - (hyphen).
The TrainingJobDefinition object that describes the training job that Amazon SageMaker runs to validate your algorithm.
The input mode used by the algorithm for the training job. For the input modes that Amazon SageMaker algorithms support, see Algorithms .
If an algorithm supports the File input mode, Amazon SageMaker downloads the training data from S3 to the provisioned ML storage Volume, and mounts the directory to docker volume for training container. If an algorithm supports the Pipe input mode, Amazon SageMaker streams data directly from S3 to the container.
The hyperparameters used for the training job.
An array of Channel objects, each of which specifies an input source.
A channel is a named input source that training algorithms can consume.
The name of the channel.
The location of the channel data.
The S3 location of the data source that is associated with a channel.
If you choose S3Prefix , S3Uri identifies a key name prefix. Amazon SageMaker uses all objects that match the specified key name prefix for model training.
If you choose ManifestFile , S3Uri identifies an object that is a manifest file containing a list of object keys that you want Amazon SageMaker to use for model training.
If you choose AugmentedManifestFile , S3Uri identifies an object that is an augmented manifest file in JSON lines format. This file contains the data you want to use for model training. AugmentedManifestFile can only be used if the Channel's input mode is Pipe .
Depending on the value specified for the S3DataType , identifies either a key name prefix or a manifest. For example:
If you want Amazon SageMaker to replicate the entire dataset on each ML compute instance that is launched for model training, specify FullyReplicated .
If you want Amazon SageMaker to replicate a subset of data on each ML compute instance that is launched for model training, specify ShardedByS3Key . If there are n ML compute instances launched for a training job, each instance gets approximately 1/n of the number of S3 objects. In this case, model training on each machine uses only the subset of training data.
Don't choose more ML compute instances for training than available S3 objects. If you do, some nodes won't get any data and you will pay for nodes that aren't getting any training data. This applies in both File and Pipemodes. Keep this in mind when developing algorithms.
In distributed training, where you use multiple ML compute EC2 instances, you might choose ShardedByS3Key . If the algorithm requires copying training data to the ML storage volume (when TrainingInputMode is set to File ), this copies 1/n of the number of objects.
A list of one or more attribute names to use that are found in a specified augmented manifest file.
The MIME type of the data.
If training data is compressed, the compression type. The default value is None . CompressionType is used only in Pipe input mode. In File mode, leave this field unset or set it to None.
Specify RecordIO as the value when input data is in raw format but the training algorithm requires the RecordIO format. In this case, Amazon SageMaker wraps each individual S3 object in a RecordIO record. If the input data is already in RecordIO format, you don't need to set this attribute. For more information, see Create a Dataset Using RecordIO .
In File mode, leave this field unset or set it to None.
(Optional) The input mode to use for the data channel in a training job. If you don't set a value for InputMode , Amazon SageMaker uses the value set for TrainingInputMode . Use this parameter to override the TrainingInputMode setting in a AlgorithmSpecification request when you have a channel that needs a different input mode from the training job's general setting. To download the data from Amazon Simple Storage Service (Amazon S3) to the provisioned ML storage volume, and mount the directory to a Docker volume, use File input mode. To stream data directly from Amazon S3 to the container, choose Pipe input mode.
To use a model for incremental training, choose File input model.
A configuration for a shuffle option for input data in a channel. If you use S3Prefix for S3DataType , this shuffles the results of the S3 key prefix matches. If you use ManifestFile , the order of the S3 object references in the ManifestFile is shuffled. If you use AugmentedManifestFile , the order of the JSON lines in the AugmentedManifestFile is shuffled. The shuffling order is determined using the Seed value.
For Pipe input mode, shuffling is done at the start of every epoch. With large datasets this ensures that the order of the training data is different for each epoch, it helps reduce bias and possible overfitting. In a multi-node training job when ShuffleConfig is combined with S3DataDistributionType of ShardedByS3Key , the data is shuffled across nodes so that the content sent to a particular node on the first epoch might be sent to a different node on the second epoch.
Determines the shuffling order in ShuffleConfig . value.
the path to the S3 bucket where you want to store model artifacts. Amazon SageMaker creates subfolders for the artifacts.
The AWS Key Management Service (AWS KMS) key that Amazon SageMaker uses to encrypt the model artifacts at rest using Amazon S3 server-side encryption. The KmsKeyId can be any of the following formats:
If you don't provide the KMS key ID, Amazon SageMaker uses the default KMS key for Amazon S3 for your role's account. For more information, see KMS-Managed Encryption Keys in Amazon Simple Storage Service Developer Guide.
Note
The KMS key policy must grant permission to the IAM role that you specify in your CreateTrainingJob request. Using Key Policies in AWS KMS in the AWS Key Management Service Developer Guide .
Identifies the S3 path where you want Amazon SageMaker to store the model artifacts. For example, s3://bucket-name/key-name-prefix .
The resources, including the ML compute instances and ML storage volumes, to use for model training.
The ML compute instance type.
The number of ML compute instances to use. For distributed training, provide a value greater than 1.
The size of the ML storage volume that you want to provision.
ML storage volumes store model artifacts and incremental states. Training algorithms might also use the ML storage volume for scratch space. If you want to store the training data in the ML storage volume, choose File as the TrainingInputMode in the algorithm specification.
You must specify sufficient ML storage for your scenario.
Note
Amazon SageMaker supports only the General Purpose SSD (gp2) ML storage volume type.
The AWS Key Management Service (AWS KMS) key that Amazon SageMaker uses to encrypt data on the storage volume attached to the ML compute instance(s) that run the training job. The VolumeKmsKeyId can be any of the following formats:
Sets a duration for training. Use this parameter to cap model training costs.
To stop a job, Amazon SageMaker sends the algorithm the SIGTERM signal, which delays job termination for 120 seconds. Algorithms might use this 120-second window to save the model artifacts.
The maximum length of time, in seconds, that the training or compilation job can run. If the job does not complete during this time, Amazon SageMaker ends the job. If value is not specified, default value is 1 day. Maximum value is 5 days.
The TransformJobDefinition object that describes the transform job that Amazon SageMaker runs to validate your algorithm.
The maximum number of parallel requests that can be sent to each instance in a transform job. The default value is 1.
The maximum payload size allowed, in MB. A payload is the data portion of a record (without metadata).
A string that determines the number of records included in a single mini-batch.
SingleRecord means only one record is used per mini-batch. MultiRecord means a mini-batch is set to contain as many records that can fit within the MaxPayloadInMB limit.
The environment variables to set in the Docker container. We support up to 16 key and values entries in the map.
A description of the input source and the way the transform job consumes it.
Describes the location of the channel data, meaning the S3 location of the input data that the model can consume.
The S3 location of the data source that is associated with a channel.
If you choose S3Prefix , S3Uri identifies a key name prefix. Amazon SageMaker uses all objects with the specified key name prefix for batch transform.
If you choose ManifestFile , S3Uri identifies an object that is a manifest file containing a list of object keys that you want Amazon SageMaker to use for batch transform.
Depending on the value specified for the S3DataType , identifies either a key name prefix or a manifest. For example:
The multipurpose internet mail extension (MIME) type of the data. Amazon SageMaker uses the MIME type with each http call to transfer data to the transform job.
Compressing data helps save on storage space. If your transform data is compressed, specify the compression type. Amazon SageMaker automatically decompresses the data for the transform job accordingly. The default value is None .
The method to use to split the transform job's data into smaller batches. The default value is None . If you don't want to split the data, specify None . If you want to split records on a newline character boundary, specify Line . To split records according to the RecordIO format, specify RecordIO .
Amazon SageMaker will send maximum number of records per batch in each request up to the MaxPayloadInMB limit. For more information, see RecordIO data format .
Note
For information about the RecordIO format, see Data Format .
Identifies the Amazon S3 location where you want Amazon SageMaker to save the results from the transform job.
The Amazon S3 path where you want Amazon SageMaker to store the results of the transform job. For example, s3://bucket-name/key-name-prefix .
For every S3 object used as input for the transform job, the transformed data is stored in a corresponding subfolder in the location under the output prefix. For example, the input data s3://bucket-name/input-name-prefix/dataset01/data.csv will have the transformed data stored at s3://bucket-name/key-name-prefix/dataset01/ , based on the original name, as a series of .part files (.part0001, part0002, etc).
The MIME type used to specify the output data. Amazon SageMaker uses the MIME type with each http call to transfer data from the transform job.
Defines how to assemble the results of the transform job as a single S3 object. You should select a format that is most convenient to you. To concatenate the results in binary format, specify None . To add a newline character at the end of every transformed record, specify Line .
The AWS Key Management Service (AWS KMS) key that Amazon SageMaker uses to encrypt the model artifacts at rest using Amazon S3 server-side encryption. The KmsKeyId can be any of the following formats:
If you don't provide a KMS key ID, Amazon SageMaker uses the default KMS key for Amazon S3 for your role's account. For more information, see KMS-Managed Encryption Keys in the Amazon Simple Storage Service Developer Guide.
The KMS key policy must grant permission to the IAM role that you specify in your CreateTramsformJob request. For more information, see Using Key Policies in AWS KMS in the AWS Key Management Service Developer Guide .
Identifies the ML compute instances for the transform job.
The ML compute instance type for the transform job. For using built-in algorithms to transform moderately sized datasets, ml.m4.xlarge or ml.m5.large should suffice. There is no default value for InstanceType .
The number of ML compute instances to use in the transform job. For distributed transform, provide a value greater than 1. The default value is 1 .
The AWS Key Management Service (AWS KMS) key that Amazon SageMaker uses to encrypt data on the storage volume attached to the ML compute instance(s) that run the batch transform job. The VolumeKmsKeyId can be any of the following formats:
The current status of the algorithm.
Details about the current status of the algorithm.
The status of the validation of the algorithm.
Represents the overall status of an algorithm.
The name of the algorithm for which the overall status is being repoorted.
The current status.
The reason for failure, if the overall status is a failed state.
The status of the scan of the algorithm's Docker image container.
Represents the overall status of an algorithm.
The name of the algorithm for which the overall status is being repoorted.
The current status.
The reason for failure, if the overall status is a failed state.
The product identifier of the algorithm.
Whether the algorithm is certified to be listed in AWS Marektplace.
Gets details about the specified git repository.
See also: AWS API Documentation
Request Syntax
response = client.describe_code_repository(
CodeRepositoryName='string'
)
[REQUIRED]
The name of the git repository to describe.
{
'CodeRepositoryName': 'string',
'CodeRepositoryArn': 'string',
'CreationTime': datetime(2015, 1, 1),
'LastModifiedTime': datetime(2015, 1, 1),
'GitConfig': {
'RepositoryUrl': 'string',
'Branch': 'string',
'SecretArn': 'string'
}
}
Response Structure
The name of the git repository.
The Amazon Resource Name (ARN) of the git repository.
The date and time that the repository was created.
The date and time that the repository was last changed.
Configuration details about the repository, including the URL where the repository is located, the default branch, and the Amazon Resource Name (ARN) of the AWS Secrets Manager secret that contains the credentials used to access the repository.
The URL where the git repository is located.
The default brach for the git repository.
The Amazon Resource Name (ARN) of the AWS Secrets Manager secret that contains the credentials used to access the git repository. The secret must have a staging label of AWSCURRENT and must be in the following format:
{"username": *UserName* , "password": *Password* }
Returns information about a model compilation job.
To create a model compilation job, use CreateCompilationJob . To get information about multiple model compilation jobs, use ListCompilationJobs .
See also: AWS API Documentation
Request Syntax
response = client.describe_compilation_job(
CompilationJobName='string'
)
[REQUIRED]
The name of the model compilation job that you want information about.
{
'CompilationJobName': 'string',
'CompilationJobArn': 'string',
'CompilationJobStatus': 'INPROGRESS'|'COMPLETED'|'FAILED'|'STARTING'|'STOPPING'|'STOPPED',
'CompilationStartTime': datetime(2015, 1, 1),
'CompilationEndTime': datetime(2015, 1, 1),
'StoppingCondition': {
'MaxRuntimeInSeconds': 123
},
'CreationTime': datetime(2015, 1, 1),
'LastModifiedTime': datetime(2015, 1, 1),
'FailureReason': 'string',
'ModelArtifacts': {
'S3ModelArtifacts': 'string'
},
'RoleArn': 'string',
'InputConfig': {
'S3Uri': 'string',
'DataInputConfig': 'string',
'Framework': 'TENSORFLOW'|'MXNET'|'ONNX'|'PYTORCH'|'XGBOOST'
},
'OutputConfig': {
'S3OutputLocation': 'string',
'TargetDevice': 'ml_m4'|'ml_m5'|'ml_c4'|'ml_c5'|'ml_p2'|'ml_p3'|'jetson_tx1'|'jetson_tx2'|'rasp3b'|'deeplens'
}
}
Response Structure
The name of the model compilation job.
The Amazon Resource Name (ARN) of an IAM role that Amazon SageMaker assumes to perform the model compilation job.
The status of the model compilation job.
The time when the model compilation job started the CompilationJob instances.
You are billed for the time between this timestamp and the timestamp in the DescribeCompilationJobResponse$CompilationEndTime field. In Amazon CloudWatch Logs, the start time might be later than this time. That's because it takes time to download the compilation job, which depends on the size of the compilation job container.
The time when the model compilation job on a compilation job instance ended. For a successful or stopped job, this is when the job's model artifacts have finished uploading. For a failed job, this is when Amazon SageMaker detected that the job failed.
The duration allowed for model compilation.
The maximum length of time, in seconds, that the training or compilation job can run. If the job does not complete during this time, Amazon SageMaker ends the job. If value is not specified, default value is 1 day. Maximum value is 5 days.
The time that the model compilation job was created.
The time that the status of the model compilation job was last modified.
If a model compilation job failed, the reason it failed.
Information about the location in Amazon S3 that has been configured for storing the model artifacts used in the compilation job.
The path of the S3 object that contains the model artifacts. For example, s3://bucket-name/keynameprefix/model.tar.gz .
The Amazon Resource Name (ARN) of the model compilation job.
Information about the location in Amazon S3 of the input model artifacts, the name and shape of the expected data inputs, and the framework in which the model was trained.
The S3 path where the model artifacts, which result from model training, are stored. This path must point to a single gzip compressed tar archive (.tar.gz suffix).
Specifies the name and shape of the expected data inputs for your trained model with a JSON dictionary form. The data inputs are InputConfig$Framework specific.
Identifies the framework in which the model was trained. For example: TENSORFLOW.
Information about the output location for the compiled model and the target device that the model runs on.
Identifies the S3 path where you want Amazon SageMaker to store the model artifacts. For example, s3://bucket-name/key-name-prefix.
Identifies the device that you want to run your model on after it has been compiled. For example: ml_c5.
Returns the description of an endpoint.
See also: AWS API Documentation
Request Syntax
response = client.describe_endpoint(
EndpointName='string'
)
[REQUIRED]
The name of the endpoint.
{
'EndpointName': 'string',
'EndpointArn': 'string',
'EndpointConfigName': 'string',
'ProductionVariants': [
{
'VariantName': 'string',
'DeployedImages': [
{
'SpecifiedImage': 'string',
'ResolvedImage': 'string',
'ResolutionTime': datetime(2015, 1, 1)
},
],
'CurrentWeight': ...,
'DesiredWeight': ...,
'CurrentInstanceCount': 123,
'DesiredInstanceCount': 123
},
],
'EndpointStatus': 'OutOfService'|'Creating'|'Updating'|'SystemUpdating'|'RollingBack'|'InService'|'Deleting'|'Failed',
'FailureReason': 'string',
'CreationTime': datetime(2015, 1, 1),
'LastModifiedTime': datetime(2015, 1, 1)
}
Response Structure
Name of the endpoint.
The Amazon Resource Name (ARN) of the endpoint.
The name of the endpoint configuration associated with this endpoint.
An array of ProductionVariantSummary objects, one for each model hosted behind this endpoint.
Describes weight and capacities for a production variant associated with an endpoint. If you sent a request to the UpdateEndpointWeightsAndCapacities API and the endpoint status is Updating , you get different desired and current values.
The name of the variant.
An array of DeployedImage objects that specify the Amazon EC2 Container Registry paths of the inference images deployed on instances of this ProductionVariant .
Gets the Amazon EC2 Container Registry path of the docker image of the model that is hosted in this ProductionVariant .
If you used the registry/repository[:tag] form to specify the image path of the primary container when you created the model hosted in this ProductionVariant , the path resolves to a path of the form registry/repository[@digest] . A digest is a hash value that identifies a specific version of an image. For information about Amazon ECR paths, see Pulling an Image in the Amazon ECR User Guide .
The image path you specified when you created the model.
The specific digest path of the image hosted in this ProductionVariant .
The date and time when the image path for the model resolved to the ResolvedImage
The weight associated with the variant.
The requested weight, as specified in the UpdateEndpointWeightsAndCapacities request.
The number of instances associated with the variant.
The number of instances requested in the UpdateEndpointWeightsAndCapacities request.
The status of the endpoint.
If the status of the endpoint is Failed , the reason why it failed.
A timestamp that shows when the endpoint was created.
A timestamp that shows when the endpoint was last modified.
Returns the description of an endpoint configuration created using the CreateEndpointConfig API.
See also: AWS API Documentation
Request Syntax
response = client.describe_endpoint_config(
EndpointConfigName='string'
)
[REQUIRED]
The name of the endpoint configuration.
{
'EndpointConfigName': 'string',
'EndpointConfigArn': 'string',
'ProductionVariants': [
{
'VariantName': 'string',
'ModelName': 'string',
'InitialInstanceCount': 123,
'InstanceType': 'ml.t2.medium'|'ml.t2.large'|'ml.t2.xlarge'|'ml.t2.2xlarge'|'ml.m4.xlarge'|'ml.m4.2xlarge'|'ml.m4.4xlarge'|'ml.m4.10xlarge'|'ml.m4.16xlarge'|'ml.m5.large'|'ml.m5.xlarge'|'ml.m5.2xlarge'|'ml.m5.4xlarge'|'ml.m5.12xlarge'|'ml.m5.24xlarge'|'ml.c4.large'|'ml.c4.xlarge'|'ml.c4.2xlarge'|'ml.c4.4xlarge'|'ml.c4.8xlarge'|'ml.p2.xlarge'|'ml.p2.8xlarge'|'ml.p2.16xlarge'|'ml.p3.2xlarge'|'ml.p3.8xlarge'|'ml.p3.16xlarge'|'ml.c5.large'|'ml.c5.xlarge'|'ml.c5.2xlarge'|'ml.c5.4xlarge'|'ml.c5.9xlarge'|'ml.c5.18xlarge',
'InitialVariantWeight': ...,
'AcceleratorType': 'ml.eia1.medium'|'ml.eia1.large'|'ml.eia1.xlarge'
},
],
'KmsKeyId': 'string',
'CreationTime': datetime(2015, 1, 1)
}
Response Structure
Name of the Amazon SageMaker endpoint configuration.
The Amazon Resource Name (ARN) of the endpoint configuration.
An array of ProductionVariant objects, one for each model that you want to host at this endpoint.
Identifies a model that you want to host and the resources to deploy for hosting it. If you are deploying multiple models, tell Amazon SageMaker how to distribute traffic among the models by specifying variant weights.
The name of the production variant.
The name of the model that you want to host. This is the name that you specified when creating the model.
Number of instances to launch initially.
The ML compute instance type.
Determines initial traffic distribution among all of the models that you specify in the endpoint configuration. The traffic to a production variant is determined by the ratio of the VariantWeight to the sum of all VariantWeight values across all ProductionVariants. If unspecified, it defaults to 1.0.
The size of the Elastic Inference (EI) instance to use for the production variant. EI instances provide on-demand GPU computing for inference. For more information, see Using Elastic Inference in Amazon SageMaker . For more information, see Using Elastic Inference in Amazon SageMaker .
AWS KMS key ID Amazon SageMaker uses to encrypt data when storing it on the ML storage volume attached to the instance.
A timestamp that shows when the endpoint configuration was created.
Gets a description of a hyperparameter tuning job.
See also: AWS API Documentation
Request Syntax
response = client.describe_hyper_parameter_tuning_job(
HyperParameterTuningJobName='string'
)
[REQUIRED]
The name of the tuning job to describe.
{
'HyperParameterTuningJobName': 'string',
'HyperParameterTuningJobArn': 'string',
'HyperParameterTuningJobConfig': {
'Strategy': 'Bayesian',
'HyperParameterTuningJobObjective': {
'Type': 'Maximize'|'Minimize',
'MetricName': 'string'
},
'ResourceLimits': {
'MaxNumberOfTrainingJobs': 123,
'MaxParallelTrainingJobs': 123
},
'ParameterRanges': {
'IntegerParameterRanges': [
{
'Name': 'string',
'MinValue': 'string',
'MaxValue': 'string'
},
],
'ContinuousParameterRanges': [
{
'Name': 'string',
'MinValue': 'string',
'MaxValue': 'string'
},
],
'CategoricalParameterRanges': [
{
'Name': 'string',
'Values': [
'string',
]
},
]
}
},
'TrainingJobDefinition': {
'StaticHyperParameters': {
'string': 'string'
},
'AlgorithmSpecification': {
'TrainingImage': 'string',
'TrainingInputMode': 'Pipe'|'File',
'AlgorithmName': 'string',
'MetricDefinitions': [
{
'Name': 'string',
'Regex': 'string'
},
]
},
'RoleArn': 'string',
'InputDataConfig': [
{
'ChannelName': 'string',
'DataSource': {
'S3DataSource': {
'S3DataType': 'ManifestFile'|'S3Prefix'|'AugmentedManifestFile',
'S3Uri': 'string',
'S3DataDistributionType': 'FullyReplicated'|'ShardedByS3Key',
'AttributeNames': [
'string',
]
}
},
'ContentType': 'string',
'CompressionType': 'None'|'Gzip',
'RecordWrapperType': 'None'|'RecordIO',
'InputMode': 'Pipe'|'File',
'ShuffleConfig': {
'Seed': 123
}
},
],
'VpcConfig': {
'SecurityGroupIds': [
'string',
],
'Subnets': [
'string',
]
},
'OutputDataConfig': {
'KmsKeyId': 'string',
'S3OutputPath': 'string'
},
'ResourceConfig': {
'InstanceType': 'ml.m4.xlarge'|'ml.m4.2xlarge'|'ml.m4.4xlarge'|'ml.m4.10xlarge'|'ml.m4.16xlarge'|'ml.m5.large'|'ml.m5.xlarge'|'ml.m5.2xlarge'|'ml.m5.4xlarge'|'ml.m5.12xlarge'|'ml.m5.24xlarge'|'ml.c4.xlarge'|'ml.c4.2xlarge'|'ml.c4.4xlarge'|'ml.c4.8xlarge'|'ml.p2.xlarge'|'ml.p2.8xlarge'|'ml.p2.16xlarge'|'ml.p3.2xlarge'|'ml.p3.8xlarge'|'ml.p3.16xlarge'|'ml.c5.xlarge'|'ml.c5.2xlarge'|'ml.c5.4xlarge'|'ml.c5.9xlarge'|'ml.c5.18xlarge',
'InstanceCount': 123,
'VolumeSizeInGB': 123,
'VolumeKmsKeyId': 'string'
},
'StoppingCondition': {
'MaxRuntimeInSeconds': 123
},
'EnableNetworkIsolation': True|False
},
'HyperParameterTuningJobStatus': 'Completed'|'InProgress'|'Failed'|'Stopped'|'Stopping',
'CreationTime': datetime(2015, 1, 1),
'HyperParameterTuningEndTime': datetime(2015, 1, 1),
'LastModifiedTime': datetime(2015, 1, 1),
'TrainingJobStatusCounters': {
'Completed': 123,
'InProgress': 123,
'RetryableError': 123,
'NonRetryableError': 123,
'Stopped': 123
},
'ObjectiveStatusCounters': {
'Succeeded': 123,
'Pending': 123,
'Failed': 123
},
'BestTrainingJob': {
'TrainingJobName': 'string',
'TrainingJobArn': 'string',
'TuningJobName': 'string',
'CreationTime': datetime(2015, 1, 1),
'TrainingStartTime': datetime(2015, 1, 1),
'TrainingEndTime': datetime(2015, 1, 1),
'TrainingJobStatus': 'InProgress'|'Completed'|'Failed'|'Stopping'|'Stopped',
'TunedHyperParameters': {
'string': 'string'
},
'FailureReason': 'string',
'FinalHyperParameterTuningJobObjectiveMetric': {
'Type': 'Maximize'|'Minimize',
'MetricName': 'string',
'Value': ...
},
'ObjectiveStatus': 'Succeeded'|'Pending'|'Failed'
},
'OverallBestTrainingJob': {
'TrainingJobName': 'string',
'TrainingJobArn': 'string',
'TuningJobName': 'string',
'CreationTime': datetime(2015, 1, 1),
'TrainingStartTime': datetime(2015, 1, 1),
'TrainingEndTime': datetime(2015, 1, 1),
'TrainingJobStatus': 'InProgress'|'Completed'|'Failed'|'Stopping'|'Stopped',
'TunedHyperParameters': {
'string': 'string'
},
'FailureReason': 'string',
'FinalHyperParameterTuningJobObjectiveMetric': {
'Type': 'Maximize'|'Minimize',
'MetricName': 'string',
'Value': ...
},
'ObjectiveStatus': 'Succeeded'|'Pending'|'Failed'
},
'WarmStartConfig': {
'ParentHyperParameterTuningJobs': [
{
'HyperParameterTuningJobName': 'string'
},
],
'WarmStartType': 'IdenticalDataAndAlgorithm'|'TransferLearning'
},
'FailureReason': 'string'
}
Response Structure
The name of the tuning job.
The Amazon Resource Name (ARN) of the tuning job.
The HyperParameterTuningJobConfig object that specifies the configuration of the tuning job.
Specifies the search strategy for hyperparameters. Currently, the only valid value is Bayesian .
The HyperParameterTuningJobObjective object that specifies the objective metric for this tuning job.
Whether to minimize or maximize the objective metric.
The name of the metric to use for the objective metric.
The ResourceLimits object that specifies the maximum number of training jobs and parallel training jobs for this tuning job.
The maximum number of training jobs that a hyperparameter tuning job can launch.
The maximum number of concurrent training jobs that a hyperparameter tuning job can launch.
The ParameterRanges object that specifies the ranges of hyperparameters that this tuning job searches.
The array of IntegerParameterRange objects that specify ranges of integer hyperparameters that a hyperparameter tuning job searches.
For a hyperparameter of the integer type, specifies the range that a hyperparameter tuning job searches.
The name of the hyperparameter to search.
The minimum value of the hyperparameter to search.
The maximum value of the hyperparameter to search.
The array of ContinuousParameterRange objects that specify ranges of continuous hyperparameters that a hyperparameter tuning job searches.
A list of continuous hyperparameters to tune.
The name of the continuous hyperparameter to tune.
The minimum value for the hyperparameter. The tuning job uses floating-point values between this value and MaxValue for tuning.
The maximum value for the hyperparameter. The tuning job uses floating-point values between MinValue value and this value for tuning.
The array of CategoricalParameterRange objects that specify ranges of categorical hyperparameters that a hyperparameter tuning job searches.
A list of categorical hyperparameters to tune.
The name of the categorical hyperparameter to tune.
A list of the categories for the hyperparameter.
The HyperParameterTrainingJobDefinition object that specifies the definition of the training jobs that this tuning job launches.
Specifies the values of hyperparameters that do not change for the tuning job.
The HyperParameterAlgorithmSpecification object that specifies the algorithm to use for the training jobs that the tuning job launches.
The registry path of the Docker image that contains the training algorithm. For information about Docker registry paths for built-in algorithms, see Algorithms Provided by Amazon SageMaker: Common Parameters .
The input mode that the algorithm supports: File or Pipe. In File input mode, Amazon SageMaker downloads the training data from Amazon S3 to the storage volume that is attached to the training instance and mounts the directory to the Docker volume for the training container. In Pipe input mode, Amazon SageMaker streams data directly from Amazon S3 to the container.
If you specify File mode, make sure that you provision the storage volume that is attached to the training instance with enough capacity to accommodate the training data downloaded from Amazon S3, the model artifacts, and intermediate information.
For more information about input modes, see Algorithms .
The name of the resource algorithm to use for the hyperparameter tuning job. If you specify a value for this parameter, do not specify a value for TrainingImage .
An array of MetricDefinition objects that specify the metrics that the algorithm emits.
Specifies a metric that the training algorithm writes to stderr or stdout . Amazon SageMakerhyperparameter tuning captures all defined metrics. You specify one metric that a hyperparameter tuning job uses as its objective metric to choose the best training job.
The name of the metric.
A regular expression that searches the output of a training job and gets the value of the metric. For more information about using regular expressions to define metrics, see Defining Objective Metrics .
The Amazon Resource Name (ARN) of the IAM role associated with the training jobs that the tuning job launches.
An array of Channel objects that specify the input for the training jobs that the tuning job launches.
A channel is a named input source that training algorithms can consume.
The name of the channel.
The location of the channel data.
The S3 location of the data source that is associated with a channel.
If you choose S3Prefix , S3Uri identifies a key name prefix. Amazon SageMaker uses all objects that match the specified key name prefix for model training.
If you choose ManifestFile , S3Uri identifies an object that is a manifest file containing a list of object keys that you want Amazon SageMaker to use for model training.
If you choose AugmentedManifestFile , S3Uri identifies an object that is an augmented manifest file in JSON lines format. This file contains the data you want to use for model training. AugmentedManifestFile can only be used if the Channel's input mode is Pipe .
Depending on the value specified for the S3DataType , identifies either a key name prefix or a manifest. For example:
If you want Amazon SageMaker to replicate the entire dataset on each ML compute instance that is launched for model training, specify FullyReplicated .
If you want Amazon SageMaker to replicate a subset of data on each ML compute instance that is launched for model training, specify ShardedByS3Key . If there are n ML compute instances launched for a training job, each instance gets approximately 1/n of the number of S3 objects. In this case, model training on each machine uses only the subset of training data.
Don't choose more ML compute instances for training than available S3 objects. If you do, some nodes won't get any data and you will pay for nodes that aren't getting any training data. This applies in both File and Pipemodes. Keep this in mind when developing algorithms.
In distributed training, where you use multiple ML compute EC2 instances, you might choose ShardedByS3Key . If the algorithm requires copying training data to the ML storage volume (when TrainingInputMode is set to File ), this copies 1/n of the number of objects.
A list of one or more attribute names to use that are found in a specified augmented manifest file.
The MIME type of the data.
If training data is compressed, the compression type. The default value is None . CompressionType is used only in Pipe input mode. In File mode, leave this field unset or set it to None.
Specify RecordIO as the value when input data is in raw format but the training algorithm requires the RecordIO format. In this case, Amazon SageMaker wraps each individual S3 object in a RecordIO record. If the input data is already in RecordIO format, you don't need to set this attribute. For more information, see Create a Dataset Using RecordIO .
In File mode, leave this field unset or set it to None.
(Optional) The input mode to use for the data channel in a training job. If you don't set a value for InputMode , Amazon SageMaker uses the value set for TrainingInputMode . Use this parameter to override the TrainingInputMode setting in a AlgorithmSpecification request when you have a channel that needs a different input mode from the training job's general setting. To download the data from Amazon Simple Storage Service (Amazon S3) to the provisioned ML storage volume, and mount the directory to a Docker volume, use File input mode. To stream data directly from Amazon S3 to the container, choose Pipe input mode.
To use a model for incremental training, choose File input model.
A configuration for a shuffle option for input data in a channel. If you use S3Prefix for S3DataType , this shuffles the results of the S3 key prefix matches. If you use ManifestFile , the order of the S3 object references in the ManifestFile is shuffled. If you use AugmentedManifestFile , the order of the JSON lines in the AugmentedManifestFile is shuffled. The shuffling order is determined using the Seed value.
For Pipe input mode, shuffling is done at the start of every epoch. With large datasets this ensures that the order of the training data is different for each epoch, it helps reduce bias and possible overfitting. In a multi-node training job when ShuffleConfig is combined with S3DataDistributionType of ShardedByS3Key , the data is shuffled across nodes so that the content sent to a particular node on the first epoch might be sent to a different node on the second epoch.
Determines the shuffling order in ShuffleConfig . value.
The VpcConfig object that specifies the VPC that you want the training jobs that this hyperparameter tuning job launches to connect to. Control access to and from your training container by configuring the VPC. For more information, see Protect Training Jobs by Using an Amazon Virtual Private Cloud .
The VPC security group IDs, in the form sg-xxxxxxxx. Specify the security groups for the VPC that is specified in the Subnets field.
The ID of the subnets in the VPC to which you want to connect your training job or model.
Specifies the path to the Amazon S3 bucket where you store model artifacts from the training jobs that the tuning job launches.
The AWS Key Management Service (AWS KMS) key that Amazon SageMaker uses to encrypt the model artifacts at rest using Amazon S3 server-side encryption. The KmsKeyId can be any of the following formats:
If you don't provide the KMS key ID, Amazon SageMaker uses the default KMS key for Amazon S3 for your role's account. For more information, see KMS-Managed Encryption Keys in Amazon Simple Storage Service Developer Guide.
Note
The KMS key policy must grant permission to the IAM role that you specify in your CreateTrainingJob request. Using Key Policies in AWS KMS in the AWS Key Management Service Developer Guide .
Identifies the S3 path where you want Amazon SageMaker to store the model artifacts. For example, s3://bucket-name/key-name-prefix .
The resources, including the compute instances and storage volumes, to use for the training jobs that the tuning job launches.
Storage volumes store model artifacts and incremental states. Training algorithms might also use storage volumes for scratch space. If you want Amazon SageMaker to use the storage volume to store the training data, choose File as the TrainingInputMode in the algorithm specification. For distributed training algorithms, specify an instance count greater than 1.
The ML compute instance type.
The number of ML compute instances to use. For distributed training, provide a value greater than 1.
The size of the ML storage volume that you want to provision.
ML storage volumes store model artifacts and incremental states. Training algorithms might also use the ML storage volume for scratch space. If you want to store the training data in the ML storage volume, choose File as the TrainingInputMode in the algorithm specification.
You must specify sufficient ML storage for your scenario.
Note
Amazon SageMaker supports only the General Purpose SSD (gp2) ML storage volume type.
The AWS Key Management Service (AWS KMS) key that Amazon SageMaker uses to encrypt data on the storage volume attached to the ML compute instance(s) that run the training job. The VolumeKmsKeyId can be any of the following formats:
Sets a maximum duration for the training jobs that the tuning job launches. Use this parameter to limit model training costs.
To stop a job, Amazon SageMaker sends the algorithm the SIGTERM signal. This delays job termination for 120 seconds. Algorithms might use this 120-second window to save the model artifacts.
When Amazon SageMaker terminates a job because the stopping condition has been met, training algorithms provided by Amazon SageMaker save the intermediate results of the job.
The maximum length of time, in seconds, that the training or compilation job can run. If the job does not complete during this time, Amazon SageMaker ends the job. If value is not specified, default value is 1 day. Maximum value is 5 days.
Isolates the training container. No inbound or outbound network calls can be made, except for calls between peers within a training cluster for distributed training. If network isolation is used for training jobs that are configured to use a VPC, Amazon SageMaker downloads and uploads customer data and model artifacts through the specifed VPC, but the training container does not have network access.
Note
The Semantic Segmentation built-in algorithm does not support network isolation.
The status of the tuning job: InProgress, Completed, Failed, Stopping, or Stopped.
The date and time that the tuning job started.
The date and time that the tuning job ended.
The date and time that the status of the tuning job was modified.
The TrainingJobStatusCounters object that specifies the number of training jobs, categorized by status, that this tuning job launched.
The number of completed training jobs launched by the hyperparameter tuning job.
The number of in-progress training jobs launched by a hyperparameter tuning job.
The number of training jobs that failed, but can be retried. A failed training job can be retried only if it failed because an internal service error occurred.
The number of training jobs that failed and can't be retried. A failed training job can't be retried if it failed because a client error occurred.
The number of training jobs launched by a hyperparameter tuning job that were manually stopped.
The ObjectiveStatusCounters object that specifies the number of training jobs, categorized by the status of their final objective metric, that this tuning job launched.
The number of training jobs whose final objective metric was evaluated by the hyperparameter tuning job and used in the hyperparameter tuning process.
The number of training jobs that are in progress and pending evaluation of their final objective metric.
The number of training jobs whose final objective metric was not evaluated and used in the hyperparameter tuning process. This typically occurs when the training job failed or did not emit an objective metric.
A TrainingJobSummary object that describes the training job that completed with the best current HyperParameterTuningJobObjective .
The name of the training job.
The Amazon Resource Name (ARN) of the training job.
The name of the hyperparameter tuning job that launched this training job.
The date and time that the training job was created.
The date and time that the training job started.
The date and time that the training job ended.
The status of the training job.
A list of the hyperparameters for which you specified ranges to search.
The reason that the training job failed.
The FinalHyperParameterTuningJobObjectiveMetric object that specifies the value of the objective metric of the tuning job that launched this training job.
Whether to minimize or maximize the objective metric. Valid values are Minimize and Maximize.
The name of the objective metric.
The value of the objective metric.
The status of the objective metric for the training job:
If the hyperparameter tuning job is an warm start tuning job with a WarmStartType of IDENTICAL_DATA_AND_ALGORITHM , this is the TrainingJobSummary for the training job with the best objective metric value of all training jobs launched by this tuning job and all parent jobs specified for the warm start tuning job.
The name of the training job.
The Amazon Resource Name (ARN) of the training job.
The name of the hyperparameter tuning job that launched this training job.
The date and time that the training job was created.
The date and time that the training job started.
The date and time that the training job ended.
The status of the training job.
A list of the hyperparameters for which you specified ranges to search.
The reason that the training job failed.
The FinalHyperParameterTuningJobObjectiveMetric object that specifies the value of the objective metric of the tuning job that launched this training job.
Whether to minimize or maximize the objective metric. Valid values are Minimize and Maximize.
The name of the objective metric.
The value of the objective metric.
The status of the objective metric for the training job:
The configuration for starting the hyperparameter parameter tuning job using one or more previous tuning jobs as a starting point. The results of previous tuning jobs are used to inform which combinations of hyperparameters to search over in the new tuning job.
An array of hyperparameter tuning jobs that are used as the starting point for the new hyperparameter tuning job. For more information about warm starting a hyperparameter tuning job, see Using a Previous Hyperparameter Tuning Job as a Starting Point .
Hyperparameter tuning jobs created before October 1, 2018 cannot be used as parent jobs for warm start tuning jobs.
A previously completed or stopped hyperparameter tuning job to be used as a starting point for a new hyperparameter tuning job.
The name of the hyperparameter tuning job to be used as a starting point for a new hyperparameter tuning job.
Specifies one of the following:
IDENTICAL_DATA_AND_ALGORITHM
The new hyperparameter tuning job uses the same input data and training image as the parent tuning jobs. You can change the hyperparameter ranges to search and the maximum number of training jobs that the hyperparameter tuning job launches. You cannot use a new version of the training algorithm, unless the changes in the new version do not affect the algorithm itself. For example, changes that improve logging or adding support for a different data format are allowed. You can also change hyperparameters from tunable to static, and from static to tunable, but the total number of static plus tunable hyperparameters must remain the same as it is in all parent jobs. The objective metric for the new tuning job must be the same as for all parent jobs.
TRANSFER_LEARNING
The new hyperparameter tuning job can include input data, hyperparameter ranges, maximum number of concurrent training jobs, and maximum number of training jobs that are different than those of its parent hyperparameter tuning jobs. The training image can also be a different version from the version used in the parent hyperparameter tuning job. You can also change hyperparameters from tunable to static, and from static to tunable, but the total number of static plus tunable hyperparameters must remain the same as it is in all parent jobs. The objective metric for the new tuning job must be the same as for all parent jobs.
If the tuning job failed, the reason it failed.
Gets information about a labeling job.
See also: AWS API Documentation
Request Syntax
response = client.describe_labeling_job(
LabelingJobName='string'
)
[REQUIRED]
The name of the labeling job to return information for.
{
'LabelingJobStatus': 'InProgress'|'Completed'|'Failed'|'Stopping'|'Stopped',
'LabelCounters': {
'TotalLabeled': 123,
'HumanLabeled': 123,
'MachineLabeled': 123,
'FailedNonRetryableError': 123,
'Unlabeled': 123
},
'FailureReason': 'string',
'CreationTime': datetime(2015, 1, 1),
'LastModifiedTime': datetime(2015, 1, 1),
'JobReferenceCode': 'string',
'LabelingJobName': 'string',
'LabelingJobArn': 'string',
'LabelAttributeName': 'string',
'InputConfig': {
'DataSource': {
'S3DataSource': {
'ManifestS3Uri': 'string'
}
},
'DataAttributes': {
'ContentClassifiers': [
'FreeOfPersonallyIdentifiableInformation'|'FreeOfAdultContent',
]
}
},
'OutputConfig': {
'S3OutputPath': 'string',
'KmsKeyId': 'string'
},
'RoleArn': 'string',
'LabelCategoryConfigS3Uri': 'string',
'StoppingConditions': {
'MaxHumanLabeledObjectCount': 123,
'MaxPercentageOfInputDatasetLabeled': 123
},
'LabelingJobAlgorithmsConfig': {
'LabelingJobAlgorithmSpecificationArn': 'string',
'InitialActiveLearningModelArn': 'string',
'LabelingJobResourceConfig': {
'VolumeKmsKeyId': 'string'
}
},
'HumanTaskConfig': {
'WorkteamArn': 'string',
'UiConfig': {
'UiTemplateS3Uri': 'string'
},
'PreHumanTaskLambdaArn': 'string',
'TaskKeywords': [
'string',
],
'TaskTitle': 'string',
'TaskDescription': 'string',
'NumberOfHumanWorkersPerDataObject': 123,
'TaskTimeLimitInSeconds': 123,
'TaskAvailabilityLifetimeInSeconds': 123,
'MaxConcurrentTaskCount': 123,
'AnnotationConsolidationConfig': {
'AnnotationConsolidationLambdaArn': 'string'
},
'PublicWorkforceTaskPrice': {
'AmountInUsd': {
'Dollars': 123,
'Cents': 123,
'TenthFractionsOfACent': 123
}
}
},
'Tags': [
{
'Key': 'string',
'Value': 'string'
},
],
'LabelingJobOutput': {
'OutputDatasetS3Uri': 'string',
'FinalActiveLearningModelArn': 'string'
}
}
Response Structure
The processing status of the labeling job.
Provides a breakdown of the number of data objects labeled by humans, the number of objects labeled by machine, the number of objects than couldn't be labeled, and the total number of objects labeled.
The total number of objects labeled.
The total number of objects labeled by a human worker.
The total number of objects labeled by automated data labeling.
The total number of objects that could not be labeled due to an error.
The total number of objects not yet labeled.
If the job failed, the reason that it failed.
The date and time that the labeling job was created.
The date and time that the labeling job was last updated.
A unique identifier for work done as part of a labeling job.
The name assigned to the labeling job when it was created.
The Amazon Resource Name (ARN) of the labeling job.
The attribute used as the label in the output manifest file.
Input configuration information for the labeling job, such as the Amazon S3 location of the data objects and the location of the manifest file that describes the data objects.
The location of the input data.
The Amazon S3 location of the input data objects.
The Amazon S3 location of the manifest file that describes the input data objects.
Attributes of the data specified by the customer.
Declares that your content is free of personally identifiable information or adult content. Amazon SageMaker may restrict the Amazon Mechanical Turk workers that can view your task based on this information.
The location of the job's output data and the AWS Key Management Service key ID for the key used to encrypt the output data, if any.
The Amazon S3 location to write output data.
The AWS Key Management Service ID of the key used to encrypt the output data, if any.
The Amazon Resource Name (ARN) that Amazon SageMaker assumes to perform tasks on your behalf during data labeling.
The S3 location of the JSON file that defines the categories used to label data objects.
A set of conditions for stopping a labeling job. If any of the conditions are met, the job is automatically stopped.
The maximum number of objects that can be labeled by human workers.
The maximum number of input data objects that should be labeled.
Configuration information for automated data labeling.
Specifies the Amazon Resource Name (ARN) of the algorithm used for auto-labeling. You must select one of the following ARNs:
At the end of an auto-label job Amazon SageMaker Ground Truth sends the Amazon Resource Nam (ARN) of the final model used for auto-labeling. You can use this model as the starting point for subsequent similar jobs by providing the ARN of the model here.
Provides configuration information for a labeling job.
The AWS Key Management Service key ID for the key used to encrypt the output data, if any.
Configuration information required for human workers to complete a labeling task.
The Amazon Resource Name (ARN) of the work team assigned to complete the tasks.
Information about the user interface that workers use to complete the labeling task.
The Amazon S3 bucket location of the UI template.
The Amazon Resource Name (ARN) of a Lambda function that is run before a data object is sent to a human worker. Use this function to provide input to a custom labeling job.
Keywords used to describe the task so that workers on Amazon Mechanical Turk can discover the task.
A title for the task for your human workers.
A description of the task for your human workers.
The number of human workers that will label an object.
The amount of time that a worker has to complete a task.
The length of time that a task remains available for labelling by human workers.
Defines the maximum number of data objects that can be labeled by human workers at the same time. Each object may have more than one worker at one time.
Configures how labels are consolidated across human workers.
The Amazon Resource Name (ARN) of a Lambda function implements the logic for annotation consolidation.
Amazon SageMaker Ground Truth provides three annotation consolidation functions that you can choose to use. They are:
For more information, see Annotation Consolidation .
The price that you pay for each task performed by a public worker.
Defines the amount of money paid to a worker in United States dollars.
The whole number of dollars in the amount.
The fractional portion, in cents, of the amount.
Fractions of a cent, in tenths.
An array of key/value pairs. For more information, see Using Cost Allocation Tags in the AWS Billing and Cost Management User Guide .
Describes a tag.
The tag key.
The tag value.
The location of the output produced by the labeling job.
The Amazon S3 bucket location of the manifest file for labeled data.
The Amazon Resource Name (ARN) for the most recent Amazon SageMaker model trained as part of automated data labeling.
Describes a model that you created using the CreateModel API.
See also: AWS API Documentation
Request Syntax
response = client.describe_model(
ModelName='string'
)
[REQUIRED]
The name of the model.
{
'ModelName': 'string',
'PrimaryContainer': {
'ContainerHostname': 'string',
'Image': 'string',
'ModelDataUrl': 'string',
'Environment': {
'string': 'string'
},
'ModelPackageName': 'string'
},
'Containers': [
{
'ContainerHostname': 'string',
'Image': 'string',
'ModelDataUrl': 'string',
'Environment': {
'string': 'string'
},
'ModelPackageName': 'string'
},
],
'ExecutionRoleArn': 'string',
'VpcConfig': {
'SecurityGroupIds': [
'string',
],
'Subnets': [
'string',
]
},
'CreationTime': datetime(2015, 1, 1),
'ModelArn': 'string',
'EnableNetworkIsolation': True|False
}
Response Structure
Name of the Amazon SageMaker model.
The location of the primary inference code, associated artifacts, and custom environment map that the inference code uses when it is deployed in production.
The DNS host name for the container after Amazon SageMaker deploys it.
The Amazon EC2 Container Registry (Amazon ECR) path where inference code is stored. If you are using your own custom algorithm instead of an algorithm provided by Amazon SageMaker, the inference code must meet Amazon SageMaker requirements. Amazon SageMaker supports both registry/repository[:tag] and registry/repository[@digest] image path formats. For more information, see Using Your Own Algorithms with Amazon SageMaker
The S3 path where the model artifacts, which result from model training, are stored. This path must point to a single gzip compressed tar archive (.tar.gz suffix).
If you provide a value for this parameter, Amazon SageMaker uses AWS Security Token Service to download model artifacts from the S3 path you provide. AWS STS is activated in your IAM user account by default. If you previously deactivated AWS STS for a region, you need to reactivate AWS STS for that region. For more information, see Activating and Deactivating AWS STS i an AWS Region in the AWS Identity and Access Management User Guide .
The environment variables to set in the Docker container. Each key and value in the Environment string to string map can have length of up to 1024. We support up to 16 entries in the map.
The name of the model package in this container.
The containers in the inference pipeline.
Describes the container, as part of model definition.
The DNS host name for the container after Amazon SageMaker deploys it.
The Amazon EC2 Container Registry (Amazon ECR) path where inference code is stored. If you are using your own custom algorithm instead of an algorithm provided by Amazon SageMaker, the inference code must meet Amazon SageMaker requirements. Amazon SageMaker supports both registry/repository[:tag] and registry/repository[@digest] image path formats. For more information, see Using Your Own Algorithms with Amazon SageMaker
The S3 path where the model artifacts, which result from model training, are stored. This path must point to a single gzip compressed tar archive (.tar.gz suffix).
If you provide a value for this parameter, Amazon SageMaker uses AWS Security Token Service to download model artifacts from the S3 path you provide. AWS STS is activated in your IAM user account by default. If you previously deactivated AWS STS for a region, you need to reactivate AWS STS for that region. For more information, see Activating and Deactivating AWS STS i an AWS Region in the AWS Identity and Access Management User Guide .
The environment variables to set in the Docker container. Each key and value in the Environment string to string map can have length of up to 1024. We support up to 16 entries in the map.
The name of the model package in this container.
The Amazon Resource Name (ARN) of the IAM role that you specified for the model.
A VpcConfig object that specifies the VPC that this model has access to. For more information, see Protect Endpoints by Using an Amazon Virtual Private Cloud
The VPC security group IDs, in the form sg-xxxxxxxx. Specify the security groups for the VPC that is specified in the Subnets field.
The ID of the subnets in the VPC to which you want to connect your training job or model.
A timestamp that shows when the model was created.
The Amazon Resource Name (ARN) of the model.
If True , no inbound or outbound network calls can be made to or from the model container.
Note
The Semantic Segmentation built-in algorithm does not support network isolation.
Returns a description of the specified model package, which is used to create Amazon SageMaker models or list on AWS Marketplace.
Buyers can subscribe to model packages listed on AWS Marketplace to create models in Amazon SageMaker.
See also: AWS API Documentation
Request Syntax
response = client.describe_model_package(
ModelPackageName='string'
)
[REQUIRED]
The name of the model package to describe.
{
'ModelPackageName': 'string',
'ModelPackageArn': 'string',
'ModelPackageDescription': 'string',
'CreationTime': datetime(2015, 1, 1),
'InferenceSpecification': {
'Containers': [
{
'ContainerHostname': 'string',
'Image': 'string',
'ImageDigest': 'string',
'ModelDataUrl': 'string',
'ProductId': 'string'
},
],
'SupportedTransformInstanceTypes': [
'ml.m4.xlarge'|'ml.m4.2xlarge'|'ml.m4.4xlarge'|'ml.m4.10xlarge'|'ml.m4.16xlarge'|'ml.c4.xlarge'|'ml.c4.2xlarge'|'ml.c4.4xlarge'|'ml.c4.8xlarge'|'ml.p2.xlarge'|'ml.p2.8xlarge'|'ml.p2.16xlarge'|'ml.p3.2xlarge'|'ml.p3.8xlarge'|'ml.p3.16xlarge'|'ml.c5.xlarge'|'ml.c5.2xlarge'|'ml.c5.4xlarge'|'ml.c5.9xlarge'|'ml.c5.18xlarge'|'ml.m5.large'|'ml.m5.xlarge'|'ml.m5.2xlarge'|'ml.m5.4xlarge'|'ml.m5.12xlarge'|'ml.m5.24xlarge',
],
'SupportedRealtimeInferenceInstanceTypes': [
'ml.t2.medium'|'ml.t2.large'|'ml.t2.xlarge'|'ml.t2.2xlarge'|'ml.m4.xlarge'|'ml.m4.2xlarge'|'ml.m4.4xlarge'|'ml.m4.10xlarge'|'ml.m4.16xlarge'|'ml.m5.large'|'ml.m5.xlarge'|'ml.m5.2xlarge'|'ml.m5.4xlarge'|'ml.m5.12xlarge'|'ml.m5.24xlarge'|'ml.c4.large'|'ml.c4.xlarge'|'ml.c4.2xlarge'|'ml.c4.4xlarge'|'ml.c4.8xlarge'|'ml.p2.xlarge'|'ml.p2.8xlarge'|'ml.p2.16xlarge'|'ml.p3.2xlarge'|'ml.p3.8xlarge'|'ml.p3.16xlarge'|'ml.c5.large'|'ml.c5.xlarge'|'ml.c5.2xlarge'|'ml.c5.4xlarge'|'ml.c5.9xlarge'|'ml.c5.18xlarge',
],
'SupportedContentTypes': [
'string',
],
'SupportedResponseMIMETypes': [
'string',
]
},
'SourceAlgorithmSpecification': {
'SourceAlgorithms': [
{
'ModelDataUrl': 'string',
'AlgorithmName': 'string'
},
]
},
'ValidationSpecification': {
'ValidationRole': 'string',
'ValidationProfiles': [
{
'ProfileName': 'string',
'TransformJobDefinition': {
'MaxConcurrentTransforms': 123,
'MaxPayloadInMB': 123,
'BatchStrategy': 'MultiRecord'|'SingleRecord',
'Environment': {
'string': 'string'
},
'TransformInput': {
'DataSource': {
'S3DataSource': {
'S3DataType': 'ManifestFile'|'S3Prefix'|'AugmentedManifestFile',
'S3Uri': 'string'
}
},
'ContentType': 'string',
'CompressionType': 'None'|'Gzip',
'SplitType': 'None'|'Line'|'RecordIO'
},
'TransformOutput': {
'S3OutputPath': 'string',
'Accept': 'string',
'AssembleWith': 'None'|'Line',
'KmsKeyId': 'string'
},
'TransformResources': {
'InstanceType': 'ml.m4.xlarge'|'ml.m4.2xlarge'|'ml.m4.4xlarge'|'ml.m4.10xlarge'|'ml.m4.16xlarge'|'ml.c4.xlarge'|'ml.c4.2xlarge'|'ml.c4.4xlarge'|'ml.c4.8xlarge'|'ml.p2.xlarge'|'ml.p2.8xlarge'|'ml.p2.16xlarge'|'ml.p3.2xlarge'|'ml.p3.8xlarge'|'ml.p3.16xlarge'|'ml.c5.xlarge'|'ml.c5.2xlarge'|'ml.c5.4xlarge'|'ml.c5.9xlarge'|'ml.c5.18xlarge'|'ml.m5.large'|'ml.m5.xlarge'|'ml.m5.2xlarge'|'ml.m5.4xlarge'|'ml.m5.12xlarge'|'ml.m5.24xlarge',
'InstanceCount': 123,
'VolumeKmsKeyId': 'string'
}
}
},
]
},
'ModelPackageStatus': 'Pending'|'InProgress'|'Completed'|'Failed'|'Deleting',
'ModelPackageStatusDetails': {
'ValidationStatuses': [
{
'Name': 'string',
'Status': 'NotStarted'|'InProgress'|'Completed'|'Failed',
'FailureReason': 'string'
},
],
'ImageScanStatuses': [
{
'Name': 'string',
'Status': 'NotStarted'|'InProgress'|'Completed'|'Failed',
'FailureReason': 'string'
},
]
},
'CertifyForMarketplace': True|False
}
Response Structure
The name of the model package being described.
The Amazon Resource Name (ARN) of the model package.
A brief summary about the model package.
A timestamp specifying when the model package was created.
Details about inference jobs that can be run with models based on this model package.
The Amazon ECR registry path of the Docker image that contains the inference code.
Describes the Docker container for the model package.
The DNS host name for the Docker container.
The Amazon EC2 Container Registry path where inference code is stored.
If you are using your own custom algorithm instead of an algorithm provided by Amazon SageMaker, the inference code must meet Amazon SageMaker requirements. Amazon SageMaker supports both registry/repository[:tag] and registry/repository[@digest] image path formats. For more information, see Using Your Own Algorithms with Amazon SageMaker .
An MD5 hash of the training algorithm that identifies the Docker image used for training.
The Amazon S3 path where the model artifacts, which result from model training, are stored. This path must point to a single gzip compressed tar archive (.tar.gz suffix).
The ID of the model package.
A list of the instance types on which a transformation job can be run or on which an endpoint can be deployed.
A list of the instance types that are used to generate inferences in real-time.
The supported MIME types for the input data.
The supported MIME types for the output data.
Details about the algorithm that was used to create the model package.
A list of the algorithms that were used to create a model package.
Specifies an algorithm that was used to create the model package. The algorithm must be either an algorithm resource in your Amazon SageMaker account or an algorithm in AWS Marketplace that you are subscribed to.
The Amazon S3 path where the model artifacts, which result from model training, are stored. This path must point to a single gzip compressed tar archive (.tar.gz suffix).
The name of an algorithm that was used to create the model package. The algorithm must be either an algorithm resource in your Amazon SageMaker account or an algorithm in AWS Marketplace that you are subscribed to.
Configurations for one or more transform jobs that Amazon SageMaker runs to test the model package.
The IAM roles to be used for the validation of a model package.
An array of ModelPackageValidationProfile objects, each of which specifies a batch transform job that Amazon SageMaker runs to validate your model package.
Contains data such as the inputs and targeted instance types that are used in the process of validating the model package.
The data provided in the validation profile is made available to your buyers on AWS Marketplace.
The name of the profile for the model package.
The TransformJobDefinition object that describes the transform job used for the validation of the model package.
The maximum number of parallel requests that can be sent to each instance in a transform job. The default value is 1.
The maximum payload size allowed, in MB. A payload is the data portion of a record (without metadata).
A string that determines the number of records included in a single mini-batch.
SingleRecord means only one record is used per mini-batch. MultiRecord means a mini-batch is set to contain as many records that can fit within the MaxPayloadInMB limit.
The environment variables to set in the Docker container. We support up to 16 key and values entries in the map.
A description of the input source and the way the transform job consumes it.
Describes the location of the channel data, meaning the S3 location of the input data that the model can consume.
The S3 location of the data source that is associated with a channel.
If you choose S3Prefix , S3Uri identifies a key name prefix. Amazon SageMaker uses all objects with the specified key name prefix for batch transform.
If you choose ManifestFile , S3Uri identifies an object that is a manifest file containing a list of object keys that you want Amazon SageMaker to use for batch transform.
Depending on the value specified for the S3DataType , identifies either a key name prefix or a manifest. For example:
The multipurpose internet mail extension (MIME) type of the data. Amazon SageMaker uses the MIME type with each http call to transfer data to the transform job.
Compressing data helps save on storage space. If your transform data is compressed, specify the compression type. Amazon SageMaker automatically decompresses the data for the transform job accordingly. The default value is None .
The method to use to split the transform job's data into smaller batches. The default value is None . If you don't want to split the data, specify None . If you want to split records on a newline character boundary, specify Line . To split records according to the RecordIO format, specify RecordIO .
Amazon SageMaker will send maximum number of records per batch in each request up to the MaxPayloadInMB limit. For more information, see RecordIO data format .
Note
For information about the RecordIO format, see Data Format .
Identifies the Amazon S3 location where you want Amazon SageMaker to save the results from the transform job.
The Amazon S3 path where you want Amazon SageMaker to store the results of the transform job. For example, s3://bucket-name/key-name-prefix .
For every S3 object used as input for the transform job, the transformed data is stored in a corresponding subfolder in the location under the output prefix. For example, the input data s3://bucket-name/input-name-prefix/dataset01/data.csv will have the transformed data stored at s3://bucket-name/key-name-prefix/dataset01/ , based on the original name, as a series of .part files (.part0001, part0002, etc).
The MIME type used to specify the output data. Amazon SageMaker uses the MIME type with each http call to transfer data from the transform job.
Defines how to assemble the results of the transform job as a single S3 object. You should select a format that is most convenient to you. To concatenate the results in binary format, specify None . To add a newline character at the end of every transformed record, specify Line .
The AWS Key Management Service (AWS KMS) key that Amazon SageMaker uses to encrypt the model artifacts at rest using Amazon S3 server-side encryption. The KmsKeyId can be any of the following formats:
If you don't provide a KMS key ID, Amazon SageMaker uses the default KMS key for Amazon S3 for your role's account. For more information, see KMS-Managed Encryption Keys in the Amazon Simple Storage Service Developer Guide.
The KMS key policy must grant permission to the IAM role that you specify in your CreateTramsformJob request. For more information, see Using Key Policies in AWS KMS in the AWS Key Management Service Developer Guide .
Identifies the ML compute instances for the transform job.
The ML compute instance type for the transform job. For using built-in algorithms to transform moderately sized datasets, ml.m4.xlarge or ml.m5.large should suffice. There is no default value for InstanceType .
The number of ML compute instances to use in the transform job. For distributed transform, provide a value greater than 1. The default value is 1 .
The AWS Key Management Service (AWS KMS) key that Amazon SageMaker uses to encrypt data on the storage volume attached to the ML compute instance(s) that run the batch transform job. The VolumeKmsKeyId can be any of the following formats:
The current status of the model package.
Details about the current status of the model package.
The status of the validation of the model package.
Represents the overall status of a model package.
The name of the model package for which the overall status is being repoorted.
The current status.
The reason for failure, if the overall status is a failed state.
The status of the scan of the Docker image container for the model package.
Represents the overall status of a model package.
The name of the model package for which the overall status is being repoorted.
The current status.
The reason for failure, if the overall status is a failed state.
Whether the model package is certified for listing on AWS Marketplace.
Returns information about a notebook instance.
See also: AWS API Documentation
Request Syntax
response = client.describe_notebook_instance(
NotebookInstanceName='string'
)
[REQUIRED]
The name of the notebook instance that you want information about.
{
'NotebookInstanceArn': 'string',
'NotebookInstanceName': 'string',
'NotebookInstanceStatus': 'Pending'|'InService'|'Stopping'|'Stopped'|'Failed'|'Deleting'|'Updating',
'FailureReason': 'string',
'Url': 'string',
'InstanceType': 'ml.t2.medium'|'ml.t2.large'|'ml.t2.xlarge'|'ml.t2.2xlarge'|'ml.t3.medium'|'ml.t3.large'|'ml.t3.xlarge'|'ml.t3.2xlarge'|'ml.m4.xlarge'|'ml.m4.2xlarge'|'ml.m4.4xlarge'|'ml.m4.10xlarge'|'ml.m4.16xlarge'|'ml.m5.xlarge'|'ml.m5.2xlarge'|'ml.m5.4xlarge'|'ml.m5.12xlarge'|'ml.m5.24xlarge'|'ml.c4.xlarge'|'ml.c4.2xlarge'|'ml.c4.4xlarge'|'ml.c4.8xlarge'|'ml.c5.xlarge'|'ml.c5.2xlarge'|'ml.c5.4xlarge'|'ml.c5.9xlarge'|'ml.c5.18xlarge'|'ml.c5d.xlarge'|'ml.c5d.2xlarge'|'ml.c5d.4xlarge'|'ml.c5d.9xlarge'|'ml.c5d.18xlarge'|'ml.p2.xlarge'|'ml.p2.8xlarge'|'ml.p2.16xlarge'|'ml.p3.2xlarge'|'ml.p3.8xlarge'|'ml.p3.16xlarge',
'SubnetId': 'string',
'SecurityGroups': [
'string',
],
'RoleArn': 'string',
'KmsKeyId': 'string',
'NetworkInterfaceId': 'string',
'LastModifiedTime': datetime(2015, 1, 1),
'CreationTime': datetime(2015, 1, 1),
'NotebookInstanceLifecycleConfigName': 'string',
'DirectInternetAccess': 'Enabled'|'Disabled',
'VolumeSizeInGB': 123,
'AcceleratorTypes': [
'ml.eia1.medium'|'ml.eia1.large'|'ml.eia1.xlarge',
],
'DefaultCodeRepository': 'string',
'AdditionalCodeRepositories': [
'string',
]
}
Response Structure
The Amazon Resource Name (ARN) of the notebook instance.
Name of the Amazon SageMaker notebook instance.
The status of the notebook instance.
If status is failed, the reason it failed.
The URL that you use to connect to the Jupyter notebook that is running in your notebook instance.
The type of ML compute instance running on the notebook instance.
The ID of the VPC subnet.
The IDs of the VPC security groups.
Amazon Resource Name (ARN) of the IAM role associated with the instance.
AWS KMS key ID Amazon SageMaker uses to encrypt data when storing it on the ML storage volume attached to the instance.
Network interface IDs that Amazon SageMaker created at the time of creating the instance.
A timestamp. Use this parameter to retrieve the time when the notebook instance was last modified.
A timestamp. Use this parameter to return the time when the notebook instance was created
Returns the name of a notebook instance lifecycle configuration.
For information about notebook instance lifestyle configurations, see Step 2.1: (Optional) Customize a Notebook Instance
Describes whether Amazon SageMaker provides internet access to the notebook instance. If this value is set to Disabled, he notebook instance does not have internet access, and cannot connect to Amazon SageMaker training and endpoint services .
For more information, see Notebook Instances Are Internet-Enabled by Default .
The size, in GB, of the ML storage volume attached to the notebook instance.
A list of the Elastic Inference (EI) instance types associated with this notebook instance. Currently only one EI instance type can be associated with a notebook instance. For more information, see Using Elastic Inference in Amazon SageMaker .
The git repository associated with the notebook instance as its default code repository. This can be either the name of a git repository stored as a resource in your account, or the URL of a git repository in AWS CodeCommit or in any other git repository. When you open a notebook instance, it opens in the directory that contains this repository. For more information, see Associating Git Repositories with Amazon SageMaker Notebook Instances .
An array of up to 3 git repositories associated with the notebook instance. These can be either the names of git repositories stored as resources in your account, or the URL of git repositories in AWS CodeCommit or in any other git repository. These repositories are cloned at the same level as the default repository of your notebook instance. For more information, see Associating Git Repositories with Amazon SageMaker Notebook Instances .
Returns a description of a notebook instance lifecycle configuration.
For information about notebook instance lifestyle configurations, see Step 2.1: (Optional) Customize a Notebook Instance .
See also: AWS API Documentation
Request Syntax
response = client.describe_notebook_instance_lifecycle_config(
NotebookInstanceLifecycleConfigName='string'
)
[REQUIRED]
The name of the lifecycle configuration to describe.
{
'NotebookInstanceLifecycleConfigArn': 'string',
'NotebookInstanceLifecycleConfigName': 'string',
'OnCreate': [
{
'Content': 'string'
},
],
'OnStart': [
{
'Content': 'string'
},
],
'LastModifiedTime': datetime(2015, 1, 1),
'CreationTime': datetime(2015, 1, 1)
}
Response Structure
The Amazon Resource Name (ARN) of the lifecycle configuration.
The name of the lifecycle configuration.
The shell script that runs only once, when you create a notebook instance.
Contains the notebook instance lifecycle configuration script.
Each lifecycle configuration script has a limit of 16384 characters.
The value of the $PATH environment variable that is available to both scripts is /sbin:bin:/usr/sbin:/usr/bin .
View CloudWatch Logs for notebook instance lifecycle configurations in log group /aws/sagemaker/NotebookInstances in log stream [notebook-instance-name]/[LifecycleConfigHook] .
Lifecycle configuration scripts cannot run for longer than 5 minutes. If a script runs for longer than 5 minutes, it fails and the notebook instance is not created or started.
For information about notebook instance lifestyle configurations, see Step 2.1: (Optional) Customize a Notebook Instance .
A base64-encoded string that contains a shell script for a notebook instance lifecycle configuration.
The shell script that runs every time you start a notebook instance, including when you create the notebook instance.
Contains the notebook instance lifecycle configuration script.
Each lifecycle configuration script has a limit of 16384 characters.
The value of the $PATH environment variable that is available to both scripts is /sbin:bin:/usr/sbin:/usr/bin .
View CloudWatch Logs for notebook instance lifecycle configurations in log group /aws/sagemaker/NotebookInstances in log stream [notebook-instance-name]/[LifecycleConfigHook] .
Lifecycle configuration scripts cannot run for longer than 5 minutes. If a script runs for longer than 5 minutes, it fails and the notebook instance is not created or started.
For information about notebook instance lifestyle configurations, see Step 2.1: (Optional) Customize a Notebook Instance .
A base64-encoded string that contains a shell script for a notebook instance lifecycle configuration.
A timestamp that tells when the lifecycle configuration was last modified.
A timestamp that tells when the lifecycle configuration was created.
Gets information about a work team provided by a vendor. It returns details about the subscription with a vendor in the AWS Marketplace.
See also: AWS API Documentation
Request Syntax
response = client.describe_subscribed_workteam(
WorkteamArn='string'
)
[REQUIRED]
The Amazon Resource Name (ARN) of the subscribed work team to describe.
{
'SubscribedWorkteam': {
'WorkteamArn': 'string',
'MarketplaceTitle': 'string',
'SellerName': 'string',
'MarketplaceDescription': 'string',
'ListingId': 'string'
}
}
Response Structure
A Workteam instance that contains information about the work team.
The Amazon Resource Name (ARN) of the vendor that you have subscribed.
The title of the service provided by the vendor in the Amazon Marketplace.
The name of the vendor in the Amazon Marketplace.
The description of the vendor from the Amazon Marketplace.
Returns information about a training job.
See also: AWS API Documentation
Request Syntax
response = client.describe_training_job(
TrainingJobName='string'
)
[REQUIRED]
The name of the training job.
{
'TrainingJobName': 'string',
'TrainingJobArn': 'string',
'TuningJobArn': 'string',
'LabelingJobArn': 'string',
'ModelArtifacts': {
'S3ModelArtifacts': 'string'
},
'TrainingJobStatus': 'InProgress'|'Completed'|'Failed'|'Stopping'|'Stopped',
'SecondaryStatus': 'Starting'|'LaunchingMLInstances'|'PreparingTrainingStack'|'Downloading'|'DownloadingTrainingImage'|'Training'|'Uploading'|'Stopping'|'Stopped'|'MaxRuntimeExceeded'|'Completed'|'Failed',
'FailureReason': 'string',
'HyperParameters': {
'string': 'string'
},
'AlgorithmSpecification': {
'TrainingImage': 'string',
'AlgorithmName': 'string',
'TrainingInputMode': 'Pipe'|'File',
'MetricDefinitions': [
{
'Name': 'string',
'Regex': 'string'
},
]
},
'RoleArn': 'string',
'InputDataConfig': [
{
'ChannelName': 'string',
'DataSource': {
'S3DataSource': {
'S3DataType': 'ManifestFile'|'S3Prefix'|'AugmentedManifestFile',
'S3Uri': 'string',
'S3DataDistributionType': 'FullyReplicated'|'ShardedByS3Key',
'AttributeNames': [
'string',
]
}
},
'ContentType': 'string',
'CompressionType': 'None'|'Gzip',
'RecordWrapperType': 'None'|'RecordIO',
'InputMode': 'Pipe'|'File',
'ShuffleConfig': {
'Seed': 123
}
},
],
'OutputDataConfig': {
'KmsKeyId': 'string',
'S3OutputPath': 'string'
},
'ResourceConfig': {
'InstanceType': 'ml.m4.xlarge'|'ml.m4.2xlarge'|'ml.m4.4xlarge'|'ml.m4.10xlarge'|'ml.m4.16xlarge'|'ml.m5.large'|'ml.m5.xlarge'|'ml.m5.2xlarge'|'ml.m5.4xlarge'|'ml.m5.12xlarge'|'ml.m5.24xlarge'|'ml.c4.xlarge'|'ml.c4.2xlarge'|'ml.c4.4xlarge'|'ml.c4.8xlarge'|'ml.p2.xlarge'|'ml.p2.8xlarge'|'ml.p2.16xlarge'|'ml.p3.2xlarge'|'ml.p3.8xlarge'|'ml.p3.16xlarge'|'ml.c5.xlarge'|'ml.c5.2xlarge'|'ml.c5.4xlarge'|'ml.c5.9xlarge'|'ml.c5.18xlarge',
'InstanceCount': 123,
'VolumeSizeInGB': 123,
'VolumeKmsKeyId': 'string'
},
'VpcConfig': {
'SecurityGroupIds': [
'string',
],
'Subnets': [
'string',
]
},
'StoppingCondition': {
'MaxRuntimeInSeconds': 123
},
'CreationTime': datetime(2015, 1, 1),
'TrainingStartTime': datetime(2015, 1, 1),
'TrainingEndTime': datetime(2015, 1, 1),
'LastModifiedTime': datetime(2015, 1, 1),
'SecondaryStatusTransitions': [
{
'Status': 'Starting'|'LaunchingMLInstances'|'PreparingTrainingStack'|'Downloading'|'DownloadingTrainingImage'|'Training'|'Uploading'|'Stopping'|'Stopped'|'MaxRuntimeExceeded'|'Completed'|'Failed',
'StartTime': datetime(2015, 1, 1),
'EndTime': datetime(2015, 1, 1),
'StatusMessage': 'string'
},
],
'FinalMetricDataList': [
{
'MetricName': 'string',
'Value': ...,
'Timestamp': datetime(2015, 1, 1)
},
],
'EnableNetworkIsolation': True|False
}
Response Structure
Name of the model training job.
The Amazon Resource Name (ARN) of the training job.
The Amazon Resource Name (ARN) of the associated hyperparameter tuning job if the training job was launched by a hyperparameter tuning job.
The Amazon Resource Name (ARN) of the Amazon SageMaker Ground Truth labeling job that created the transform or training job.
Information about the Amazon S3 location that is configured for storing model artifacts.
The path of the S3 object that contains the model artifacts. For example, s3://bucket-name/keynameprefix/model.tar.gz .
The status of the training job.
Amazon SageMaker provides the following training job statuses:
For more detailed information, see SecondaryStatus .
Provides detailed information about the state of the training job. For detailed information on the secondary status of the training job, see StatusMessage under SecondaryStatusTransition .
Amazon SageMaker provides primary statuses and secondary statuses that apply to each of them:
InProgress
Completed
Failed
Stopped
Stopping
Warning
Valid values for SecondaryStatus are subject to change.
We no longer support the following secondary statuses:
If the training job failed, the reason it failed.
Algorithm-specific parameters.
Information about the algorithm used for training, and algorithm metadata.
The registry path of the Docker image that contains the training algorithm. For information about docker registry paths for built-in algorithms, see Algorithms Provided by Amazon SageMaker: Common Parameters .
The name of the algorithm resource to use for the training job. This must be an algorithm resource that you created or subscribe to on AWS Marketplace. If you specify a value for this parameter, you can't specify a value for TrainingImage .
The input mode that the algorithm supports. For the input modes that Amazon SageMaker algorithms support, see Algorithms . If an algorithm supports the File input mode, Amazon SageMaker downloads the training data from S3 to the provisioned ML storage Volume, and mounts the directory to docker volume for training container. If an algorithm supports the Pipe input mode, Amazon SageMaker streams data directly from S3 to the container.
In File mode, make sure you provision ML storage volume with sufficient capacity to accommodate the data download from S3. In addition to the training data, the ML storage volume also stores the output model. The algorithm container use ML storage volume to also store intermediate information, if any.
For distributed algorithms using File mode, training data is distributed uniformly, and your training duration is predictable if the input data objects size is approximately same. Amazon SageMaker does not split the files any further for model training. If the object sizes are skewed, training won't be optimal as the data distribution is also skewed where one host in a training cluster is overloaded, thus becoming bottleneck in training.
A list of metric definition objects. Each object specifies the metric name and regular expressions used to parse algorithm logs. Amazon SageMaker publishes each metric to Amazon CloudWatch.
Specifies a metric that the training algorithm writes to stderr or stdout . Amazon SageMakerhyperparameter tuning captures all defined metrics. You specify one metric that a hyperparameter tuning job uses as its objective metric to choose the best training job.
The name of the metric.
A regular expression that searches the output of a training job and gets the value of the metric. For more information about using regular expressions to define metrics, see Defining Objective Metrics .
The AWS Identity and Access Management (IAM) role configured for the training job.
An array of Channel objects that describes each data input channel.
A channel is a named input source that training algorithms can consume.
The name of the channel.
The location of the channel data.
The S3 location of the data source that is associated with a channel.
If you choose S3Prefix , S3Uri identifies a key name prefix. Amazon SageMaker uses all objects that match the specified key name prefix for model training.
If you choose ManifestFile , S3Uri identifies an object that is a manifest file containing a list of object keys that you want Amazon SageMaker to use for model training.
If you choose AugmentedManifestFile , S3Uri identifies an object that is an augmented manifest file in JSON lines format. This file contains the data you want to use for model training. AugmentedManifestFile can only be used if the Channel's input mode is Pipe .
Depending on the value specified for the S3DataType , identifies either a key name prefix or a manifest. For example:
If you want Amazon SageMaker to replicate the entire dataset on each ML compute instance that is launched for model training, specify FullyReplicated .
If you want Amazon SageMaker to replicate a subset of data on each ML compute instance that is launched for model training, specify ShardedByS3Key . If there are n ML compute instances launched for a training job, each instance gets approximately 1/n of the number of S3 objects. In this case, model training on each machine uses only the subset of training data.
Don't choose more ML compute instances for training than available S3 objects. If you do, some nodes won't get any data and you will pay for nodes that aren't getting any training data. This applies in both File and Pipemodes. Keep this in mind when developing algorithms.
In distributed training, where you use multiple ML compute EC2 instances, you might choose ShardedByS3Key . If the algorithm requires copying training data to the ML storage volume (when TrainingInputMode is set to File ), this copies 1/n of the number of objects.
A list of one or more attribute names to use that are found in a specified augmented manifest file.
The MIME type of the data.
If training data is compressed, the compression type. The default value is None . CompressionType is used only in Pipe input mode. In File mode, leave this field unset or set it to None.
Specify RecordIO as the value when input data is in raw format but the training algorithm requires the RecordIO format. In this case, Amazon SageMaker wraps each individual S3 object in a RecordIO record. If the input data is already in RecordIO format, you don't need to set this attribute. For more information, see Create a Dataset Using RecordIO .
In File mode, leave this field unset or set it to None.
(Optional) The input mode to use for the data channel in a training job. If you don't set a value for InputMode , Amazon SageMaker uses the value set for TrainingInputMode . Use this parameter to override the TrainingInputMode setting in a AlgorithmSpecification request when you have a channel that needs a different input mode from the training job's general setting. To download the data from Amazon Simple Storage Service (Amazon S3) to the provisioned ML storage volume, and mount the directory to a Docker volume, use File input mode. To stream data directly from Amazon S3 to the container, choose Pipe input mode.
To use a model for incremental training, choose File input model.
A configuration for a shuffle option for input data in a channel. If you use S3Prefix for S3DataType , this shuffles the results of the S3 key prefix matches. If you use ManifestFile , the order of the S3 object references in the ManifestFile is shuffled. If you use AugmentedManifestFile , the order of the JSON lines in the AugmentedManifestFile is shuffled. The shuffling order is determined using the Seed value.
For Pipe input mode, shuffling is done at the start of every epoch. With large datasets this ensures that the order of the training data is different for each epoch, it helps reduce bias and possible overfitting. In a multi-node training job when ShuffleConfig is combined with S3DataDistributionType of ShardedByS3Key , the data is shuffled across nodes so that the content sent to a particular node on the first epoch might be sent to a different node on the second epoch.
Determines the shuffling order in ShuffleConfig . value.
The S3 path where model artifacts that you configured when creating the job are stored. Amazon SageMaker creates subfolders for model artifacts.
The AWS Key Management Service (AWS KMS) key that Amazon SageMaker uses to encrypt the model artifacts at rest using Amazon S3 server-side encryption. The KmsKeyId can be any of the following formats:
If you don't provide the KMS key ID, Amazon SageMaker uses the default KMS key for Amazon S3 for your role's account. For more information, see KMS-Managed Encryption Keys in Amazon Simple Storage Service Developer Guide.
Note
The KMS key policy must grant permission to the IAM role that you specify in your CreateTrainingJob request. Using Key Policies in AWS KMS in the AWS Key Management Service Developer Guide .
Identifies the S3 path where you want Amazon SageMaker to store the model artifacts. For example, s3://bucket-name/key-name-prefix .
Resources, including ML compute instances and ML storage volumes, that are configured for model training.
The ML compute instance type.
The number of ML compute instances to use. For distributed training, provide a value greater than 1.
The size of the ML storage volume that you want to provision.
ML storage volumes store model artifacts and incremental states. Training algorithms might also use the ML storage volume for scratch space. If you want to store the training data in the ML storage volume, choose File as the TrainingInputMode in the algorithm specification.
You must specify sufficient ML storage for your scenario.
Note
Amazon SageMaker supports only the General Purpose SSD (gp2) ML storage volume type.
The AWS Key Management Service (AWS KMS) key that Amazon SageMaker uses to encrypt data on the storage volume attached to the ML compute instance(s) that run the training job. The VolumeKmsKeyId can be any of the following formats:
A VpcConfig object that specifies the VPC that this training job has access to. For more information, see Protect Training Jobs by Using an Amazon Virtual Private Cloud .
The VPC security group IDs, in the form sg-xxxxxxxx. Specify the security groups for the VPC that is specified in the Subnets field.
The ID of the subnets in the VPC to which you want to connect your training job or model.
The condition under which to stop the training job.
The maximum length of time, in seconds, that the training or compilation job can run. If the job does not complete during this time, Amazon SageMaker ends the job. If value is not specified, default value is 1 day. Maximum value is 5 days.
A timestamp that indicates when the training job was created.
Indicates the time when the training job starts on training instances. You are billed for the time interval between this time and the value of TrainingEndTime . The start time in CloudWatch Logs might be later than this time. The difference is due to the time it takes to download the training data and to the size of the training container.
Indicates the time when the training job ends on training instances. You are billed for the time interval between the value of TrainingStartTime and this time. For successful jobs and stopped jobs, this is the time after model artifacts are uploaded. For failed jobs, this is the time when Amazon SageMaker detects a job failure.
A timestamp that indicates when the status of the training job was last modified.
A history of all of the secondary statuses that the training job has transitioned through.
An array element of DescribeTrainingJobResponse$SecondaryStatusTransitions . It provides additional details about a status that the training job has transitioned through. A training job can be in one of several states, for example, starting, downloading, training, or uploading. Within each state, there are a number of intermediate states. For example, within the starting state, Amazon SageMaker could be starting the training job or launching the ML instances. These transitional states are referred to as the job's secondary status.
Contains a secondary status information from a training job.
Status might be one of the following secondary statuses:
InProgress
Completed
Failed
Stopped
Stopping
We no longer support the following secondary statuses:
A timestamp that shows when the training job transitioned to the current secondary status state.
A timestamp that shows when the training job transitioned out of this secondary status state into another secondary status state or when the training job has ended.
A detailed description of the progress within a secondary status.
Amazon SageMaker provides secondary statuses and status messages that apply to each of them:
Starting
Training
Warning
Status messages are subject to change. Therefore, we recommend not including them in code that programmatically initiates actions. For examples, don't use status messages in if statements.
To have an overview of your training job's progress, view TrainingJobStatus and SecondaryStatus in DescribeTrainingJobResponse , and StatusMessage together. For example, at the start of a training job, you might see the following:
A collection of MetricData objects that specify the names, values, and dates and times that the training algorithm emitted to Amazon CloudWatch.
The name, value, and date and time of a metric that was emitted to Amazon CloudWatch.
The name of the metric.
The value of the metric.
The date and time that the algorithm emitted the metric.
If True , inbound or outbound network calls can be made, except for calls between peers within a training cluster for distributed training. If network isolation is used for training jobs that are configured to use a VPC, Amazon SageMaker downloads and uploads customer data and model artifacts through the specifed VPC, but the training container does not have network access.
Note
The Semantic Segmentation built-in algorithm does not support network isolation.
Returns information about a transform job.
See also: AWS API Documentation
Request Syntax
response = client.describe_transform_job(
TransformJobName='string'
)
[REQUIRED]
The name of the transform job that you want to view details of.
{
'TransformJobName': 'string',
'TransformJobArn': 'string',
'TransformJobStatus': 'InProgress'|'Completed'|'Failed'|'Stopping'|'Stopped',
'FailureReason': 'string',
'ModelName': 'string',
'MaxConcurrentTransforms': 123,
'MaxPayloadInMB': 123,
'BatchStrategy': 'MultiRecord'|'SingleRecord',
'Environment': {
'string': 'string'
},
'TransformInput': {
'DataSource': {
'S3DataSource': {
'S3DataType': 'ManifestFile'|'S3Prefix'|'AugmentedManifestFile',
'S3Uri': 'string'
}
},
'ContentType': 'string',
'CompressionType': 'None'|'Gzip',
'SplitType': 'None'|'Line'|'RecordIO'
},
'TransformOutput': {
'S3OutputPath': 'string',
'Accept': 'string',
'AssembleWith': 'None'|'Line',
'KmsKeyId': 'string'
},
'TransformResources': {
'InstanceType': 'ml.m4.xlarge'|'ml.m4.2xlarge'|'ml.m4.4xlarge'|'ml.m4.10xlarge'|'ml.m4.16xlarge'|'ml.c4.xlarge'|'ml.c4.2xlarge'|'ml.c4.4xlarge'|'ml.c4.8xlarge'|'ml.p2.xlarge'|'ml.p2.8xlarge'|'ml.p2.16xlarge'|'ml.p3.2xlarge'|'ml.p3.8xlarge'|'ml.p3.16xlarge'|'ml.c5.xlarge'|'ml.c5.2xlarge'|'ml.c5.4xlarge'|'ml.c5.9xlarge'|'ml.c5.18xlarge'|'ml.m5.large'|'ml.m5.xlarge'|'ml.m5.2xlarge'|'ml.m5.4xlarge'|'ml.m5.12xlarge'|'ml.m5.24xlarge',
'InstanceCount': 123,
'VolumeKmsKeyId': 'string'
},
'CreationTime': datetime(2015, 1, 1),
'TransformStartTime': datetime(2015, 1, 1),
'TransformEndTime': datetime(2015, 1, 1),
'LabelingJobArn': 'string'
}
Response Structure
The name of the transform job.
The Amazon Resource Name (ARN) of the transform job.
The status of the transform job. If the transform job failed, the reason is returned in the FailureReason field.
If the transform job failed, the reason that it failed.
The name of the model used in the transform job.
The maximum number of parallel requests on each instance node that can be launched in a transform job. The default value is 1.
The maximum payload size , in MB used in the transform job.
SingleRecord means only one record was used per a batch. MultiRecord means batches contained as many records that could possibly fit within the MaxPayloadInMB limit.
Describes the dataset to be transformed and the Amazon S3 location where it is stored.
Describes the location of the channel data, meaning the S3 location of the input data that the model can consume.
The S3 location of the data source that is associated with a channel.
If you choose S3Prefix , S3Uri identifies a key name prefix. Amazon SageMaker uses all objects with the specified key name prefix for batch transform.
If you choose ManifestFile , S3Uri identifies an object that is a manifest file containing a list of object keys that you want Amazon SageMaker to use for batch transform.
Depending on the value specified for the S3DataType , identifies either a key name prefix or a manifest. For example:
The multipurpose internet mail extension (MIME) type of the data. Amazon SageMaker uses the MIME type with each http call to transfer data to the transform job.
Compressing data helps save on storage space. If your transform data is compressed, specify the compression type. Amazon SageMaker automatically decompresses the data for the transform job accordingly. The default value is None .
The method to use to split the transform job's data into smaller batches. The default value is None . If you don't want to split the data, specify None . If you want to split records on a newline character boundary, specify Line . To split records according to the RecordIO format, specify RecordIO .
Amazon SageMaker will send maximum number of records per batch in each request up to the MaxPayloadInMB limit. For more information, see RecordIO data format .
Note
For information about the RecordIO format, see Data Format .
Identifies the Amazon S3 location where you want Amazon SageMaker to save the results from the transform job.
The Amazon S3 path where you want Amazon SageMaker to store the results of the transform job. For example, s3://bucket-name/key-name-prefix .
For every S3 object used as input for the transform job, the transformed data is stored in a corresponding subfolder in the location under the output prefix. For example, the input data s3://bucket-name/input-name-prefix/dataset01/data.csv will have the transformed data stored at s3://bucket-name/key-name-prefix/dataset01/ , based on the original name, as a series of .part files (.part0001, part0002, etc).
The MIME type used to specify the output data. Amazon SageMaker uses the MIME type with each http call to transfer data from the transform job.
Defines how to assemble the results of the transform job as a single S3 object. You should select a format that is most convenient to you. To concatenate the results in binary format, specify None . To add a newline character at the end of every transformed record, specify Line .
The AWS Key Management Service (AWS KMS) key that Amazon SageMaker uses to encrypt the model artifacts at rest using Amazon S3 server-side encryption. The KmsKeyId can be any of the following formats:
If you don't provide a KMS key ID, Amazon SageMaker uses the default KMS key for Amazon S3 for your role's account. For more information, see KMS-Managed Encryption Keys in the Amazon Simple Storage Service Developer Guide.
The KMS key policy must grant permission to the IAM role that you specify in your CreateTramsformJob request. For more information, see Using Key Policies in AWS KMS in the AWS Key Management Service Developer Guide .
Describes the resources, including ML instance types and ML instance count, to use for the transform job.
The ML compute instance type for the transform job. For using built-in algorithms to transform moderately sized datasets, ml.m4.xlarge or ml.m5.large should suffice. There is no default value for InstanceType .
The number of ML compute instances to use in the transform job. For distributed transform, provide a value greater than 1. The default value is 1 .
The AWS Key Management Service (AWS KMS) key that Amazon SageMaker uses to encrypt data on the storage volume attached to the ML compute instance(s) that run the batch transform job. The VolumeKmsKeyId can be any of the following formats:
A timestamp that shows when the transform Job was created.
Indicates when the transform job starts on ML instances. You are billed for the time interval between this time and the value of TransformEndTime .
Indicates when the transform job is Completed , Stopped , or Failed . You are billed for the time interval between this time and the value of TransformStartTime .
The Amazon Resource Name (ARN) of the Amazon SageMaker Ground Truth labeling job that created the transform or training job.
Gets information about a specific work team. You can see information such as the create date, the last updated date, membership information, and the work team's Amazon Resource Name (ARN).
See also: AWS API Documentation
Request Syntax
response = client.describe_workteam(
WorkteamName='string'
)
[REQUIRED]
The name of the work team to return a description of.
{
'Workteam': {
'WorkteamName': 'string',
'MemberDefinitions': [
{
'CognitoMemberDefinition': {
'UserPool': 'string',
'UserGroup': 'string',
'ClientId': 'string'
}
},
],
'WorkteamArn': 'string',
'ProductListingIds': [
'string',
],
'Description': 'string',
'SubDomain': 'string',
'CreateDate': datetime(2015, 1, 1),
'LastUpdatedDate': datetime(2015, 1, 1)
}
}
Response Structure
A Workteam instance that contains information about the work team.
The name of the work team.
The Amazon Cognito user groups that make up the work team.
Defines the Amazon Cognito user group that is part of a work team.
The Amazon Cognito user group that is part of the work team.
An identifier for a user pool. The user pool must be in the same region as the service that you are calling.
An identifier for a user group.
An identifier for an application client. You must create the app client ID using Amazon Cognito.
The Amazon Resource Name (ARN) that identifies the work team.
The Amazon Marketplace identifier for a vendor's work team.
A description of the work team.
The URI of the labeling job's user interface. Workers open this URI to start labeling your data objects.
The date and time that the work team was created (timestamp).
The date and time that the work team was last updated (timestamp).
Generate a presigned url given a client, its method, and arguments
The presigned url
Create a paginator for an operation.
Returns suggestions for the property name to use in Search queries. Provides suggestions for HyperParameters , Tags , and Metrics .
See also: AWS API Documentation
Request Syntax
response = client.get_search_suggestions(
Resource='TrainingJob',
SuggestionQuery={
'PropertyNameQuery': {
'PropertyNameHint': 'string'
}
}
)
[REQUIRED]
The name of the Amazon SageMaker resource to Search for. The only valid Resource value is TrainingJob .
Limits the property names that are included in the response.
Defines a property name hint. Only property names that match the specified hint are included in the response.
The hyperparameter, metric, and tag key property names that begin with the specified hint.
dict
Response Syntax
{
'PropertyNameSuggestions': [
{
'PropertyName': 'string'
},
]
}
Response Structure
(dict) --
PropertyNameSuggestions (list) --
A list of property names for a Resource that match a SuggestionQuery .
(dict) --
A property name returned from a GetSearchSuggestions call that specifies a value in the PropertyNameQuery field.
PropertyName (string) --
A suggested property name.
Returns an object that can wait for some condition.
Lists the machine learning algorithms that have been created.
See also: AWS API Documentation
Request Syntax
response = client.list_algorithms(
CreationTimeAfter=datetime(2015, 1, 1),
CreationTimeBefore=datetime(2015, 1, 1),
MaxResults=123,
NameContains='string',
NextToken='string',
SortBy='Name'|'CreationTime',
SortOrder='Ascending'|'Descending'
)
dict
Response Syntax
{
'AlgorithmSummaryList': [
{
'AlgorithmName': 'string',
'AlgorithmArn': 'string',
'AlgorithmDescription': 'string',
'CreationTime': datetime(2015, 1, 1),
'AlgorithmStatus': 'Pending'|'InProgress'|'Completed'|'Failed'|'Deleting'
},
],
'NextToken': 'string'
}
Response Structure
(dict) --
AlgorithmSummaryList (list) --
>An array of AlgorithmSummary objects, each of which lists an algorithm.
(dict) --
Provides summary information about an algorithm.
AlgorithmName (string) --
The name of the algorithm which is described by the summary.
AlgorithmArn (string) --
The Amazon Resource Name (ARN) of the algorithm.
AlgorithmDescription (string) --
A brief statement describing the algorithm.
CreationTime (datetime) --
A timestamp that shows when the algorithm was created.
AlgorithmStatus (string) --
The overall status of the algorithm.
NextToken (string) --
If the response is truncated, Amazon SageMaker returns this token. To retrieve the next set of algorithms, use it in the subsequent request.
Gets a list of the git repositories in your account.
See also: AWS API Documentation
Request Syntax
response = client.list_code_repositories(
CreationTimeAfter=datetime(2015, 1, 1),
CreationTimeBefore=datetime(2015, 1, 1),
LastModifiedTimeAfter=datetime(2015, 1, 1),
LastModifiedTimeBefore=datetime(2015, 1, 1),
MaxResults=123,
NameContains='string',
NextToken='string',
SortBy='Name'|'CreationTime'|'LastModifiedTime',
SortOrder='Ascending'|'Descending'
)
dict
Response Syntax
{
'CodeRepositorySummaryList': [
{
'CodeRepositoryName': 'string',
'CodeRepositoryArn': 'string',
'CreationTime': datetime(2015, 1, 1),
'LastModifiedTime': datetime(2015, 1, 1),
'GitConfig': {
'RepositoryUrl': 'string',
'Branch': 'string',
'SecretArn': 'string'
}
},
],
'NextToken': 'string'
}
Response Structure
(dict) --
CodeRepositorySummaryList (list) --
Gets a list of summaries of the git repositories. Each summary specifies the following values for the repository:
(dict) --
Specifies summary information about a git repository.
CodeRepositoryName (string) --
The name of the git repository.
CodeRepositoryArn (string) --
The Amazon Resource Name (ARN) of the git repository.
CreationTime (datetime) --
The date and time that the git repository was created.
LastModifiedTime (datetime) --
The date and time that the git repository was last modified.
GitConfig (dict) --
Configuration details for the git repository, including the URL where it is located and the ARN of the AWS Secrets Manager secret that contains the credentials used to access the repository.
RepositoryUrl (string) --
The URL where the git repository is located.
Branch (string) --
The default brach for the git repository.
SecretArn (string) --
The Amazon Resource Name (ARN) of the AWS Secrets Manager secret that contains the credentials used to access the git repository. The secret must have a staging label of AWSCURRENT and must be in the following format:
{"username": *UserName* , "password": *Password* }
NextToken (string) --
If the result of a ListCodeRepositoriesOutput request was truncated, the response includes a NextToken . To get the next set of git repositories, use the token in the next request.
Lists model compilation jobs that satisfy various filters.
To create a model compilation job, use CreateCompilationJob . To get information about a particular model compilation job you have created, use DescribeCompilationJob .
See also: AWS API Documentation
Request Syntax
response = client.list_compilation_jobs(
NextToken='string',
MaxResults=123,
CreationTimeAfter=datetime(2015, 1, 1),
CreationTimeBefore=datetime(2015, 1, 1),
LastModifiedTimeAfter=datetime(2015, 1, 1),
LastModifiedTimeBefore=datetime(2015, 1, 1),
NameContains='string',
StatusEquals='INPROGRESS'|'COMPLETED'|'FAILED'|'STARTING'|'STOPPING'|'STOPPED'
)
dict
Response Syntax
{
'CompilationJobSummaries': [
{
'CompilationJobName': 'string',
'CompilationJobArn': 'string',
'CreationTime': datetime(2015, 1, 1),
'CompilationEndTime': datetime(2015, 1, 1),
'CompilationTargetDevice': 'ml_m4'|'ml_m5'|'ml_c4'|'ml_c5'|'ml_p2'|'ml_p3'|'jetson_tx1'|'jetson_tx2'|'rasp3b'|'deeplens',
'LastModifiedTime': datetime(2015, 1, 1),
'CompilationJobStatus': 'INPROGRESS'|'COMPLETED'|'FAILED'|'STARTING'|'STOPPING'|'STOPPED'
},
],
'NextToken': 'string'
}
Response Structure
(dict) --
CompilationJobSummaries (list) --
An array of CompilationJobSummary objects, each describing a model compilation job.
(dict) --
A summary of a model compilation job.
CompilationJobName (string) --
The name of the model compilation job that you want a summary for.
CompilationJobArn (string) --
The Amazon Resource Name (ARN) of the model compilation job.
CreationTime (datetime) --
The time when the model compilation job was created.
CompilationEndTime (datetime) --
The time when the model compilation job completed.
CompilationTargetDevice (string) --
The type of device that the model will run on after compilation has completed.
LastModifiedTime (datetime) --
The time when the model compilation job was last modified.
CompilationJobStatus (string) --
The status of the model compilation job.
NextToken (string) --
If the response is truncated, Amazon SageMaker returns this NextToken . To retrieve the next set of model compilation jobs, use this token in the next request.
Lists endpoint configurations.
See also: AWS API Documentation
Request Syntax
response = client.list_endpoint_configs(
SortBy='Name'|'CreationTime',
SortOrder='Ascending'|'Descending',
NextToken='string',
MaxResults=123,
NameContains='string',
CreationTimeBefore=datetime(2015, 1, 1),
CreationTimeAfter=datetime(2015, 1, 1)
)
dict
Response Syntax
{
'EndpointConfigs': [
{
'EndpointConfigName': 'string',
'EndpointConfigArn': 'string',
'CreationTime': datetime(2015, 1, 1)
},
],
'NextToken': 'string'
}
Response Structure
(dict) --
EndpointConfigs (list) --
An array of endpoint configurations.
(dict) --
Provides summary information for an endpoint configuration.
EndpointConfigName (string) --
The name of the endpoint configuration.
EndpointConfigArn (string) --
The Amazon Resource Name (ARN) of the endpoint configuration.
CreationTime (datetime) --
A timestamp that shows when the endpoint configuration was created.
NextToken (string) --
If the response is truncated, Amazon SageMaker returns this token. To retrieve the next set of endpoint configurations, use it in the subsequent request
Lists endpoints.
See also: AWS API Documentation
Request Syntax
response = client.list_endpoints(
SortBy='Name'|'CreationTime'|'Status',
SortOrder='Ascending'|'Descending',
NextToken='string',
MaxResults=123,
NameContains='string',
CreationTimeBefore=datetime(2015, 1, 1),
CreationTimeAfter=datetime(2015, 1, 1),
LastModifiedTimeBefore=datetime(2015, 1, 1),
LastModifiedTimeAfter=datetime(2015, 1, 1),
StatusEquals='OutOfService'|'Creating'|'Updating'|'SystemUpdating'|'RollingBack'|'InService'|'Deleting'|'Failed'
)
dict
Response Syntax
{
'Endpoints': [
{
'EndpointName': 'string',
'EndpointArn': 'string',
'CreationTime': datetime(2015, 1, 1),
'LastModifiedTime': datetime(2015, 1, 1),
'EndpointStatus': 'OutOfService'|'Creating'|'Updating'|'SystemUpdating'|'RollingBack'|'InService'|'Deleting'|'Failed'
},
],
'NextToken': 'string'
}
Response Structure
(dict) --
Endpoints (list) --
An array or endpoint objects.
(dict) --
Provides summary information for an endpoint.
EndpointName (string) --
The name of the endpoint.
EndpointArn (string) --
The Amazon Resource Name (ARN) of the endpoint.
CreationTime (datetime) --
A timestamp that shows when the endpoint was created.
LastModifiedTime (datetime) --
A timestamp that shows when the endpoint was last modified.
EndpointStatus (string) --
The status of the endpoint.
To get a list of endpoints with a specified status, use the ListEndpointsInput$StatusEquals filter.
NextToken (string) --
If the response is truncated, Amazon SageMaker returns this token. To retrieve the next set of training jobs, use it in the subsequent request.
Gets a list of HyperParameterTuningJobSummary objects that describe the hyperparameter tuning jobs launched in your account.
See also: AWS API Documentation
Request Syntax
response = client.list_hyper_parameter_tuning_jobs(
NextToken='string',
MaxResults=123,
SortBy='Name'|'Status'|'CreationTime',
SortOrder='Ascending'|'Descending',
NameContains='string',
CreationTimeAfter=datetime(2015, 1, 1),
CreationTimeBefore=datetime(2015, 1, 1),
LastModifiedTimeAfter=datetime(2015, 1, 1),
LastModifiedTimeBefore=datetime(2015, 1, 1),
StatusEquals='Completed'|'InProgress'|'Failed'|'Stopped'|'Stopping'
)
dict
Response Syntax
{
'HyperParameterTuningJobSummaries': [
{
'HyperParameterTuningJobName': 'string',
'HyperParameterTuningJobArn': 'string',
'HyperParameterTuningJobStatus': 'Completed'|'InProgress'|'Failed'|'Stopped'|'Stopping',
'Strategy': 'Bayesian',
'CreationTime': datetime(2015, 1, 1),
'HyperParameterTuningEndTime': datetime(2015, 1, 1),
'LastModifiedTime': datetime(2015, 1, 1),
'TrainingJobStatusCounters': {
'Completed': 123,
'InProgress': 123,
'RetryableError': 123,
'NonRetryableError': 123,
'Stopped': 123
},
'ObjectiveStatusCounters': {
'Succeeded': 123,
'Pending': 123,
'Failed': 123
},
'ResourceLimits': {
'MaxNumberOfTrainingJobs': 123,
'MaxParallelTrainingJobs': 123
}
},
],
'NextToken': 'string'
}
Response Structure
(dict) --
HyperParameterTuningJobSummaries (list) --
A list of HyperParameterTuningJobSummary objects that describe the tuning jobs that the ListHyperParameterTuningJobs request returned.
(dict) --
Provides summary information about a hyperparameter tuning job.
HyperParameterTuningJobName (string) --
The name of the tuning job.
HyperParameterTuningJobArn (string) --
The Amazon Resource Name (ARN) of the tuning job.
HyperParameterTuningJobStatus (string) --
The status of the tuning job.
Strategy (string) --
Specifies the search strategy hyperparameter tuning uses to choose which hyperparameters to use for each iteration. Currently, the only valid value is Bayesian.
CreationTime (datetime) --
The date and time that the tuning job was created.
HyperParameterTuningEndTime (datetime) --
The date and time that the tuning job ended.
LastModifiedTime (datetime) --
The date and time that the tuning job was modified.
TrainingJobStatusCounters (dict) --
The TrainingJobStatusCounters object that specifies the numbers of training jobs, categorized by status, that this tuning job launched.
Completed (integer) --
The number of completed training jobs launched by the hyperparameter tuning job.
InProgress (integer) --
The number of in-progress training jobs launched by a hyperparameter tuning job.
RetryableError (integer) --
The number of training jobs that failed, but can be retried. A failed training job can be retried only if it failed because an internal service error occurred.
NonRetryableError (integer) --
The number of training jobs that failed and can't be retried. A failed training job can't be retried if it failed because a client error occurred.
Stopped (integer) --
The number of training jobs launched by a hyperparameter tuning job that were manually stopped.
ObjectiveStatusCounters (dict) --
The ObjectiveStatusCounters object that specifies the numbers of training jobs, categorized by objective metric status, that this tuning job launched.
Succeeded (integer) --
The number of training jobs whose final objective metric was evaluated by the hyperparameter tuning job and used in the hyperparameter tuning process.
Pending (integer) --
The number of training jobs that are in progress and pending evaluation of their final objective metric.
Failed (integer) --
The number of training jobs whose final objective metric was not evaluated and used in the hyperparameter tuning process. This typically occurs when the training job failed or did not emit an objective metric.
ResourceLimits (dict) --
The ResourceLimits object that specifies the maximum number of training jobs and parallel training jobs allowed for this tuning job.
MaxNumberOfTrainingJobs (integer) --
The maximum number of training jobs that a hyperparameter tuning job can launch.
MaxParallelTrainingJobs (integer) --
The maximum number of concurrent training jobs that a hyperparameter tuning job can launch.
NextToken (string) --
If the result of this ListHyperParameterTuningJobs request was truncated, the response includes a NextToken . To retrieve the next set of tuning jobs, use the token in the next request.
Gets a list of labeling jobs.
See also: AWS API Documentation
Request Syntax
response = client.list_labeling_jobs(
CreationTimeAfter=datetime(2015, 1, 1),
CreationTimeBefore=datetime(2015, 1, 1),
LastModifiedTimeAfter=datetime(2015, 1, 1),
LastModifiedTimeBefore=datetime(2015, 1, 1),
MaxResults=123,
NextToken='string',
NameContains='string',
SortBy='Name'|'CreationTime'|'Status',
SortOrder='Ascending'|'Descending',
StatusEquals='InProgress'|'Completed'|'Failed'|'Stopping'|'Stopped'
)
dict
Response Syntax
{
'LabelingJobSummaryList': [
{
'LabelingJobName': 'string',
'LabelingJobArn': 'string',
'CreationTime': datetime(2015, 1, 1),
'LastModifiedTime': datetime(2015, 1, 1),
'LabelingJobStatus': 'InProgress'|'Completed'|'Failed'|'Stopping'|'Stopped',
'LabelCounters': {
'TotalLabeled': 123,
'HumanLabeled': 123,
'MachineLabeled': 123,
'FailedNonRetryableError': 123,
'Unlabeled': 123
},
'WorkteamArn': 'string',
'PreHumanTaskLambdaArn': 'string',
'AnnotationConsolidationLambdaArn': 'string',
'FailureReason': 'string',
'LabelingJobOutput': {
'OutputDatasetS3Uri': 'string',
'FinalActiveLearningModelArn': 'string'
},
'InputConfig': {
'DataSource': {
'S3DataSource': {
'ManifestS3Uri': 'string'
}
},
'DataAttributes': {
'ContentClassifiers': [
'FreeOfPersonallyIdentifiableInformation'|'FreeOfAdultContent',
]
}
}
},
],
'NextToken': 'string'
}
Response Structure
(dict) --
LabelingJobSummaryList (list) --
An array of LabelingJobSummary objects, each describing a labeling job.
(dict) --
Provides summary information about a labeling job.
LabelingJobName (string) --
The name of the labeling job.
LabelingJobArn (string) --
The Amazon Resource Name (ARN) assigned to the labeling job when it was created.
CreationTime (datetime) --
The date and time that the job was created (timestamp).
LastModifiedTime (datetime) --
The date and time that the job was last modified (timestamp).
LabelingJobStatus (string) --
The current status of the labeling job.
LabelCounters (dict) --
Counts showing the progress of the labeling job.
TotalLabeled (integer) --
The total number of objects labeled.
HumanLabeled (integer) --
The total number of objects labeled by a human worker.
MachineLabeled (integer) --
The total number of objects labeled by automated data labeling.
FailedNonRetryableError (integer) --
The total number of objects that could not be labeled due to an error.
Unlabeled (integer) --
The total number of objects not yet labeled.
WorkteamArn (string) --
The Amazon Resource Name (ARN) of the work team assigned to the job.
PreHumanTaskLambdaArn (string) --
The Amazon Resource Name (ARN) of a Lambda function. The function is run before each data object is sent to a worker.
AnnotationConsolidationLambdaArn (string) --
The Amazon Resource Name (ARN) of the Lambda function used to consolidate the annotations from individual workers into a label for a data object. For more information, see Annotation Consolidation .
FailureReason (string) --
If the LabelingJobStatus field is Failed , this field contains a description of the error.
LabelingJobOutput (dict) --
The location of the output produced by the labeling job.
OutputDatasetS3Uri (string) --
The Amazon S3 bucket location of the manifest file for labeled data.
FinalActiveLearningModelArn (string) --
The Amazon Resource Name (ARN) for the most recent Amazon SageMaker model trained as part of automated data labeling.
InputConfig (dict) --
Input configuration for the labeling job.
DataSource (dict) --
The location of the input data.
S3DataSource (dict) --
The Amazon S3 location of the input data objects.
ManifestS3Uri (string) --
The Amazon S3 location of the manifest file that describes the input data objects.
DataAttributes (dict) --
Attributes of the data specified by the customer.
ContentClassifiers (list) --
Declares that your content is free of personally identifiable information or adult content. Amazon SageMaker may restrict the Amazon Mechanical Turk workers that can view your task based on this information.
NextToken (string) --
If the response is truncated, Amazon SageMaker returns this token. To retrieve the next set of labeling jobs, use it in the subsequent request.
Gets a list of labeling jobs assigned to a specified work team.
See also: AWS API Documentation
Request Syntax
response = client.list_labeling_jobs_for_workteam(
WorkteamArn='string',
MaxResults=123,
NextToken='string',
CreationTimeAfter=datetime(2015, 1, 1),
CreationTimeBefore=datetime(2015, 1, 1),
JobReferenceCodeContains='string',
SortBy='CreationTime',
SortOrder='Ascending'|'Descending'
)
[REQUIRED]
The Amazon Resource Name (ARN) of the work team for which you want to see labeling jobs for.
dict
Response Syntax
{
'LabelingJobSummaryList': [
{
'LabelingJobName': 'string',
'JobReferenceCode': 'string',
'WorkRequesterAccountId': 'string',
'CreationTime': datetime(2015, 1, 1),
'LabelCounters': {
'HumanLabeled': 123,
'PendingHuman': 123,
'Total': 123
}
},
],
'NextToken': 'string'
}
Response Structure
(dict) --
LabelingJobSummaryList (list) --
An array of LabelingJobSummary objects, each describing a labeling job.
(dict) --
Provides summary information for a work team.
LabelingJobName (string) --
The name of the labeling job that the work team is assigned to.
JobReferenceCode (string) --
A unique identifier for a labeling job. You can use this to refer to a specific labeling job.
WorkRequesterAccountId (string) --
CreationTime (datetime) --
The date and time that the labeling job was created.
LabelCounters (dict) --
Provides information about the progress of a labeling job.
HumanLabeled (integer) --
The total number of data objects labeled by a human worker.
PendingHuman (integer) --
The total number of data objects that need to be labeled by a human worker.
Total (integer) --
The total number of tasks in the labeling job.
NextToken (string) --
If the response is truncated, Amazon SageMaker returns this token. To retrieve the next set of labeling jobs, use it in the subsequent request.
Lists the model packages that have been created.
See also: AWS API Documentation
Request Syntax
response = client.list_model_packages(
CreationTimeAfter=datetime(2015, 1, 1),
CreationTimeBefore=datetime(2015, 1, 1),
MaxResults=123,
NameContains='string',
NextToken='string',
SortBy='Name'|'CreationTime',
SortOrder='Ascending'|'Descending'
)
dict
Response Syntax
{
'ModelPackageSummaryList': [
{
'ModelPackageName': 'string',
'ModelPackageArn': 'string',
'ModelPackageDescription': 'string',
'CreationTime': datetime(2015, 1, 1),
'ModelPackageStatus': 'Pending'|'InProgress'|'Completed'|'Failed'|'Deleting'
},
],
'NextToken': 'string'
}
Response Structure
(dict) --
ModelPackageSummaryList (list) --
An array of ModelPackageSummary objects, each of which lists a model package.
(dict) --
Provides summary information about a model package.
ModelPackageName (string) --
The name of the model package.
ModelPackageArn (string) --
The Amazon Resource Name (ARN) of the model package.
ModelPackageDescription (string) --
A brief statement describing the model package.
CreationTime (datetime) --
A timestamp that shows when the model package was created.
ModelPackageStatus (string) --
The overall status of the model package.
NextToken (string) --
If the response is truncated, Amazon SageMaker returns this token. To retrieve the next set of model packages, use it in the subsequent request.
Lists models created with the CreateModel API.
See also: AWS API Documentation
Request Syntax
response = client.list_models(
SortBy='Name'|'CreationTime',
SortOrder='Ascending'|'Descending',
NextToken='string',
MaxResults=123,
NameContains='string',
CreationTimeBefore=datetime(2015, 1, 1),
CreationTimeAfter=datetime(2015, 1, 1)
)
dict
Response Syntax
{
'Models': [
{
'ModelName': 'string',
'ModelArn': 'string',
'CreationTime': datetime(2015, 1, 1)
},
],
'NextToken': 'string'
}
Response Structure
(dict) --
Models (list) --
An array of ModelSummary objects, each of which lists a model.
(dict) --
Provides summary information about a model.
ModelName (string) --
The name of the model that you want a summary for.
ModelArn (string) --
The Amazon Resource Name (ARN) of the model.
CreationTime (datetime) --
A timestamp that indicates when the model was created.
NextToken (string) --
If the response is truncated, Amazon SageMaker returns this token. To retrieve the next set of models, use it in the subsequent request.
Lists notebook instance lifestyle configurations created with the CreateNotebookInstanceLifecycleConfig API.
See also: AWS API Documentation
Request Syntax
response = client.list_notebook_instance_lifecycle_configs(
NextToken='string',
MaxResults=123,
SortBy='Name'|'CreationTime'|'LastModifiedTime',
SortOrder='Ascending'|'Descending',
NameContains='string',
CreationTimeBefore=datetime(2015, 1, 1),
CreationTimeAfter=datetime(2015, 1, 1),
LastModifiedTimeBefore=datetime(2015, 1, 1),
LastModifiedTimeAfter=datetime(2015, 1, 1)
)
dict
Response Syntax
{
'NextToken': 'string',
'NotebookInstanceLifecycleConfigs': [
{
'NotebookInstanceLifecycleConfigName': 'string',
'NotebookInstanceLifecycleConfigArn': 'string',
'CreationTime': datetime(2015, 1, 1),
'LastModifiedTime': datetime(2015, 1, 1)
},
]
}
Response Structure
(dict) --
NextToken (string) --
If the response is truncated, Amazon SageMaker returns this token. To get the next set of lifecycle configurations, use it in the next request.
NotebookInstanceLifecycleConfigs (list) --
An array of NotebookInstanceLifecycleConfiguration objects, each listing a lifecycle configuration.
(dict) --
Provides a summary of a notebook instance lifecycle configuration.
NotebookInstanceLifecycleConfigName (string) --
The name of the lifecycle configuration.
NotebookInstanceLifecycleConfigArn (string) --
The Amazon Resource Name (ARN) of the lifecycle configuration.
CreationTime (datetime) --
A timestamp that tells when the lifecycle configuration was created.
LastModifiedTime (datetime) --
A timestamp that tells when the lifecycle configuration was last modified.
Returns a list of the Amazon SageMaker notebook instances in the requester's account in an AWS Region.
See also: AWS API Documentation
Request Syntax
response = client.list_notebook_instances(
NextToken='string',
MaxResults=123,
SortBy='Name'|'CreationTime'|'Status',
SortOrder='Ascending'|'Descending',
NameContains='string',
CreationTimeBefore=datetime(2015, 1, 1),
CreationTimeAfter=datetime(2015, 1, 1),
LastModifiedTimeBefore=datetime(2015, 1, 1),
LastModifiedTimeAfter=datetime(2015, 1, 1),
StatusEquals='Pending'|'InService'|'Stopping'|'Stopped'|'Failed'|'Deleting'|'Updating',
NotebookInstanceLifecycleConfigNameContains='string',
DefaultCodeRepositoryContains='string',
AdditionalCodeRepositoryEquals='string'
)
If the previous call to the ListNotebookInstances is truncated, the response includes a NextToken . You can use this token in your subsequent ListNotebookInstances request to fetch the next set of notebook instances.
Note
You might specify a filter or a sort order in your request. When response is truncated, you must use the same values for the filer and sort order in the next request.
dict
Response Syntax
{
'NextToken': 'string',
'NotebookInstances': [
{
'NotebookInstanceName': 'string',
'NotebookInstanceArn': 'string',
'NotebookInstanceStatus': 'Pending'|'InService'|'Stopping'|'Stopped'|'Failed'|'Deleting'|'Updating',
'Url': 'string',
'InstanceType': 'ml.t2.medium'|'ml.t2.large'|'ml.t2.xlarge'|'ml.t2.2xlarge'|'ml.t3.medium'|'ml.t3.large'|'ml.t3.xlarge'|'ml.t3.2xlarge'|'ml.m4.xlarge'|'ml.m4.2xlarge'|'ml.m4.4xlarge'|'ml.m4.10xlarge'|'ml.m4.16xlarge'|'ml.m5.xlarge'|'ml.m5.2xlarge'|'ml.m5.4xlarge'|'ml.m5.12xlarge'|'ml.m5.24xlarge'|'ml.c4.xlarge'|'ml.c4.2xlarge'|'ml.c4.4xlarge'|'ml.c4.8xlarge'|'ml.c5.xlarge'|'ml.c5.2xlarge'|'ml.c5.4xlarge'|'ml.c5.9xlarge'|'ml.c5.18xlarge'|'ml.c5d.xlarge'|'ml.c5d.2xlarge'|'ml.c5d.4xlarge'|'ml.c5d.9xlarge'|'ml.c5d.18xlarge'|'ml.p2.xlarge'|'ml.p2.8xlarge'|'ml.p2.16xlarge'|'ml.p3.2xlarge'|'ml.p3.8xlarge'|'ml.p3.16xlarge',
'CreationTime': datetime(2015, 1, 1),
'LastModifiedTime': datetime(2015, 1, 1),
'NotebookInstanceLifecycleConfigName': 'string',
'DefaultCodeRepository': 'string',
'AdditionalCodeRepositories': [
'string',
]
},
]
}
Response Structure
(dict) --
NextToken (string) --
If the response to the previous ListNotebookInstances request was truncated, Amazon SageMaker returns this token. To retrieve the next set of notebook instances, use the token in the next request.
NotebookInstances (list) --
An array of NotebookInstanceSummary objects, one for each notebook instance.
(dict) --
Provides summary information for an Amazon SageMaker notebook instance.
NotebookInstanceName (string) --
The name of the notebook instance that you want a summary for.
NotebookInstanceArn (string) --
The Amazon Resource Name (ARN) of the notebook instance.
NotebookInstanceStatus (string) --
The status of the notebook instance.
Url (string) --
The URL that you use to connect to the Jupyter instance running in your notebook instance.
InstanceType (string) --
The type of ML compute instance that the notebook instance is running on.
CreationTime (datetime) --
A timestamp that shows when the notebook instance was created.
LastModifiedTime (datetime) --
A timestamp that shows when the notebook instance was last modified.
NotebookInstanceLifecycleConfigName (string) --
The name of a notebook instance lifecycle configuration associated with this notebook instance.
For information about notebook instance lifestyle configurations, see Step 2.1: (Optional) Customize a Notebook Instance .
DefaultCodeRepository (string) --
The git repository associated with the notebook instance as its default code repository. This can be either the name of a git repository stored as a resource in your account, or the URL of a git repository in AWS CodeCommit or in any other git repository. When you open a notebook instance, it opens in the directory that contains this repository. For more information, see Associating Git Repositories with Amazon SageMaker Notebook Instances .
AdditionalCodeRepositories (list) --
An array of up to 3 git repositories associated with the notebook instance. These can be either the names of git repositories stored as resources in your account, or the URL of git repositories in AWS CodeCommit or in any other git repository. These repositories are cloned at the same level as the default repository of your notebook instance. For more information, see Associating Git Repositories with Amazon SageMaker Notebook Instances .
Gets a list of the work teams that you are subscribed to in the AWS Marketplace. The list may be empty if no work team satisfies the filter specified in the NameContains parameter.
See also: AWS API Documentation
Request Syntax
response = client.list_subscribed_workteams(
NameContains='string',
NextToken='string',
MaxResults=123
)
dict
Response Syntax
{
'SubscribedWorkteams': [
{
'WorkteamArn': 'string',
'MarketplaceTitle': 'string',
'SellerName': 'string',
'MarketplaceDescription': 'string',
'ListingId': 'string'
},
],
'NextToken': 'string'
}
Response Structure
(dict) --
SubscribedWorkteams (list) --
An array of Workteam objects, each describing a work team.
(dict) --
Describes a work team of a vendor that does the a labelling job.
WorkteamArn (string) --
The Amazon Resource Name (ARN) of the vendor that you have subscribed.
MarketplaceTitle (string) --
The title of the service provided by the vendor in the Amazon Marketplace.
SellerName (string) --
The name of the vendor in the Amazon Marketplace.
MarketplaceDescription (string) --
The description of the vendor from the Amazon Marketplace.
ListingId (string) --
NextToken (string) --
If the response is truncated, Amazon SageMaker returns this token. To retrieve the next set of work teams, use it in the subsequent request.
Returns the tags for the specified Amazon SageMaker resource.
See also: AWS API Documentation
Request Syntax
response = client.list_tags(
ResourceArn='string',
NextToken='string',
MaxResults=123
)
[REQUIRED]
The Amazon Resource Name (ARN) of the resource whose tags you want to retrieve.
dict
Response Syntax
{
'Tags': [
{
'Key': 'string',
'Value': 'string'
},
],
'NextToken': 'string'
}
Response Structure
(dict) --
Tags (list) --
An array of Tag objects, each with a tag key and a value.
(dict) --
Describes a tag.
Key (string) --
The tag key.
Value (string) --
The tag value.
NextToken (string) --
If response is truncated, Amazon SageMaker includes a token in the response. You can use this token in your subsequent request to fetch next set of tokens.
Lists training jobs.
See also: AWS API Documentation
Request Syntax
response = client.list_training_jobs(
NextToken='string',
MaxResults=123,
CreationTimeAfter=datetime(2015, 1, 1),
CreationTimeBefore=datetime(2015, 1, 1),
LastModifiedTimeAfter=datetime(2015, 1, 1),
LastModifiedTimeBefore=datetime(2015, 1, 1),
NameContains='string',
StatusEquals='InProgress'|'Completed'|'Failed'|'Stopping'|'Stopped',
SortBy='Name'|'CreationTime'|'Status',
SortOrder='Ascending'|'Descending'
)
dict
Response Syntax
{
'TrainingJobSummaries': [
{
'TrainingJobName': 'string',
'TrainingJobArn': 'string',
'CreationTime': datetime(2015, 1, 1),
'TrainingEndTime': datetime(2015, 1, 1),
'LastModifiedTime': datetime(2015, 1, 1),
'TrainingJobStatus': 'InProgress'|'Completed'|'Failed'|'Stopping'|'Stopped'
},
],
'NextToken': 'string'
}
Response Structure
(dict) --
TrainingJobSummaries (list) --
An array of TrainingJobSummary objects, each listing a training job.
(dict) --
Provides summary information about a training job.
TrainingJobName (string) --
The name of the training job that you want a summary for.
TrainingJobArn (string) --
The Amazon Resource Name (ARN) of the training job.
CreationTime (datetime) --
A timestamp that shows when the training job was created.
TrainingEndTime (datetime) --
A timestamp that shows when the training job ended. This field is set only if the training job has one of the terminal statuses (Completed , Failed , or Stopped ).
LastModifiedTime (datetime) --
Timestamp when the training job was last modified.
TrainingJobStatus (string) --
The status of the training job.
NextToken (string) --
If the response is truncated, Amazon SageMaker returns this token. To retrieve the next set of training jobs, use it in the subsequent request.
Gets a list of TrainingJobSummary objects that describe the training jobs that a hyperparameter tuning job launched.
See also: AWS API Documentation
Request Syntax
response = client.list_training_jobs_for_hyper_parameter_tuning_job(
HyperParameterTuningJobName='string',
NextToken='string',
MaxResults=123,
StatusEquals='InProgress'|'Completed'|'Failed'|'Stopping'|'Stopped',
SortBy='Name'|'CreationTime'|'Status'|'FinalObjectiveMetricValue',
SortOrder='Ascending'|'Descending'
)
[REQUIRED]
The name of the tuning job whose training jobs you want to list.
The field to sort results by. The default is Name .
If the value of this field is FinalObjectiveMetricValue , any training jobs that did not return an objective metric are not listed.
dict
Response Syntax
{
'TrainingJobSummaries': [
{
'TrainingJobName': 'string',
'TrainingJobArn': 'string',
'TuningJobName': 'string',
'CreationTime': datetime(2015, 1, 1),
'TrainingStartTime': datetime(2015, 1, 1),
'TrainingEndTime': datetime(2015, 1, 1),
'TrainingJobStatus': 'InProgress'|'Completed'|'Failed'|'Stopping'|'Stopped',
'TunedHyperParameters': {
'string': 'string'
},
'FailureReason': 'string',
'FinalHyperParameterTuningJobObjectiveMetric': {
'Type': 'Maximize'|'Minimize',
'MetricName': 'string',
'Value': ...
},
'ObjectiveStatus': 'Succeeded'|'Pending'|'Failed'
},
],
'NextToken': 'string'
}
Response Structure
(dict) --
TrainingJobSummaries (list) --
A list of TrainingJobSummary objects that describe the training jobs that the ListTrainingJobsForHyperParameterTuningJob request returned.
(dict) --
Specifies summary information about a training job.
TrainingJobName (string) --
The name of the training job.
TrainingJobArn (string) --
The Amazon Resource Name (ARN) of the training job.
TuningJobName (string) --
The name of the hyperparameter tuning job that launched this training job.
CreationTime (datetime) --
The date and time that the training job was created.
TrainingStartTime (datetime) --
The date and time that the training job started.
TrainingEndTime (datetime) --
The date and time that the training job ended.
TrainingJobStatus (string) --
The status of the training job.
TunedHyperParameters (dict) --
A list of the hyperparameters for which you specified ranges to search.
FailureReason (string) --
The reason that the training job failed.
FinalHyperParameterTuningJobObjectiveMetric (dict) --
The FinalHyperParameterTuningJobObjectiveMetric object that specifies the value of the objective metric of the tuning job that launched this training job.
Type (string) --
Whether to minimize or maximize the objective metric. Valid values are Minimize and Maximize.
MetricName (string) --
The name of the objective metric.
Value (float) --
The value of the objective metric.
ObjectiveStatus (string) --
The status of the objective metric for the training job:
NextToken (string) --
If the result of this ListTrainingJobsForHyperParameterTuningJob request was truncated, the response includes a NextToken . To retrieve the next set of training jobs, use the token in the next request.
Lists transform jobs.
See also: AWS API Documentation
Request Syntax
response = client.list_transform_jobs(
CreationTimeAfter=datetime(2015, 1, 1),
CreationTimeBefore=datetime(2015, 1, 1),
LastModifiedTimeAfter=datetime(2015, 1, 1),
LastModifiedTimeBefore=datetime(2015, 1, 1),
NameContains='string',
StatusEquals='InProgress'|'Completed'|'Failed'|'Stopping'|'Stopped',
SortBy='Name'|'CreationTime'|'Status',
SortOrder='Ascending'|'Descending',
NextToken='string',
MaxResults=123
)
dict
Response Syntax
{
'TransformJobSummaries': [
{
'TransformJobName': 'string',
'TransformJobArn': 'string',
'CreationTime': datetime(2015, 1, 1),
'TransformEndTime': datetime(2015, 1, 1),
'LastModifiedTime': datetime(2015, 1, 1),
'TransformJobStatus': 'InProgress'|'Completed'|'Failed'|'Stopping'|'Stopped',
'FailureReason': 'string'
},
],
'NextToken': 'string'
}
Response Structure
(dict) --
TransformJobSummaries (list) --
An array of TransformJobSummary objects.
(dict) --
Provides a summary of a transform job. Multiple TransformJobSummary objects are returned as a list after calling ListTransformJobs .
TransformJobName (string) --
The name of the transform job.
TransformJobArn (string) --
The Amazon Resource Name (ARN) of the transform job.
CreationTime (datetime) --
A timestamp that shows when the transform Job was created.
TransformEndTime (datetime) --
Indicates when the transform job ends on compute instances. For successful jobs and stopped jobs, this is the exact time recorded after the results are uploaded. For failed jobs, this is when Amazon SageMaker detected that the job failed.
LastModifiedTime (datetime) --
Indicates when the transform job was last modified.
TransformJobStatus (string) --
The status of the transform job.
FailureReason (string) --
If the transform job failed, the reason it failed.
NextToken (string) --
If the response is truncated, Amazon SageMaker returns this token. To retrieve the next set of transform jobs, use it in the next request.
Gets a list of work teams that you have defined in a region. The list may be empty if no work team satisfies the filter specified in the NameContains parameter.
See also: AWS API Documentation
Request Syntax
response = client.list_workteams(
SortBy='Name'|'CreateDate',
SortOrder='Ascending'|'Descending',
NameContains='string',
NextToken='string',
MaxResults=123
)
dict
Response Syntax
{
'Workteams': [
{
'WorkteamName': 'string',
'MemberDefinitions': [
{
'CognitoMemberDefinition': {
'UserPool': 'string',
'UserGroup': 'string',
'ClientId': 'string'
}
},
],
'WorkteamArn': 'string',
'ProductListingIds': [
'string',
],
'Description': 'string',
'SubDomain': 'string',
'CreateDate': datetime(2015, 1, 1),
'LastUpdatedDate': datetime(2015, 1, 1)
},
],
'NextToken': 'string'
}
Response Structure
(dict) --
Workteams (list) --
An array of Workteam objects, each describing a work team.
(dict) --
Provides details about a labeling work team.
WorkteamName (string) --
The name of the work team.
MemberDefinitions (list) --
The Amazon Cognito user groups that make up the work team.
(dict) --
Defines the Amazon Cognito user group that is part of a work team.
CognitoMemberDefinition (dict) --
The Amazon Cognito user group that is part of the work team.
UserPool (string) --
An identifier for a user pool. The user pool must be in the same region as the service that you are calling.
UserGroup (string) --
An identifier for a user group.
ClientId (string) --
An identifier for an application client. You must create the app client ID using Amazon Cognito.
WorkteamArn (string) --
The Amazon Resource Name (ARN) that identifies the work team.
ProductListingIds (list) --
The Amazon Marketplace identifier for a vendor's work team.
Description (string) --
A description of the work team.
SubDomain (string) --
The URI of the labeling job's user interface. Workers open this URI to start labeling your data objects.
CreateDate (datetime) --
The date and time that the work team was created (timestamp).
LastUpdatedDate (datetime) --
The date and time that the work team was last updated (timestamp).
NextToken (string) --
If the response is truncated, Amazon SageMaker returns this token. To retrieve the next set of work teams, use it in the subsequent request.
Renders the UI template so that you can preview the worker's experience.
See also: AWS API Documentation
Request Syntax
response = client.render_ui_template(
UiTemplate={
'Content': 'string'
},
Task={
'Input': 'string'
},
RoleArn='string'
)
[REQUIRED]
A Tempate object containing the worker UI template to render.
The content of the Liquid template for the worker user interface.
[REQUIRED]
A RenderableTask object containing a representative task to render.
A JSON object that contains values for the variables defined in the template. It is made available to the template under the substitution variable task.input . For example, if you define a variable task.input.text in your template, you can supply the variable in the JSON object as "text": "sample text" .
[REQUIRED]
The Amazon Resource Name (ARN) that has access to the S3 objects that are used by the template.
dict
Response Syntax
{
'RenderedContent': 'string',
'Errors': [
{
'Code': 'string',
'Message': 'string'
},
]
}
Response Structure
(dict) --
RenderedContent (string) --
A Liquid template that renders the HTML for the worker UI.
Errors (list) --
A list of one or more RenderingError objects if any were encountered while rendering the template. If there were no errors, the list is empty.
(dict) --
A description of an error that occurred while rendering the template.
Code (string) --
A unique identifier for a specific class of errors.
Message (string) --
A human-readable message describing the error.
Finds Amazon SageMaker resources that match a search query. Matching resource objects are returned as a list of SearchResult objects in the response. The search results can be sorted by any resrouce property in a ascending or descending order.
You can query against the following value types: numerical, text, Booleans, and timestamps.
See also: AWS API Documentation
Request Syntax
response = client.search(
Resource='TrainingJob',
SearchExpression={
'Filters': [
{
'Name': 'string',
'Operator': 'Equals'|'NotEquals'|'GreaterThan'|'GreaterThanOrEqualTo'|'LessThan'|'LessThanOrEqualTo'|'Contains',
'Value': 'string'
},
],
'NestedFilters': [
{
'NestedPropertyName': 'string',
'Filters': [
{
'Name': 'string',
'Operator': 'Equals'|'NotEquals'|'GreaterThan'|'GreaterThanOrEqualTo'|'LessThan'|'LessThanOrEqualTo'|'Contains',
'Value': 'string'
},
]
},
],
'SubExpressions': [
{'... recursive ...'},
],
'Operator': 'And'|'Or'
},
SortBy='string',
SortOrder='Ascending'|'Descending',
NextToken='string',
MaxResults=123
)
[REQUIRED]
The name of the Amazon SageMaker resource to search for. Currently, the only valid Resource value is TrainingJob .
A Boolean conditional statement. Resource objects must satisfy this condition to be included in search results.
A list of filter objects.
A conditional statement for a search expression that includes a Boolean operator, a resource property, and a value.
If you don't specify an Operator and a Value , the filter searches for only the specified property. For example, defining a Filter for the FailureReason for the TrainingJob Resource searches for training job objects that have a value in the FailureReason field.
If you specify a Value , but not an Operator , Amazon SageMaker uses the equals operator as a default.
In search, there are several property types:
Metrics
To define a metric filter, enter a value using the form "Metrics.<name>" , where <name> is a metric name. For example, the following filter searches for training jobs with an "accuracy" metric greater than "0.9" :
{"Name": "Metrics.accuracy",
"Operator": "GREATER_THAN",
"Value": "0.9"
}
HyperParameters
To define a hyperparameter filter, enter a value with the form "HyperParamters.<name>" . Decimal hyperparameter values are treated as a decimal in a comparison if the specified Value is also a decimal value. If the specified Value is an integer, the decimal hyperparameter values are treated as integers. For example, the following filter is satisfied by training jobs with a "learning_rate" hyperparameter that is less than "0.5" :
{"Name": "HyperParameters.learning_rate",
"Operator": "LESS_THAN",
"Value": "0.5"
}
Tags
To define a tag filter, enter a value with the form "Tags.<key>" .
A property name. For example, TrainingJobName . See TrainingJob properties for the list of valid property names for each supported resource.
A Boolean binary operator that is used to evaluate the filter. The operator field contains one of the following values:
Equals
The specified resource in Name equals the specified Value .
NotEquals
The specified resource in Name does not equal the specified Value .
GreaterThan
The specified resource in Name is greater than the specified Value . Not supported for text-based properties.
GreaterThanOrEqualTo
The specified resource in Name is greater than or equal to the specified Value . Not supported for text-based properties.
LessThan
The specified resource in Name is less than the specified Value . Not supported for text-based properties.
LessThanOrEqualTo
The specified resource in Name is less than or equal to the specified Value . Not supported for text-based properties.
Contains
Only supported for text-based properties. The word-list of the property contains the specified Value .
A value used with Resource and Operator to determin if objects statisfy the filter's condition. For numerical properties, Value must be an integer or floating-point decimal. For timestamp properties, Value must be an ISO 8601 date-time string of the following format: YYYY-mm-dd'T'HH:MM:SS .
A list of nested filter objects.
A NestedFilter is defined by using a resource name under NestedPropertyName , which entries in a list that properties must match to be included in the results. To satisfy the conditions specified in the NestedFilters call, each object in the list must satisfy the conditions of all of the filters.
For example, a NestedFilters could be defined using the training job's InputDataConfig property, this would be defined as a list of Channel objects.
A NestedFilters object contains multiple filters. For example, to find all training jobs that have train in their name, and have cat/data in their S3Uri (under InputDataConfig ), you need to create a NestedFilters object that specfies the InputDataConfig property with the following Filter objects:
.The name of the property used in the nested filters.
A list of filters. Each filter acts on a property. For example, a NestedFilters call might include a filter on the PropertyName parameter fof the InputDataConfig property: InputDataConfig.DataSource.S3DataSource.S3Uri .
A conditional statement for a search expression that includes a Boolean operator, a resource property, and a value.
If you don't specify an Operator and a Value , the filter searches for only the specified property. For example, defining a Filter for the FailureReason for the TrainingJob Resource searches for training job objects that have a value in the FailureReason field.
If you specify a Value , but not an Operator , Amazon SageMaker uses the equals operator as a default.
In search, there are several property types:
Metrics
To define a metric filter, enter a value using the form "Metrics.<name>" , where <name> is a metric name. For example, the following filter searches for training jobs with an "accuracy" metric greater than "0.9" :
{"Name": "Metrics.accuracy",
"Operator": "GREATER_THAN",
"Value": "0.9"
}
HyperParameters
To define a hyperparameter filter, enter a value with the form "HyperParamters.<name>" . Decimal hyperparameter values are treated as a decimal in a comparison if the specified Value is also a decimal value. If the specified Value is an integer, the decimal hyperparameter values are treated as integers. For example, the following filter is satisfied by training jobs with a "learning_rate" hyperparameter that is less than "0.5" :
{"Name": "HyperParameters.learning_rate",
"Operator": "LESS_THAN",
"Value": "0.5"
}
Tags
To define a tag filter, enter a value with the form "Tags.<key>" .
A property name. For example, TrainingJobName . See TrainingJob properties for the list of valid property names for each supported resource.
A Boolean binary operator that is used to evaluate the filter. The operator field contains one of the following values:
Equals
The specified resource in Name equals the specified Value .
NotEquals
The specified resource in Name does not equal the specified Value .
GreaterThan
The specified resource in Name is greater than the specified Value . Not supported for text-based properties.
GreaterThanOrEqualTo
The specified resource in Name is greater than or equal to the specified Value . Not supported for text-based properties.
LessThan
The specified resource in Name is less than the specified Value . Not supported for text-based properties.
LessThanOrEqualTo
The specified resource in Name is less than or equal to the specified Value . Not supported for text-based properties.
Contains
Only supported for text-based properties. The word-list of the property contains the specified Value .
A value used with Resource and Operator to determin if objects statisfy the filter's condition. For numerical properties, Value must be an integer or floating-point decimal. For timestamp properties, Value must be an ISO 8601 date-time string of the following format: YYYY-mm-dd'T'HH:MM:SS .
A list of search expression objects.
A multi-expression that searches for the specified resource or resources. All resource objects that satisfy the expression's condition are included in the search results.
A SearchExpression contains the following components:
A Boolean operator used to evaluate the search expression. If you want every conditional statement in all lists to be satisfied for the entire search expression to be true, specify And . If only a single conditional statement needs to be true for the entire search expression to be true, specify Or .
dict
Response Syntax
{
'Results': [
{
'TrainingJob': {
'TrainingJobName': 'string',
'TrainingJobArn': 'string',
'TuningJobArn': 'string',
'LabelingJobArn': 'string',
'ModelArtifacts': {
'S3ModelArtifacts': 'string'
},
'TrainingJobStatus': 'InProgress'|'Completed'|'Failed'|'Stopping'|'Stopped',
'SecondaryStatus': 'Starting'|'LaunchingMLInstances'|'PreparingTrainingStack'|'Downloading'|'DownloadingTrainingImage'|'Training'|'Uploading'|'Stopping'|'Stopped'|'MaxRuntimeExceeded'|'Completed'|'Failed',
'FailureReason': 'string',
'HyperParameters': {
'string': 'string'
},
'AlgorithmSpecification': {
'TrainingImage': 'string',
'AlgorithmName': 'string',
'TrainingInputMode': 'Pipe'|'File',
'MetricDefinitions': [
{
'Name': 'string',
'Regex': 'string'
},
]
},
'RoleArn': 'string',
'InputDataConfig': [
{
'ChannelName': 'string',
'DataSource': {
'S3DataSource': {
'S3DataType': 'ManifestFile'|'S3Prefix'|'AugmentedManifestFile',
'S3Uri': 'string',
'S3DataDistributionType': 'FullyReplicated'|'ShardedByS3Key',
'AttributeNames': [
'string',
]
}
},
'ContentType': 'string',
'CompressionType': 'None'|'Gzip',
'RecordWrapperType': 'None'|'RecordIO',
'InputMode': 'Pipe'|'File',
'ShuffleConfig': {
'Seed': 123
}
},
],
'OutputDataConfig': {
'KmsKeyId': 'string',
'S3OutputPath': 'string'
},
'ResourceConfig': {
'InstanceType': 'ml.m4.xlarge'|'ml.m4.2xlarge'|'ml.m4.4xlarge'|'ml.m4.10xlarge'|'ml.m4.16xlarge'|'ml.m5.large'|'ml.m5.xlarge'|'ml.m5.2xlarge'|'ml.m5.4xlarge'|'ml.m5.12xlarge'|'ml.m5.24xlarge'|'ml.c4.xlarge'|'ml.c4.2xlarge'|'ml.c4.4xlarge'|'ml.c4.8xlarge'|'ml.p2.xlarge'|'ml.p2.8xlarge'|'ml.p2.16xlarge'|'ml.p3.2xlarge'|'ml.p3.8xlarge'|'ml.p3.16xlarge'|'ml.c5.xlarge'|'ml.c5.2xlarge'|'ml.c5.4xlarge'|'ml.c5.9xlarge'|'ml.c5.18xlarge',
'InstanceCount': 123,
'VolumeSizeInGB': 123,
'VolumeKmsKeyId': 'string'
},
'VpcConfig': {
'SecurityGroupIds': [
'string',
],
'Subnets': [
'string',
]
},
'StoppingCondition': {
'MaxRuntimeInSeconds': 123
},
'CreationTime': datetime(2015, 1, 1),
'TrainingStartTime': datetime(2015, 1, 1),
'TrainingEndTime': datetime(2015, 1, 1),
'LastModifiedTime': datetime(2015, 1, 1),
'SecondaryStatusTransitions': [
{
'Status': 'Starting'|'LaunchingMLInstances'|'PreparingTrainingStack'|'Downloading'|'DownloadingTrainingImage'|'Training'|'Uploading'|'Stopping'|'Stopped'|'MaxRuntimeExceeded'|'Completed'|'Failed',
'StartTime': datetime(2015, 1, 1),
'EndTime': datetime(2015, 1, 1),
'StatusMessage': 'string'
},
],
'FinalMetricDataList': [
{
'MetricName': 'string',
'Value': ...,
'Timestamp': datetime(2015, 1, 1)
},
],
'EnableNetworkIsolation': True|False,
'Tags': [
{
'Key': 'string',
'Value': 'string'
},
]
}
},
],
'NextToken': 'string'
}
Response Structure
(dict) --
Results (list) --
A list of SearchResult objects.
(dict) --
An individual search result record that contains a single resource object.
TrainingJob (dict) --
A TrainingJob object that is returned as part of a Search request.
TrainingJobName (string) --
The name of the training job.
TrainingJobArn (string) --
The Amazon Resource Name (ARN) of the training job.
TuningJobArn (string) --
The Amazon Resource Name (ARN) of the associated hyperparameter tuning job if the training job was launched by a hyperparameter tuning job.
LabelingJobArn (string) --
The Amazon Resource Name (ARN) of the labeling job.
ModelArtifacts (dict) --
Information about the Amazon S3 location that is configured for storing model artifacts.
S3ModelArtifacts (string) --
The path of the S3 object that contains the model artifacts. For example, s3://bucket-name/keynameprefix/model.tar.gz .
TrainingJobStatus (string) --
The status of the training job.
Training job statuses are:
For more detailed information, see SecondaryStatus .
SecondaryStatus (string) --
Provides detailed information about the state of the training job. For detailed information about the secondary status of the training job, see StatusMessage under SecondaryStatusTransition .
Amazon SageMaker provides primary statuses and secondary statuses that apply to each of them:
InProgress
Starting - Starting the training job.
Downloading - An optional stage for algorithms that support File training input mode. It indicates that data is being downloaded to the ML storage volumes.
Training - Training is in progress.
Uploading - Training is complete and the model artifacts are being uploaded to the S3 location.
Completed
Completed - The training job has completed.
Failed
Failed - The training job has failed. The reason for the failure is returned in the FailureReason field of DescribeTrainingJobResponse .
Stopped
MaxRuntimeExceeded - The job stopped because it exceeded the maximum allowed runtime.
Stopped - The training job has stopped.
Stopping
Stopping - Stopping the training job.
Warning
Valid values for SecondaryStatus are subject to change.
We no longer support the following secondary statuses:
FailureReason (string) --
If the training job failed, the reason it failed.
HyperParameters (dict) --
Algorithm-specific parameters.
AlgorithmSpecification (dict) --
Information about the algorithm used for training, and algorithm metadata.
TrainingImage (string) --
The registry path of the Docker image that contains the training algorithm. For information about docker registry paths for built-in algorithms, see Algorithms Provided by Amazon SageMaker: Common Parameters .
AlgorithmName (string) --
The name of the algorithm resource to use for the training job. This must be an algorithm resource that you created or subscribe to on AWS Marketplace. If you specify a value for this parameter, you can't specify a value for TrainingImage .
TrainingInputMode (string) --
The input mode that the algorithm supports. For the input modes that Amazon SageMaker algorithms support, see Algorithms . If an algorithm supports the File input mode, Amazon SageMaker downloads the training data from S3 to the provisioned ML storage Volume, and mounts the directory to docker volume for training container. If an algorithm supports the Pipe input mode, Amazon SageMaker streams data directly from S3 to the container.
In File mode, make sure you provision ML storage volume with sufficient capacity to accommodate the data download from S3. In addition to the training data, the ML storage volume also stores the output model. The algorithm container use ML storage volume to also store intermediate information, if any.
For distributed algorithms using File mode, training data is distributed uniformly, and your training duration is predictable if the input data objects size is approximately same. Amazon SageMaker does not split the files any further for model training. If the object sizes are skewed, training won't be optimal as the data distribution is also skewed where one host in a training cluster is overloaded, thus becoming bottleneck in training.
MetricDefinitions (list) --
A list of metric definition objects. Each object specifies the metric name and regular expressions used to parse algorithm logs. Amazon SageMaker publishes each metric to Amazon CloudWatch.
(dict) --
Specifies a metric that the training algorithm writes to stderr or stdout . Amazon SageMakerhyperparameter tuning captures all defined metrics. You specify one metric that a hyperparameter tuning job uses as its objective metric to choose the best training job.
Name (string) --
The name of the metric.
Regex (string) --
A regular expression that searches the output of a training job and gets the value of the metric. For more information about using regular expressions to define metrics, see Defining Objective Metrics .
RoleArn (string) --
The AWS Identity and Access Management (IAM) role configured for the training job.
InputDataConfig (list) --
An array of Channel objects that describes each data input channel.
(dict) --
A channel is a named input source that training algorithms can consume.
ChannelName (string) --
The name of the channel.
DataSource (dict) --
The location of the channel data.
S3DataSource (dict) --
The S3 location of the data source that is associated with a channel.
S3DataType (string) --
If you choose S3Prefix , S3Uri identifies a key name prefix. Amazon SageMaker uses all objects that match the specified key name prefix for model training.
If you choose ManifestFile , S3Uri identifies an object that is a manifest file containing a list of object keys that you want Amazon SageMaker to use for model training.
If you choose AugmentedManifestFile , S3Uri identifies an object that is an augmented manifest file in JSON lines format. This file contains the data you want to use for model training. AugmentedManifestFile can only be used if the Channel's input mode is Pipe .
S3Uri (string) --
Depending on the value specified for the S3DataType , identifies either a key name prefix or a manifest. For example:
S3DataDistributionType (string) --
If you want Amazon SageMaker to replicate the entire dataset on each ML compute instance that is launched for model training, specify FullyReplicated .
If you want Amazon SageMaker to replicate a subset of data on each ML compute instance that is launched for model training, specify ShardedByS3Key . If there are n ML compute instances launched for a training job, each instance gets approximately 1/n of the number of S3 objects. In this case, model training on each machine uses only the subset of training data.
Don't choose more ML compute instances for training than available S3 objects. If you do, some nodes won't get any data and you will pay for nodes that aren't getting any training data. This applies in both File and Pipemodes. Keep this in mind when developing algorithms.
In distributed training, where you use multiple ML compute EC2 instances, you might choose ShardedByS3Key . If the algorithm requires copying training data to the ML storage volume (when TrainingInputMode is set to File ), this copies 1/n of the number of objects.
AttributeNames (list) --
A list of one or more attribute names to use that are found in a specified augmented manifest file.
ContentType (string) --
The MIME type of the data.
CompressionType (string) --
If training data is compressed, the compression type. The default value is None . CompressionType is used only in Pipe input mode. In File mode, leave this field unset or set it to None.
RecordWrapperType (string) --
Specify RecordIO as the value when input data is in raw format but the training algorithm requires the RecordIO format. In this case, Amazon SageMaker wraps each individual S3 object in a RecordIO record. If the input data is already in RecordIO format, you don't need to set this attribute. For more information, see Create a Dataset Using RecordIO .
In File mode, leave this field unset or set it to None.
InputMode (string) --
(Optional) The input mode to use for the data channel in a training job. If you don't set a value for InputMode , Amazon SageMaker uses the value set for TrainingInputMode . Use this parameter to override the TrainingInputMode setting in a AlgorithmSpecification request when you have a channel that needs a different input mode from the training job's general setting. To download the data from Amazon Simple Storage Service (Amazon S3) to the provisioned ML storage volume, and mount the directory to a Docker volume, use File input mode. To stream data directly from Amazon S3 to the container, choose Pipe input mode.
To use a model for incremental training, choose File input model.
ShuffleConfig (dict) --
A configuration for a shuffle option for input data in a channel. If you use S3Prefix for S3DataType , this shuffles the results of the S3 key prefix matches. If you use ManifestFile , the order of the S3 object references in the ManifestFile is shuffled. If you use AugmentedManifestFile , the order of the JSON lines in the AugmentedManifestFile is shuffled. The shuffling order is determined using the Seed value.
For Pipe input mode, shuffling is done at the start of every epoch. With large datasets this ensures that the order of the training data is different for each epoch, it helps reduce bias and possible overfitting. In a multi-node training job when ShuffleConfig is combined with S3DataDistributionType of ShardedByS3Key , the data is shuffled across nodes so that the content sent to a particular node on the first epoch might be sent to a different node on the second epoch.
Seed (integer) --
Determines the shuffling order in ShuffleConfig . value.
OutputDataConfig (dict) --
The S3 path where model artifacts that you configured when creating the job are stored. Amazon SageMaker creates subfolders for model artifacts.
KmsKeyId (string) --
The AWS Key Management Service (AWS KMS) key that Amazon SageMaker uses to encrypt the model artifacts at rest using Amazon S3 server-side encryption. The KmsKeyId can be any of the following formats:
If you don't provide the KMS key ID, Amazon SageMaker uses the default KMS key for Amazon S3 for your role's account. For more information, see KMS-Managed Encryption Keys in Amazon Simple Storage Service Developer Guide.
Note
The KMS key policy must grant permission to the IAM role that you specify in your CreateTrainingJob request. Using Key Policies in AWS KMS in the AWS Key Management Service Developer Guide .
S3OutputPath (string) --
Identifies the S3 path where you want Amazon SageMaker to store the model artifacts. For example, s3://bucket-name/key-name-prefix .
ResourceConfig (dict) --
Resources, including ML compute instances and ML storage volumes, that are configured for model training.
InstanceType (string) --
The ML compute instance type.
InstanceCount (integer) --
The number of ML compute instances to use. For distributed training, provide a value greater than 1.
VolumeSizeInGB (integer) --
The size of the ML storage volume that you want to provision.
ML storage volumes store model artifacts and incremental states. Training algorithms might also use the ML storage volume for scratch space. If you want to store the training data in the ML storage volume, choose File as the TrainingInputMode in the algorithm specification.
You must specify sufficient ML storage for your scenario.
Note
Amazon SageMaker supports only the General Purpose SSD (gp2) ML storage volume type.
VolumeKmsKeyId (string) --
The AWS Key Management Service (AWS KMS) key that Amazon SageMaker uses to encrypt data on the storage volume attached to the ML compute instance(s) that run the training job. The VolumeKmsKeyId can be any of the following formats:
VpcConfig (dict) --
A VpcConfig object that specifies the VPC that this training job has access to. For more information, see Protect Training Jobs by Using an Amazon Virtual Private Cloud .
SecurityGroupIds (list) --
The VPC security group IDs, in the form sg-xxxxxxxx. Specify the security groups for the VPC that is specified in the Subnets field.
Subnets (list) --
The ID of the subnets in the VPC to which you want to connect your training job or model.
StoppingCondition (dict) --
The condition under which to stop the training job.
MaxRuntimeInSeconds (integer) --
The maximum length of time, in seconds, that the training or compilation job can run. If the job does not complete during this time, Amazon SageMaker ends the job. If value is not specified, default value is 1 day. Maximum value is 5 days.
CreationTime (datetime) --
A timestamp that indicates when the training job was created.
TrainingStartTime (datetime) --
Indicates the time when the training job starts on training instances. You are billed for the time interval between this time and the value of TrainingEndTime . The start time in CloudWatch Logs might be later than this time. The difference is due to the time it takes to download the training data and to the size of the training container.
TrainingEndTime (datetime) --
Indicates the time when the training job ends on training instances. You are billed for the time interval between the value of TrainingStartTime and this time. For successful jobs and stopped jobs, this is the time after model artifacts are uploaded. For failed jobs, this is the time when Amazon SageMaker detects a job failure.
LastModifiedTime (datetime) --
A timestamp that indicates when the status of the training job was last modified.
SecondaryStatusTransitions (list) --
A history of all of the secondary statuses that the training job has transitioned through.
(dict) --
An array element of DescribeTrainingJobResponse$SecondaryStatusTransitions . It provides additional details about a status that the training job has transitioned through. A training job can be in one of several states, for example, starting, downloading, training, or uploading. Within each state, there are a number of intermediate states. For example, within the starting state, Amazon SageMaker could be starting the training job or launching the ML instances. These transitional states are referred to as the job's secondary status.
Status (string) --
Contains a secondary status information from a training job.
Status might be one of the following secondary statuses:
InProgress
Starting - Starting the training job.
Downloading - An optional stage for algorithms that support File training input mode. It indicates that data is being downloaded to the ML storage volumes.
Training - Training is in progress.
Uploading - Training is complete and the model artifacts are being uploaded to the S3 location.
Completed
Completed - The training job has completed.
Failed
Failed - The training job has failed. The reason for the failure is returned in the FailureReason field of DescribeTrainingJobResponse .
Stopped
MaxRuntimeExceeded - The job stopped because it exceeded the maximum allowed runtime.
Stopped - The training job has stopped.
Stopping
Stopping - Stopping the training job.
We no longer support the following secondary statuses:
StartTime (datetime) --
A timestamp that shows when the training job transitioned to the current secondary status state.
EndTime (datetime) --
A timestamp that shows when the training job transitioned out of this secondary status state into another secondary status state or when the training job has ended.
StatusMessage (string) --
A detailed description of the progress within a secondary status.
Amazon SageMaker provides secondary statuses and status messages that apply to each of them:
Starting
Starting the training job.
Launching requested ML instances.
Insufficient capacity error from EC2 while launching instances, retrying!
Launched instance was unhealthy, replacing it!
Preparing the instances for training.
Training
Downloading the training image.
Training image download completed. Training in progress.
Warning
Status messages are subject to change. Therefore, we recommend not including them in code that programmatically initiates actions. For examples, don't use status messages in if statements.
To have an overview of your training job's progress, view TrainingJobStatus and SecondaryStatus in DescribeTrainingJobResponse , and StatusMessage together. For example, at the start of a training job, you might see the following:
FinalMetricDataList (list) --
A list of final metric values that are set when the Training Job completes. Used only if the training job was configured to use metrics.
(dict) --
The name, value, and date and time of a metric that was emitted to Amazon CloudWatch.
MetricName (string) --
The name of the metric.
Value (float) --
The value of the metric.
Timestamp (datetime) --
The date and time that the algorithm emitted the metric.
EnableNetworkIsolation (boolean) --
If the TrainingJob was created with network isolation, the value is set to true . If network isolation is enabled, nodes can't communicate beyond the VPC they run in.
Tags (list) --
An array of key-value pairs. For more information, see Using Cost Allocation Tags in the AWS Billing and Cost Management User Guide .
(dict) --
Describes a tag.
Key (string) --
The tag key.
Value (string) --
The tag value.
NextToken (string) --
If the result of the previous Search request was truncated, the response includes a NextToken. To retrieve the next set of results, use the token in the next request.
Launches an ML compute instance with the latest version of the libraries and attaches your ML storage volume. After configuring the notebook instance, Amazon SageMaker sets the notebook instance status to InService . A notebook instance's status must be InService before you can connect to your Jupyter notebook.
See also: AWS API Documentation
Request Syntax
response = client.start_notebook_instance(
NotebookInstanceName='string'
)
[REQUIRED]
The name of the notebook instance to start.
Stops a model compilation job.
To stop a job, Amazon SageMaker sends the algorithm the SIGTERM signal. This gracefully shuts the job down. If the job hasn’t stopped, it sends the SIGKILL signal.
When it receives a StopCompilationJob request, Amazon SageMaker changes the CompilationJobSummary$CompilationJobStatus of the job to Stopping . After Amazon SageMaker stops the job, it sets the CompilationJobSummary$CompilationJobStatus to Stopped .
See also: AWS API Documentation
Request Syntax
response = client.stop_compilation_job(
CompilationJobName='string'
)
[REQUIRED]
The name of the model compilation job to stop.
Stops a running hyperparameter tuning job and all running training jobs that the tuning job launched.
All model artifacts output from the training jobs are stored in Amazon Simple Storage Service (Amazon S3). All data that the training jobs write to Amazon CloudWatch Logs are still available in CloudWatch. After the tuning job moves to the Stopped state, it releases all reserved resources for the tuning job.
See also: AWS API Documentation
Request Syntax
response = client.stop_hyper_parameter_tuning_job(
HyperParameterTuningJobName='string'
)
[REQUIRED]
The name of the tuning job to stop.
Stops a running labeling job. A job that is stopped cannot be restarted. Any results obtained before the job is stopped are placed in the Amazon S3 output bucket.
See also: AWS API Documentation
Request Syntax
response = client.stop_labeling_job(
LabelingJobName='string'
)
[REQUIRED]
The name of the labeling job to stop.
Terminates the ML compute instance. Before terminating the instance, Amazon SageMaker disconnects the ML storage volume from it. Amazon SageMaker preserves the ML storage volume.
To access data on the ML storage volume for a notebook instance that has been terminated, call the StartNotebookInstance API. StartNotebookInstance launches another ML compute instance, configures it, and attaches the preserved ML storage volume so you can continue your work.
See also: AWS API Documentation
Request Syntax
response = client.stop_notebook_instance(
NotebookInstanceName='string'
)
[REQUIRED]
The name of the notebook instance to terminate.
Stops a training job. To stop a job, Amazon SageMaker sends the algorithm the SIGTERM signal, which delays job termination for 120 seconds. Algorithms might use this 120-second window to save the model artifacts, so the results of the training is not lost.
Training algorithms provided by Amazon SageMaker save the intermediate results of a model training job. This intermediate data is a valid model artifact. You can use the model artifacts that are saved when Amazon SageMaker stops a training job to create a model.
When it receives a StopTrainingJob request, Amazon SageMaker changes the status of the job to Stopping . After Amazon SageMaker stops the job, it sets the status to Stopped .
See also: AWS API Documentation
Request Syntax
response = client.stop_training_job(
TrainingJobName='string'
)
[REQUIRED]
The name of the training job to stop.
Stops a transform job.
When Amazon SageMaker receives a StopTransformJob request, the status of the job changes to Stopping . After Amazon SageMaker stops the job, the status is set to Stopped . When you stop a transform job before it is completed, Amazon SageMaker doesn't store the job's output in Amazon S3.
See also: AWS API Documentation
Request Syntax
response = client.stop_transform_job(
TransformJobName='string'
)
[REQUIRED]
The name of the transform job to stop.
Updates the specified git repository with the specified values.
See also: AWS API Documentation
Request Syntax
response = client.update_code_repository(
CodeRepositoryName='string',
GitConfig={
'SecretArn': 'string'
}
)
[REQUIRED]
The name of the git repository to update.
The configuration of the git repository, including the URL and the Amazon Resource Name (ARN) of the AWS Secrets Manager secret that contains the credentials used to access the repository. The secret must have a staging label of AWSCURRENT and must be in the following format:
{"username": *UserName* , "password": *Password* }
The Amazon Resource Name (ARN) of the AWS Secrets Manager secret that contains the credentials used to access the git repository. The secret must have a staging label of AWSCURRENT and must be in the following format:
{"username": *UserName* , "password": *Password* }
dict
Response Syntax
{
'CodeRepositoryArn': 'string'
}
Response Structure
(dict) --
CodeRepositoryArn (string) --
The ARN of the git repository.
Deploys the new EndpointConfig specified in the request, switches to using newly created endpoint, and then deletes resources provisioned for the endpoint using the previous EndpointConfig (there is no availability loss).
When Amazon SageMaker receives the request, it sets the endpoint status to Updating . After updating the endpoint, it sets the status to InService . To check the status of an endpoint, use the DescribeEndpoint API.
Note
You cannot update an endpoint with the current EndpointConfig . To update an endpoint, you must create a new EndpointConfig .
See also: AWS API Documentation
Request Syntax
response = client.update_endpoint(
EndpointName='string',
EndpointConfigName='string'
)
[REQUIRED]
The name of the endpoint whose configuration you want to update.
[REQUIRED]
The name of the new endpoint configuration.
dict
Response Syntax
{
'EndpointArn': 'string'
}
Response Structure
(dict) --
EndpointArn (string) --
The Amazon Resource Name (ARN) of the endpoint.
Updates variant weight of one or more variants associated with an existing endpoint, or capacity of one variant associated with an existing endpoint. When it receives the request, Amazon SageMaker sets the endpoint status to Updating . After updating the endpoint, it sets the status to InService . To check the status of an endpoint, use the DescribeEndpoint API.
See also: AWS API Documentation
Request Syntax
response = client.update_endpoint_weights_and_capacities(
EndpointName='string',
DesiredWeightsAndCapacities=[
{
'VariantName': 'string',
'DesiredWeight': ...,
'DesiredInstanceCount': 123
},
]
)
[REQUIRED]
The name of an existing Amazon SageMaker endpoint.
[REQUIRED]
An object that provides new capacity and weight values for a variant.
Specifies weight and capacity values for a production variant.
The name of the variant to update.
The variant's weight.
The variant's capacity.
dict
Response Syntax
{
'EndpointArn': 'string'
}
Response Structure
(dict) --
EndpointArn (string) --
The Amazon Resource Name (ARN) of the updated endpoint.
Updates a notebook instance. NotebookInstance updates include upgrading or downgrading the ML compute instance used for your notebook instance to accommodate changes in your workload requirements. You can also update the VPC security groups.
See also: AWS API Documentation
Request Syntax
response = client.update_notebook_instance(
NotebookInstanceName='string',
InstanceType='ml.t2.medium'|'ml.t2.large'|'ml.t2.xlarge'|'ml.t2.2xlarge'|'ml.t3.medium'|'ml.t3.large'|'ml.t3.xlarge'|'ml.t3.2xlarge'|'ml.m4.xlarge'|'ml.m4.2xlarge'|'ml.m4.4xlarge'|'ml.m4.10xlarge'|'ml.m4.16xlarge'|'ml.m5.xlarge'|'ml.m5.2xlarge'|'ml.m5.4xlarge'|'ml.m5.12xlarge'|'ml.m5.24xlarge'|'ml.c4.xlarge'|'ml.c4.2xlarge'|'ml.c4.4xlarge'|'ml.c4.8xlarge'|'ml.c5.xlarge'|'ml.c5.2xlarge'|'ml.c5.4xlarge'|'ml.c5.9xlarge'|'ml.c5.18xlarge'|'ml.c5d.xlarge'|'ml.c5d.2xlarge'|'ml.c5d.4xlarge'|'ml.c5d.9xlarge'|'ml.c5d.18xlarge'|'ml.p2.xlarge'|'ml.p2.8xlarge'|'ml.p2.16xlarge'|'ml.p3.2xlarge'|'ml.p3.8xlarge'|'ml.p3.16xlarge',
RoleArn='string',
LifecycleConfigName='string',
DisassociateLifecycleConfig=True|False,
VolumeSizeInGB=123,
DefaultCodeRepository='string',
AdditionalCodeRepositories=[
'string',
],
AcceleratorTypes=[
'ml.eia1.medium'|'ml.eia1.large'|'ml.eia1.xlarge',
],
DisassociateAcceleratorTypes=True|False,
DisassociateDefaultCodeRepository=True|False,
DisassociateAdditionalCodeRepositories=True|False
)
[REQUIRED]
The name of the notebook instance to update.
The Amazon Resource Name (ARN) of the IAM role that Amazon SageMaker can assume to access the notebook instance. For more information, see Amazon SageMaker Roles .
Note
To be able to pass this role to Amazon SageMaker, the caller of this API must have the iam:PassRole permission.
An array of up to 3 git repositories to associate with the notebook instance. These can be either the names of git repositories stored as resources in your account, or the URL of git repositories in AWS CodeCommit or in any other git repository.. These repositories are cloned at the same level as the default repository of your notebook instance. For more information, see Associating Git Repositories with Amazon SageMaker Notebook Instances .
A list of the Elastic Inference (EI) instance types to associate with this notebook instance. Currently only one EI instance type can be associated with a notebook instance. For more information, see Using Elastic Inference in Amazon SageMaker .
dict
Response Syntax
{}
Response Structure
Updates a notebook instance lifecycle configuration created with the CreateNotebookInstanceLifecycleConfig API.
See also: AWS API Documentation
Request Syntax
response = client.update_notebook_instance_lifecycle_config(
NotebookInstanceLifecycleConfigName='string',
OnCreate=[
{
'Content': 'string'
},
],
OnStart=[
{
'Content': 'string'
},
]
)
[REQUIRED]
The name of the lifecycle configuration.
The shell script that runs only once, when you create a notebook instance
Contains the notebook instance lifecycle configuration script.
Each lifecycle configuration script has a limit of 16384 characters.
The value of the $PATH environment variable that is available to both scripts is /sbin:bin:/usr/sbin:/usr/bin .
View CloudWatch Logs for notebook instance lifecycle configurations in log group /aws/sagemaker/NotebookInstances in log stream [notebook-instance-name]/[LifecycleConfigHook] .
Lifecycle configuration scripts cannot run for longer than 5 minutes. If a script runs for longer than 5 minutes, it fails and the notebook instance is not created or started.
For information about notebook instance lifestyle configurations, see Step 2.1: (Optional) Customize a Notebook Instance .
A base64-encoded string that contains a shell script for a notebook instance lifecycle configuration.
The shell script that runs every time you start a notebook instance, including when you create the notebook instance.
Contains the notebook instance lifecycle configuration script.
Each lifecycle configuration script has a limit of 16384 characters.
The value of the $PATH environment variable that is available to both scripts is /sbin:bin:/usr/sbin:/usr/bin .
View CloudWatch Logs for notebook instance lifecycle configurations in log group /aws/sagemaker/NotebookInstances in log stream [notebook-instance-name]/[LifecycleConfigHook] .
Lifecycle configuration scripts cannot run for longer than 5 minutes. If a script runs for longer than 5 minutes, it fails and the notebook instance is not created or started.
For information about notebook instance lifestyle configurations, see Step 2.1: (Optional) Customize a Notebook Instance .
A base64-encoded string that contains a shell script for a notebook instance lifecycle configuration.
dict
Response Syntax
{}
Response Structure
Updates an existing work team with new member definitions or description.
See also: AWS API Documentation
Request Syntax
response = client.update_workteam(
WorkteamName='string',
MemberDefinitions=[
{
'CognitoMemberDefinition': {
'UserPool': 'string',
'UserGroup': 'string',
'ClientId': 'string'
}
},
],
Description='string'
)
[REQUIRED]
The name of the work team to update.
A list of MemberDefinition objects that contain the updated work team members.
Defines the Amazon Cognito user group that is part of a work team.
The Amazon Cognito user group that is part of the work team.
An identifier for a user pool. The user pool must be in the same region as the service that you are calling.
An identifier for a user group.
An identifier for an application client. You must create the app client ID using Amazon Cognito.
dict
Response Syntax
{
'Workteam': {
'WorkteamName': 'string',
'MemberDefinitions': [
{
'CognitoMemberDefinition': {
'UserPool': 'string',
'UserGroup': 'string',
'ClientId': 'string'
}
},
],
'WorkteamArn': 'string',
'ProductListingIds': [
'string',
],
'Description': 'string',
'SubDomain': 'string',
'CreateDate': datetime(2015, 1, 1),
'LastUpdatedDate': datetime(2015, 1, 1)
}
}
Response Structure
(dict) --
Workteam (dict) --
A Workteam object that describes the updated work team.
WorkteamName (string) --
The name of the work team.
MemberDefinitions (list) --
The Amazon Cognito user groups that make up the work team.
(dict) --
Defines the Amazon Cognito user group that is part of a work team.
CognitoMemberDefinition (dict) --
The Amazon Cognito user group that is part of the work team.
UserPool (string) --
An identifier for a user pool. The user pool must be in the same region as the service that you are calling.
UserGroup (string) --
An identifier for a user group.
ClientId (string) --
An identifier for an application client. You must create the app client ID using Amazon Cognito.
WorkteamArn (string) --
The Amazon Resource Name (ARN) that identifies the work team.
ProductListingIds (list) --
The Amazon Marketplace identifier for a vendor's work team.
Description (string) --
A description of the work team.
SubDomain (string) --
The URI of the labeling job's user interface. Workers open this URI to start labeling your data objects.
CreateDate (datetime) --
The date and time that the work team was created (timestamp).
LastUpdatedDate (datetime) --
The date and time that the work team was last updated (timestamp).
The available paginators are:
paginator = client.get_paginator('list_endpoint_configs')
Creates an iterator that will paginate through responses from SageMaker.Client.list_endpoint_configs().
See also: AWS API Documentation
Request Syntax
response_iterator = paginator.paginate(
SortBy='Name'|'CreationTime',
SortOrder='Ascending'|'Descending',
NameContains='string',
CreationTimeBefore=datetime(2015, 1, 1),
CreationTimeAfter=datetime(2015, 1, 1),
PaginationConfig={
'MaxItems': 123,
'PageSize': 123,
'StartingToken': 'string'
}
)
A dictionary that provides parameters to control pagination.
The total number of items to return. If the total number of items available is more than the value specified in max-items then a NextToken will be provided in the output that you can use to resume pagination.
The size of each page.
A token to specify where to start paginating. This is the NextToken from a previous response.
dict
Response Syntax
{
'EndpointConfigs': [
{
'EndpointConfigName': 'string',
'EndpointConfigArn': 'string',
'CreationTime': datetime(2015, 1, 1)
},
],
}
Response Structure
(dict) --
EndpointConfigs (list) --
An array of endpoint configurations.
(dict) --
Provides summary information for an endpoint configuration.
EndpointConfigName (string) --
The name of the endpoint configuration.
EndpointConfigArn (string) --
The Amazon Resource Name (ARN) of the endpoint configuration.
CreationTime (datetime) --
A timestamp that shows when the endpoint configuration was created.
paginator = client.get_paginator('list_endpoints')
Creates an iterator that will paginate through responses from SageMaker.Client.list_endpoints().
See also: AWS API Documentation
Request Syntax
response_iterator = paginator.paginate(
SortBy='Name'|'CreationTime'|'Status',
SortOrder='Ascending'|'Descending',
NameContains='string',
CreationTimeBefore=datetime(2015, 1, 1),
CreationTimeAfter=datetime(2015, 1, 1),
LastModifiedTimeBefore=datetime(2015, 1, 1),
LastModifiedTimeAfter=datetime(2015, 1, 1),
StatusEquals='OutOfService'|'Creating'|'Updating'|'SystemUpdating'|'RollingBack'|'InService'|'Deleting'|'Failed',
PaginationConfig={
'MaxItems': 123,
'PageSize': 123,
'StartingToken': 'string'
}
)
A dictionary that provides parameters to control pagination.
The total number of items to return. If the total number of items available is more than the value specified in max-items then a NextToken will be provided in the output that you can use to resume pagination.
The size of each page.
A token to specify where to start paginating. This is the NextToken from a previous response.
dict
Response Syntax
{
'Endpoints': [
{
'EndpointName': 'string',
'EndpointArn': 'string',
'CreationTime': datetime(2015, 1, 1),
'LastModifiedTime': datetime(2015, 1, 1),
'EndpointStatus': 'OutOfService'|'Creating'|'Updating'|'SystemUpdating'|'RollingBack'|'InService'|'Deleting'|'Failed'
},
],
}
Response Structure
(dict) --
Endpoints (list) --
An array or endpoint objects.
(dict) --
Provides summary information for an endpoint.
EndpointName (string) --
The name of the endpoint.
EndpointArn (string) --
The Amazon Resource Name (ARN) of the endpoint.
CreationTime (datetime) --
A timestamp that shows when the endpoint was created.
LastModifiedTime (datetime) --
A timestamp that shows when the endpoint was last modified.
EndpointStatus (string) --
The status of the endpoint.
To get a list of endpoints with a specified status, use the ListEndpointsInput$StatusEquals filter.
paginator = client.get_paginator('list_models')
Creates an iterator that will paginate through responses from SageMaker.Client.list_models().
See also: AWS API Documentation
Request Syntax
response_iterator = paginator.paginate(
SortBy='Name'|'CreationTime',
SortOrder='Ascending'|'Descending',
NameContains='string',
CreationTimeBefore=datetime(2015, 1, 1),
CreationTimeAfter=datetime(2015, 1, 1),
PaginationConfig={
'MaxItems': 123,
'PageSize': 123,
'StartingToken': 'string'
}
)
A dictionary that provides parameters to control pagination.
The total number of items to return. If the total number of items available is more than the value specified in max-items then a NextToken will be provided in the output that you can use to resume pagination.
The size of each page.
A token to specify where to start paginating. This is the NextToken from a previous response.
dict
Response Syntax
{
'Models': [
{
'ModelName': 'string',
'ModelArn': 'string',
'CreationTime': datetime(2015, 1, 1)
},
],
}
Response Structure
(dict) --
Models (list) --
An array of ModelSummary objects, each of which lists a model.
(dict) --
Provides summary information about a model.
ModelName (string) --
The name of the model that you want a summary for.
ModelArn (string) --
The Amazon Resource Name (ARN) of the model.
CreationTime (datetime) --
A timestamp that indicates when the model was created.
paginator = client.get_paginator('list_notebook_instances')
Creates an iterator that will paginate through responses from SageMaker.Client.list_notebook_instances().
See also: AWS API Documentation
Request Syntax
response_iterator = paginator.paginate(
SortBy='Name'|'CreationTime'|'Status',
SortOrder='Ascending'|'Descending',
NameContains='string',
CreationTimeBefore=datetime(2015, 1, 1),
CreationTimeAfter=datetime(2015, 1, 1),
LastModifiedTimeBefore=datetime(2015, 1, 1),
LastModifiedTimeAfter=datetime(2015, 1, 1),
StatusEquals='Pending'|'InService'|'Stopping'|'Stopped'|'Failed'|'Deleting'|'Updating',
NotebookInstanceLifecycleConfigNameContains='string',
DefaultCodeRepositoryContains='string',
AdditionalCodeRepositoryEquals='string',
PaginationConfig={
'MaxItems': 123,
'PageSize': 123,
'StartingToken': 'string'
}
)
A dictionary that provides parameters to control pagination.
The total number of items to return. If the total number of items available is more than the value specified in max-items then a NextToken will be provided in the output that you can use to resume pagination.
The size of each page.
A token to specify where to start paginating. This is the NextToken from a previous response.
dict
Response Syntax
{
'NotebookInstances': [
{
'NotebookInstanceName': 'string',
'NotebookInstanceArn': 'string',
'NotebookInstanceStatus': 'Pending'|'InService'|'Stopping'|'Stopped'|'Failed'|'Deleting'|'Updating',
'Url': 'string',
'InstanceType': 'ml.t2.medium'|'ml.t2.large'|'ml.t2.xlarge'|'ml.t2.2xlarge'|'ml.t3.medium'|'ml.t3.large'|'ml.t3.xlarge'|'ml.t3.2xlarge'|'ml.m4.xlarge'|'ml.m4.2xlarge'|'ml.m4.4xlarge'|'ml.m4.10xlarge'|'ml.m4.16xlarge'|'ml.m5.xlarge'|'ml.m5.2xlarge'|'ml.m5.4xlarge'|'ml.m5.12xlarge'|'ml.m5.24xlarge'|'ml.c4.xlarge'|'ml.c4.2xlarge'|'ml.c4.4xlarge'|'ml.c4.8xlarge'|'ml.c5.xlarge'|'ml.c5.2xlarge'|'ml.c5.4xlarge'|'ml.c5.9xlarge'|'ml.c5.18xlarge'|'ml.c5d.xlarge'|'ml.c5d.2xlarge'|'ml.c5d.4xlarge'|'ml.c5d.9xlarge'|'ml.c5d.18xlarge'|'ml.p2.xlarge'|'ml.p2.8xlarge'|'ml.p2.16xlarge'|'ml.p3.2xlarge'|'ml.p3.8xlarge'|'ml.p3.16xlarge',
'CreationTime': datetime(2015, 1, 1),
'LastModifiedTime': datetime(2015, 1, 1),
'NotebookInstanceLifecycleConfigName': 'string',
'DefaultCodeRepository': 'string',
'AdditionalCodeRepositories': [
'string',
]
},
]
}
Response Structure
(dict) --
NotebookInstances (list) --
An array of NotebookInstanceSummary objects, one for each notebook instance.
(dict) --
Provides summary information for an Amazon SageMaker notebook instance.
NotebookInstanceName (string) --
The name of the notebook instance that you want a summary for.
NotebookInstanceArn (string) --
The Amazon Resource Name (ARN) of the notebook instance.
NotebookInstanceStatus (string) --
The status of the notebook instance.
Url (string) --
The URL that you use to connect to the Jupyter instance running in your notebook instance.
InstanceType (string) --
The type of ML compute instance that the notebook instance is running on.
CreationTime (datetime) --
A timestamp that shows when the notebook instance was created.
LastModifiedTime (datetime) --
A timestamp that shows when the notebook instance was last modified.
NotebookInstanceLifecycleConfigName (string) --
The name of a notebook instance lifecycle configuration associated with this notebook instance.
For information about notebook instance lifestyle configurations, see Step 2.1: (Optional) Customize a Notebook Instance .
DefaultCodeRepository (string) --
The git repository associated with the notebook instance as its default code repository. This can be either the name of a git repository stored as a resource in your account, or the URL of a git repository in AWS CodeCommit or in any other git repository. When you open a notebook instance, it opens in the directory that contains this repository. For more information, see Associating Git Repositories with Amazon SageMaker Notebook Instances .
AdditionalCodeRepositories (list) --
An array of up to 3 git repositories associated with the notebook instance. These can be either the names of git repositories stored as resources in your account, or the URL of git repositories in AWS CodeCommit or in any other git repository. These repositories are cloned at the same level as the default repository of your notebook instance. For more information, see Associating Git Repositories with Amazon SageMaker Notebook Instances .
paginator = client.get_paginator('list_tags')
Creates an iterator that will paginate through responses from SageMaker.Client.list_tags().
See also: AWS API Documentation
Request Syntax
response_iterator = paginator.paginate(
ResourceArn='string',
PaginationConfig={
'MaxItems': 123,
'PageSize': 123,
'StartingToken': 'string'
}
)
[REQUIRED]
The Amazon Resource Name (ARN) of the resource whose tags you want to retrieve.
A dictionary that provides parameters to control pagination.
The total number of items to return. If the total number of items available is more than the value specified in max-items then a NextToken will be provided in the output that you can use to resume pagination.
The size of each page.
A token to specify where to start paginating. This is the NextToken from a previous response.
dict
Response Syntax
{
'Tags': [
{
'Key': 'string',
'Value': 'string'
},
],
}
Response Structure
(dict) --
Tags (list) --
An array of Tag objects, each with a tag key and a value.
(dict) --
Describes a tag.
Key (string) --
The tag key.
Value (string) --
The tag value.
paginator = client.get_paginator('list_training_jobs')
Creates an iterator that will paginate through responses from SageMaker.Client.list_training_jobs().
See also: AWS API Documentation
Request Syntax
response_iterator = paginator.paginate(
CreationTimeAfter=datetime(2015, 1, 1),
CreationTimeBefore=datetime(2015, 1, 1),
LastModifiedTimeAfter=datetime(2015, 1, 1),
LastModifiedTimeBefore=datetime(2015, 1, 1),
NameContains='string',
StatusEquals='InProgress'|'Completed'|'Failed'|'Stopping'|'Stopped',
SortBy='Name'|'CreationTime'|'Status',
SortOrder='Ascending'|'Descending',
PaginationConfig={
'MaxItems': 123,
'PageSize': 123,
'StartingToken': 'string'
}
)
A dictionary that provides parameters to control pagination.
The total number of items to return. If the total number of items available is more than the value specified in max-items then a NextToken will be provided in the output that you can use to resume pagination.
The size of each page.
A token to specify where to start paginating. This is the NextToken from a previous response.
dict
Response Syntax
{
'TrainingJobSummaries': [
{
'TrainingJobName': 'string',
'TrainingJobArn': 'string',
'CreationTime': datetime(2015, 1, 1),
'TrainingEndTime': datetime(2015, 1, 1),
'LastModifiedTime': datetime(2015, 1, 1),
'TrainingJobStatus': 'InProgress'|'Completed'|'Failed'|'Stopping'|'Stopped'
},
],
}
Response Structure
(dict) --
TrainingJobSummaries (list) --
An array of TrainingJobSummary objects, each listing a training job.
(dict) --
Provides summary information about a training job.
TrainingJobName (string) --
The name of the training job that you want a summary for.
TrainingJobArn (string) --
The Amazon Resource Name (ARN) of the training job.
CreationTime (datetime) --
A timestamp that shows when the training job was created.
TrainingEndTime (datetime) --
A timestamp that shows when the training job ended. This field is set only if the training job has one of the terminal statuses (Completed , Failed , or Stopped ).
LastModifiedTime (datetime) --
Timestamp when the training job was last modified.
TrainingJobStatus (string) --
The status of the training job.
The available waiters are:
waiter = client.get_waiter('endpoint_deleted')
Polls SageMaker.Client.describe_endpoint() every 30 seconds until a successful state is reached. An error is returned after 60 failed checks.
See also: AWS API Documentation
Request Syntax
waiter.wait(
EndpointName='string',
WaiterConfig={
'Delay': 123,
'MaxAttempts': 123
}
)
[REQUIRED]
The name of the endpoint.
A dictionary that provides parameters to control waiting behavior.
The amount of time in seconds to wait between attempts. Default: 30
The maximum number of attempts to be made. Default: 60
None
waiter = client.get_waiter('endpoint_in_service')
Polls SageMaker.Client.describe_endpoint() every 30 seconds until a successful state is reached. An error is returned after 120 failed checks.
See also: AWS API Documentation
Request Syntax
waiter.wait(
EndpointName='string',
WaiterConfig={
'Delay': 123,
'MaxAttempts': 123
}
)
[REQUIRED]
The name of the endpoint.
A dictionary that provides parameters to control waiting behavior.
The amount of time in seconds to wait between attempts. Default: 30
The maximum number of attempts to be made. Default: 120
None
waiter = client.get_waiter('notebook_instance_deleted')
Polls SageMaker.Client.describe_notebook_instance() every 30 seconds until a successful state is reached. An error is returned after 60 failed checks.
See also: AWS API Documentation
Request Syntax
waiter.wait(
NotebookInstanceName='string',
WaiterConfig={
'Delay': 123,
'MaxAttempts': 123
}
)
[REQUIRED]
The name of the notebook instance that you want information about.
A dictionary that provides parameters to control waiting behavior.
The amount of time in seconds to wait between attempts. Default: 30
The maximum number of attempts to be made. Default: 60
None
waiter = client.get_waiter('notebook_instance_in_service')
Polls SageMaker.Client.describe_notebook_instance() every 30 seconds until a successful state is reached. An error is returned after 60 failed checks.
See also: AWS API Documentation
Request Syntax
waiter.wait(
NotebookInstanceName='string',
WaiterConfig={
'Delay': 123,
'MaxAttempts': 123
}
)
[REQUIRED]
The name of the notebook instance that you want information about.
A dictionary that provides parameters to control waiting behavior.
The amount of time in seconds to wait between attempts. Default: 30
The maximum number of attempts to be made. Default: 60
None
waiter = client.get_waiter('notebook_instance_stopped')
Polls SageMaker.Client.describe_notebook_instance() every 30 seconds until a successful state is reached. An error is returned after 60 failed checks.
See also: AWS API Documentation
Request Syntax
waiter.wait(
NotebookInstanceName='string',
WaiterConfig={
'Delay': 123,
'MaxAttempts': 123
}
)
[REQUIRED]
The name of the notebook instance that you want information about.
A dictionary that provides parameters to control waiting behavior.
The amount of time in seconds to wait between attempts. Default: 30
The maximum number of attempts to be made. Default: 60
None
waiter = client.get_waiter('training_job_completed_or_stopped')
Polls SageMaker.Client.describe_training_job() every 120 seconds until a successful state is reached. An error is returned after 180 failed checks.
See also: AWS API Documentation
Request Syntax
waiter.wait(
TrainingJobName='string',
WaiterConfig={
'Delay': 123,
'MaxAttempts': 123
}
)
[REQUIRED]
The name of the training job.
A dictionary that provides parameters to control waiting behavior.
The amount of time in seconds to wait between attempts. Default: 120
The maximum number of attempts to be made. Default: 180
None
waiter = client.get_waiter('transform_job_completed_or_stopped')
Polls SageMaker.Client.describe_transform_job() every 60 seconds until a successful state is reached. An error is returned after 60 failed checks.
See also: AWS API Documentation
Request Syntax
waiter.wait(
TransformJobName='string',
WaiterConfig={
'Delay': 123,
'MaxAttempts': 123
}
)
[REQUIRED]
The name of the transform job that you want to view details of.
A dictionary that provides parameters to control waiting behavior.
The amount of time in seconds to wait between attempts. Default: 60
The maximum number of attempts to be made. Default: 60
None