Object / Action / initiate_multipart_upload

initiate_multipart_upload#

S3.Object.initiate_multipart_upload(**kwargs)#

This action initiates a multipart upload and returns an upload ID. This upload ID is used to associate all of the parts in the specific multipart upload. You specify this upload ID in each of your subsequent upload part requests (see UploadPart). You also include this upload ID in the final request to either complete or abort the multipart upload request. For more information about multipart uploads, see Multipart Upload Overview in the Amazon S3 User Guide.

Note

After you initiate a multipart upload and upload one or more parts, to stop being charged for storing the uploaded parts, you must either complete or abort the multipart upload. Amazon S3 frees up the space used to store the parts and stops charging you for storing them only after you either complete or abort a multipart upload.

If you have configured a lifecycle rule to abort incomplete multipart uploads, the created multipart upload must be completed within the number of days specified in the bucket lifecycle configuration. Otherwise, the incomplete multipart upload becomes eligible for an abort action and Amazon S3 aborts the multipart upload. For more information, see Aborting Incomplete Multipart Uploads Using a Bucket Lifecycle Configuration.

Note

Request signing

For request signing, multipart upload is just a series of regular requests. You initiate a multipart upload, send one or more requests to upload parts, and then complete the multipart upload process. You sign each request individually. There is nothing special about signing multipart upload requests. For more information about signing, see Authenticating Requests (Amazon Web Services Signature Version 4) in the Amazon S3 User Guide.

Permissions

  • General purpose bucket permissions - To perform a multipart upload with encryption using an Key Management Service (KMS) KMS key, the requester must have permission to the kms:Decrypt and kms:GenerateDataKey actions on the key. The requester must also have permissions for the kms:GenerateDataKey action for the CreateMultipartUpload API. Then, the requester needs permissions for the kms:Decrypt action on the UploadPart and UploadPartCopy APIs. These permissions are required because Amazon S3 must decrypt and read data from the encrypted file parts before it completes the multipart upload. For more information, see Multipart upload API and permissions and Protecting data using server-side encryption with Amazon Web Services KMS in the Amazon S3 User Guide.

  • Directory bucket permissions - To grant access to this API operation on a directory bucket, we recommend that you use the CreateSession API operation for session-based authorization. Specifically, you grant the s3express:CreateSession permission to the directory bucket in a bucket policy or an IAM identity-based policy. Then, you make the CreateSession API call on the bucket to obtain a session token. With the session token in your request header, you can make API requests to this operation. After the session token expires, you make another CreateSession API call to generate a new session token for use. Amazon Web Services CLI or SDKs create session and refresh the session token automatically to avoid service interruptions when a session expires. For more information about authorization, see CreateSession.

    Encryption

  • General purpose buckets - Server-side encryption is for data encryption at rest. Amazon S3 encrypts your data as it writes it to disks in its data centers and decrypts it when you access it. Amazon S3 automatically encrypts all new objects that are uploaded to an S3 bucket. When doing a multipart upload, if you don’t specify encryption information in your request, the encryption setting of the uploaded parts is set to the default encryption configuration of the destination bucket. By default, all buckets have a base level of encryption configuration that uses server-side encryption with Amazon S3 managed keys (SSE-S3). If the destination bucket has a default encryption configuration that uses server-side encryption with an Key Management Service (KMS) key (SSE-KMS), or a customer-provided encryption key (SSE-C), Amazon S3 uses the corresponding KMS key, or a customer-provided key to encrypt the uploaded parts. When you perform a CreateMultipartUpload operation, if you want to use a different type of encryption setting for the uploaded parts, you can request that Amazon S3 encrypts the object with a different encryption key (such as an Amazon S3 managed key, a KMS key, or a customer-provided key). When the encryption setting in your request is different from the default encryption configuration of the destination bucket, the encryption setting in your request takes precedence. If you choose to provide your own encryption key, the request headers you provide in UploadPart and UploadPartCopy requests must match the headers you used in the CreateMultipartUpload request.

    • Use KMS keys (SSE-KMS) that include the Amazon Web Services managed key ( aws/s3) and KMS customer managed keys stored in Key Management Service (KMS) – If you want Amazon Web Services to manage the keys used to encrypt data, specify the following headers in the request.

      • x-amz-server-side-encryption

      • x-amz-server-side-encryption-aws-kms-key-id

      • x-amz-server-side-encryption-context

    Note

    • If you specify x-amz-server-side-encryption:aws:kms, but don’t provide x-amz-server-side-encryption-aws-kms-key-id, Amazon S3 uses the Amazon Web Services managed key ( aws/s3 key) in KMS to protect the data.

    • To perform a multipart upload with encryption by using an Amazon Web Services KMS key, the requester must have permission to the kms:Decrypt and kms:GenerateDataKey* actions on the key. These permissions are required because Amazon S3 must decrypt and read data from the encrypted file parts before it completes the multipart upload. For more information, see Multipart upload API and permissions and Protecting data using server-side encryption with Amazon Web Services KMS in the Amazon S3 User Guide.

    • If your Identity and Access Management (IAM) user or role is in the same Amazon Web Services account as the KMS key, then you must have these permissions on the key policy. If your IAM user or role is in a different account from the key, then you must have the permissions on both the key policy and your IAM user or role.

    • All GET and PUT requests for an object protected by KMS fail if you don’t make them by using Secure Sockets Layer (SSL), Transport Layer Security (TLS), or Signature Version 4. For information about configuring any of the officially supported Amazon Web Services SDKs and Amazon Web Services CLI, see Specifying the Signature Version in Request Authentication in the Amazon S3 User Guide.

    For more information about server-side encryption with KMS keys (SSE-KMS), see Protecting Data Using Server-Side Encryption with KMS keys in the Amazon S3 User Guide.

    • Use customer-provided encryption keys (SSE-C) – If you want to manage your own encryption keys, provide all the following headers in the request.

      • x-amz-server-side-encryption-customer-algorithm

      • x-amz-server-side-encryption-customer-key

      • x-amz-server-side-encryption-customer-key-MD5

    For more information about server-side encryption with customer-provided encryption keys (SSE-C), see Protecting data using server-side encryption with customer-provided encryption keys (SSE-C) in the Amazon S3 User Guide.

  • Directory buckets - For directory buckets, there are only two supported options for server-side encryption: server-side encryption with Amazon S3 managed keys (SSE-S3) ( AES256) and server-side encryption with KMS keys (SSE-KMS) ( aws:kms). We recommend that the bucket’s default encryption uses the desired encryption configuration and you don’t override the bucket default encryption in your CreateSession requests or PUT object requests. Then, new objects are automatically encrypted with the desired encryption settings. For more information, see Protecting data with server-side encryption in the Amazon S3 User Guide. For more information about the encryption overriding behaviors in directory buckets, see Specifying server-side encryption with KMS for new object uploads. In the Zonal endpoint API calls (except CopyObject and UploadPartCopy) using the REST API, the encryption request headers must match the encryption settings that are specified in the CreateSession request. You can’t override the values of the encryption settings ( x-amz-server-side-encryption, x-amz-server-side-encryption-aws-kms-key-id, x-amz-server-side-encryption-context, and x-amz-server-side-encryption-bucket-key-enabled) that are specified in the CreateSession request. You don’t need to explicitly specify these encryption settings values in Zonal endpoint API calls, and Amazon S3 will use the encryption settings values from the CreateSession request to protect new objects in the directory bucket.

Note

When you use the CLI or the Amazon Web Services SDKs, for CreateSession, the session token refreshes automatically to avoid service interruptions when a session expires. The CLI or the Amazon Web Services SDKs use the bucket’s default encryption configuration for the CreateSession request. It’s not supported to override the encryption settings values in the CreateSession request. So in the Zonal endpoint API calls (except CopyObject and UploadPartCopy), the encryption request headers must match the default encryption configuration of the directory bucket.

Note

For directory buckets, when you perform a CreateMultipartUpload operation and an UploadPartCopy operation, the request headers you provide in the CreateMultipartUpload request must match the default encryption configuration of the destination bucket.

HTTP Host header syntax

Directory buckets - The HTTP Host header syntax is Bucket_name.s3express-az_id.region.amazonaws.com.

The following operations are related to CreateMultipartUpload:

See also: AWS API Documentation

Request Syntax

multipart_upload = object.initiate_multipart_upload(
    ACL='private'|'public-read'|'public-read-write'|'authenticated-read'|'aws-exec-read'|'bucket-owner-read'|'bucket-owner-full-control',
    CacheControl='string',
    ContentDisposition='string',
    ContentEncoding='string',
    ContentLanguage='string',
    ContentType='string',
    Expires=datetime(2015, 1, 1),
    GrantFullControl='string',
    GrantRead='string',
    GrantReadACP='string',
    GrantWriteACP='string',
    Metadata={
        'string': 'string'
    },
    ServerSideEncryption='AES256'|'aws:kms'|'aws:kms:dsse',
    StorageClass='STANDARD'|'REDUCED_REDUNDANCY'|'STANDARD_IA'|'ONEZONE_IA'|'INTELLIGENT_TIERING'|'GLACIER'|'DEEP_ARCHIVE'|'OUTPOSTS'|'GLACIER_IR'|'SNOW'|'EXPRESS_ONEZONE',
    WebsiteRedirectLocation='string',
    SSECustomerAlgorithm='string',
    SSECustomerKey='string',
    SSEKMSKeyId='string',
    SSEKMSEncryptionContext='string',
    BucketKeyEnabled=True|False,
    RequestPayer='requester',
    Tagging='string',
    ObjectLockMode='GOVERNANCE'|'COMPLIANCE',
    ObjectLockRetainUntilDate=datetime(2015, 1, 1),
    ObjectLockLegalHoldStatus='ON'|'OFF',
    ExpectedBucketOwner='string',
    ChecksumAlgorithm='CRC32'|'CRC32C'|'SHA1'|'SHA256'
)
Parameters:

  • ACL (string) –

    The canned ACL to apply to the object. Amazon S3 supports a set of predefined ACLs, known as canned ACLs. Each canned ACL has a predefined set of grantees and permissions. For more information, see Canned ACL in the Amazon S3 User Guide.

    By default, all objects are private. Only the owner has full access control. When uploading an object, you can grant access permissions to individual Amazon Web Services accounts or to predefined groups defined by Amazon S3. These permissions are then added to the access control list (ACL) on the new object. For more information, see Using ACLs. One way to grant the permissions using the request headers is to specify a canned ACL with the x-amz-acl request header.

    Note

    • This functionality is not supported for directory buckets.

    • This functionality is not supported for Amazon S3 on Outposts.

  • CacheControl (string) – Specifies caching behavior along the request/reply chain.

  • ContentDisposition (string) – Specifies presentational information for the object.

  • ContentEncoding (string) –

    Specifies what content encodings have been applied to the object and thus what decoding mechanisms must be applied to obtain the media-type referenced by the Content-Type header field.

    Note

    For directory buckets, only the aws-chunked value is supported in this header field.

  • ContentLanguage (string) – The language that the content is in.

  • ContentType (string) – A standard MIME type describing the format of the object data.

  • Expires (datetime) – The date and time at which the object is no longer cacheable.

  • GrantFullControl (string) –

    Specify access permissions explicitly to give the grantee READ, READ_ACP, and WRITE_ACP permissions on the object.

    By default, all objects are private. Only the owner has full access control. When uploading an object, you can use this header to explicitly grant access permissions to specific Amazon Web Services accounts or groups. This header maps to specific permissions that Amazon S3 supports in an ACL. For more information, see Access Control List (ACL) Overview in the Amazon S3 User Guide.

    You specify each grantee as a type=value pair, where the type is one of the following:

    • id – if the value specified is the canonical user ID of an Amazon Web Services account

    • uri – if you are granting permissions to a predefined group

    • emailAddress – if the value specified is the email address of an Amazon Web Services account

    Note

    Using email addresses to specify a grantee is only supported in the following Amazon Web Services Regions:

    • US East (N. Virginia)

    • US West (N. California)

    • US West (Oregon)

    • Asia Pacific (Singapore)

    • Asia Pacific (Sydney)

    • Asia Pacific (Tokyo)

    • Europe (Ireland)

    • South America (São Paulo)

    For a list of all the Amazon S3 supported Regions and endpoints, see Regions and Endpoints in the Amazon Web Services General Reference.

    For example, the following x-amz-grant-read header grants the Amazon Web Services accounts identified by account IDs permissions to read object data and its metadata:

    x-amz-grant-read: id="11112222333", id="444455556666"

    Note

    • This functionality is not supported for directory buckets.

    • This functionality is not supported for Amazon S3 on Outposts.

  • GrantRead (string) –

    Specify access permissions explicitly to allow grantee to read the object data and its metadata.

    By default, all objects are private. Only the owner has full access control. When uploading an object, you can use this header to explicitly grant access permissions to specific Amazon Web Services accounts or groups. This header maps to specific permissions that Amazon S3 supports in an ACL. For more information, see Access Control List (ACL) Overview in the Amazon S3 User Guide.

    You specify each grantee as a type=value pair, where the type is one of the following:

    • id – if the value specified is the canonical user ID of an Amazon Web Services account

    • uri – if you are granting permissions to a predefined group

    • emailAddress – if the value specified is the email address of an Amazon Web Services account

    Note

    Using email addresses to specify a grantee is only supported in the following Amazon Web Services Regions:

    • US East (N. Virginia)

    • US West (N. California)

    • US West (Oregon)

    • Asia Pacific (Singapore)

    • Asia Pacific (Sydney)

    • Asia Pacific (Tokyo)

    • Europe (Ireland)

    • South America (São Paulo)

    For a list of all the Amazon S3 supported Regions and endpoints, see Regions and Endpoints in the Amazon Web Services General Reference.

    For example, the following x-amz-grant-read header grants the Amazon Web Services accounts identified by account IDs permissions to read object data and its metadata:

    x-amz-grant-read: id="11112222333", id="444455556666"

    Note

    • This functionality is not supported for directory buckets.

    • This functionality is not supported for Amazon S3 on Outposts.

  • GrantReadACP (string) –

    Specify access permissions explicitly to allows grantee to read the object ACL.

    By default, all objects are private. Only the owner has full access control. When uploading an object, you can use this header to explicitly grant access permissions to specific Amazon Web Services accounts or groups. This header maps to specific permissions that Amazon S3 supports in an ACL. For more information, see Access Control List (ACL) Overview in the Amazon S3 User Guide.

    You specify each grantee as a type=value pair, where the type is one of the following:

    • id – if the value specified is the canonical user ID of an Amazon Web Services account

    • uri – if you are granting permissions to a predefined group

    • emailAddress – if the value specified is the email address of an Amazon Web Services account

    Note

    Using email addresses to specify a grantee is only supported in the following Amazon Web Services Regions:

    • US East (N. Virginia)

    • US West (N. California)

    • US West (Oregon)

    • Asia Pacific (Singapore)

    • Asia Pacific (Sydney)

    • Asia Pacific (Tokyo)

    • Europe (Ireland)

    • South America (São Paulo)

    For a list of all the Amazon S3 supported Regions and endpoints, see Regions and Endpoints in the Amazon Web Services General Reference.

    For example, the following x-amz-grant-read header grants the Amazon Web Services accounts identified by account IDs permissions to read object data and its metadata:

    x-amz-grant-read: id="11112222333", id="444455556666"

    Note

    • This functionality is not supported for directory buckets.

    • This functionality is not supported for Amazon S3 on Outposts.

  • GrantWriteACP (string) –

    Specify access permissions explicitly to allows grantee to allow grantee to write the ACL for the applicable object.

    By default, all objects are private. Only the owner has full access control. When uploading an object, you can use this header to explicitly grant access permissions to specific Amazon Web Services accounts or groups. This header maps to specific permissions that Amazon S3 supports in an ACL. For more information, see Access Control List (ACL) Overview in the Amazon S3 User Guide.

    You specify each grantee as a type=value pair, where the type is one of the following:

    • id – if the value specified is the canonical user ID of an Amazon Web Services account

    • uri – if you are granting permissions to a predefined group

    • emailAddress – if the value specified is the email address of an Amazon Web Services account

    Note

    Using email addresses to specify a grantee is only supported in the following Amazon Web Services Regions:

    • US East (N. Virginia)

    • US West (N. California)

    • US West (Oregon)

    • Asia Pacific (Singapore)

    • Asia Pacific (Sydney)

    • Asia Pacific (Tokyo)

    • Europe (Ireland)

    • South America (São Paulo)

    For a list of all the Amazon S3 supported Regions and endpoints, see Regions and Endpoints in the Amazon Web Services General Reference.

    For example, the following x-amz-grant-read header grants the Amazon Web Services accounts identified by account IDs permissions to read object data and its metadata:

    x-amz-grant-read: id="11112222333", id="444455556666"

    Note

    • This functionality is not supported for directory buckets.

    • This functionality is not supported for Amazon S3 on Outposts.

  • Metadata (dict) –

    A map of metadata to store with the object in S3.

    • (string) –

      • (string) –

  • ServerSideEncryption (string) –

    The server-side encryption algorithm used when you store this object in Amazon S3 (for example, AES256, aws:kms).

    • Directory buckets - For directory buckets, there are only two supported options for server-side encryption: server-side encryption with Amazon S3 managed keys (SSE-S3) ( AES256) and server-side encryption with KMS keys (SSE-KMS) ( aws:kms). We recommend that the bucket’s default encryption uses the desired encryption configuration and you don’t override the bucket default encryption in your CreateSession requests or PUT object requests. Then, new objects are automatically encrypted with the desired encryption settings. For more information, see Protecting data with server-side encryption in the Amazon S3 User Guide. For more information about the encryption overriding behaviors in directory buckets, see Specifying server-side encryption with KMS for new object uploads. In the Zonal endpoint API calls (except CopyObject and UploadPartCopy) using the REST API, the encryption request headers must match the encryption settings that are specified in the CreateSession request. You can’t override the values of the encryption settings ( x-amz-server-side-encryption, x-amz-server-side-encryption-aws-kms-key-id, x-amz-server-side-encryption-context, and x-amz-server-side-encryption-bucket-key-enabled) that are specified in the CreateSession request. You don’t need to explicitly specify these encryption settings values in Zonal endpoint API calls, and Amazon S3 will use the encryption settings values from the CreateSession request to protect new objects in the directory bucket.

    Note

    When you use the CLI or the Amazon Web Services SDKs, for CreateSession, the session token refreshes automatically to avoid service interruptions when a session expires. The CLI or the Amazon Web Services SDKs use the bucket’s default encryption configuration for the CreateSession request. It’s not supported to override the encryption settings values in the CreateSession request. So in the Zonal endpoint API calls (except CopyObject and UploadPartCopy), the encryption request headers must match the default encryption configuration of the directory bucket.

  • StorageClass (string) –

    By default, Amazon S3 uses the STANDARD Storage Class to store newly created objects. The STANDARD storage class provides high durability and high availability. Depending on performance needs, you can specify a different Storage Class. For more information, see Storage Classes in the Amazon S3 User Guide.

    Note

    • For directory buckets, only the S3 Express One Zone storage class is supported to store newly created objects.

    • Amazon S3 on Outposts only uses the OUTPOSTS Storage Class.

  • WebsiteRedirectLocation (string) –

    If the bucket is configured as a website, redirects requests for this object to another object in the same bucket or to an external URL. Amazon S3 stores the value of this header in the object metadata.

    Note

    This functionality is not supported for directory buckets.

  • SSECustomerAlgorithm (string) –

    Specifies the algorithm to use when encrypting the object (for example, AES256).

    Note

    This functionality is not supported for directory buckets.

  • SSECustomerKey (string) –

    Specifies the customer-provided encryption key for Amazon S3 to use in encrypting data. This value is used to store the object and then it is discarded; Amazon S3 does not store the encryption key. The key must be appropriate for use with the algorithm specified in the x-amz-server-side-encryption-customer-algorithm header.

    Note

    This functionality is not supported for directory buckets.

  • SSECustomerKeyMD5 (string) –

    Specifies the 128-bit MD5 digest of the customer-provided encryption key according to RFC 1321. Amazon S3 uses this header for a message integrity check to ensure that the encryption key was transmitted without error.

    Note

    This functionality is not supported for directory buckets.

    Please note that this parameter is automatically populated if it is not provided. Including this parameter is not required

  • SSEKMSKeyId (string) –

    Specifies the KMS key ID (Key ID, Key ARN, or Key Alias) to use for object encryption. If the KMS key doesn’t exist in the same account that’s issuing the command, you must use the full Key ARN not the Key ID.

    General purpose buckets - If you specify x-amz-server-side-encryption with aws:kms or aws:kms:dsse, this header specifies the ID (Key ID, Key ARN, or Key Alias) of the KMS key to use. If you specify x-amz-server-side-encryption:aws:kms or x-amz-server-side-encryption:aws:kms:dsse, but do not provide x-amz-server-side-encryption-aws-kms-key-id, Amazon S3 uses the Amazon Web Services managed key ( aws/s3) to protect the data.

    Directory buckets - If you specify x-amz-server-side-encryption with aws:kms, you must specify the x-amz-server-side-encryption-aws-kms-key-id header with the ID (Key ID or Key ARN) of the KMS symmetric encryption customer managed key to use. Otherwise, you get an HTTP 400 Bad Request error. Only use the key ID or key ARN. The key alias format of the KMS key isn’t supported. Your SSE-KMS configuration can only support 1 customer managed key per directory bucket for the lifetime of the bucket. Amazon Web Services managed key ( aws/s3) isn’t supported.

  • SSEKMSEncryptionContext (string) –

    Specifies the Amazon Web Services KMS Encryption Context to use for object encryption. The value of this header is a Base64-encoded string of a UTF-8 encoded JSON, which contains the encryption context as key-value pairs.

    Directory buckets - You can optionally provide an explicit encryption context value. The value must match the default encryption context - the bucket Amazon Resource Name (ARN). An additional encryption context value is not supported.

  • BucketKeyEnabled (boolean) –

    Specifies whether Amazon S3 should use an S3 Bucket Key for object encryption with server-side encryption using Key Management Service (KMS) keys (SSE-KMS).

    General purpose buckets - Setting this header to true causes Amazon S3 to use an S3 Bucket Key for object encryption with SSE-KMS. Also, specifying this header with a PUT action doesn’t affect bucket-level settings for S3 Bucket Key.

    Directory buckets - S3 Bucket Keys are always enabled for GET and PUT operations in a directory bucket and can’t be disabled. S3 Bucket Keys aren’t supported, when you copy SSE-KMS encrypted objects from general purpose buckets to directory buckets, from directory buckets to general purpose buckets, or between directory buckets, through CopyObject, UploadPartCopy, the Copy operation in Batch Operations, or the import jobs. In this case, Amazon S3 makes a call to KMS every time a copy request is made for a KMS-encrypted object.

  • RequestPayer (string) –

    Confirms that the requester knows that they will be charged for the request. Bucket owners need not specify this parameter in their requests. If either the source or destination S3 bucket has Requester Pays enabled, the requester will pay for corresponding charges to copy the object. For information about downloading objects from Requester Pays buckets, see Downloading Objects in Requester Pays Buckets in the Amazon S3 User Guide.

    Note

    This functionality is not supported for directory buckets.

  • Tagging (string) –

    The tag-set for the object. The tag-set must be encoded as URL Query parameters.

    Note

    This functionality is not supported for directory buckets.

  • ObjectLockMode (string) –

    Specifies the Object Lock mode that you want to apply to the uploaded object.

    Note

    This functionality is not supported for directory buckets.

  • ObjectLockRetainUntilDate (datetime) –

    Specifies the date and time when you want the Object Lock to expire.

    Note

    This functionality is not supported for directory buckets.

  • ObjectLockLegalHoldStatus (string) –

    Specifies whether you want to apply a legal hold to the uploaded object.

    Note

    This functionality is not supported for directory buckets.

  • ExpectedBucketOwner (string) – The account ID of the expected bucket owner. If the account ID that you provide does not match the actual owner of the bucket, the request fails with the HTTP status code 403 Forbidden (access denied).

  • ChecksumAlgorithm (string) – Indicates the algorithm that you want Amazon S3 to use to create the checksum for the object. For more information, see Checking object integrity in the Amazon S3 User Guide.

Return type:

s3.MultipartUpload

Returns:

MultipartUpload resource