S3#

Client#

class S3.Client#

A low-level client representing Amazon Simple Storage Service (S3)

import boto3

client = boto3.client('s3')

These are the available methods:

Client Exceptions#

Client exceptions are available on a client instance via the exceptions property. For more detailed instructions and examples on the exact usage of client exceptions, see the error handling user guide.

The available client exceptions are:

Paginators#

Paginators are available on a client instance via the get_paginator method. For more detailed instructions and examples on the usage of paginators, see the paginators user guide.

The available paginators are:

Waiters#

Waiters are available on a client instance via the get_waiter method. For more detailed instructions and examples on the usage or waiters, see the waiters user guide.

The available waiters are:

Resources#

Resources are available in boto3 via the resource method. For more detailed instructions and examples on the usage of resources, see the resources user guide.

The available resources are:

Examples#

List objects in an Amazon S3 bucket#

The following example shows how to use an Amazon S3 bucket resource to list the objects in the bucket.

import boto3

s3 = boto3.resource('s3')
bucket = s3.Bucket('amzn-s3-demo-bucket')
for obj in bucket.objects.all():
    print(obj.key)

List top-level common prefixes in Amazon S3 bucket#

This example shows how to list all of the top-level common prefixes in an Amazon S3 bucket:

import boto3

client = boto3.client('s3')
paginator = client.get_paginator('list_objects')
result = paginator.paginate(Bucket='amzn-s3-demo-bucket', Delimiter='/')
for prefix in result.search('CommonPrefixes'):
    print(prefix.get('Prefix'))

Restore Glacier objects in an Amazon S3 bucket#

The following example shows how to initiate restoration of glacier objects in an Amazon S3 bucket, determine if a restoration is on-going, and determine if a restoration is finished.

import boto3

s3 = boto3.resource('s3')
bucket = s3.Bucket('amzn-s3-demo-bucket')
for obj_sum in bucket.objects.all():
    obj = s3.Object(obj_sum.bucket_name, obj_sum.key)
    if obj.storage_class == 'GLACIER':
        # Try to restore the object if the storage class is glacier and
        # the object does not have a completed or ongoing restoration
        # request.
        if obj.restore is None:
            print('Submitting restoration request: %s' % obj.key)
            obj.restore_object(RestoreRequest={'Days': 1})
        # Print out objects whose restoration is on-going
        elif 'ongoing-request="true"' in obj.restore:
            print('Restoration in-progress: %s' % obj.key)
        # Print out objects whose restoration is complete
        elif 'ongoing-request="false"' in obj.restore:
            print('Restoration complete: %s' % obj.key)

Uploading/downloading files using SSE KMS#

This example shows how to use SSE-KMS to upload objects using server side encryption with a key managed by KMS.

We can either use the default KMS master key, or create a custom key in AWS and use it to encrypt the object by passing in its key id.

With KMS, nothing else needs to be provided for getting the object; S3 already knows how to decrypt the object.

import boto3
import os

BUCKET = 'amzn-s3-demo-bucket'
s3 = boto3.client('s3')
keyid = '<the key id>'

print("Uploading S3 object with SSE-KMS")
s3.put_object(Bucket=BUCKET,
              Key='encrypt-key',
              Body=b'foobar',
              ServerSideEncryption='aws:kms',
              # Optional: SSEKMSKeyId
              SSEKMSKeyId=keyid)
print("Done")

# Getting the object:
print("Getting S3 object...")
response = s3.get_object(Bucket=BUCKET,
                         Key='encrypt-key')
print("Done, response body:")
print(response['Body'].read())

Uploading/downloading files using SSE Customer Keys#

This example shows how to use SSE-C to upload objects using server side encryption with a customer provided key.

First, we’ll need a 32 byte key. For this example, we’ll randomly generate a key but you can use any 32 byte key you want. Remember, you must the same key to download the object. If you lose the encryption key, you lose the object.

Also note how we don’t have to provide the SSECustomerKeyMD5. Boto3 will automatically compute this value for us.

import boto3
import os

BUCKET = 'amzn-s3-demo-bucket'
KEY = os.urandom(32)
s3 = boto3.client('s3')

print("Uploading S3 object with SSE-C")
s3.put_object(Bucket=BUCKET,
              Key='encrypt-key',
              Body=b'foobar',
              SSECustomerKey=KEY,
              SSECustomerAlgorithm='AES256')
print("Done")

# Getting the object:
print("Getting S3 object...")
# Note how we're using the same ``KEY`` we
# created earlier.
response = s3.get_object(Bucket=BUCKET,
                         Key='encrypt-key',
                         SSECustomerKey=KEY,
                         SSECustomerAlgorithm='AES256')
print("Done, response body:")
print(response['Body'].read())

Downloading a specific version of an S3 object#

This example shows how to download a specific version of an S3 object.

import boto3
s3 = boto3.client('s3')

s3.download_file(
    "amzn-s3-demo-bucket", "key-name", "tmp.txt",
    ExtraArgs={"VersionId": "my-version-id"}
)

Filter objects by last modified time using JMESPath#

This example shows how to filter objects by last modified time using JMESPath.

import boto3
s3 = boto3.client("s3")

s3_paginator = s3.get_paginator('list_objects_v2')
s3_iterator = s3_paginator.paginate(Bucket='amzn-s3-demo-bucket')

filtered_iterator = s3_iterator.search(
    "Contents[?to_string(LastModified)>='\"2022-01-05 08:05:37+00:00\"'].Key"
)

for key_data in filtered_iterator:
    print(key_data)

Client Context Parameters#

Client context parameters are configurable on a client instance via the client_context_params parameter in the Config object. For more detailed instructions and examples on the exact usage of context params see the configuration guide.

The available s3 client context params are:

  • disable_s3_express_session_auth (boolean) - Disables this client’s usage of Session Auth for S3Express

    buckets and reverts to using conventional SigV4 for those.