aws resource access note

boto-note.md

Boto Note

Quick Start

S3

Before start using boto3, we should set up an user on AWS IAM and grant S3 access permission.

List Buckets

import boto3

s3 = boto3.resource('s3')

for bucket in s3.buckets.all():
    print(bucket.name)

Upload a file to S3

import boto3

s3 = boto3.resource('s3')

bucket = 'mybucket'
fname = 'test.txt'

with open(fname, 'rb') as f:
    s3.Bucket(bucket).put_object(Key=fname, Body=f)

Upload Directory

ref: How to create a directory in a bucket using boto3 ? #377

import boto3

from pathlib import Path

s3 = boto3.resource('s3')

bucket = 'mybucket'
directory = Path('.')

for p in directory.glob('**/*'):

    if p.is_dir():
        continue

    path = str(p)
    with open(path, 'rb') as f:
        s3.Bucket(bucket).put_object(Key=path, Body=f)

Download file

ref: download_file

import boto3

from pathlib import Path

s3 = boto3.resource('s3')

bucket = 'mybucket'
fname = 'test.txt'
fpath = Path('.') / fname

s3.meta.client.download_file(bucket, fname, str(fpath))

Filter objects in Bucket

import boto3

s3 = boto3.resource('s3')

bn = 'mybucket'
bucket = s3.Bucket(bn)

for l in bucket.objects.filter(Prefix='dir'):
    print(l)

Futher reading:

• Boto 3 Documentation
• boto/boto3