AWS-CLI commands to work with S3 bucket

aws_s3.md

Login

sudo $(aws ecr get-login --no-include-email --region us-east-1)

AWS Simple Storage Service (s3)

General

List buckets

aws s3 ls

or

aws s3 ls path/to/lookat

Create, copy, move

NOTE :

The order is --exclude first followed by --included Switching the order doesn't work !

Create

Create bucket

aws s3 mb s3://bucket

Copy

Copy files starting with specific prefix (or suffix)

aws s3 cp s3://bucket/subdirectory ./ --exclude "*" --include "*prefix*" --recursive

Copy directory to local

Preferred way

aws s3 sync s3://path/to/dir localpath

Alternative way

aws s3 cp s3://path/to/dir localpath --recursive

Move

Move files from subfolder one step up i.e. containing folder

aws s3 mv s3://bucketname/firstSubfolder/secondSubfolder/ s3://bucketname/firstSubfolder/ --recursive

Delete

NOTE:

Doesn't hurt to use --dryrun when trying these commands to see the list of files to be deleted !

Delete empty bucket

aws s3 rb s3://bucketname

Delete non-empty bucket and everythin within it recursively

aws s3 rb s3://bucketname --force

Delete only specific subfolder and everything within in

aws s3 rm s3://bucketname/ --exclude "*" --include "*subfoldername/*" --recursive

Delete specific file(s) within bucket

Single file

aws s3 rm s3://bucketname/filename

All files within bucket (i.e. empty it)

aws s3 rm s3://bucketname/filename --recursive

All files within bucket except those with specific extension

aws s3 rm s3://bucketname/filename --recursive --exclude "*.jpeg"

All files within bucket except those with specific prefix or within specific subfolder

aws s3 rm s3://bucketname/filename --recursive --exclude "subfolder/*"

Delete couple 'o specific files only

aws s3 rm s3://bucketname/ --recursive --exclude "*" --include "fullFilename1" --include "fullFilename2"

Hundreds of files

Don't have a single command yet. Figure out a smart combination of --exclude and --include flags