Skip to content

Instantly share code, notes, and snippets.

@chrisyour
Last active March 15, 2018 21:41
  • Star 1 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
Star You must be signed in to star a gist
Embed
What would you like to do?
AWS S3 Command Line Tool for Backups

How to quickly copy an AWS S3 bucket to your local hard drive (without FTP):

aws s3 sync s3://mybucket .

The Problem

Using FTP to backup a large S3 bucket is slow. That's because FTP creates a new request for each file, downloads the file, and closes the request. That adds a lot of time to your download.

The Solution

Using the AWS S3 command line interface (CLI), I was able to download a 3GB bucket in a few minutes. A lot faster than a couple of hours using FTP software on the same internet connection.

Here's a simple walk-through that is plain and simple, even for people who may be uncomfortable with command line tools.


The Set Up (macOS)

Step 1. Open Terminal (It comes pre-installed on your Mac - just search for "terminal" on your Mac or get Siri to find it for you by saying "open terminal".)

Step 2. Install the AWS S3 command line tool:

sudo easy_install awscli

(Note: This is a sudo command - which means "super user do" - it will ask you for your computer password, when prompted for your password, type it in and press return.)

Step 3. Configure your account:

aws configure

Step 4. When prompted, enter your root access key and press return.

Step 5. When prompted, enter your secret access key and press return.

Step 6. In Terminal, use the cd command to navigate to the your backup folder:

cd ~/Documents/path/to/the/backup/folder

Step 7. Then download the bucket to the current directory:

aws s3 sync s3://mybucket .

(Note: the "." that comes after "s3://mybucket" tells the CLI to download the files directly into the current directory.)

Step 8. Sit back and watch those downloads!

Step 9. View the files you just downloaded in Finder:

open .

(Note: the "." tells the Finder to open the current directory.)


The Set Up (Windows)

Get the Windows installer here:

http://aws.amazon.com/cli/

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment