AWS CLI Crib Sheet

AWS-CLI-Crib-Sheet.md

AWS CLI Crib Sheet

I might go for several months without accessing my personal AWS account configuration, then when I need to do something I find that I have forgotten how to do even the most basic of things. I created this crib sheet to save me from having to learn it all from first principles every time (especially that MFA tempoarary access token part that bites me again and again).

Check the AWS CLI Version

The latest available version can be found here: https://github.com/aws/aws-cli/releases

Check the installed version thus:

aws --version

Install or Upgrade to Latest CLI Version

Using the AWS reccomended "Bundled Installer" (see Installing the AWS CLI version 1):

1. Download and unpack the ZIP file from https://s3.amazonaws.com/aws-cli/awscli-bundle.zip

2. Run the installer

sudo ./awscli-bundle/install -i /usr/local/aws -b /usr/local/bin/aws

Configure Your Access Keys

Run the following and answer the prompts to provide your access key ID, secret key, default region, and preferred output format:

aws configure

This should create or update the ~/.aws/credentials file.

You can confirm that all is well with these credentals using the following command to display basic information about your user:

aws sts get-caller-identity

This command will work no matter how limited your access rights might be.

Obtaining Temporary Credentials for MFA Authentication

Do you keep getting You are not authorized to perform this operation errors for almost every AWS CLI command that you try even though you have double checked your ~/.aws/credentials file has been configured?

Do you have MFA (Multi-Factor Authentication) enabled for your user? If yes, that is probably your issue.

See How do I use an MFA token to authenticate access to my AWS resources through the AWS CLI? for the solution.

List all S3 buckets

aws s3 ls

NOTE: If you have MFA configured for your user and have obtained temporary access keys as described by the "How do I use an MFA token to authenticate access to my AWS resources through the AWS CLI?" knowledge base article, and have stored this in a non-default section of your ~/.aws/credentials file, then you will need to provide the name of that non-default section for this and most other commands. Something like this:

aws s3 ls --profile mfa

List all files in an S3 bucket

aws s3 ls s3://bucket-name --recursive

Delete files that match a pattern

aws s3 rm s3://bucket-name --recursive --only-show-errors --exclude "*" --include "your-pattern-here/*"

NOTE: --dryrun displays what would happen but does not do it

Copy all files in current directory files to S3

aws s3 cp . s3://bucket-name --recursive

Copy all matching files between S3 buckets

aws s3 cp s3://source-bucket s3://target-bucket --recursive --exclude "*" --include "*onlye-this-pattern*"

Retrieve log entries for a given time period (up to a maximum response size of 1MB)

aws logs filter-log-events --log-group-name PUT_LOG_GROUP_NAME_HERE --interleaved --start-time 1440616500000 --end-time 1440619200000

JavaScript to Pull and Dump AWS Logs To A Local System

// Load the Amazon AWS API module
var AWS = require('aws-sdk');

// Set the default AWS region
AWS.config.update({region: 'us-east-1'});

// Get an instance of CloudWatchLogs
var cwlogs = new AWS.CloudWatchLogs();

// Define the parameters for the initial request
var params = {
    "logGroupName": "PUT_LOG_GROUP_NAME_HERE",
    "startTime": 1440696300000,
    "endTime": 1440697800000,
    "limit": 500,
    "interleaved": true
}

// Declare function that repeatedly calls itself until all of the log data has been read
var fetchLogs = function(params) {

    // Fetch a page of log data
    cwlogs.filterLogEvents(params, function(err, data) {
        if (err) {
            // An error occurred
            console.log(err, err.stack);
        } else {
            // Successful response: dump out the AWS log events as single line tab separeted data set
            data.events.forEach(function(event) {

                // Log the event minus any trailing carriage returns on the message
                console.log(event.ingestionTime + "\t" + event.message.replace(/[\r\n]+$/, ""));
            });

            // If there are more AWS log entries to process, recurse
            if (typeof data.nextToken !== 'undefined') {
                params.nextToken = data.nextToken;
                fetchLogs(params);
            }
        }
    });
}

// Kick of the recursive log fetch
fetchLogs(params);

List Metrics In Namespace

aws cloudwatch list-metrics --namespace "PUT_NAMESPACE_NAME_HERE"