Manual Snapshot (Backup) of an Elastic Cloud instance to AWS S3

Manual Snapshot (Backup) of an Elastic Cloud instance.md

Manual Snapshot (Backup) of an Elastic Cloud instance to AWS S3

Summary

This post explains how to manually create a Snapshot of an Elastic Cloud instance to an AWS S3 Bucket using what Elasticsearch calls a custom repository.

This is based on the following:

• Configure a snapshot repository using AWS S3
• AWS Elasticsearch Manual Snapshot and Restore on AWS S3

License

This content is shared under a Creative Commons Attribution-ShareAlike 4.0 International.

Create the AWS Resources

Create an S3 Bucket

On the AWS S3 console, create a new bucket, making sure to note the name. Also, be sure to secure the bucket (allowing no public access). Also, the bucket should be created in the same region as your Elastic Cloud instance!

Create an IAM User and give the user access

On the AWS S3 console, create a new user.

• Give the user "programmatic" access.

• When prompted or a security policy, create a new one with the following:

  {
    "Version": "2012-10-17",
    "Statement": [
      {
        "Effect": "Allow",
        "Action": "s3:*",
        "Resource": [
          "arn:aws:s3:::your-bucket-name-here/*",
          "arn:aws:s3:::your-bucket-name-here"
        ]
      }
    ]
  }

• Be sure to attach the new policy to the new user!

• Once the user is created, take note of the Access Key Id and Secret Access Key.

Configure your Elasticsearch Instance to use your AWS Resource

Add your AWS Credentials to your Cloud Instance

• Log in to your Elastic Cloud dashboard and drill into the instance you wish to Snapshot.
• In the Security tab, add your AWS Credentials to the Elasticsearch keystore:
  • Click Create settings
  • Set type to Single string.
  • Add the following keys and their values to match your AWS credentials: s3.client.default.access_key, s3.client.default.secret_key

Add the bucket to Elasticsearch

Still logged into your Elastic Cloud dashboard for your instance, click Elasticsearch then API console:

• Select POST from the dropdown.

• Enter _snapshot/enter-a-name-for-your-repo-here in the POST text box (replacing the string with a name for your repo. This can be any value you like, but keep it short to make it easier to enter it into the snapshot command!)

• In the Body of the post, enter:

  {
    "type": "s3",
    "settings": {
      "bucket": "your-bucket-name-here",
      "region": "us-east-1"
    }
  }

  (Be sure to replace region with the proper region for your bucket.)

Snapshot(Backup) your Instance

Still logged into your Elastic Cloud dashboard for your instance, still in Elasticsearch then API console:

• Select POST from the dropdown.
• Enter _snapshot/your-repo-name-here/name-your-backup-to-suit-your-needs in the POST text box.

Note: Snapshots are not immediate. You can verify that a Snapshot started in your Elasticsearch Logs tab. You should see something similar to "[instance-0000000000] snapshot [your-repo-name-here:your-backup-name-here/tlLWPFGQTC-Ay_ASoVtwaw] started". When the Snapshot completes, you'll see a message similar to "[instance-0000000000] snapshot [your-repo-name-here:your-backup-name-here/tlLWPFGQTC-Ay_ASoVtwaw] completed with state [SUCCESS]".