Skip to content

Instantly share code, notes, and snippets.

Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save gregorynicholas/858ddbf1c3a45bd6c97354ad4d770131 to your computer and use it in GitHub Desktop.
Save gregorynicholas/858ddbf1c3a45bd6c97354ad4d770131 to your computer and use it in GitHub Desktop.
How to use Amazon AWS Elasticsearch

How to use Amazon AWS Elasticsearch

The good news: you can get it running on the free tier (with a tiny instance).

The bad news: it's stuck on Elasticsearch 1.5.2 and dynamic scripting (Groovy) is disabled.

http://docs.aws.amazon.com/elasticsearch-service/latest/developerguide/aes-limits.html

Authentication: the safest option is to create a brand new IAM user (using the tool at https://console.aws.amazon.com/iam/home?region=us-east-1 ) with its own access key and secret key. Then when you create the Elasticsearch instance you can paste in the following IAM string:

arn:aws:iam::YOUR_AWS_ACCOUNT_ID:user/YOUR_IAM_USERNAME

You'll need to look up YOUR_AWS_ACCOUNT_ID - http://docs.aws.amazon.com/IAM/latest/UserGuide/console_account-alias.html

Having done all of the above, here's the secret recipe to getting Python to talk to your new Elasticsearch instance:

import requests
from requests_aws4auth import AWS4Auth

endpoint = 'https://search-your-endpoint.us-east-1.es.amazonaws.com'
auth=AWS4Auth(ACCESS_ID, ACCESS_SECRET, 'us-east-1', 'es')

print requests.get(endpoint, auth=auth).json()
print requests.get(endpoint + '/_aliases', auth=auth).json()
# etc

If you're using the official Python client for Elasticsearch, the recipe looks like this:

from elasticsearch import Elasticsearch, RequestsHttpConnection
from requests_aws4auth import AWS4Auth

es = Elasticsearch(
    'default',
    hosts=['search-your-endpoint.us-east-1.es.amazonaws.com'],
    http_auth=AWS4Auth(ACCESS_ID, ACCESS_SECRET, 'us-east-1', 'es'),
    use_ssl=True,
    verify_certs=True,
    connection_class=RequestsHttpConnection
)

See also elastic/elasticsearch-py#280

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment