Skip to content

Instantly share code, notes, and snippets.

@darklow
Created February 24, 2018 03:42
Show Gist options
  • Save darklow/21ed4b0036185a0dc9322e00672023f9 to your computer and use it in GitHub Desktop.
Save darklow/21ed4b0036185a0dc9322e00672023f9 to your computer and use it in GitHub Desktop.
ElasticSearch cluster of 3x nodes (2 master/data + 1x master) on AWS EC2 using ec2 discovery plugin
# FILE: ./elastic-master/Dockerfile
FROM docker.elastic.co/elasticsearch/elasticsearch:6.2.2
RUN /usr/share/elasticsearch/bin/elasticsearch-plugin install --batch discovery-ec2
ARG AWS_ACCESS_KEY
ARG AWS_ACCESS_SECRET
RUN echo $AWS_ACCESS_KEY | /usr/share/elasticsearch/bin/elasticsearch-keystore add discovery.ec2.access_key
RUN echo $AWS_ACCESS_SECRET | /usr/share/elasticsearch/bin/elasticsearch-keystore add discovery.ec2.secret_key
ADD ./elastic-master/elasticsearch.yml /usr/share/elasticsearch/config/
# --------------------------------------------
# FILE: ./elastic-data/Dockerfile
FROM docker.elastic.co/elasticsearch/elasticsearch:6.2.2
RUN /usr/share/elasticsearch/bin/elasticsearch-plugin install --batch discovery-ec2
ARG AWS_ACCESS_KEY
ARG AWS_ACCESS_SECRET
RUN echo $AWS_ACCESS_KEY | /usr/share/elasticsearch/bin/elasticsearch-keystore add discovery.ec2.access_key
RUN echo $AWS_ACCESS_SECRET | /usr/share/elasticsearch/bin/elasticsearch-keystore add discovery.ec2.secret_key
ADD ./docker/elastic-data/elasticsearch.yml /usr/share/elasticsearch/config/
# --------------------------------------------
# FILE: elastic-master/elasticsearch.yml
cluster.name: cp-cluster
node.name: ${NODE_HOSTNAME}
action.destructive_requires_name: true
node.data: false
node.master: true
bootstrap.memory_lock: true
xpack.security.enabled: false
discovery.zen.hosts_provider: ec2
discovery.zen.minimum_master_nodes: 2
discovery.zen.ping_timeout: 60s
discovery.zen.fd.ping_retries: 20
network.host: 0.0.0.0
network.publish_host: _ec2:privateIp_
discovery.ec2.host_type: private_ip
discovery.ec2.availability_zones: us-east-1b,us-east-1c,us-east-1d
discovery.ec2.endpoint: ec2.us-east-1.amazonaws.com
discovery.ec2.protocol: https
discovery.ec2.groups: sg-ad5912d9
cloud.node.auto_attributes: true
cluster.routing.allocation.awareness.attributes: aws_availability_zone
plugin.mandatory:
- discovery-ec2
# --------------------------------------------
# FILE: elastic-master/elasticsearch.yml
cluster.name: cp-cluster
node.name: ${NODE_HOSTNAME}
action.destructive_requires_name: true
node.data: true
node.master: true
bootstrap.memory_lock: true
xpack.security.enabled: false
discovery.zen.hosts_provider: ec2
discovery.zen.minimum_master_nodes: 2
discovery.zen.ping_timeout: 60s
discovery.zen.fd.ping_retries: 20
network.host: 0.0.0.0
network.publish_host: _ec2:privateIp_
discovery.ec2.host_type: private_ip
discovery.ec2.availability_zones: us-east-1b,us-east-1c,us-east-1d
discovery.ec2.endpoint: ec2.us-east-1.amazonaws.com
discovery.ec2.protocol: https
discovery.ec2.groups: sg-ad5912d9
cloud.node.auto_attributes: true
cluster.routing.allocation.awareness.attributes: aws_availability_zone
plugin.mandatory:
- discovery-ec2
@ductrinhhuu
Copy link

Hello,
If I turn xpack.security.enabled to TRUE, it seems these nodes are not able to see each other.
Do you have any idea?
Thanks

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment