Skip to content

Instantly share code, notes, and snippets.

@bschaatsbergen
Last active May 20, 2024 07:54
Show Gist options
  • Save bschaatsbergen/cc88b52cc0783ce87969c25b9256ec47 to your computer and use it in GitHub Desktop.
Save bschaatsbergen/cc88b52cc0783ce87969c25b9256ec47 to your computer and use it in GitHub Desktop.
multi-node elasticsearch cluster
version: '2.2'
services:
es01:
image: docker.elastic.co/elasticsearch/elasticsearch:7.12.0
container_name: es01
environment:
- node.name=es01
- cluster.name=es-docker-cluster
- discovery.seed_hosts=es02,es03
- cluster.initial_master_nodes=es01,es02,es03
- bootstrap.memory_lock=true
- "ES_JAVA_OPTS=-Xms512m -Xmx512m"
ulimits:
memlock:
soft: -1
hard: -1
volumes:
- data01:/usr/share/elasticsearch/data
ports:
- 9200:9200
networks:
- elastic
es02:
image: docker.elastic.co/elasticsearch/elasticsearch:7.12.0
container_name: es02
environment:
- node.name=es02
- cluster.name=es-docker-cluster
- discovery.seed_hosts=es01,es03
- cluster.initial_master_nodes=es01,es02,es03
- bootstrap.memory_lock=true
- "ES_JAVA_OPTS=-Xms512m -Xmx512m"
ulimits:
memlock:
soft: -1
hard: -1
volumes:
- data02:/usr/share/elasticsearch/data
networks:
- elastic
es03:
image: docker.elastic.co/elasticsearch/elasticsearch:7.12.0
container_name: es03
environment:
- node.name=es03
- cluster.name=es-docker-cluster
- discovery.seed_hosts=es01,es02
- cluster.initial_master_nodes=es01,es02,es03
- bootstrap.memory_lock=true
- "ES_JAVA_OPTS=-Xms512m -Xmx512m"
ulimits:
memlock:
soft: -1
hard: -1
volumes:
- data03:/usr/share/elasticsearch/data
networks:
- elastic
kib01:
image: docker.elastic.co/kibana/kibana:7.12.0
container_name: kib01
ports:
- 5601:5601
environment:
ELASTICSEARCH_URL: http://es01:9200
ELASTICSEARCH_HOSTS: '["http://es01:9200","http://es02:9200","http://es03:9200"]'
networks:
- elastic
volumes:
data01:
driver: local
data02:
driver: local
data03:
driver: local
networks:
elastic:
driver: bridge
@bschaatsbergen
Copy link
Author

Important: when running this on your machine and you plan on mimicking certain load, you might want to set the following variable: vm.max_map_count. Setting this variable depends on the host machine.

Linux
Setting the variable temporarily: sudo sysctl -w vm.max_map_count=262144
Setting the variable permanently in /etc/sysctl.conf:

grep vm.max_map_count /etc/sysctl.conf
vm.max_map_count=262144

macOS with Docker for Mac
The vm.max_map_count setting must be set within the xhyve virtual machine:

From the command line, run:

screen ~/Library/Containers/com.docker.docker/Data/vms/0/tty

Press enter and use sysctl to configure vm.max_map_count:

sysctl -w vm.max_map_count=262144

To exit the screen session, type: Ctrl a d

Windows and macOS with Docker Desktop
The vm.max_map_count setting must be set via docker-machine:

docker-machine ssh
sudo sysctl -w vm.max_map_count=262144

Windows with Docker Desktop WSL 2 backend
The vm.max_map_count setting must be set in the docker-desktop container:

wsl -d docker-desktop
sysctl -w vm.max_map_count=262144

@Anthonymatta
Copy link

if I lost data with a restart or something happened in the container how i can retrieve the data ?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment