minio-docker-swarm

minio-docker-steps.md

• Pre-Conditions: https://docs.docker.com/engine/swarm/swarm-tutorial/#/three-networked-host-machines For distributed Minio to run, you need 4 networked host machines.

• Create a new swarm and set the manager. SSH to one of the host machine, which you want to set as manager and run: docker swarm init --advertise-addr <MANAGER-IP>

• Current node should become the manager. Check using: docker node ls

• Open a terminal and ssh into the machine where you want to run a worker node.

• Run the command as output by the step where master is created. It will add the current machine (as a worker) to the swarm. Add all the workers similarly.

• Check if all the machines are added as workers, SSH to the master and run: docker node ls

Create an overlay network:

  docker network create \
  --driver overlay \
  minio_distributed

Create Minio service

docker service create \
  --name minio_distributed_service \
  --network minio_distributed \
  --replicas 4 \
  --mount type=volume,source=data,destination=/root/export \
  --env MINIO_ACCESS_KEY=minio \
  --env MINIO_SECRET_KEY=minio123 \
  minio/minio \
  server http://10.0.0.3/root/export http://10.0.0.4/root/export \
  http://10.0.0.5/root/export http://10.0.0.6/root/export

Current Hack

You need to first create the minio service, find the IPs, delete the service, update the IPs in the minio server command and create the service again. Currently you need to docker network inspect minio_distributed on all the nodes to find the overlay network IP (of each container) and use those IPs to create the Minio service.

TODO:

• The IP addresses of the minio servers will change based on overlay network. Need to find out the host name of the containers and use that instead of the IP address.
• Check if --mount is required at all. What are the persistence options available.

You can define minio hostname in docker-stack configuration