consul setup on mesos / coreos

consul.md

We are having troubles talking to Mesos DNS if our Marathon application / service has two ports.

We could use mesos consul https://hub.docker.com/r/ciscocloud/mesos-consul/ but that would require a consul cluster.

The following are instructions to install consul on dcos/mesos/coreos. This may not be the best way. Coreos does have a package manager (but we don't know how to use it). At a minimum we should script this with ansible. (We are not ansible experts either).

So far, I have been unable to get mesos-consul to work but consul is running on the mesos-masters and mesos-slaves in a special snowflake fashion.

Get Consul

wget https://releases.hashicorp.com/consul/0.6.4/consul_0.6.4_linux_amd64.zip

Copy to master and ssh in.

scp -i ~/"US-WEST-2-KEY-XXX-D.pem" *.zip core@52.27.164.201:/home/core/consul.zip
ssh -i ~/"US-WEST-2-KEY-XXX-D.pem" core@52.27.164.201

Install consul

sudo mkdir -p /opt/consul/bin /opt/consul/data /opt/consul/conf /var/log/consul
sudo unzip consul.zip -d /opt/consul/bin
sudo chown -R core /opt/consul/data/

Run bootstrap on master1

sudo systemd-run  /opt/consul/bin/consul agent -server -bootstrap-expect 3 \
    -data-dir /opt/consul/data -node=agent1 -bind=`/opt/mesosphere/bin/detect_ip` \
    -config-dir /opt/consul/conf  -syslog

View consul logs

# view consul log via syslog
journalctl -f  | grep "consul"

Copy consul to other to masters

scp -i ~/"US-WEST-2-KEY-RBSS-001-D.pem" *.zip core@52.38.250.75:/home/core/consul.zip
scp -i ~/"US-WEST-2-KEY-RBSS-001-D.pem" *.zip core@52.40.60.164:/home/core/consul.zip

Install consul on other two masters

ssh -i ~/"US-WEST-2-KEY-RBSS-001-D.pem" core@52.38.250.75
# repeat consul install steps

ssh -i ~/"US-WEST-2-KEY-RBSS-001-D.pem" core@52.40.60.164
# repeat consul install steps

Run bootstrap for master 2 on master2

sudo systemd-run  /opt/consul/bin/consul agent -server -bootstrap-expect 3 \
    -data-dir /opt/consul/data -node=agent2 -bind=`/opt/mesosphere/bin/detect_ip` \
    -config-dir /opt/consul/conf  -syslog

Run bootstrap for master 3 on master3

sudo systemd-run  /opt/consul/bin/consul agent -server -bootstrap-expect 3 \
    -data-dir /opt/consul/data -node=agent3 -bind=`/opt/mesosphere/bin/detect_ip` \
    -config-dir /opt/consul/conf  -syslog

Join them all

/opt/consul/bin/consul join 10.16.205.237 10.16.205.238 10.16.205.239

ssh into the master using dcos

 dcos node ssh --master-proxy --leader

Copy consul.zip to the slaves From one of the masters

scp *.zip 10.16.204.92:/home/core/consul.zip
scp *.zip 10.16.204.93:/home/core/consul.zip
scp *.zip 10.16.204.94:/home/core/consul.zip
scp *.zip 10.16.204.95:/home/core/consul.zip
scp *.zip 10.16.204.96:/home/core/consul.zip

Run consul on slaves

sudo systemd-run  /opt/consul/bin/consul agent  -join=10.16.205.237 \
    -join=10.16.205.238 -join=10.16.205.239 \
    -data-dir /opt/consul/data   -bind=`/opt/mesosphere/bin/detect_ip` \
    -config-dir /opt/consul/conf  -syslog 

ssh into each box and install consul (same as before)

ssh 10.16.204.92
# install consul
# run consul

ssh 10.16.204.93
# install consul
# run consul

ssh 10.16.204.94
# install consul
# run consul

ssh 10.16.204.95
# install consul
# run consul

ssh 10.16.204.96
# install consul
# run consul