Wait until ETCD Cluster is up and running before trying to do anything that might use ETCD

cloud-config.yaml

#cloud-config

---
write_files:
- path: /opt/bin/waiter.sh
  owner: root
  content: |
    #! /usr/bin/bash
    until curl http://127.0.0.1:4001/v2/machines; do sleep 2; done
coreos:
  etcd:
    discovery: https://discovery.etcd.io/something
    addr: $public_ipv4:4001
    peer-addr: $public_ipv4:7001
  fleet:
    public-ip: $public_ipv4
  units:
  - name: etcd.service
    command: start
  - name: fleet.service
    command: start
  - name: docker-tcp.socket
    command: start
    enable: true
    content: |
      [Unit]
      Description=Docker Socket for the API

      [Socket]
      ListenStream=2375
      Service=docker.service
      BindIPv6Only=both

      [Install]
      WantedBy=sockets.target
  - name: etcd-waiter.service
    command: start
    content: "[Unit]\nWants=network-online.target\nWants=etcd.service\nAfter=etcd.service\nAfter=network-online.target\nBefore=flannel.service\nBefore=docker.service\nBefore=setup-network-environment.service\n\n[Service]\nExecStartPre=/usr/bin/chmod
      +x /opt/bin/waiter.sh\nExecStart=/usr/bin/bash /opt/bin/waiter.sh \nRemainAfterExit=true\nType=oneshot\n"
  - name: flannel.service
    command: start
    content: |
      [Unit]
      After=network-online.target
      Wants=network-online.target
      Wants=etcd-waiter.service
      After=etcd-waiter.service
      Wants=etcd.service
      After=etcd.service
      Before=docker.service
      Description=flannel is an etcd backed overlay network for containers

      [Service]
      ExecStartPre=/usr/bin/etcdctl mk /coreos.com/network/config '{"Network":"10.0.0.0/16"}'
      ExecStartPre=/usr/bin/mkdir -p /opt/bin
      ExecStartPre=/usr/bin/wget -N -P /opt/bin http://storage.googleapis.com/flannel/flanneld
      ExecStartPre=/usr/bin/chmod +x /opt/bin/flanneld
      ExecStart=/opt/bin/flanneld
      Restart=always
      RestartSec=30
  - name: docker.service
    command: start
    content: |
      [Unit]
      After=flannel.service
      After=setup-network-environment.service
      Wants=setup-network-environment.service
      Wants=flannel.service
      After=etcd-waiter.service
      Wants=etcd-waiter.service
      Description=Docker Application Container Engine
      Documentation=http://docs.docker.io

      [Service]
      EnvironmentFile=/run/flannel/subnet.env
      ExecStartPre=/bin/mount --make-rprivate /
      ExecStart=/usr/bin/docker -d --bip=${FLANNEL_SUBNET} --mtu=${FLANNEL_MTU} -s=btrfs -H fd://
      Restart=always
      RestartSec=30

      [Install]
      WantedBy=multi-user.target
  - name: setup-network-environment.service
    command: start
    content: |
      [Unit]
      Description=Setup Network Environment
      Documentation=https://github.com/kelseyhightower/setup-network-environment
      Requires=network-online.target
      After=network-online.target
      After=etcd-waiter.service
      Requires=etcd-waiter.service
      Before=docker.service

      [Service]
      ExecStartPre=/usr/bin/mkdir -p /opt/bin
      ExecStartPre=/usr/bin/wget -N -P /opt/bin http://storage.googleapis.com/snenv/setup-network-environment
      ExecStartPre=/usr/bin/chmod +x /opt/bin/setup-network-environment
      ExecStart=/opt/bin/setup-network-environment
      RemainAfterExit=yes
      Restart=no
      Type=oneshot

this probably needs some timeout tweaks. But it works with a small 3 node cluster. When quorum is reached, the rest of the unit files start.

This helped a lot with CoreOS on GCE and AWS. Thanks.

Btw, just to make it easier to read, consider changing to

    - name: etcd-waiter.service
      command: start
      content: |
        [Unit]
        Description=etcd waiter
        Wants=network-online.target
        Wants=etcd.service
        After=etcd.service
        After=network-online.target
        Before=flannel.service
        Before=setup-network-environment.service

        [Service]
        ExecStartPre=/usr/bin/chmod +x /opt/bin/waiter.sh
        ExecStart=/usr/bin/bash /opt/bin/waiter.sh
        RemainAfterExit=true
        Type=oneshot

What's the license for this?

Public Domain