CoreOS cloud-config for DigitalOcean with iptables firewall

cloud-config.yml

#cloud-config

coreos:
  etcd:
    # generate a new token for each unique cluster from https://discovery.etcd.io/new
    discovery: https://discovery.etcd.io/<token>
    # multi-region deployments, multi-cloud deployments, and droplets without
    # private networking need to use $public_ipv4
    addr: $private_ipv4:4001
    peer-addr: $private_ipv4:7001
  fleet:
    public-ip: $private_ipv4
  units:
    - name: etcd.service
      command: start
    - name: fleet.service
      command: start
    - name: iptables-restore.service
      enable: true
write_files:
  - path: /var/lib/iptables/rules-save
    permissions: 0644
    owner: root:root
    content: |
      *filter
      :INPUT DROP [0:0]
      :FORWARD DROP [0:0]
      :OUTPUT ACCEPT [0:0]
      -A INPUT -i lo -j ACCEPT
      -A INPUT -i eth1 -j ACCEPT
      -A INPUT -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
      -A INPUT -p tcp -m tcp --dport 22 -j ACCEPT
      -A INPUT -p tcp -m tcp --dport 80 -j ACCEPT
      -A INPUT -p tcp -m tcp --dport 443 -j ACCEPT
      -A INPUT -p icmp -m icmp --icmp-type 0 -j ACCEPT
      -A INPUT -p icmp -m icmp --icmp-type 3 -j ACCEPT
      -A INPUT -p icmp -m icmp --icmp-type 11 -j ACCEPT
      COMMIT

Two things are off here:

• name: iptables-restore.service is missing command: start - without that it doesn't work (at least as per of CoreOS 717 and 723)
• more serious issue: this iptables setup doesn't protect your CoreOS cluster from anybody connecting to it via the private networking interface (-A INPUT -i eth1 -j ACCEPT) on eg. DigitalOcean. I'm still looking for ways how to have the iptables rules change dynamically depending on the nodes in the cluster.

@jaren - the config should work cause it allows for all traffic on the private networking interface (eth1 - which also is a problem as per above) - I sometimes had to wait for a bit till the cluster fully formed.

Additional note, unrelated to the subject at hand (Iptables) is that this file references Etcd and not the more recent Etcd2. This can result in a very weird situation. Be explicit in what version you want (Etcd2 is preferred) and don't reference the other version or you might get two Etcd instances fighting for the cluster.

See also: etcd-io/etcd#3103

@oliver006 did you ever find a solution to locking down the private networking interface?

I came up with a file that addresses some of the concerns mentioned here and fixes another problem that I was having. iptables doesn't seem to want to start if the rules-save file doesn't end with a newline. I was not successful in coming up with a cloud-config that would write a newline but it turns out that a comment line works as well: https://gist.github.com/socketwiz/d6fe23d19180a1ad8b5a

If anyone sees any problems with my cloud-config, please let me know.

@sjlu @socketwiz @oliver006 Any solution found for the private network security concern?