minimum viable yak shave to make docker's iptables management play nice with /etc/iptables.d/ based systems

idea.md

Overview:

1. disable docker's internal iptables management
2. write a simple utility 'pantheon-docker-iptables' to use after starting and stopping the container via systemd's ExecStartPre= and ExecStartPost=
   • on start this utility would run after docker starts the container, use docker inspect to figure out the IpAddress assigned to the container on the docker0 bridge. Also, use docker inspect to get the tcp/udp port mappings assigned to the container. Then, it would create the appropriate /etc/iptables.d/docker-container-<container_name> file and call restart iptables to load new rules.
   • on stop, remove the /etc/iptables.d/docker-container-<container_name> file and call restart iptables

Pros:

• maintains parity of most common method of using docker containers in dev/test cycles and production (ie: mapping host port to container port. Allows re-use of community containers since port collisions can be avoided.)
• systemd is pretty good about ensuring Exec* commands are always run, so we can rely on the Pre and Post getting executed as expected
• ensures /etc/iptables.d/ is canonical source of rules, iptables restarts will not blow away docker specific iptables rules

Cons:

• we are disabling docker's iptables management and doing it on our own. We may miss some non-standard use cases, or future use cases. In fairness, everything of any usefulness (eg: kubernetes, maybe deis, fleet, flynn) all have to disable docker's iptables or networking features and implement their own network management too.

Pseudo-code

chef:

  pantheon_docker_container 'hello-world' do
   ...
   ports [ '5001:5000' ]
  end

/etc/systemd/system/hello-world.service

  ExecStart=/usr/bin/docker run -n hello-world ...
  ExecStartPost=-/usr/local/bin/pantheon-docker-iptables create hello-world
  ExecStopPost=-/usr/local/bin/pantheon-docker-iptables destroy hello-world

/usr/local/bin/pantheon-docker-iptables

    #!/usr/bin/env joe-psuedo-code-language
    
    action = argv[1]
    container_name = argv[2]

    if action == create
        container_metadata = json.parse(`docker inspect hello-world`)
        container_ip = container_metadata['NetworkSettings']['IpAddress']  # eg: 172.17.0.2
        for each in container_metadata['NetworkSettings']['Ports']:
            # build up iptables rules for any tcp and udp port mappings on the container, then:
            file.write('/etc/iptables.d/docker-container_#{container_name}')
            exec('systemctl restart iptables')

    if action == destroy
        file.delete('/etc/iptables.d/docker-container_#{container_name}')
        exec('systemctl restart iptables')

example of docker inspect output (just the networking settings that we would be using). The trickiest part is iterating thru the port mappings but it actually doesn't look too hard, and at least it is json so no brittle regex tom-foolery is needed

$ docker inspect hello-world | jq -r .[].NetworkSettings
{
  "Ports": {
    "5001/tcp": [
      {
        "HostPort": "5000",
        "HostIp": "0.0.0.0"
      }
    ]
  },
  "PortMapping": null,
  "IPPrefixLen": 16,
  "IPAddress": "172.17.0.2",
  "Gateway": "172.17.42.1",
  "Bridge": "docker0"
}