The first thing when you come to this topic is that when it comes to iptables, docker just behaves like a spoiled child that takes what (s)he want, although the child is been banned to do so.
In other words, as you may already know, by default, docker ignores any pre-existing iptables' rules when adding its own Chain in the tables. The result of this is a useless firewall when it comes to protecting the access to your services hosted in containers.
So, at the office, at the moment we had this problem we were migrating our projects into containers. Since we wanted to do it right, we started with the private machines and internal propose inside our toolchain: jenkins, nexus, etc. The basic team for any development team nowadays.
And so we started deploying the containers into the (physical) servers, everything seemed right until we saw that those machines could be accessed from any place, although we had a iptables-based firewall. The IP filtering rules we were applying to the internal tools for been reachable only from a subset of IPs was not been respected anymore and docker was the responsible for that.
What happened? Docker, that little bastard and its own dynamic Chain was making the containers and its services public to the world and creating a security problem, a big one.
So, we dive into SO, Github's issues and basically any existing post about this, getting the following conclusions:
- People was pissed off with this default behavior.
- Apart from the problem itself, the bigger problem was the lack of an easy solution that would preserve security and containers in the same recipe. And by easy we mean that could be automatable.
- In many cases, this thing was not a problem if you were already using AWS or equivalent, the solution is provided by the platform on a upper level. Or even if you have several machines, you and create your own NAT on a single purpose machine that will filter and redirect the traffic to any other machine.
- If your infrastructure is not specialized, virtualized but you want it automated (or just, working under good implementations) , you, well, you're basically up to have a bad time.
Or not.
On a first attempt we tried creating a docker container with nginx and iptables in it. Iptables would filter the traffic and nginx would forward it to the rest of the containers.
Since we were pretty on the run for getting a solution, we thought it could be enough for some time.
But, after trying to implement the solution for some of our containers we found out that we could not use this solution on many cases. If you wanted to do SSL for your services, the development of each docker-compose was becoming long, manual and not generic at all. And that was in the best case scenario, on the bad ones, the nginx would not even work good enough as for been a valid option on each situation.
After seeing the restrictions on the first solution we step back to the origin of the problem: the iptables.
In the end, it was the best place where to focus on, the problem was there and so our solution should be implemented, despite of the quality of any solution implemented on a lower layer, removing the problem on the root felt like the best scope.
What did we do? Well, after disabling docker's iptables option, we started fixing the ip addresses for the docker networks we were creating in our docker-compose files. Knowing the ip address in advance, we could append as many rules as networks at the end of our INPUT chain and the problem would been solve.
The solution was nice, it solve every single problem that we had, but, it was not enough. The problem in this case was development becoming tedious and for sure the solution was not smooth enough.
It worked and could have stayed like that for some time, but it was not sharp enough for our aim and we knew that after some time fighting against fixed ip addresses and creating to much code for declaring networks, we would like to use something better.
After some time scratching our minds on how to improve the previous solution we realized what we were missing.
Docker uses an IP range for creating its own networks, and that IP range is always the same, it can use any address for a network, but by default, will use a free address in the range. And we could knew the range in advance.
So, instead of fixing the address and declaring them in the iptables, we would just append at the end of the rules the range that docker was using, accepting any traffic on it, and that would be all.
This way we were getting everything we were aiming for:
- It can be automated.
- It does not affect to development, no extra code or infrastructure in our docker compositions.
- Easy to maintain and track.
And so we did. Now it's working and mostly everyone already forgot that one day we were exposed to the world because of a spoiled child.
Read this. Realize it's not enough and not scalable over the time and systems.
Use this code:
> cat /etc/default/docker
DOCKER_OPTS=" \
--iptables=false"
> sudo service restart docker
> cat /etc/iptables.d/10-docker
-A FORWARD -d 172.16.0.0/12 -j ACCEPT
-A FORWARD -s 172.16.0.0/12 -j ACCEPT
> flush-and-load-iptables-from-files.sh
Now, forget about the problem of docker and iptables, it would be better for your health.
http://stackoverflow.com/a/28368435 https://developer.ibm.com/recipes/tutorials/networking-your-docker-containers-using-docker0-bridge/ https://docs.docker.com/v1.5/articles/networking/ https://docs.docker.com/engine/userguide/networking/default_network/custom-docker0/ https://svenv.nl/unixandlinux/dockerufw https://fralef.me/docker-and-iptables.html
moby/moby#1871 (comment) moby/moby#22054 (comment)
https://fordodone.com/2016/03/30/docker-compose-static-ip-address-in-docker-compose-yml/ https://jpetazzo.github.io/2013/10/16/configure-docker-bridge-network/
http://rudijs.github.io/2015-07/docker-restricting-container-access-with-iptables/