Skip to content

Instantly share code, notes, and snippets.

@lukasnellen
Last active September 23, 2024 04:19
Show Gist options
  • Save lukasnellen/20761a20286f32efc396e207d986295d to your computer and use it in GitHub Desktop.
Save lukasnellen/20761a20286f32efc396e207d986295d to your computer and use it in GitHub Desktop.
setup shorewall for docker networking beyond the default bridge network, e.g., for docker-compose

Docker(-compose) with shorewall

The shorewall documentation explains in http://shorewall.org/Docker.html how to configure shorewall for use with docker. The problem with the configuration is that it only allows connections from the host to the main bridge docker0. Connections to other networks on dynamically created bridges, with names starting by default with br-, is blocked. Instead of the recommended contents of /etc/shorewall/interfaces, use wild-card interface names as follows:

#ZONE	INTERFACE	OPTIONS
#dock	docker0		bridge     # disabled default recommendation
dock 	docker0		physical=docker+,routeback=1
dock 	br		physical=br-+,routeback=1

This declares interfaces with names starting with docker, including the default docker0, and starting with br- to be in the dock zone.

For the rest of the configuration, follow the shorewall documentation as is.

This setup fixes problems running composite apps set up manually or using docker-compose.

###############################################################################
?FORMAT 2
###############################################################################
#ZONE INTERFACE OPTIONS
net eth0 dhcp,tcpflags,nosmurfs,routefilter,logmartians,sourceroute=0
#dock docker0 bridge
dock docker0 physical=docker+,routeback=1
dock br physical=br-+,routeback=1
# and others ...
###############################################################################
#SOURCE DEST POLICY LOG LEVEL LIMIT:BURST
$FW net ACCEPT
dock $FW REJECT
dock all ACCEPT
# Some applications or use case require uncommenting the next line
$FW dock ACCEPT
# THE FOLLOWING POLICY MUST BE LAST
all all REJECT info
# apply the change below, as
# ...
# LN:
#DOCKER=No
DOCKER=Yes
# ...
###############################################################################
#ZONE TYPE OPTIONS IN OUT
# OPTIONS OPTIONS
fw firewall
dock ipv4
# and others ...
@lukasnellen
Copy link
Author

Instead of using the DOCKER-USER chain, I set up rules in shorewall that filter connections to or from the dock zone. For more fine-grained control, you can use some of docker's advanced options:

  • create separate networks, maybe with a named bridge so you can create a special zone on that bridge
  • set the IP of the containers in docker run or in docker-compose.yml
  • user shorewall's dynamic zones (https://shorewall.org/Dynamic.html), put your container's IP into the correct, dynamic zone, and filter on those zones.

The point here is to control the interface or IP address used by a container. That way, you can write specific filter rules in shorewall w/o having to inspect the container's configuration, which might change when the container gets re-created. This is very similar to how I set up filtering for full-blown virtual machines.

The alternative is to use iptables to manipulate the DOCKER-USER table, which is not managed directly by shorewall.

@phlax
Copy link

phlax commented Apr 7, 2020

related to above comments about iptables rules on debian, it seems that the versions of shorewall and docker in debian stable and above do not work (at least not without always restarting docker after shorewall is restarted)

i rebuilt the debian shorewall packages (shorewall + shorewall-core) with the 5.2.4 version and this fixed the problem

i have raised an issue on the shorewall gitlab issue tracker

https://gitlab.com/shorewall/debian/-/issues/1

@interso
Copy link

interso commented Aug 4, 2020

Thank you very much.

@grharry
Copy link

grharry commented Oct 31, 2020

Great post!
However in my case it's missing a vital detail !
I disabled docker iptables in /etc/docker/daemon.json
{
"iptables":false
}
And after that all works Great! Docker containers are accessible from net if enabled in shorewall ( rules, zones, policy ) etc !!
Regards,
Harry

@sgrayban
Copy link

sgrayban commented Sep 9, 2021

This line is giving a Broadcast error when starting Shorewall 5.2.3.2

dock docker0 physical=docker+,routeback=1

ERROR: Invalid BROADCAST address /etc/shorewall/interfaces

@leo-fg
Copy link

leo-fg commented Sep 16, 2021

I added different and it worked as expected.

dock docker physical=docker+,routeback=1

@lukasnellen
Copy link
Author

The actual name should not matter, since the wildcard physical=docker+ is there to match any interface with a name starting with docker.

Details might also depend on the way how the distribution sets up networking and on the order in which components get started.

Disclaimer: My production machines are debian, versions 9-11. So details might vary. I'm glad to get additional information on what to change/fix on other distributions.

@sgrayban
Copy link

I am using debian 10 and just cannot get this to work -- the only way to get it working for me, and I still don't think its right, is using::

dock    docker0
dock    br

shorewall version
5.2.3.2

@lukasnellen
Copy link
Author

Did you try running shorewall compile? This gives better diagnostics than trying to restart the service. And it allows you to detect and fix problems rather than leaving the firewall in an unreliable or diabled state (especially when restarting).

I recently migrated my debian 10 docker host to 11 and don't have a debian 10 docker host at hand to verify the behaviour.

@sgrayban
Copy link

shorewall compile

Compiling using Shorewall 5.2.3.2...
Processing /etc/shorewall/params ...
Processing /etc/shorewall/shorewall.conf...
Loading Modules...
Compiling /etc/shorewall/zones...
Compiling /etc/shorewall/interfaces...
ERROR: Invalid BROADCAST address /etc/shorewall/interfaces (line 191)

@lukasnellen
Copy link
Author

Do you have the format specifier at the top of the interfaces file? The line

?FORMAT 2

is required. If I leave it out, I can reproduce the error message about the BROADCAST address in my setup. It almost looks like a comment i the file I have included at the top.

@sgrayban
Copy link

Crap... missed that. It's working now!! Thanks mate.

@athimel
Copy link

athimel commented Mar 25, 2022

Great post! However in my case it's missing a vital detail ! I disabled docker iptables in /etc/docker/daemon.json { "iptables":false } And after that all works Great! Docker containers are accessible from net if enabled in shorewall ( rules, zones, policy ) etc !! Regards, Harry

@grharry This is the ""detail"" I was missing! Thank you so much!

@kickerofelves
Copy link

kickerofelves commented Feb 21, 2023

Thanks! New docker user and this gist and comments allowed me to connect to running containers.

But now running containers cannot reach the wider internet. How do I do that?

Edit: resolved with help from serverfault.com/a/631877 specifically creating a /etc/shorewall/masq file and turning on ip forwarding.

@vshuraeff
Copy link

Has anyone succeeded in restricting network access to containers for the entire Internet and only allowing access to a specific list of addresses (ipset)?

@robozb
Copy link

robozb commented Sep 11, 2023

You saved my day, thank you so much!

@abidibo
Copy link

abidibo commented Nov 17, 2023

Hero!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment