Skip to content

Instantly share code, notes, and snippets.

Last active August 22, 2020 02:14
Show Gist options
  • Save amirkdv/e9f47fa3209d54f5978c to your computer and use it in GitHub Desktop.
Save amirkdv/e9f47fa3209d54f5978c to your computer and use it in GitHub Desktop.
Transparent Proxy to docker containers

This is an example of using Linux Kernel's Transparent Proxy to route all TCP traffic to docker containers without having to resort to PROXY protocol which is not supported by some applications (e.g. sshd). To get the demo to work you only need vagrant installed:

git clone [this-gist] tproxy-demo
cd tproxy-demo
vagrant up
# follow instructions in the very last few lines of vagrant provisioner:

# tab #1
vagrant ssh -- sudo make -C /vagrant start_nc
# tab #2
vagrant ssh -- sudo make -C /vagrant start_haproxy
# tab #3
nc 9000 # tab 1 logs must show (vagrant host) and not (proxy IP)


FROM ubuntu:14.04
RUN apt-get update && apt-get install -y netcat-openbsd
CMD ["nc", "-vkl", "7000"]
log local0
mode tcp
log global
option tcplog
retries 3
maxconn 1000
timeout connect 5s
timeout client 15min
timeout server 15min
listen tcp-in *:9000
mode tcp
log global
# the following line is the magic, remove this and nc logs will show
# proxy ip ( as opposed to original client ip.
source usesrc clientip
server srv-tcp # hardcodes docker container IP, might break
@echo '============================[fixing locale]============================'
locale-gen en_CA.utf8
update-locale LANG=en_CA.utf8
@echo '==========================[installing docker]========================='
apt-get update
apt-get install -y apt-transport-https
apt-key adv --keyserver hkp:// --recv-keys 36A1D7869245C8950F966E92D8576A8BA88D21E9
echo 'deb docker main' > /etc/apt/sources.list.d/docker.list
apt-get update
apt-get install -y lxc-docker
@echo '==================[enabling xt_TPROXY and xt_socket]=================='
find /lib/modules/`uname -r` | grep -q xt_TPROXY
find /lib/modules/`uname -r` | grep -q xt_socket
printf "xt_TPROXY\nxt_socket\n" | tee -a /etc/modules | xargs modprobe
@echo '================[configuring iptables rules for TPROXY]==============='
iptables -t mangle -N DIVERT
iptables -t mangle -A PREROUTING -p tcp -m socket -j DIVERT
iptables -t mangle -A DIVERT -j MARK --set-mark 111
iptables -t mangle -A DIVERT -j ACCEPT
ip rule add fwmark 111 lookup 100
ip route add local dev lo table 100
echo 1 > /proc/sys/net/ipv4/conf/all/forwarding
dependencies: docker mods net_config
@echo '======================[building netcat container]====================='
docker build -q -t amirkdv/nc .
@echo '================[installing HAProxy w/ TPROXY support]================'
apt-get update
apt-get install -y build-essential make gcc
tar -zxf haproxy-1.5-dev26.tar.gz
make -C haproxy-1.5-dev26 TARGET=linux26 CPU=x86_64 USE_LINUX_TPROXY=1
make -C haproxy-1.5-dev26 install target=linux26
build: fix_locale dependencies build_haproxy build_nc net_config
@echo '==============[starting netcat container (port: 9876)]================'
docker run -i -t --expose 7000 amirkdv/nc
@echo '===================[starting haproxy in debug mode]==================='
haproxy -f /vagrant/haproxy.cfg -d
@echo '======================[transparent proxy demo]========================'
@echo '1. in the first tab start the netcat container:'
@echo ' vagrant ssh -- sudo make -C /vagrant start_nc'
@echo '2. in a second tab start HAProxy:'
@echo ' vagrant ssh -- sudo make -C /vagrant start_haproxy'
@echo '3. in a third tab connect to the proxied netcat container and watch logs on tab 1 and 2:'
@echo ' nc 9000'
@echo ' you must see (originating IP) in nc logs and not (proxy ip)'
.PHONY: all
# -*- mode: ruby -*-
# vi: set ft=ruby :
Vagrant.configure('2') do |config| = 'trusty64'
config.vm.box_url = ''
config.vm.hostname = 'transparent-proxy' :private_network, ip: ''
config.vm.provision 'shell', inline: 'make -C /vagrant build'
config.vm.provision 'shell', inline: 'make -C /vagrant demo'
# works around network slowness inside vagrant, see
config.vm.provider :virtualbox do |vb|
vb.customize ["modifyvm", :id, "--natdnshostresolver1", "on"]
vb.customize ["modifyvm", :id, "--natdnsproxy1", "on"]
Copy link

aledbf commented May 5, 2016

@amirkdv thank you for this guide!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment