Skip to content

Instantly share code, notes, and snippets.

Embed
What would you like to do?
Notes about using point-to-point links for routing IP addresses in virtual machines
Some experiments, which worked but were eventually not used.
See http://www.greenhills.co.uk/2011/06/10/lxc.html
Now, the next big milestone is to get networking working.
In the config, we told lxc-create to use br0 as the link,
and inside the container we have eth0:
root@thunder:/# brctl show br0
bridge name bridge id STP enabled interfaces
br0 8000.962e6bb72480 no vethP70LNx
root@thunder:/# ip link list br0
7: br0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN
link/ether 96:2e:6b:b7:24:80 brd ff:ff:ff:ff:ff:ff
root@natty1:/# ip link list eth0
68: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
link/ether ee:4d:fd:34:c2:59 brd ff:ff:ff:ff:ff:ff
By way of initial test I did a simple internal routing setup akin to [this description](http://www.mail-archive.com/lxc-users@lists.sourceforge.net/msg01969.html),
to a RFC1918 address on the container.
root@thunder:/# ifconfig br0 up
root@thunder:/# ip route add 192.168.0.0/24 dev br0
root@thunder:/# ip addr add dev br0 172.16.0.1/24 brd +
root@thunder:/# ip route add to 192.168.0.99 via 172.16.0.2
root@thunder:/# echo 1 > /proc/sys/net/ipv4/ip_forward
root@natty1:/# ip addr add 192.168.0.99/32 dev eth0
root@natty1:/# ip route add default via 172.16.0.1
root@natty1:/# ping 46.43.35.202
PING 46.43.35.202 (46.43.35.202): 48 data bytes
56 bytes from 46.43.35.202: icmp_seq=0 ttl=64 time=0.047 ms
root@thunder:/# ping 192.168.0.99
PING 192.168.0.99 (192.168.0.99) 56(84) bytes of data.
64 bytes from 192.168.0.99: icmp_req=1 ttl=64 time=9.26 ms
and after copying /etc/resolv.conf from the controlling host to the container and
starting sshd in the container (<code>service sh start</code>) I could <code>ssh 192.168.0.99</code> in.
Now with the actual IP addresses I've been allocated by my ISP:
root@natty1:/# ip addr add 46.43.55.73 dev eth0
root@thunder:/# ip route add to 46.43.55.73 via 172.16.0.2
This allows me to ssh in over the internet.
Next problem: making outbound connections (e.g. ping www.google.com) fails.
tcpdump on the controlling host shows:
14:15:13.182730 IP 172.16.0.2 > ez-in-f105.1e100.net: ICMP echo request, id 49409, seq 512, length 56
Clearly that's not going to work; it needs to have 46.43.55.73.
You can force that with (from the console, not an ssh session):
root@natty1:/# ip route del default dev eth0
root@natty1:/# ip route add default via 172.16.0.1 dev eth0 src 46.43.55.73
or by the looks of it by making the internal address non-global:
root@natty1:/# ip addr del 172.16.0.2/4 dev eth0
root@natty1:/# ip addr add 172.16.0.2/4 brd + scope link dev eth0
root@natty1:/# ip route del default
root@natty1:/# ip route add default via 172.16.0.1 dev eth0
root@natty1:/# ping -q -c 1 www.google.com
PING www.l.google.com (209.85.146.103): 48 data bytes
--- www.l.google.com ping statistics ---
1 packets transmitted, 1 packets received, 0% packet loss
round-trip min/avg/max/stddev = 13.763/13.763/13.763/0.000 ms
This is all good, but do I need Proxy-ARP here?
It appears not: if I ping 46.43.55.74 from the internet the hosting provider duly
forwards it to the main IP address, as shown by tcpdump on the controlling host.
So, let's start with a new container, natty2 on 46.43.55.73,
and copy some more in, and optimise some.
NAME=natty2
IP=46.43.55.73
TRANSFER_SOURCE=172.16.73.2
TRANSFER_ROUTER=172.16.73.1
ip addr add dev br0 $TRANSFER_ROUTER/24 brd +
ip route add to $IP via $TRANSFER_SOURCE
echo 1 > /proc/sys/net/ipv4/ip_forward
TRANSFER_MASK=255.255.255.0
LXCDIR=/var/lib/lxc
ROOTFS=$LXCDIR/${NAME}/rootfs
CONFIG=/root/lxc-${NAME}-config.tmp
cat > $CONFIG <<EOM
lxc.network.type = veth
lxc.network.link = br0
lxc.network.name = eth0
EOM
cat > $ROOTFS/etc/network/interfaces <<EOM
auto lo
iface lo inet loopback
# public routable address
auto eth0
iface eth0 inet static
address $IP
netmask 255.255.255.255
# transfer network
auto eth0:0
iface eth0:0 inet static
name transfer network
address $TRANSFER_SOURCE
netmask $TRANSFER_MASK
pointopoint $TRANSFER_ROUTER
post-up ip route add default via $TRANSFER_ROUTER dev eth0 src $IP
pre-down ip route del default dev eth0
EOM
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.