Skip to content

Instantly share code, notes, and snippets.

@dbathgate
Last active July 7, 2022 14:13
Show Gist options
  • Star 8 You must be signed in to star a gist
  • Fork 1 You must be signed in to fork a gist
  • Save dbathgate/dc94fa88c73dd32bec8fcb503e9c6294 to your computer and use it in GitHub Desktop.
Save dbathgate/dc94fa88c73dd32bec8fcb503e9c6294 to your computer and use it in GitHub Desktop.
Assign VPC addresses to Docker containers associated as secondary IP addresses to an ENI in AWS
# Description: How to assign VPC addresses to Docker containers associated as secondary IP addresses to an ENI in AWS
# Operating System: Amazon Linux AMI 2016.09.1 (HVM)
# Prerequisites:
# - Assign N number of secondary IP addresses to network interface on Docker host
# - Each new container requires additional assignment of secondary IP addresses
# - Containers can be launched with --net=none to avoid adding a Docker networked NIC (docker run --net=none -d imageId)
# Credit to https://github.com/jpetazzo/pipework for steps on linking IP address
################# Setup Bridge of eth0 ######################
yum install bridge-utils -y
cat >> /etc/sysconfig/network-scripts/ifcfg-br0 <<EOF
DEVICE=br0
TYPE=Bridge
BOOTPROTO=dhcp
ONBOOT=yes
STP=no
EOF
ifup br0
echo "BRIDGE=br0" >> /etc/sysconfig/network-scripts/ifcfg-eth0
/etc/init.d/network restart
#############################################################
################## Collect parameters #######################
container_id=010337e45492 # Target container
ip=10.1.0.59 # Secondary IP address of ENI (to be assigned to container)
gateway_ip=$(ifconfig br0 | grep "inet addr" | awk '{print $2}' | sed -e "s/addr:\(.*\)/\1/") # Primary IP of ENI (IP assigned to eth0)
mtu=$(ifconfig br0 | grep MTU | awk '{print $5}' | sed -e "s/MTU:\(.*\)/\1/") # MTU of eth0 (typically 9001 for Ubuntu AMI)
pid=$(docker inspect --format='{{ .State.Pid }}' ${container_id})
#############################################################
############ Setup virtual ethernet pair of br0 #############
ip link add name veth0pl${pid} mtu ${mtu} type veth peer name veth0pg${pid} mtu ${mtu}
ip link set veth0pl${pid} master br0
ip link set veth0pl${pid} up
#############################################################
############ Link veth to container using netns #############
mkdir -p /var/run/netns/
ln -s /proc/${pid}/ns/net /var/run/netns/${pid}
ip link set veth0pg${pid} netns ${pid}
ip netns exec ${pid} ip link set veth0pg${pid} name eth1
ip netns exec ${pid} ip link set dev eth1 up
#############################################################
####### Setup routing and static IP for container ###########
ip netns exec ${pid} ip -4 route delete default
ip netns exec ${pid} ip -4 route add ${gateway_ip}/32 dev eth1
ip netns exec ${pid} ip -4 route replace default via ${gateway_ip} dev eth1
ip netns exec ${pid} ip -4 addr add ${ip}/32 dev eth1
#############################################################
# Description: How to assign VPC addresses to Docker containers associated as secondary IP addresses to an ENI in AWS
# Operating System: Ubuntu Server 14.04 LTS (HVM)
# Prerequisites:
# - Assign N number of secondary IP addresses to network interface on Docker host
# - Each new container requires additional assignment of secondary IP addresses
# - Containers can be launched with --net=none to avoid adding a Docker networked NIC (docker run --net=none -d imageId)
# Credit to https://github.com/jpetazzo/pipework for steps on linking IP address
################# Setup Bridge of eth0 ######################
apt-get update && apt-get install bridge-utils
cat >> /etc/network/interfaces.d/eth0.cfg <<EOF
auto br0
iface br0 inet dhcp
bridge_ports eth0
bridge_stp off
bridge_fd 0
bridge_maxwait 0
EOF
ifup br0
#############################################################
################## Collect parameters #######################
container_id=010337e45492 # Target container
ip=10.1.0.59 # Secondary IP address of ENI (to be assigned to container)
gateway_ip=$(ifconfig br0 | grep "inet addr" | awk '{print $2}' | sed -e "s/addr:\(.*\)/\1/") # Primary IP of ENI (IP assigned to eth0)
mtu=$(ifconfig br0 | grep MTU | awk '{print $5}' | sed -e "s/MTU:\(.*\)/\1/") # MTU of eth0 (typically 9001 for Ubuntu AMI)
pid=$(docker inspect --format='{{ .State.Pid }}' ${container_id})
#############################################################
############ Setup virtual ethernet pair of br0 #############
ip link add name veth0pl${pid} mtu ${mtu} type veth peer name veth0pg${pid} mtu ${mtu}
ip link set veth0pl${pid} master br0
ip link set veth0pl${pid} up
#############################################################
############ Link veth to container using netns #############
mkdir -p /var/run/netns/
ln -s /proc/${pid}/ns/net /var/run/netns/${pid}
ip link set veth0pg${pid} netns ${pid}
ip netns exec ${pid} ip link set veth0pg${pid} name eth1
ip netns exec ${pid} ip link set dev eth1 up
#############################################################
####### Setup routing and static IP for container ###########
ip netns exec ${pid} ip -4 route delete default
ip netns exec ${pid} ip -4 route add ${gateway_ip}/32 dev eth1
ip netns exec ${pid} ip -4 route replace default via ${gateway_ip} dev eth1
ip netns exec ${pid} ip -4 addr add ${ip}/32 dev eth1
#############################################################
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment