Qemu/KVM with VDE networking

vdeq.sh

#!/bin/bash

#
#To start a linux QEMU instance with an ext2 image rootfs and VDE networking
#
#
macstr=("`printf 'DE:AD:BE:EF:%02X:%02X\n' $((RANDOM%256)) $((RANDOM%256))`")

LOCATION="/home/eerihug/dev/buildroot/output/images"
KERNEL="bzImage"
DISK="rootfs.ext2"
				
vdeq qemu-system-x86_64 -kernel $LOCATION/$KERNEL \
-boot c \
-m 512 \
-append "root=/dev/sda rw panic=1 console=ttyS0, 115200" \
-localtime \
-no-reboot \
-name Megatron \
-net nic,vlan=0,model=e1000,macaddr=$macstr \
-net vde,sock=/var/run/kvm0.ctl \
-net user,net=192.168.0.0/24 \
-nographic \
-hda $LOCATION/$DISK

Guide to set up qemuctl (from Ubuntu wiki)
http://tjworld.net/wiki/Linux/Ubuntu/VirtualMachinesWithVDENetworking

Virtual Machines With VDE Networking

For a long time now I've used kvm, the hardware-accelerated alternative to QEMU, for running Intel x86-based VM (Virtual Machine) guests. After a lot of investigation I decided to use VDE2 (Virtual Distributed Ethernet) to provide the networking support. I chose it because it provided the closest to a regular network experience of any of the alternatives, especially since it is a full Ethernet emulation. I use dnsmasq to provide DNS and DHCP services to the virtual network.

There are many nuances to getting a perfect VDE configuration so it is time I documented the configuration I've used with Ubuntu Gutsy and Hardy.

For generic situations libvirt and virt-manager are useful tools to help manage VM clusters. ubuntu-vm-builder is useful for creating VMs and adding them to libvirt hosts.
VDE and libvirt Don't Play Together

Unfortunately libvirt doesn't currently support VDE networks, although it is possible for someone to implement a VDE interface using the libvirt network API.
Install Packages

These are the required packages:

sudo apt-get install vde2 dnsmasq kvm

Optional simple GUI VM management:

sudo apt-get install qemuctl

Optional libvirt support (currently no use with VDE, and limited if the VMs are started with lots of customised options):

sudo apt-get install libvirt0 libvirt-bin virt-manager ubuntu-vm-builder

Add User(s) to Groups

Add the current user to network and virtual machine groups:

sudo adduser $(id -un) kvm
sudo adduser $(id -un) vde2-net
sudo adduser $(id -un) tun

Optional libvirt support:

sudo adduser $(id -un) libvirtd

Notice: At this point the user should log-out and log-in to make these new group memberships active. Check them with:

groups
tj adm disk dialout cdrom floppy audio dip video plugdev fuse lpadmin admin kvm sambashare vde2-net libvirtd tun

Give New Devices tun Group Ownership

This is a udev rule that is fired when a new tun/tap device is created:

cat <<EOF | sudo tee /etc/udev/rules.d/41-permissions-tun.rules

provide non-root user access to tun/tap devices for Virtual Machines using VDE etc

KERNEL=="tun", GROUP="tun"
EOF

Because the user is a member of the 'tun' group they can read and write to tun/tap devices. This makes it possible for VM guests to run with regular user permissions and still use VDE networking.

To reread the udev rules do:

sudo udevadm control --reload_rules

Define the VDE Network Interface

I chose to name the virtual network interface kvm0 but you might call it anything that doesn't conflict with other devices. This stanza will ensure the interface is set-up at system start-up (auto kvm0). The absolute values I've defined such as the IP address should be changed to suit:

cat <<EOF | sudo tee -a /etc/network/interfaces

KVM/QEMU Virtual Machine Network

auto kvm0
iface kvm0 inet static
address 10.254.1.254
netmask 255.255.255.0
pre-up /usr/bin/vde_switch --tap ${IFACE} --daemon --group vde2-net --sock /var/run/${IFACE}.ctl
--mod 775 --mgmtmode 770 --mgmt /var/run/${IFACE}-manage --pidfile /var/run/${IFACE}_vde.pid
up /usr/sbin/dnsmasq --interface=${IFACE} --except-interface=lo --bind-interfaces --user=nobody
--dhcp-range=kvm,10.254.1.1,10.254.1.253,255.255.255.0,10.254.1.255,8h
--domain=lan.tjworld.net --pid-file=/var/run/${IFACE}_dnsmasq.pid --conf-file
up echo "route -n | sed -n 's/^0\.0\.0\.0 .* \(.*\)$/\1/p'" > /var/run/${IFACE}_route
up iptables -t nat -A POSTROUTING -o cat /var/run/${IFACE}_route -j MASQUERADE
down iptables -t nat -D POSTROUTING -o cat /var/run/${IFACE}_route -j MASQUERADE
down kill -s TERM cat /var/run/${IFACE}_dnsmasq.pid && rm -f /var/run/${IFACE}_dnsmasq.pid
post-down kill -s HUP cat /var/run/${IFACE}_vde.pid
post-down rm -f /var/run/${IFACE}_route
EOF

The chosen sub-net and interface IP address shouldn't be the same as the host or other sub-nets in use. In this case the physical LAN sub-net is 10.254.251.0/24 and there are remote VPNs on other /24 sub-nets all routable locally.

dnsmasq is given its settings directly to avoid it conflicting with any other instance running that is using the default /etc/dnsmasq.conf settings, which all instances of dnsmasq will read unless the --conf-file (-C) option is given.

In this case it only attaches and binds to the kvm0 interface, so it won't interfere with other dnsmasq instances or, for example, bind, if that is running on the host's primary interfaces.

on the kvm0 interface any VMs that connect and request DHCP leases will get an IP v4 address in the range 10.254.1.1 - 253. The gateway is the address given to kvm0 - 10.254.1.254.

iptables is used to create a netfilters Network Address Translation (NAT) rule that routes all outbound packets to the current default route interface. Because this can change between kvm0 starting and stopping, the current default-route interface is written to a text file in /var/run/kvm0_route and used later to correctly remove the netfilters rule.

To control the interface manually do:

start the interface

sudo ifup kvm0

stop the interface

sudo ifdown kvm0

When it is running it should show:

ifconfig kvm0

kvm0 Link encap:Ethernet HWaddr 00:ff:84:1d:b3:f0
inet addr:10.254.1.254 Bcast:10.254.1.255 Mask:255.255.255.0
inet6 addr: fe80::2ff:84ff:fe1d:b3f0/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:1 errors:0 dropped:0 overruns:0 frame:0
TX packets:22 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:500
RX bytes:53 (53.0 B) TX bytes:6478 (6.3 KB)

And the netfilters rule:

sudo iptables -t nat -nvL POSTROUTING

Chain POSTROUTING (policy ACCEPT 2233 packets, 205K bytes)
pkts bytes target prot opt in out source destination
3 193 MASQUERADE all -- * eth0 0.0.0.0/0 0.0.0.0/0