QEMU usermode virtual vlan using -netdev socket

qemu_netdev_socket_vlan.md

Goal

How to launch multiple QEMU based VM's that can communicate with the outside world and between each other other without any setup on the host.

This uses two features available in qemu

• User Mode Networking stack - SLIRP
• Socket networking backend allows you to create a network of guests that can see each other

This allows us to have

• SLIRP to communicate to the outside world
• multicast to communicate between the VMs
• And it uses the multicast to allow the host to communicate with the guests

Virtual Machine launch

VMN=1
qemu-system-x86_64 \
    -machine pc,accel=kvm,kernel_irqchip \
    -enable-kvm \
    -bios OVMF.fd \
    -smp sockets=1,cpus=4,cores=2 -cpu host \
    -m 1024 \
    -vga none -nographic \
    -drive file="$IMAGE",if=virtio,aio=threads,format=raw \
    -netdev user,id=mynet0,hostfwd=tcp::${VMN}0022-:22,hostfwd=tcp::${VMN}2375-:2375 \
    -device virtio-net-pci,netdev=mynet0 \
    -netdev socket,id=vlan,mcast=230.0.0.1:1234 \
    -device virtio-net-pci,netdev=vlan \

You can add any number of VM's to this socket network, choosing a different value for VMN

Connecting to this network from the host

You can use socat to create a tap interface on the host with an IP in the same subnet as you want to VLAN to be on and

sudo socat UDP4-DATAGRAM:230.0.0.1:1234,sourceport=1234,reuseaddr,ip-add-membership=230.0.0.1:127.0.0.1 TUN:10.0.3.1/24,tun-type=tap,iff-no-pi,iff-up,tun-name=vmvlan0

This will create a tap interface vmvlan0 on the host with IP: 10.0.3.1

We can run a DHCP server on this tap interface to allow for seamless configuration of the VLAN driven by the host

DHCP based VLAN configuration

To assign IP addresss to these VM's from the host you can use dnsmasq serving DHCP on the vmvlan0 interface.

sudo -E dnsmasq --conf-file=dnsmasq.conf --leasefile-ro

dnsmasq.conf

strict-order
pid-file=default.pid
except-interface=lo
bind-dynamic
interface=vmvlan0
dhcp-range=10.0.3.100,10.0.3.200
dhcp-no-override
dhcp-authoritative
dhcp-lease-max=253

• https://wiki.qemu.org/Documentation/Networking
• https://archive.fosdem.org/2018/schedule/event/vai_qemu_jungle/attachments/slides/2539/export/events/attachments/vai_qemu_jungle/slides/2539/qemu_cli_jungle.pdf
• https://lists.gnu.org/archive/html/qemu-discuss/2018-01/msg00036.html
• https://people.gnome.org/~markmc/qemu-networking.html

To potentially solve the manual IP address setup

https://unix.stackexchange.com/questions/430727/accessing-qemu-multicast-networks-from-docker-containers

Ok, I had a look at the format of the multicast packets QEMU sends, and it's just the ethernet frame. So you can do something like

socat UDP4-DATAGRAM:230.0.0.1:1234,sourceport=1234,ip-add-membership=230.0.0.1:10.0.0.2 TUN:10.2.3.1/24,tun-type=tap,iff-no-pi,iff-up
on the host that has the docker container, where 10.0.0.2 is a valid IP address for the host with respect to the multicast group, e.g. the IP address of eth0. This will give you a tap (level 2) interface where you can send ethernet packets to and receive them from the multicast group. Now you only have to bridge/route those to your Docker container.

Instead of socat, you can also write your own short C program etc., if you want and think it's more efficient.

I guess we should call you multicastelino! Thanks for this. 🙂

Can confirm this worked for me on arch hosts and guests, except for the dhcp server part. But might have been an issue with network-manager config on the guests. Got it to work with static IPs though.

Also using spice for shared clipboard so found that incrementing spicechannel{0,1,2,...} kept the clipboard working on all guests

-display spice-app \
-device virtio-serial-pci \
-spice port=593${VMN},disable-ticketing=on \
-device virtserialport,chardev=spicechannel${VMN},name=com.redhat.spice.0 \
-chardev spicevmc,id=spicechannel${VMN},name=vdagent \