LXC environment setup for live migration assignment

notes.md

Launching host containers

lxc launch image:alpine:3.15 host1
lxc launch image:alpine:3.15 host2

Essential packages in alpine

• qemu-system-x86_64
• qemu-img
• qemu-ui-curses
• busybox-extras (for telnet)
• bridge

SSH extra configs in guest

• Set permit root login to yes
• Listen to all interfaces (maybe not needed)

Adding shared mount and kvm devices

lxc storage volume create default nfs
lxc config device add host1 nfs disk pool=default source=nfs path=/mnt/nfs
lxc config device add host2 nfs disk pool=default source=nfs path=/mnt/nfs
lxc config device add host1 kvm unix-char path=/dev/kvm
lxc config device add host2 kvm unix-char path=/dev/kvm

Setting up bridge and tap

This has to be done on both hosts, or else ssh will die after migration (due to lost of internet connection).

Automatically

In /etc/network/interfaces:

auto tap0 inet manual
        pre-up tunctl -t tap0

auto br0
iface br0 inet dhcp
        bridge-ports eth0 tap0
        bridge-stp 0

hostname $(hostname)

Alternatively, the tap device can be setup manually:

tunctl
brctl addif br0 tap0
ip link set tap0 up

Manually

brctl addbr br0
brctl addif br0 eth0

IP4=$(ip -f inet -o a | awk '/eth0/ { print $4 }')
ip addr add dev br0 $IP4
ip addr del $IP4 dev eth0

GATEWAY=$(ip r | awk '/^default/ { print $3 }')
ip link set br0 up
ip r del default
ip r add default via $GATEWAY dev br0

tunctl
brctl addif br0 tap0
ip link set tap0 up

Launching qemu for install

qemu-system-x86_64 -cpu host -enable-kvm -m 1G -smp 1 \
    -drive if=virtio,format=raw,file=/mnt/nfs/alpine.img \
    -boot d -cdrom alpine-virt-3.15.4-x86_64.iso \
    -curses -nographic \
    -netdev tap,id=tap0,ifname=tap0,script=no,downscript=no \
    -device virtio-net-pci,netdev=tap0

Launching qemu for migrate

On host1:

qemu-system-x86_64 -cpu host -enable-kvm -m 1G -smp 1 \
    -drive if=virtio,format=raw,file=/mnt/nfs/alpine.img \
    -netdev tap,id=tap0,ifname=tap0,script=no,downscript=no \
    -device virtio-net-pci,netdev=tap0 \
    -curses -nographic \
    -monitor telnet:127.0.0.1:5500,server,nowait

On host2:

qemu-system-x86_64 -cpu host -enable-kvm -m 1G -smp 1 \
   -drive if=virtio,format=raw,file=/mnt/nfs/alpine.img \
   -netdev tap,id=tap0,ifname=tap0,script=no,downscript=no \
   -device virtio-net-pci,netdev=tap0 \
   -curses -nographic \
   -incoming tcp:0:4400

After that just do telnet localhost:5500 and migrate tcp:<ip of host2>:4400.