NetworkManager bridged network for KVM guests

nmbrnetkvm.md

Setting up a bridged network for KVM guests

This will work with either networkd or NetworkManager as a resolver. In fact, this is the only way to do bridged KVM (libvirtd) networking with NetworkManager.

If you're using NetworkManager (on a desktop or laptop, for example) on your KVM host, follow these instructions to set up a bridge interface.

Once you have the host bridge set up, proceed as follows:

1. Create a bridge network device inside KVM. Edit and save the below text as file host-bridge.xml:

<network>
   <name>host-bridge</name>
   <forward mode="bridge"/>
   <bridge name="br0"/>
</network>

Then execute these commands (as a user in the libvirt group):

$ virsh net-define host-bridge.xml
$ virsh net-start host-bridge
$ virsh net-autostart host-bridge

2. Make it possible for hosts outside of KVM to talk to your bridged guest by making the following changes on the KVM host.

Load the br_netfilter module:

$ sudo modprobe br_netfilter

Persist on reboot by creating /etc/modules-load.d/br_netfilter.conf:

$ sudo echo "br_netfilter" > /etc/modules-load.d/br_netfilter.conf

Create /etc/sysctl.d/10-bridge.conf:

# Do not filter packets crossing a bridge
net.bridge.bridge-nf-call-ip6tables=0
net.bridge.bridge-nf-call-iptables=0
net.bridge.bridge-nf-call-arptables=0

Apply the config now:

$ sudo sysctl -p /etc/sysctl.d/10-bridge.conf

Check result:

$ sudo sysctl -a | grep "bridge-nf-call"

3. Configure the guest to use host-bridge. Open up the Virtual Machine Manager and then select the target guest. Go to the NIC device. The drop down for "Network Source" should now include a device called "Virtual netowrk 'host-bridge'". The "Bridge network device model" will be "virtio" if that's your KVM configuration's default.

Select that "host-bridge" device.

If you inspect the guest's XML (by using virsh dumplxml guestname), it shoud look something like this:

<interface type='network'>
   <mac address='52:54:8b:d9:bf:a2'/>
   <source network='host-bridge'/>
   <model type='virtio'/>
   <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>
</interface>'

Be sure to save your changes!

4. Go up to your router and add a DHCP reservation and DNS mapping for the guest (assuming you want a dynamic address and want to be able to easily find the guest later). Otherwise, be prepared to manually configure networking on the guest.

5. Start (or restart) the guest.

I tried this instruction set end to end. With minor corrections to the commands. I also made a shell file with these commands.

The guest os ubuntu22.04 server installs. And I am able to use the host bridge to launch it. But it does not connect to any network.

not lan, not a bridged network within the host machine not to internet. It does not get any ip address even if I run dhclient.

have no idea what to do.

I am able to change to the NAT network and do a dhclient command to get it running. did not try the macvtap, but macvtap worked in another machine so I guess it will work.

I am running ubuntu 22.04 desktop with virtual machine manager 4.0.0

here is the script if it matters any bit.

---

#got instructions from the following urls

#https://gist.github.com/plembo/a7b69f92953a76ab2d06533754b5e2bb

#https://gist.github.com/plembo/f7abd2d9b6f76e7afdece02dae7e5097

#We must know apriori what connections we want to use for the virtual server so we hard code these. We may be able to pass in this information in a later version of this script. This script needs much more development before distribution

#echo what is happening so we can trace what went right or wrong
#just comment the line if we dont want to print the commands that are in this script.

#can use exec command https://www.baeldung.com/linux/exec-command-in-shell-script to later turn off all output if this shell is not in debug mode. This will take some thought.

if [[ $1 =~ debug ]]
then
echo turning debug on
set -x

else
echo turning debug off
set +x
fi

#backing up these variables for future use.
#not all of these variables are used in the script. I added some
#lanConName="Mb_lport"
#lanConDev="enp37s0"

#host_bridge_con_name="host_bridge" #this is the name used by linux networking as the bridge connection name
#host_bridge_ifacename="hbr_iface" # this is the interface name given to the bridge under linux networking

#host_bridge_config_filename="HostBridge.xml" #this is the file in which the configuration information is supplied to libvirt through virsh
#host_bridge_libvirt_name="HostBridge" # this is the name used by libvirt to identify the bridge (as available in virt-manager and other virsh commands)

#host_bridge_slave_connection_name="hbr_slv"
#host_bridge_slave_dev_interface_name="hbr_slv_dev" #this variable is not used in this script. created this when typing the script expecting it to be useful. The physical network interface device name is used directly. it is not a different item.

#not all of these variables are used in the script. I added some
lanConName="enp37s0"
lanConDev="enp37s0"

host_bridge_con_name="br0" #this is the name used by linux networking as the bridge connection name
host_bridge_ifacename="br0" # this is the interface name given to the bridge under linux networking

host_bridge_config_filename="HostBridge.xml" #this is the file in which the configuration information is supplied to libvirt through virsh
host_bridge_libvirt_name="HostBridge" # this is the name used by libvirt to identify the bridge (as available in virt-manager and other virsh commands)

host_bridge_slave_connection_name="hbr_slv"
host_bridge_slave_dev_interface_name="hbr_slv_dev" #this variable is not used in this script. created this when typing the script expecting it to be useful. The physical network interface device name is used directly. it is not a different item.

#copying the connection detail of the host machine lanport connection, it maybe possible to extract this through the lanConName variable later in the development.
bridgeIPAddr="192.168.1.13/24"
bridgeGateway="192.168.1.1"
bridgeDNS="192.168.1.1"

#check if the necessary connection is present, really this needs more work. This is used to turn down the lan connection and turn up the bridge connection at the appropriate time.
if nmcli con show|grep $lanConName
then
echo the lan connection $lanConName is present
else
echo the lan connection $lanConName is missing
exit
fi

[[ $1 =~ debug ]] && read -p "press enter to continue"

#this was the original
#nmcli con show
#if the necessary host bridge is already present do nothing. Else create it.

if ! nmcli con show | grep -w $host_bridge_con_name # > /dev/null 2>&1 # develop this later if not debug discard the output just check if the thing is found.
then
[[ $1 =~ debug ]] && read -p "created linux host bridge with name ${host_bridge_con_name}. press enter to continue - "
nmcli con add ifname ${host_bridge_ifacename} type bridge con-name ${host_bridge_con_name}
else
[[ $1 =~ debug ]] && read -p "linux host bridge with name ${host_bridge_con_name} exists. press enter to continue - "
fi

if ! nmcli con show | grep -w ${host_bridge_slave_connection_name}
then
[[ $1 =~ debug ]] && read -p "inserting host bridge slave interface. Press enter to continue."
nmcli con add type bridge-slave ifname ${lanConDev} master ${host_bridge_con_name} con-name ${host_bridge_slave_connection_name} # the source does not suggest this. need to find if this is a problem
[[ $1 =~ debug ]] && read -p "check if host bridge has been created with proper slave connection and name"
else
[[ $1 =~ debug ]] && read -p "linux host bridge slave with name ${host_bridge_slave_connection_name} exists. press enter to continue - "
fi

#network connection STP (Spanning Tree Protocol) is a protocol used to avoid looping in network routing. But for some reason the instructions in my source want to turn it down. Some sources say this should not be done, others insist it is necessary. Including this in the script as an option.

nmcli con mod $host_bridge_con_name bridge.stp no

#now turn down the primary connection and also the bridge connection
nmcli con down $lanConName
nmcli con down $host_bridge_con_name

#now set the proper parameters for the bridge connection
nmcli con mod $host_bridge_con_name ipv4.method manual
nmcli con mod $host_bridge_con_name ipv4.addresses $bridgeIPAddr
nmcli con mod $host_bridge_con_name ipv4.gateway $bridgeGateway
nmcli con mod $host_bridge_con_name ipv4.dns $bridgeDNS

#turn the bridge connection up again
nmcli con up $host_bridge_con_name

[[ $1 =~ debug ]] && read -p "check host bridge status and hit enter or ctrl+c"

#restart the networking service to make sure that these settings are stored.
sudo systemctl restart NetworkManager.service

#check if the internet connection is working by querying the dns with a random name

host gnu.org
printf "\n \n checked gnu.org to check if the internet connection is working \n \n"

#create a HostBridge.xml file with the name of the bridge we want to use

#it is important here to note that the 'bridge name' to be supplied here is "host bridge slave interface device' name rather than the 'bridge connection' name. If the bridge connection name is supplied, virt-manager will refuse to create the virtual machine.
#wait a minute, this comment needs to be changed as i discover what needs to go into the tag

#the host bridge config file is used to introduce the libvirt connection name into libvirt. the libvirt name is used in further commands.
echo "
$host_bridge_libvirt_name
<forward mode="bridge"/>
<bridge name="${host_bridge_ifacename}"/>
" > $host_bridge_config_filename

#we typically dont need to do this. the libvirt group is automatically created and the user using this library has already been added to it. But this is just excess precaution. If the user is already in the group, no harm is done. It just prints some information. No checking is necessary about the result.

sudo adduser $(whoami) libvirt

#if the bridge device was used earlier, delete it from libvirt configuration it and redo it.

virsh net-destroy $host_bridge_libvirt_name
virsh net-undefine $host_bridge_libvirt_name

#now define this host bridge in the kvm configuration, start it and set autostart on this host bridge.
virsh net-define $host_bridge_config_filename #the file is used to introduce the name of the libvirt connection name. the libvirt name is used in further commands. The libvirt name is included in the config file.
[[ $1 =~ debug ]] && read -p "check if the host bridge has been inserted libvirt by trying virsh net-list"
virsh net-start $host_bridge_libvirt_name
virsh net-autostart $host_bridge_libvirt_name

#Make it possible for hosts outside of KVM to talk to your bridged guest by making the following changes on the KVM host. Load the br_netfilter module:

sudo modprobe br_netfilter

#pause here to check that our attempt has worked and a proper bridge device has been created

echo check if the bridge device has been created : virsh net-list in a separate terminal window
read -p "hit enter to continue. Or ctrl+c to break the execution."

#Persist on reboot by creating /etc/modules-load.d/br_netfilter.conf:

echo "br_netfilter" |sudo tee /etc/modules-load.d/br_netfilter.conf

#Create /etc/sysctl.d/10-bridge.conf:

Do not filter packets crossing a bridge

printf "net.bridge.bridge-nf-call-ip6tables=0 \n"
"net.bridge.bridge-nf-call-iptables=0 \n"
"net.bridge.bridge-nf-call-arptables=0 \n" | sudo tee /etc/sysctl.d/10-bridge.conf

#Apply the config now:
sudo sysctl -p /etc/sysctl.d/10-bridge.conf

#Check result:

sudo sysctl -a | grep "bridge-nf-call"

#Configure the guest to use host-bridge. Open up the Virtual Machine Manager and then select the target guest. Go to the NIC device. The drop down for "Network Source" should now include a device called "Virtual netowrk 'host-bridge'". The "Bridge network device model" will be "virtio" if that's your KVM configuration's default. Select that "host-bridge" device.

#If you inspect the guest's XML (by using virsh dumplxml guestname), it shoud look something like this:

#

#'