Skip to content

Instantly share code, notes, and snippets.

@dotSlashLu
Last active August 26, 2021 16:52
Show Gist options
  • Save dotSlashLu/97acdcf49a9641606ee00b0352f1ec24 to your computer and use it in GitHub Desktop.
Save dotSlashLu/97acdcf49a9641606ee00b0352f1ec24 to your computer and use it in GitHub Desktop.
Prepare, install Open vSwitch and DPDK rpms, and configure OVS
# Install ovs-dpdk
# Copyright Dotslash.Lu <dotslash.lu@gmail.com>
#
# NOTE:
# Please add `iommu=pt intel_iommu=on` to grub bootline and
# reboot
#
# Installation will cause NETWORK LOST, so it's better to
# execute this script within an admin console
#
# check IOMMU
echo "Checking boot command line"
if ! cat /proc/cmdline | grep intel_iommu; then
echo "Failed, please add \`iommu=pt intel_iommu=on\` to" \
"boot command line and reboot"
exit
fi
# cleaning
yum install -y pciutils kernel-devel
yum erase -y openvswitch openvswitch-devel openvswitch-kmod \
openvswitch-debuginfo openvswitch-kmod-debuginfo
rm -rf /etc/openvswitch /var/run/openvswitch /usr/share/openvswitch
# install rpms
echo
echo "Installing"
tar -xzf ovs-dpdk_rpms.tgz
yum localinstall -y *.rpm
# insert kmod
echo
echo "Inserting kmods"
modprobe vfio-pci
modprobe openvswitch
echo "vfio-pci" > /etc/modules-load.d/vfio-pci.conf
echo "openvswitch" > /etc/modules-load.d/openvswitch.conf
# bind NIC to vfio-pci
echo
echo "Binding NICs"
chmod a+x /dev/vfio
chmod 0666 /dev/vfio/*
nics=$(driverctl -v list-devices | grep -i net | gawk '{print $1}')
if [-z $nics]; then
echo "Can't get NICs to bind"
exit
fi
for nic in $nics; do
echo "Binding NIC ${nic}"
driverctl set-override ${nic} vfio-pci
done
echo
echo "Allocating hugepages"
# allocate 1024 2M hugepages
sysctl -w vm.nr_hugepages=1024
# make hugepage setting persistent
echo "vm.nr_hugepages=1024" > /etc/sysctl.d/hugepages.conf
# mount hugepages
mount -t hugetlbfs none /dev/hugepages
echo
echo "Configuring ovs"
systemctl start openvswitch
# enable dpdk
ovs-vsctl --no-wait set Open_vSwitch . other_config:dpdk-init=true
# pin pmd threads to cores 1, 2
ovs-vsctl --no-wait set Open_vSwitch . other_config:pmd-cpu-mask=6
sleep 2
systemctl restart openvswitch
echo
echo "Adding ports"
ovs-vsctl add-br ovsbr0 -- set bridge ovsbr0 datapath_type=netdev
ovs-vsctl add-bond ovsbr0 dpdkbond dpdk0 dpdk1 \
-- set Interface dpdk0 type=dpdk \
-- set Interface dpdk1 type=dpdk
ovs-vsctl add-br ovsbr1 -- set bridge ovsbr1 datapath_type=netdev
ovs-vsctl add-bond ovsbr1 dpdkbond1 dpdk2 dpdk3 \
-- set Interface dpdk2 type=dpdk \
-- set Interface dpdk3 type=dpdk
echo
ovs-vsctl show
@wittling
Copy link

The file you use in this script, ovs-dpdk_rpms.tgz, can you list the rpms that are contained in that?

@dotSlashLu
Copy link
Author

The file you use in this script, ovs-dpdk_rpms.tgz, can you list the rpms that are contained in that?

Hey wittling, this script is written years ago, I can't remember the details, sorry. But I think these rpms are built from ovs source and packaged together for easy distribution.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment