term_logs.md

Logs

Compile and Install Linux Kernel

Install dependencies

apt install flex bison bc libssl-dev libelf-dev libncurses-dev

Download the latest stable kernel

wget https://cdn.kernel.org/pub/linux/kernel/v4.x/linux-4.16.11.tar.xz
tar -xvf linux-4.16.11.tar.xz
cd linux-4.16.11/

Update current config utilising a provided .config as base and set new symbols to their default value without prompting

make olddefconfig

Update current config utilising a menu based program

make menuconfig

Compressed kernel image (arch/x86/boot/bzImage)

make bzImage

Build all modules

make modules

Install all modules to INSTALL_MOD_PATH (default: /)

sudo make modules_install

Install kernel

sudo make install

You don't have to run sudo update-grub since it's automatically updated.

Compile and Install Netmap

git clone https://github.com/luigirizzo/netmap
cd netmap
./configure --no-ext-drivers --drivers=ixgbe # use intel ixgbe driver
make

sudo insmod netmap.ko # install netmap.ko module
sudo rmmod ixgbe # remove default ixgbe module
sudo insmod ixgbe/ixgbe.ko # install netmap ixgbe module

Bring your interfaces up

sudo ip link set ens1f0 up
sudo ip link set ens1f1 up
sudo ip link set ens1f1 promisc on # set the receiver interface to promiscuous mode

Run the test

sudo ./build-apps/pkt-gen/pkt-gen -f rx -i ens1f1 # receiver side
sudo ./build-apps/pkt-gen/pkt-gen -f tx -i ens1f0 # sender side

Stop irqbalance service

Irqbalance is a daemon to help balance the cpu load generated by interrupts across all of a systems cpus. Stop irqbalance service to disable automatically cpu load balancing.

sudo service irqbalance stop

CPU frequency scaling

sudo service ondemand stop # disable ondemand CPU Frequency Scaling governor
sudo apt-get install cpufrequtils # install cpufrequtils
cpufreq-info # check current cpu frequency
sudo bash -c 'for i in {0..7}; do cpufreq-set -c $i -g performance; done'
cat /sys/devices/system/cpu/cpu0/cpufreq/scaling_governor # cpu governor should be performance now

Change NIC queue size

Queue size 1

sudo ethtool -L ens1f0 combined 1
sudo ethtool -L ens1f1 combined 1

sudo taskset -c 1 ./build-apps/pkt-gen/pkt-gen -f rx -i ens1f1 -c 1 -p 1 # receiver side
sudo taskset -c 0 ./build-apps/pkt-gen/pkt-gen -f tx -i ens1f0 -c 1 -p 1 # sender side

Queue size 2

sudo ethtool -L ens1f0 combined 2
sudo ethtool -L ens1f1 combined 2

sudo taskset -c 2-3 ./build-apps/pkt-gen/pkt-gen -f rx -i ens1f1 -c 2 -p 2 # receiver side
sudo taskset -c 0-1 ./build-apps/pkt-gen/pkt-gen -o 512 -o 1024 -f tx -i ens1f0 -c 2 -p 2 # sender side

Queue size 3

sudo ethtool -L ens1f0 combined 3
sudo ethtool -L ens1f1 combined 3

sudo taskset -c 3-5 ./build-apps/pkt-gen/pkt-gen -f rx -i ens1f1 -c 3 -p 3 # receiver side
sudo taskset -c 0-2 ./build-apps/pkt-gen/pkt-gen -o 512 -o 1024 -f tx -i ens1f0 -c 3 -p 3 # sender side

Queue size 4

sudo ethtool -L ens1f0 combined 4
sudo ethtool -L ens1f1 combined 4

sudo taskset -c 4-7 ./build-apps/pkt-gen/pkt-gen -f rx -i ens1f1 -c 4 -p 4 # receiver side
sudo taskset -c 0-3 ./build-apps/pkt-gen/pkt-gen -o 512 -o 1024 -f tx -i ens1f0 -c 4 -p 4 # sender side

Queue size 5

sudo ethtool -L ens1f0 combined 5
sudo ethtool -L ens1f1 combined 5
sudo taskset -c 5-9 ./build-apps/pkt-gen/pkt-gen -f rx -i ens1f1 -c 5 -p 5 # receiver side
sudo taskset -c 0-4 ./build-apps/pkt-gen/pkt-gen -o 512 -o 1024 -f tx -i ens1f0 -c 5 -p 5 # sender side