Skip to content

Instantly share code, notes, and snippets.

@Dieterbe
Last active January 11, 2021 11:02
Show Gist options
  • Save Dieterbe/a52c95a9603507670eb39274544ee1a8 to your computer and use it in GitHub Desktop.
Save Dieterbe/a52c95a9603507670eb39274544ee1a8 to your computer and use it in GitHub Desktop.
I use this script to set up my system for benchmarking. read it carefully, the 3 first steps are instructions to change your bootloader configuration.
#!/bin/bash
# useful articles:
# https://vstinner.github.io/journey-to-stable-benchmark-system.html
# http://perf.readthedocs.io/en/latest/system.html
# https://baiweiblog.wordpress.com/2017/11/02/how-to-set-processor-affinity-in-linux-using-taskset/
# https://wiki.archlinux.org/index.php/CPU_frequency_scaling#CPU_frequency_driver
if [[ $EUID > 0 ]]; then
echo "Please run as root"
exit 1
fi
#### 1) isolation & affinity
# isolate core 1,2,3 (using kernel boot argument: isolcpus=1,2,3,5,6,7) which we will use explicitly for stuff under test
# (note affinities are inherited by child processes)
# leaving core 0 (cpu 0 and 4) for interactive stuff
# see lscpu --extended and /proc/cpuinfo for the mappings
# or: oldlstopo-no-graphics --no-io --no-legend --of txt (hwloc package)
# on my system each real core has 2 fake cores due to hyperthreading
# see also https://www.linuxtopia.org/online_books/linux_kernel/kernel_configuration/re46.html
# we can confirm this works because:
# 1) with only desktop apps etc running, only cpu0 and cpu4 are used in top, all others are 100% idle
# 2) running: seq 8 | xargs -P0 -n1 md5sum /dev/zero
# runs 8 processes at ~25% cpu each, and cpu0 and cpu4 are maxed out in top, all others are 100% idle
# 3) to run on isolated core (for example):
# taskset --cpu-list 5 shasum /dev/zero
# taskset --cpu-list 7 shasum /dev/zero
# (again, top confirms)
echo "isolated cores:"
cat /sys/devices/system/cpu/isolated
#### 2) significantly reduce tick interruptions
# kernel boot arg: nohz_full=1,2,3,5,6,7
# see https://lwn.net/Articles/659490/ and https://lwn.net/Articles/549580/
# confirm with cat /sys/devices/system/cpu/nohz_full
# zcat /proc/config.gz | r HZ
zgrep CONFIG_NO_HZ_FULL /proc/config.gz
echo "nohz_full cores:"
cat /sys/devices/system/cpu/nohz_full
#### 3) fix cpu frequency scaling
#
# use kernel boot arg intel_pstate=disable
# needed to make changing of governors take effect (see watch grep \"cpu MHz\" /proc/cpuinfo )
# performance -> 3.2GHz, powersave 800MHz
# with pstate driver, doesn't work
# according to https://vstinner.github.io/intel-cpus.html there's more concerns with intel cpu's like turbo mode etc
# but i think that doesn't apply if we disable intel_pstate :?
# also, fun fact from https://vstinner.github.io/intel-cpus-part2.html "According to an Intel engineer, the intel_pstate driver was never tested with CPU isolation."
#### 4) disable intel boost
#
# not sure if this is needed considering we disabled the intel pstate driver, but doesn't hurt
echo 0 > /sys/devices/system/cpu/cpufreq/boost
# for the record, if we were using pstate, it would be /sys/devices/system/cpu/intel_pstate/no_turbo
# see also https://www.kernel.org/doc/html/v4.14/admin-guide/pm/cpufreq.html#the-boost-file-in-sysfs
#### 5) turbo. we don't have to worry about this anymore because we don't use pstate
# echo 1 > /sys/devices/system/cpu/intel_pstate/no_turbo
#### 6) make sure to use consistent cpu frequency.
#
# governors:
# * performance may overheat the core - though this is speculation i have no idea - and I don't want to damage my CPU, or trigger
# any safety measure kicking in and slowing down the cpu during a benchmark
# * I tried powersave but it's 800MHz and feels slow
# * userspace lets us set whatever we want. https://www.kernel.org/doc/Documentation/cpu-freq/governors.txt
# so that's what we choose below
for cpu in /sys/devices/system/cpu/cpu[0-9]*; do
echo "CPU $cpu"
govern=$cpu/cpufreq/scaling_governor
old=$(cat $govern)
echo userspace > $govern
new=$(cat $govern)
echo "governor: $old -> $new"
speed=$cpu/cpufreq/cpuinfo_cur_freq
old=$(cat $speed)
echo 2000000 > $cpu/cpufreq/scaling_setspeed
new=$(cat $speed)
echo "speed: $old -> $new"
done
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment