Skip to content

Instantly share code, notes, and snippets.

@jschwe

jschwe/2021-03-11.png Secret

Last active March 21, 2021 16:59
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save jschwe/d4c46026aec57b10a2b0e6f72258b96e to your computer and use it in GitHub Desktop.
Save jschwe/d4c46026aec57b10a2b0e6f72258b96e to your computer and use it in GitHub Desktop.
Download ZIP
Cyclictest --mainaffinity effects
Raw
Readme.md

Testing proposed cyclictest --mainaffinity option

System information:

  • 2 socket Intel Xeon E5-2643 v4 @ 3.40Ghz
  • Turbo boost is disabled.
  • Fedora 33 with kernel 5.10.1-rt20 with small patch (see cmdline isolcpus)
  • Cmdline: nosmt isolcpus=domain,managed_irq,wq,rcu,misc,kthread,3,5,7,9,11 rcu_nocbs=3,5,7,9,11 irqaffinity=0,2,4 maxcpus=12 rcu_nocb_poll nowatchdog tsc=nowatchdog processor.max_cstate=1 intel_idle.max_cstate=0 systemd.unified_cgroup_hierarchy=0
    • The additional isolcpus arguments set the HK_FLAG with the corresponding name. This cmdline adds all HK_FLAGs usually set by nohz_full, except the actual nohz flags tick and timer. This improves cyclictest latencies on my system.
  • Rteval is running on all CPUs from node 0 + CPU 1, but not on the isolated CPUs.
  • L3 Cache is reserved for the isolated CPUs via resctrl (CPU based allocation)
  • Test duration 24 hours, interval 200 µs

Test 1: 5 cyclictest instance with main pid on same cpu as the measurement thread

This test simply starts 5 cyclictest instances (via numactl) with one measurement thread each and bound to a single CPU via --affinity, so that the main thread is also bound to the same CPU.

Figure: 5 cyclictest instances with main pid pinned to same CPU as measurement thread

Test 2: Single cyclictest instance with --mainaffinity=1 for isolated CPUs 3,5,7,9,11

The main thread was placed on CPU 1 via --mainaffinity and --refresh_on_max was added for good measure to keep the logfile small.

Figure: Single cyclictest instance with --mainaffinity=1 for isolated CPUs 3,5,7,9,11

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment