Skip to content

Instantly share code, notes, and snippets.

@mika

mika/BTF_Debian.rst Secret

Last active Feb 3, 2021
Embed
What would you like to do?
BTF support in Debian/bullseye

See http://www.brendangregg.com/blog/2020-11-04/bpf-co-re-btf-libbpf.html for the underlying rational.

Request for packaging libbpf-tools in Debian is reported as https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=978727.

System environment for building:

  • Debian bullseye (current Debian/testing, upcoming stable release) [or Debian unstable]
  • >=2GB RAM

System environment for running the resulting libbpf-tools binaries:

Kernel package linux-image-amd64 >=5.10.5-1 is needed. Currently (as of 2021-01-12) this is available for a bullseye based environment from Debian/unstable via:

% echo 'deb http://deb.debian.org/debian unstable main' | sudo tee -a /etc/apt/sources.list.d/unstable.list
% echo ' http://deb.debian.org/debian unstable main' | sudo tee -a /etc/apt/sources.list.d/unstable.list
% sudo apt update
% sudo apt install linux-image-amd64=5.10.5-1

Verify that it is enabled in your running kernel via:

% grep CONFIG_DEBUG_INFO_BTF /boot/config-$(uname -r)
CONFIG_DEBUG_INFO_BTF=y

Install build dependencies:

% sudo apt build-dep bpfcc  # this requires a corresponding deb-src entry in apt's sources.list config
% sudo apt install git clang

Build:

% git clone https://github.com/iovisor/bcc/
% mkdir bcc/build
% cd bcc/build
% cmake -DPYTHON_CMD=python3 ..
% make
% cd ../libbpf-tools
% make

If you want to build a quick'n'dirty Debian package via fpm, ensure fpm is available:

% sudo apt install bundler
% bundle init
% echo 'gem "fpm"' >> Gemfile
% bundle config set --local path 'vendor/bundle'
% bundler install
% bundle binstubs --all
% sed -i '/require "digest"/a require "zlib"'  ./vendor/bundle/ruby/2.7.0/gems/fpm-1.11.0/lib/fpm/package/deb.rb  # workaround for https://github.com/jordansissel/fpm/issues/1739

Then generate a Debian package for the libbpf-tools binaries using fpm (NOTE: this relies on zsh globbing, otherwise list the according binaries instead of *(x.)):

% cp bin/bpftool ./  # to include bpftool also in the Debian package generated by fpm
% strip *(x.)        # to reduce binary size of tools from ~1.2MB to <250KB, the bpftool binary will be 2.3MB instead of 2.4MB
% ./bin/fpm -s dir -t deb -n libbpf-tools -v $(git describe | sed 's/^v//') --prefix /usr/sbin/ *(x.)

The resulting Debian package (like libbpf-tools_0.11.0-455-g1cb5026e_amd64.deb) should provide all the relevant tools for usage on a Debian system, without having to install lots of runtime dependencies.

Known issues:

% sudo readahead
libbpf: failed to find kernel BTF type ID of '__do_page_cache_readahead': -3
libbpf: failed to load object 'readahead_bpf'
libbpf: failed to load BPF skeleton 'readahead_bpf': -3
failed to open and/or load BPF ojbect

% sudo llcstat
failed to init perf sampling: No such file or directory
% sudo strace -f -s500 llcstat 2>&1 | tail
munmap(0x7f17e10f2000, 360448)          = 0
perf_event_open({type=PERF_TYPE_HARDWARE, size=0 /* PERF_ATTR_SIZE_??? */, config=PERF_COUNT_HW_CACHE_MISSES, ...}, -1, 0, -1, 0) = -1 ENOENT (No such file or directory)
write(2, "failed to init perf sampling: No such file or directory\n", 56failed to init perf sampling: No such file or directory) = 56
[...]

FTR, iff BTF support is missing in your running kernel, execution of binaries will fail with:

% sudo ./biolatency 1 10
libbpf: failed to find valid kernel BTF
libbpf: Error loading vmlinux BTF: -3
libbpf: failed to load object 'biolatency_bpf'
libbpf: failed to load BPF skeleton 'biolatency_bpf': -3
failed to load BPF object: -3
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment