Why these tools? Why eBPF?
Install:
apt install linux-tools-aws linux-tools-5.4.0-1045-aws
Try:
perf top
It's bytecode that gets run in a VM inside the Kernel.
You write BPF programs, the Kernel runs them and you can get data out of it (via ring buffers).
They're small and very fast.
The newer the Kernel, the more BPF features we've got access to, but, in general, allows tracing almost anything within the Kernel.
But not only tracing – it can also modify behavior! It can act as a driver, can replace e.g. iptables rules, firewalls, etc.
Bytecode gets compiled with LLVM.
Install:
apt install bpfcc-tools
Try:
execsnoop-bpfcc -T
opensnoop-bpfcc -T
tcpaccept-bpfcc -T
...
| Tool | Shows | Visualization |
|---|---|---|
| execsnoop | New processes (via exec(2)) | table |
| opensnoop | Files opened | table |
| ext4slower | Slow filesystem I/O | table |
| biolatency | Disk I/O latency histogram | heat map |
| biosnoop | Disk I/O per-event details | table, offset heat map |
| cachestat | File system cache statistics | line charts |
| tcplife | TCP connections | table, distributed graph |
| tcpretrans | TCP retransmissions | table |
| runqlat | CPU scheduler latency | heat map |
| profile | CPU stack trace samples | flame graph |
Two new technologies, BTF and CO-RE, are paving the way for BPF to become a billion-dollar industry. Right now there are many BPF (eBPF) startups building networking, security, and performance products (and more in stealth), yet requiring customers to install the LLVM, Clang, and kernel header dependencies – which can consume over 100 Mbytes of storage – is an adoption drag. BTF and CO-RE eliminate these dependencies at runtime, not only making BPF more practical for embedded Linux environments, but for adoption everywhere
These technologies are:
- BTF: BPF Type Format, which provides struct information to avoid needing Clang and kernel headers.
- CO-RE: BPF Compile-Once Run-Everywhere, which allows compiled BPF bytecode to be relocatable, avoiding the need for recompilation by LLVM.