Using tcp_v6_connect to know when a curl with tcp-v6 was executed on k8s

eBPF Kprobes.md

eBPF Kprobes

The goal was to receive eBPF kprobes using ebpf2go.

This will print the following traces when a tcp-v6 packet will arrive or exit:

<...>-70775   [002] d...1 14216.275232: bpf_trace_printk: sys kprobe/tcp_v6_connect exit - 70775
<...>-70775   [002] d...1 14216.275249: bpf_trace_printk: sys kprobe/tcp_v6_connect enter - 70775
<...>-70894   [003] d...1 14231.718191: bpf_trace_printk: sys kprobe/tcp_v6_connect exit - 70894
<...>-70894   [003] d...1 14231.718199: bpf_trace_printk: sys kprobe/tcp_v6_connect enter - 70894
<...>-74347   [009] d...1 14371.919322: bpf_trace_printk: sys kprobe/tcp_v6_connect exit - 74347
<...>-74347   [009] d...1 14371.919332: bpf_trace_printk: sys kprobe/tcp_v6_connect enter - 74347
<...>-74370   [000] d...1 14373.224750: bpf_trace_printk: sys kprobe/tcp_v6_connect exit - 74370
<...>-74370   [000] d...1 14373.224766: bpf_trace_printk: sys kprobe/tcp_v6_connect enter - 74370
<...>-74386   [003] d...1 14374.201845: bpf_trace_printk: sys kprobe/tcp_v6_connect exit - 74386
<...>-74386   [003] d...1 14374.201853: bpf_trace_printk: sys kprobe/tcp_v6_connect enter - 74386
<...>-74405   [002] d...1 14375.136027: bpf_trace_printk: sys kprobe/tcp_v6_connect exit - 74405
<...>-74405   [002] d...1 14375.136032: bpf_trace_printk: sys kprobe/tcp_v6_connect enter - 74405
<...>-74547   [011] d...1 14376.634997: bpf_trace_printk: sys kprobe/tcp_v6_connect exit - 74547
<...>-74547   [011] d...1 14376.635012: bpf_trace_printk: sys kprobe/tcp_v6_connect enter - 74547

This can be recrieted using the following terminal command:

kubectl exec --tty rabbitmq-0 -n rabbitmq -- curl -g -6 google.com

It will work even if the curl pod is located on a different kubernetes namespace.

Running the eBPF script on the Kernel

By executing go generate && go build && sudo ./ebpf-xdp for a couple of seconds and then using ctrl+c we wrote some traces into /sys/kernel/debug/tracing/trace_pipe.

They can be viewed by:

sudo cat /sys/kernel/debug/tracing/trace_pipe

Enabling Tracing

If your Linux distribution (e.g. Ubuntu) does not have the tracing subsystem enabled by default, you may not see any output. Use the following command to enable this feature:

• sudo mount -t debugfs none /sys/kernel/debug
• sudo echo 1 > /sys/kernel/debug/tracing/tracing_on

kprobe.c

//go:build ignore

#include <linux/sched.h>
#include<bpf/bpf_helpers.h>
#include <bpf/bpf_tracing.h>

char __license[] SEC("license") = "Dual MIT/GPL";

// Tried krobes: tcp_v4_rcv, tcp_sendmsg
SEC("kprobe/tcp_v6_connect")
int tcp_recvmsg(struct pt_regs* ctx) {
    __u64 pid_tgid = bpf_get_current_pid_tgid();

    if (pid_tgid > 0)
        bpf_printk("sys kprobe/tcp_v6_connect enter - %d", pid_tgid);

    return 0; }

SEC("kretprobe/tcp_v6_connect")
int tcp_recvmsg_ret(struct pt_regs* ctx) {
    __u64 pid_tgid = bpf_get_current_pid_tgid();

    if (pid_tgid > 0)
        bpf_printk("sys kprobe/tcp_v6_connect exit - %d", pid_tgid);

    return 0;
}

main.go

// This program demonstrates attaching an eBPF program to a kernel symbol.
// The eBPF program will be attached to the start of the sys_execve
// kernel function and prints out the number of times it has been called
// every second.
package main

import (
	"log"

	"github.com/cilium/ebpf/link"
	"github.com/cilium/ebpf/rlimit"
)

//go:generate go run github.com/cilium/ebpf/cmd/bpf2go bpf kprobe.c -- -I./headers -I/usr/include/aarch64-linux-gnu

func main() {
	// Allow the current process to lock memory for eBPF resources.
	if err := rlimit.RemoveMemlock(); err != nil {
		log.Fatal(err)
	}

	// Load pre-compiled programs and maps into the kernel.
	objs := bpfObjects{}
	if err := loadBpfObjects(&objs, nil); err != nil {
		log.Fatalf("loading objects: %v", err)
	}
	defer objs.Close()

	// Open a Kprobe at the entry point of the kernel function and attach the
	// pre-compiled program. Each time the kernel function enters, the program
	// will increment the execution counter by 1. The read loop below polls this
	// map value once per second.
	kpTcpV4Rcv, err := link.Kprobe("tcp_v6_connect", objs.TcpRecvmsg, nil)
	if err != nil {
		log.Fatalf("opening kprobe: %s", err)
	}
	defer kpTcpV4Rcv.Close()

	kpTcpV4RcvRet, err := link.Kprobe("tcp_v6_connect", objs.TcpRecvmsgRet, nil)
	if err != nil {
		log.Fatalf("opening kprobe: %s", err)
	}
	defer kpTcpV4RcvRet.Close()

	log.Println("Waiting for events..")
	for {
	}
}