Skip to content

Instantly share code, notes, and snippets.

@joestringer
Last active June 20, 2019 20:58
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save joestringer/d5313e4bf4231e2c46405bd7a3053936 to your computer and use it in GitHub Desktop.
Save joestringer/d5313e4bf4231e2c46405bd7a3053936 to your computer and use it in GitHub Desktop.
Kernel stack trace issue with skb_orphan() removal
[ 1474.586748] ------------[ cut here ]------------
[ 1474.586773] refcount_t hit zero at sk_stop_timer+0x2c/0x30 in cilium-agent[16359], uid/euid: 0/0
[ 1474.586785] WARNING: CPU: 0 PID: 16359 at kernel/panic.c:686 refcount_error_report+0x9c/0xa1
[ 1474.586787] Modules linked in: xt_TPROXY nf_tproxy_ipv6 nf_tproxy_ipv4 xt_comment xt_CT xt_mark ip6table_raw ip6table_mangle ip6_tables iptable_raw iptable_mangle cls_bpf algif_hash af_alg sch_ingress vxlan ip6_udp_tunnel udp_tunnel nfsv3 nfs_acl nfs lockd grace fscache veth cpuid xt_nat xt_tcpudp xt_MASQUERADE nf_conntrack_netlink nfnetlink xfrm_user xfrm_alg
o iptable_nat xt_addrtype iptable_filter xt_conntrack nf_nat nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 br_netfilter bridge stp llc overlay intel_rapl_perf input_leds serio_raw mac_hid sch_fq_codel binfmt_misc ib_iser rdma_cm iw_cm ib_cm ib_core iscsi_tcp libiscsi_tcp libiscsi scsi_transport_iscsi sunrpc ip_tables x_tables autofs4 btrfs zstd_compress raid10 raid4
56 async_raid6_recov async_memcpy async_pq async_xor async_tx xor raid6_pq libcrc32c raid1 raid0 multipath linear crct10dif_pclmul crc32_pclmul ghash_clmulni_intel aesni_intel aes_x86_64 crypto_simd cryptd glue_helper psmouse vboxvideo ttm ahci drm_kms_helper libahci
[ 1474.586926] syscopyarea sysfillrect sysimgblt i2c_piix4 fb_sys_fops drm e1000 pata_acpi video
[ 1474.586947] CPU: 0 PID: 16359 Comm: cilium-agent Tainted: G W 5.2.0-rc2-remove-orphan-call-v0.1 #9
[ 1474.586951] Hardware name: innotek GmbH VirtualBox/VirtualBox, BIOS VirtualBox 12/01/2006
[ 1474.586959] RIP: 0010:refcount_error_report+0x9c/0xa1
[ 1474.586964] Code: 45 8b 85 a8 08 00 00 49 8b 94 24 80 00 00 00 49 8d 8d 50 0a 00 00 41 56 41 89 c1 48 89 de 48 c7 c7 c0 4e ce 9d e8 16 00 00 00 <0f> 0b 58 eb 8c 48 c7 c7 60 4d ce 9d e8 c6 94 06 00 e9 6f fa ff ff
[ 1474.586967] RSP: 0018:ffffb3b301a23ad8 EFLAGS: 00010286
[ 1474.586972] RAX: 0000000000000000 RBX: ffffffff9dce1db3 RCX: 0000000000000006
[ 1474.586974] RDX: 0000000000000007 RSI: 0000000000000086 RDI: ffff96769ba17380
[ 1474.586977] RBP: ffffb3b301a23b00 R08: 0000000000000e0e R09: 0000000000000004
[ 1474.586979] R10: 0000000000000000 R11: 0000000000000001 R12: ffffb3b301a23c08
[ 1474.586981] R13: ffff967694485c00 R14: 0000000000000000 R15: ffffb3b301a23c08
[ 1474.586986] FS: 00007fffee9c5700(0000) GS:ffff96769ba00000(0000) knlGS:0000000000000000
[ 1474.586990] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 1474.586993] CR2: 000000c420455000 CR3: 0000000115794001 CR4: 00000000000606f0
[ 1474.587003] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 1474.587006] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
[ 1474.587008] Call Trace:
[ 1474.587022] ex_handler_refcount+0x52/0x80
[ 1474.587030] fixup_exception+0x4a/0x61
[ 1474.587045] do_trap+0x4b/0xe0
[ 1474.587052] do_error_trap+0x7c/0xc0
[ 1474.587059] ? csum_partial_copy_generic+0x210f/0x3520
[ 1474.587064] do_invalid_op+0x3b/0x50
[ 1474.587068] ? csum_partial_copy_generic+0x210f/0x3520
[ 1474.587073] invalid_op+0x14/0x20
[ 1474.587080] RIP: 0010:sk_stop_timer+0x2c/0x30
[ 1474.587084] Code: 44 00 00 55 48 89 e5 53 48 89 fb 48 89 f7 e8 cb a2 87 ff 85 c0 74 13 f0 ff 8b 80 00 00 00 0f 84 e2 16 1a 00 0f 88 dc 16 1a 00 <5b> 5d c3 90 0f 1f 44 00 00 55 48 89 e5 53 48 89 fb 48 8b bf 18 01
[ 1474.587087] RSP: 0018:ffffb3b301a23cb0 EFLAGS: 00010246
[ 1474.587091] RAX: 0000000000000001 RBX: ffff967694723480 RCX: ffff967694723500
[ 1474.587093] RDX: ffff96769ba1b990 RSI: 0000000000000246 RDI: 0000000000000246
[ 1474.587096] RBP: ffffb3b301a23cb8 R08: ffff96769ba1b990 R09: ffffffff9d33aed6
[ 1474.587098] R10: fffff85041128600 R11: 000000000000dbd0 R12: 0000000000000000
[ 1474.587101] R13: ffff967692e4bb00 R14: ffff9676963d8624 R15: 0000000000000000
[ 1474.587108] ? inet_put_port+0xa6/0xd0
[ 1474.587115] inet_csk_clear_xmit_timers+0x2e/0x50
[ 1474.587120] tcp_done+0x8b/0xf0
[ 1474.587124] tcp_reset+0x49/0xc0
[ 1474.587129] tcp_validate_incoming+0x2f7/0x410
[ 1474.587133] tcp_rcv_state_process+0x250/0xdb6
[ 1474.587139] ? tcp_v4_connect+0x46f/0x4e0
[ 1474.587146] tcp_v4_do_rcv+0xbd/0x1f0
[ 1474.587152] __release_sock+0x84/0xd0
[ 1474.587157] release_sock+0x30/0xa0
[ 1474.587164] inet_stream_connect+0x47/0x60
[ 1474.587169] __sys_connect+0xed/0x120
[ 1474.587178] ? apparmor_socket_setsockopt+0x1e/0x20
[ 1474.587184] ? fput+0x13/0x20
[ 1474.587191] ? __sys_setsockopt+0xba/0xf0
[ 1474.587195] __x64_sys_connect+0x1a/0x20
[ 1474.587202] do_syscall_64+0x5a/0x140
[ 1474.587207] entry_SYSCALL_64_after_hwframe+0x44/0xa9
[ 1474.587212] RIP: 0033:0x1159cf0
[ 1474.587217] Code: 8b 7c 24 10 48 8b 74 24 18 48 8b 54 24 20 49 c7 c2 00 00 00 00 49 c7 c0 00 00 00 00 49 c7 c1 00 00 00 00 48 8b 44 24 08 0f 05 <48> 3d 01 f0 ff ff 76 20 48 c7 44 24 28 ff ff ff ff 48 c7 44 24 30
[ 1474.587219] RSP: 002b:000000c0006832f0 EFLAGS: 00000206 ORIG_RAX: 000000000000002a
[ 1474.587223] RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 0000000001159cf0
[ 1474.587226] RDX: 0000000000000010 RSI: 000000c00053d8ac RDI: 000000000000001f
[ 1474.587228] RBP: 000000c000683338 R08: 0000000000000000 R09: 0000000000000000
[ 1474.587230] R10: 0000000000000000 R11: 0000000000000206 R12: ffffffffffffffff
[ 1474.587233] R13: 00000000000000c6 R14: 00000000000000c5 R15: 0000000000000100
[ 1474.587238] ---[ end trace b0668b91a13f5f14 ]---
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment