Skip to content

Instantly share code, notes, and snippets.

@trozet

trozet/geneve_ofproto.txt

Created August 18, 2022 14:51
Show Gist options
  • Save trozet/343abe2721e59b1980c8544ec684bc02 to your computer and use it in GitHub Desktop.
Save trozet/343abe2721e59b1980c8544ec684bc02 to your computer and use it in GitHub Desktop.
Download ZIP
ofproto trace geneve ovn
Raw
geneve_ofproto.txt
So, for the trace, I have:
# oc get pod -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
pod1 1/1 Running 0 34s 10.244.0.6 ovn-control-plane <none> <none>
pod2 1/1 Running 0 10s 10.244.2.3 ovn-worker2 <none> <none>
And:
# oc get pods -o wide -n ovn-kubernetes | grep ovn-worker2 | grep ovnkube
ovnkube-node-9pvtz 3/3 Running 0 23m 10.89.0.3 ovn-worker2 <none> <none>
Get the source node IP:
# oc get node -o wide | grep control-plane
ovn-control-plane Ready control-plane 29m v1.24.0 10.89.0.4 <none> Ubuntu 21.10 5.17.12-100.fc34.x86_64 containerd://1.6.4
Get the OVS tunnel interface's port number on the destination node:
# oc exec -n ovn-kubernetes ovnkube-node-9pvtz -- ovs-vsctl find interface options:remote_ip="10.89.0.4" | grep ofport
Defaulted container "ovnkube-node" out of: ovnkube-node, ovn-controller, ovs-metrics-exporter
ofport : 1
ofport_request : []
Get the destination logical switch SB datapath tunnel key:
# oc exec -n ovn-kubernetes ovnkube-db-5c6757846-m6f74 -- ovn-sbctl list datapath ovn-worker2 | grep tunnel_key
Defaulted container "nb-ovsdb" out of: nb-ovsdb, sb-ovsdb
tunnel_key : 4
Get the destination logical switch SB port binding towards cluster router key:
# oc exec -n ovn-kubernetes ovnkube-db-5c6757846-m6f74 -- ovn-sbctl list port_b stor-ovn-worker2 | grep tunnel
Defaulted container "nb-ovsdb" out of: nb-ovsdb, sb-ovsdb
tunnel_key : 1
Get the SB port binding of the destination pod:
# oc exec -n ovn-kubernetes ovnkube-db-5c6757846-m6f74 -- ovn-sbctl list port_b default_pod2 | grep tunnel
Defaulted container "nb-ovsdb" out of: nb-ovsdb, sb-ovsdb
tunnel_key : 3
REG14 == logical input port (in this case stor-ovn-worker2)
REG15 == logical output port (in this case default_pod2)
Summarizing:
# sb_datapath=4
# sb_inport=1
# sb_outport=3
Build the tun metadata field, sb_inport shifted 16 bits or-ed with sb_outport:
# tun_metadata=$((($sb_inport << 16) | $sb_outport))
Run the trace (smac is the router port mac, SIP is pod1's IP):
# oc exec -n ovn-kubernetes ovnkube-db-5c6757846-m6f74 -- ovn-nbctl list logical_router_port rtos-ovn-worker2 | grep mac
Defaulted container "nb-ovsdb" out of: nb-ovsdb, sb-ovsdb
mac : "0a:58:0a:f4:02:01
# oc exec -n ovn-kubernetes ovnkube-db-5c6757846-m6f74 -- ovn-nbctl list logical_switch_port default_pod1 | grep addresses
Defaulted container "nb-ovsdb" out of: nb-ovsdb, sb-ovsdb
addresses : ["0a:58:0a:f4:00:06 10.244.0.6"]
# oc exec -n ovn-kubernetes ovnkube-db-5c6757846-m6f74 -- ovn-nbctl list logical_switch_port default_pod2 | grep addresses
Defaulted container "nb-ovsdb" out of: nb-ovsdb, sb-ovsdb
addresses : ["0a:58:0a:f4:02:03 10.244.2.3"]
# smac=0a:58:0a:f4:02:01
# dmac=0a:58:0a:f4:02:03
# sip=10.244.0.6
# dip=10.244.2.3
# oc exec -n ovn-kubernetes ovnkube-node-9pvtz -- ovs-appctl ofproto/trace br-int in_port=1,tun_id=$sb_datapath,tun_metadata0=$tun_metadata,icmp,dl_src=$smac,dl_dst=$dmac,nw_src=$sip,nw_dst=$dip | oc exec -it -n ovn-kubernetes ovnkube-db-5c6757846-m6f74 -- ovn-detrace
Flow: icmp,tun_id=0x4,in_port=1,vlan_tci=0x0000,dl_src=0a:58:0a:f4:02:01,dl_dst=0a:58:0a:f4:02:03,nw_src=10.244.0.6,nw_dst=10.244.2.3,nw_tos=0,nw_ecn=0,nw_ttl=0,icmp_type=0,icmp_code=0
bridge("br-int")
----------------
0. in_port=1, priority 100
move:NXM_NX_TUN_ID[0..23]->OXM_OF_METADATA[0..23]
-> OXM_OF_METADATA[0..23] is now 0x4
move:NXM_NX_TUN_METADATA0[16..30]->NXM_NX_REG14[0..14]
-> NXM_NX_REG14[0..14] is now 0x1
move:NXM_NX_TUN_METADATA0[0..15]->NXM_NX_REG15[0..15]
-> NXM_NX_REG15[0..15] is now 0x3
resubmit(,38)
38. reg15=0x3,metadata=0x4, priority 100, cookie 0x12fd0c25
set_field:0x6->reg13
set_field:0x7->reg11
set_field:0xa->reg12
resubmit(,39)
* Logical datapath: "ovn-worker2" (7b99948f-1778-4a5e-a8bc-f8ce8c04fe05)
* Port Binding: logical_port "default_pod2", tunnel_key 3, chassis-name "f88350cf-84d5-4d61-9f67-c921f186320e", chassis-str "ovn-worker2"
39. priority 0
set_field:0->reg0
set_field:0->reg1
set_field:0->reg2
set_field:0->reg3
set_field:0->reg4
set_field:0->reg5
set_field:0->reg6
set_field:0->reg7
set_field:0->reg8
set_field:0->reg9
resubmit(,40)
40. ip,metadata=0x4, priority 100, cookie 0xf79ff809
set_field:0x4000000000000000000000000/0x4000000000000000000000000->xxreg0
resubmit(,41)
* Logical datapaths:
* "ovn-worker2" (7b99948f-1778-4a5e-a8bc-f8ce8c04fe05) [egress]
* "ovn-control-plane" (d6291cbf-97f1-4452-8782-c39b4888eeac) [egress]
* "ovn-worker" (e489b86e-bc37-4847-8684-f29ac03481e7) [egress]
* Logical flow: table=0 (ls_out_pre_lb), priority=100, match=(ip), actions=(reg0[2] = 1; next;)
41. ip,metadata=0x4, priority 100, cookie 0x3b8a680f
set_field:0x1000000000000000000000000/0x1000000000000000000000000->xxreg0
resubmit(,42)
* Logical datapaths:
* "ovn-worker2" (7b99948f-1778-4a5e-a8bc-f8ce8c04fe05) [egress]
* "ovn-control-plane" (d6291cbf-97f1-4452-8782-c39b4888eeac) [egress]
* "ovn-worker" (e489b86e-bc37-4847-8684-f29ac03481e7) [egress]
* Logical flow: table=1 (ls_out_pre_acl), priority=100, match=(ip), actions=(reg0[0] = 1; next;)
[...]
46. metadata=0x4, priority 0, cookie 0x33a24f04
resubmit(,47)
* Logical datapaths:
* "ext_ovn-control-plane" (21427d4f-1b5c-48f3-a6a2-ffe309d95137) [egress]
* "ext_ovn-worker2" (2e8d988d-4cfa-4c05-888e-5022ac843da4) [egress]
* "ovn-worker2" (7b99948f-1778-4a5e-a8bc-f8ce8c04fe05) [egress]
* "join" (83216e02-2381-44e3-bfb0-85c697545c3d) [egress]
* "ext_ovn-worker" (8719c547-76ca-47c4-b921-3f5a832b4338) [egress]
* "ovn-control-plane" (d6291cbf-97f1-4452-8782-c39b4888eeac) [egress]
* "ovn-worker" (e489b86e-bc37-4847-8684-f29ac03481e7) [egress]
* Logical flow: table=6 (ls_out_qos_meter), priority=0, match=(1), actions=(next;)
47. ip,reg0=0x2/0x2002,metadata=0x4, priority 100, cookie 0x89ed041
ct(commit,zone=NXM_NX_REG13[0..15],exec(set_field:0/0x1->ct_label))
set_field:0/0x1->ct_label
-> Sets the packet to an untracked state, and clears all the conntrack fields.
resubmit(,48)
* Logical datapaths:
* "ext_ovn-control-plane" (21427d4f-1b5c-48f3-a6a2-ffe309d95137) [egress]
* "ext_ovn-worker2" (2e8d988d-4cfa-4c05-888e-5022ac843da4) [egress]
* "ovn-worker2" (7b99948f-1778-4a5e-a8bc-f8ce8c04fe05) [egress]
* "join" (83216e02-2381-44e3-bfb0-85c697545c3d) [egress]
* "ext_ovn-worker" (8719c547-76ca-47c4-b921-3f5a832b4338) [egress]
* "ovn-control-plane" (d6291cbf-97f1-4452-8782-c39b4888eeac) [egress]
* "ovn-worker" (e489b86e-bc37-4847-8684-f29ac03481e7) [egress]
* Logical flow: table=7 (ls_out_stateful), priority=100, match=(reg0[1] == 1 && reg0[13] == 0), actions=(ct_commit { ct_label.blocked = 0; }; next;)
48. ip,reg15=0x3,metadata=0x4,dl_dst=0a:58:0a:f4:02:03,nw_dst=10.244.2.3, priority 90, cookie 0xb4ed6b8b
resubmit(,49)
* Logical datapaths:
* "ovn-worker2" (7b99948f-1778-4a5e-a8bc-f8ce8c04fe05) [egress]
* Logical flow: table=8 (ls_out_port_sec_ip), priority=90, match=(outport == "default_pod2" && eth.dst == 0a:58:0a:f4:02:03 && ip4.dst == {255.255.255.255, 224.0.0.0/4, 10.244.2.3}), actions=(next;)
* Logical Switch Port: default_pod2 type (addresses ['0a:58:0a:f4:02:03 10.244.2.3'], dynamic addresses [], security ['0a:58:0a:f4:02:03 10.244.2.3']
49. reg15=0x3,metadata=0x4,dl_dst=0a:58:0a:f4:02:03, priority 50, cookie 0xd19387c8
resubmit(,64)
* Logical datapaths:
* "ovn-worker2" (7b99948f-1778-4a5e-a8bc-f8ce8c04fe05) [egress]
* Logical flow: table=9 (ls_out_port_sec_l2), priority=50, match=(outport == "default_pod2" && eth.dst == {0a:58:0a:f4:02:03}), actions=(output;)
* Logical Switch Port: default_pod2 type (addresses ['0a:58:0a:f4:02:03 10.244.2.3'], dynamic addresses [], security ['0a:58:0a:f4:02:03 10.244.2.3']
64. priority 0
resubmit(,65)
65. reg15=0x3,metadata=0x4, priority 100, cookie 0x12fd0c25
output:5
* Logical datapath: "ovn-worker2" (7b99948f-1778-4a5e-a8bc-f8ce8c04fe05)
* Port Binding: logical_port "default_pod2", tunnel_key 3, chassis-name "f88350cf-84d5-4d61-9f67-c921f186320e", chassis-str "ovn-worker2"
Final flow: recirc_id=0xc8,eth,icmp,reg0=0x287,reg11=0x7,reg12=0xa,reg13=0x6,reg14=0x1,reg15=0x3,tun_id=0x4,metadata=0x4,in_port=1,vlan_tci=0x0000,dl_src=0a:58:0a:f4:02:01,dl_dst=0a:58:0a:f4:02:03,nw_src=10.244.0.6,nw_dst=10.244.2.3,nw_tos=0,nw_ecn=0,nw_ttl=0,icmp_type=0,icmp_code=0
Megaflow: recirc_id=0xc8,ct_state=+new-est-rel-rpl-inv+trk,ct_label=0/0x1,eth,ip,in_port=1,dl_src=0a:58:0a:f4:02:01,dl_dst=0a:58:0a:f4:02:03,nw_src=10.244.0.0/23,nw_dst=10.244.2.3,nw_frag=no
Datapath actions: ct(commit,zone=6,label=0/0x1),6
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment