Skip to content

Instantly share code, notes, and snippets.

@beardymcbeards
Last active Jan 12, 2017
Embed
What would you like to do?
LACP+VPC issue
# ovs-vsctl --version
ovs-vsctl (Open vSwitch) 2.3.2.45785
Compiled Mar 17 2016 13:32:15
DB Schema 7.6.2
# ovs-appctl bond/show bond0
---- bond0 ----
bond_mode: balance-tcp
bond may use recirculation: yes, Recirc-ID : 301
bond-hash-basis: 0
updelay: 31000 ms
downdelay: 200 ms
lacp_status: negotiated
active slave mac: 90:e2:ba:78:33:ed(eth1)
slave eth0: enabled
may_enable: true
slave eth1: enabled
active slave
may_enable: true
# ovs-appctl lacp/show bond0
---- bond0 ----
status: active negotiated
sys_id: 90:e2:ba:78:33:ec
sys_priority: 65534
aggregation key: 1
lacp_time: slow
slave: eth0: current attached
port_id: 1
port_priority: 65535
may_enable: true
actor sys_id: 90:e2:ba:78:33:ec
actor sys_priority: 65534
actor port_id: 1
actor port_priority: 65535
actor key: 1
actor state: activity aggregation synchronized collecting distributing
partner sys_id: 00:23:04:ee:be:cc
partner sys_priority: 32667
partner port_id: 281
partner port_priority: 32768
partner key: 32793
partner state: aggregation synchronized collecting distributing
slave: eth1: current attached
port_id: 2
port_priority: 65535
may_enable: true
actor sys_id: 90:e2:ba:78:33:ec
actor sys_priority: 65534
actor port_id: 2
actor port_priority: 65535
actor key: 1
actor state: activity aggregation synchronized collecting distributing
partner sys_id: 00:23:04:ee:be:cc
partner sys_priority: 32667
partner port_id: 16665
partner port_priority: 32768
partner key: 32793
partner state: aggregation synchronized collecting distributing
# tcpdump -evvnni eth0 ether proto 0x8809
20:35:57.898432 f8:c2:88:44:ff:60 > 01:80:c2:00:00:02, ethertype Slow Protocols (0x8809), length 124: LACPv1, length 110
Actor Information TLV (0x01), length 20
System 00:23:04:ee:be:cc, System Priority 32667, Key 32793, Port 281, Port Priority 32768
State Flags [Aggregation, Synchronization, Collecting, Distributing]
Partner Information TLV (0x02), length 20
System 90:e2:ba:78:33:ec, System Priority 65534, Key 1, Port 1, Port Priority 65535
State Flags [Activity, Aggregation, Synchronization, Collecting, Distributing]
20:35:58.579591 90:e2:ba:78:33:ec > 01:80:c2:00:00:02, ethertype Slow Protocols (0x8809), length 124: LACPv1, length 110
Actor Information TLV (0x01), length 20
System 90:e2:ba:78:33:ec, System Priority 65534, Key 1, Port 1, Port Priority 65535
State Flags [Activity, Aggregation, Synchronization, Collecting, Distributing]
Partner Information TLV (0x02), length 20
System 00:23:04:ee:be:cc, System Priority 32667, Key 32793, Port 281, Port Priority 32768
State Flags [Aggregation, Synchronization, Collecting, Distributing]
Above is the normal exchange of LACPDUs, we see each are advertising "Synchronization". At this point I manually down the eth1 interface. LACPDUs flow normally on eth0 while eth1 is down.
20:36:27.899711 f8:c2:88:44:ff:60 > 01:80:c2:00:00:02, ethertype Slow Protocols (0x8809), length 124: LACPv1, length 110
Actor Information TLV (0x01), length 20
System 00:23:04:ee:be:cc, System Priority 32667, Key 32793, Port 281, Port Priority 32768
State Flags [Aggregation, Synchronization, Collecting, Distributing]
Partner Information TLV (0x02), length 20
System 90:e2:ba:78:33:ec, System Priority 65534, Key 1, Port 1, Port Priority 65535
State Flags [Activity, Aggregation, Synchronization, Collecting, Distributing]
20:36:28.589470 90:e2:ba:78:33:ec > 01:80:c2:00:00:02, ethertype Slow Protocols (0x8809), length 124: LACPv1, length 110
Actor Information TLV (0x01), length 20
System 90:e2:ba:78:33:ec, System Priority 65534, Key 1, Port 1, Port Priority 65535
State Flags [Activity, Aggregation, Synchronization, Collecting, Distributing]
Partner Information TLV (0x02), length 20
System 00:23:04:ee:be:cc, System Priority 32667, Key 32793, Port 281, Port Priority 32768
State Flags [Aggregation, Synchronization, Collecting, Distributing]
...reduced for brevity...
20:37:57.902192 f8:c2:88:44:ff:60 > 01:80:c2:00:00:02, ethertype Slow Protocols (0x8809), length 124: LACPv1, length 110
Actor Information TLV (0x01), length 20
System 00:23:04:ee:be:cc, System Priority 32667, Key 32793, Port 281, Port Priority 32768
State Flags [Aggregation, Synchronization, Collecting, Distributing]
Partner Information TLV (0x02), length 20
System 90:e2:ba:78:33:ec, System Priority 65534, Key 1, Port 1, Port Priority 65535
State Flags [Activity, Aggregation, Synchronization, Collecting, Distributing]
20:37:58.869885 90:e2:ba:78:33:ec > 01:80:c2:00:00:02, ethertype Slow Protocols (0x8809), length 124: LACPv1, length 110
Actor Information TLV (0x01), length 20
System 90:e2:ba:78:33:ec, System Priority 65534, Key 1, Port 1, Port Priority 65535
State Flags [Activity, Aggregation, Synchronization, Collecting, Distributing]
Partner Information TLV (0x02), length 20
System 00:23:04:ee:be:cc, System Priority 32667, Key 32793, Port 281, Port Priority 32768
State Flags [Aggregation, Synchronization, Collecting, Distributing]
However, when I bring eth1 back up and it starts it's LACP initialization, OVS sends a rogue LACPDU out eth0, missing all of the "Synchronization, Collecting, Distributing" flags.
20:38:16.979437 90:e2:ba:78:33:ec > 01:80:c2:00:00:02, ethertype Slow Protocols (0x8809), length 124: LACPv1, length 110
Actor Information TLV (0x01), length 20
System 90:e2:ba:78:33:ec, System Priority 65534, Key 1, Port 1, Port Priority 65535
State Flags [Activity, Aggregation]
Partner Information TLV (0x02), length 20
System 00:23:04:ee:be:cc, System Priority 32667, Key 32793, Port 281, Port Priority 32768
State Flags [Aggregation, Synchronization, Collecting, Distributing]
When the switch sees the change in its partner's state flags, it responds with the timeout state to restart the LACP negotiation.
20:38:16.979924 f8:c2:88:44:ff:60 > 01:80:c2:00:00:02, ethertype Slow Protocols (0x8809), length 124: LACPv1, length 110
Actor Information TLV (0x01), length 20
System 00:23:04:ee:be:cc, System Priority 32667, Key 32793, Port 281, Port Priority 32768
State Flags [Timeout, Aggregation]
Partner Information TLV (0x02), length 20
System 90:e2:ba:78:33:ec, System Priority 65534, Key 1, Port 1, Port Priority 65535
State Flags [Activity, Aggregation]
What's odd is when OVS see's this LACPDU and responds with its state, OVS responds with a different state than it advertised previously. It advertises "Synchronization, Collecting, Distributing" again.
20:38:16.980909 90:e2:ba:78:33:ec > 01:80:c2:00:00:02, ethertype Slow Protocols (0x8809), length 124: LACPv1, length 110
Actor Information TLV (0x01), length 20
System 90:e2:ba:78:33:ec, System Priority 65534, Key 1, Port 1, Port Priority 65535
State Flags [Activity, Aggregation, Synchronization, Collecting, Distributing]
Partner Information TLV (0x02), length 20
System 00:23:04:ee:be:cc, System Priority 32667, Key 32793, Port 281, Port Priority 32768
State Flags [Timeout, Aggregation]
The rest of the exchange is the two resynchronizing the LACP state.
20:38:17.109650 00:00:00:00:00:00 > 01:80:c2:00:00:02, ethertype Slow Protocols (0x8809), length 124: LACPv1, length 110
Actor Information TLV (0x01), length 20
System 00:00:00:00:00:00, System Priority 0, Key 0, Port 0, Port Priority 0
State Flags [Timeout]
Partner Information TLV (0x02), length 20
System 00:00:00:00:00:00, System Priority 0, Key 0, Port 0, Port Priority 0
State Flags [none]
20:38:17.489756 90:e2:ba:78:33:ec > 01:80:c2:00:00:02, ethertype Slow Protocols (0x8809), length 124: LACPv1, length 110
Actor Information TLV (0x01), length 20
System 90:e2:ba:78:33:ec, System Priority 65534, Key 1, Port 1, Port Priority 65535
State Flags [Activity, Aggregation, Synchronization, Collecting, Distributing]
Partner Information TLV (0x02), length 20
System 00:00:00:00:00:00, System Priority 0, Key 0, Port 0, Port Priority 0
State Flags [Timeout]
20:38:17.490310 f8:c2:88:44:ff:60 > 01:80:c2:00:00:02, ethertype Slow Protocols (0x8809), length 124: LACPv1, length 110
Actor Information TLV (0x01), length 20
System 00:23:04:ee:be:cc, System Priority 32667, Key 32793, Port 281, Port Priority 32768
State Flags [Timeout, Aggregation]
Partner Information TLV (0x02), length 20
System 90:e2:ba:78:33:ec, System Priority 65534, Key 1, Port 1, Port Priority 65535
State Flags [Activity, Aggregation, Collecting, Distributing]
20:38:17.586266 90:e2:ba:78:33:ec > 01:80:c2:00:00:02, ethertype Slow Protocols (0x8809), length 124: LACPv1, length 110
Actor Information TLV (0x01), length 20
System 90:e2:ba:78:33:ec, System Priority 65534, Key 1, Port 1, Port Priority 65535
State Flags [Activity, Aggregation, Synchronization, Collecting, Distributing]
Partner Information TLV (0x02), length 20
System 00:23:04:ee:be:cc, System Priority 32667, Key 32793, Port 281, Port Priority 32768
State Flags [Timeout, Aggregation]
20:38:18.599023 90:e2:ba:78:33:ec > 01:80:c2:00:00:02, ethertype Slow Protocols (0x8809), length 124: LACPv1, length 110
Actor Information TLV (0x01), length 20
System 90:e2:ba:78:33:ec, System Priority 65534, Key 1, Port 1, Port Priority 65535
State Flags [Activity, Aggregation, Synchronization, Collecting, Distributing]
Partner Information TLV (0x02), length 20
System 00:23:04:ee:be:cc, System Priority 32667, Key 32793, Port 281, Port Priority 32768
State Flags [Timeout, Aggregation]
20:38:19.499912 f8:c2:88:44:ff:60 > 01:80:c2:00:00:02, ethertype Slow Protocols (0x8809), length 124: LACPv1, length 110
Actor Information TLV (0x01), length 20
System 00:23:04:ee:be:cc, System Priority 32667, Key 32793, Port 281, Port Priority 32768
State Flags [Timeout, Aggregation, Synchronization]
Partner Information TLV (0x02), length 20
System 90:e2:ba:78:33:ec, System Priority 65534, Key 1, Port 1, Port Priority 65535
State Flags [Activity, Aggregation, Synchronization, Collecting, Distributing]
20:38:19.609038 90:e2:ba:78:33:ec > 01:80:c2:00:00:02, ethertype Slow Protocols (0x8809), length 124: LACPv1, length 110
Actor Information TLV (0x01), length 20
System 90:e2:ba:78:33:ec, System Priority 65534, Key 1, Port 1, Port Priority 65535
State Flags [Activity, Aggregation, Synchronization, Collecting, Distributing]
Partner Information TLV (0x02), length 20
System 00:23:04:ee:be:cc, System Priority 32667, Key 32793, Port 281, Port Priority 32768
State Flags [Timeout, Aggregation, Synchronization]
20:38:19.864590 f8:c2:88:44:ff:60 > 01:80:c2:00:00:02, ethertype Slow Protocols (0x8809), length 124: LACPv1, length 110
Actor Information TLV (0x01), length 20
System 00:23:04:ee:be:cc, System Priority 32667, Key 32793, Port 281, Port Priority 32768
State Flags [Timeout, Aggregation, Synchronization, Collecting]
Partner Information TLV (0x02), length 20
System 90:e2:ba:78:33:ec, System Priority 65534, Key 1, Port 1, Port Priority 65535
State Flags [Activity, Aggregation, Synchronization, Collecting, Distributing]
20:38:19.871692 f8:c2:88:44:ff:60 > 01:80:c2:00:00:02, ethertype Slow Protocols (0x8809), length 124: LACPv1, length 110
Actor Information TLV (0x01), length 20
System 00:23:04:ee:be:cc, System Priority 32667, Key 32793, Port 281, Port Priority 32768
State Flags [Aggregation, Synchronization, Collecting, Distributing]
Partner Information TLV (0x02), length 20
System 90:e2:ba:78:33:ec, System Priority 65534, Key 1, Port 1, Port Priority 65535
State Flags [Activity, Aggregation, Synchronization, Collecting, Distributing]
20:38:20.619056 90:e2:ba:78:33:ec > 01:80:c2:00:00:02, ethertype Slow Protocols (0x8809), length 124: LACPv1, length 110
Actor Information TLV (0x01), length 20
System 90:e2:ba:78:33:ec, System Priority 65534, Key 1, Port 1, Port Priority 65535
State Flags [Activity, Aggregation, Synchronization, Collecting, Distributing]
Partner Information TLV (0x02), length 20
System 00:23:04:ee:be:cc, System Priority 32667, Key 32793, Port 281, Port Priority 32768
State Flags [Aggregation, Synchronization, Collecting, Distributing]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment