-
-
Save beardymcbeards/7bd9feca87c0574e996a397d90d5ff98 to your computer and use it in GitHub Desktop.
LACP+VPC issue
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| # ovs-vsctl --version | |
| ovs-vsctl (Open vSwitch) 2.3.2.45785 | |
| Compiled Mar 17 2016 13:32:15 | |
| DB Schema 7.6.2 | |
| # ovs-appctl bond/show bond0 | |
| ---- bond0 ---- | |
| bond_mode: balance-tcp | |
| bond may use recirculation: yes, Recirc-ID : 301 | |
| bond-hash-basis: 0 | |
| updelay: 31000 ms | |
| downdelay: 200 ms | |
| lacp_status: negotiated | |
| active slave mac: 90:e2:ba:78:33:ed(eth1) | |
| slave eth0: enabled | |
| may_enable: true | |
| slave eth1: enabled | |
| active slave | |
| may_enable: true | |
| # ovs-appctl lacp/show bond0 | |
| ---- bond0 ---- | |
| status: active negotiated | |
| sys_id: 90:e2:ba:78:33:ec | |
| sys_priority: 65534 | |
| aggregation key: 1 | |
| lacp_time: slow | |
| slave: eth0: current attached | |
| port_id: 1 | |
| port_priority: 65535 | |
| may_enable: true | |
| actor sys_id: 90:e2:ba:78:33:ec | |
| actor sys_priority: 65534 | |
| actor port_id: 1 | |
| actor port_priority: 65535 | |
| actor key: 1 | |
| actor state: activity aggregation synchronized collecting distributing | |
| partner sys_id: 00:23:04:ee:be:cc | |
| partner sys_priority: 32667 | |
| partner port_id: 281 | |
| partner port_priority: 32768 | |
| partner key: 32793 | |
| partner state: aggregation synchronized collecting distributing | |
| slave: eth1: current attached | |
| port_id: 2 | |
| port_priority: 65535 | |
| may_enable: true | |
| actor sys_id: 90:e2:ba:78:33:ec | |
| actor sys_priority: 65534 | |
| actor port_id: 2 | |
| actor port_priority: 65535 | |
| actor key: 1 | |
| actor state: activity aggregation synchronized collecting distributing | |
| partner sys_id: 00:23:04:ee:be:cc | |
| partner sys_priority: 32667 | |
| partner port_id: 16665 | |
| partner port_priority: 32768 | |
| partner key: 32793 | |
| partner state: aggregation synchronized collecting distributing |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| # tcpdump -evvnni eth0 ether proto 0x8809 | |
| 20:35:57.898432 f8:c2:88:44:ff:60 > 01:80:c2:00:00:02, ethertype Slow Protocols (0x8809), length 124: LACPv1, length 110 | |
| Actor Information TLV (0x01), length 20 | |
| System 00:23:04:ee:be:cc, System Priority 32667, Key 32793, Port 281, Port Priority 32768 | |
| State Flags [Aggregation, Synchronization, Collecting, Distributing] | |
| Partner Information TLV (0x02), length 20 | |
| System 90:e2:ba:78:33:ec, System Priority 65534, Key 1, Port 1, Port Priority 65535 | |
| State Flags [Activity, Aggregation, Synchronization, Collecting, Distributing] | |
| 20:35:58.579591 90:e2:ba:78:33:ec > 01:80:c2:00:00:02, ethertype Slow Protocols (0x8809), length 124: LACPv1, length 110 | |
| Actor Information TLV (0x01), length 20 | |
| System 90:e2:ba:78:33:ec, System Priority 65534, Key 1, Port 1, Port Priority 65535 | |
| State Flags [Activity, Aggregation, Synchronization, Collecting, Distributing] | |
| Partner Information TLV (0x02), length 20 | |
| System 00:23:04:ee:be:cc, System Priority 32667, Key 32793, Port 281, Port Priority 32768 | |
| State Flags [Aggregation, Synchronization, Collecting, Distributing] | |
| Above is the normal exchange of LACPDUs, we see each are advertising "Synchronization". At this point I manually down the eth1 interface. LACPDUs flow normally on eth0 while eth1 is down. | |
| 20:36:27.899711 f8:c2:88:44:ff:60 > 01:80:c2:00:00:02, ethertype Slow Protocols (0x8809), length 124: LACPv1, length 110 | |
| Actor Information TLV (0x01), length 20 | |
| System 00:23:04:ee:be:cc, System Priority 32667, Key 32793, Port 281, Port Priority 32768 | |
| State Flags [Aggregation, Synchronization, Collecting, Distributing] | |
| Partner Information TLV (0x02), length 20 | |
| System 90:e2:ba:78:33:ec, System Priority 65534, Key 1, Port 1, Port Priority 65535 | |
| State Flags [Activity, Aggregation, Synchronization, Collecting, Distributing] | |
| 20:36:28.589470 90:e2:ba:78:33:ec > 01:80:c2:00:00:02, ethertype Slow Protocols (0x8809), length 124: LACPv1, length 110 | |
| Actor Information TLV (0x01), length 20 | |
| System 90:e2:ba:78:33:ec, System Priority 65534, Key 1, Port 1, Port Priority 65535 | |
| State Flags [Activity, Aggregation, Synchronization, Collecting, Distributing] | |
| Partner Information TLV (0x02), length 20 | |
| System 00:23:04:ee:be:cc, System Priority 32667, Key 32793, Port 281, Port Priority 32768 | |
| State Flags [Aggregation, Synchronization, Collecting, Distributing] | |
| ...reduced for brevity... | |
| 20:37:57.902192 f8:c2:88:44:ff:60 > 01:80:c2:00:00:02, ethertype Slow Protocols (0x8809), length 124: LACPv1, length 110 | |
| Actor Information TLV (0x01), length 20 | |
| System 00:23:04:ee:be:cc, System Priority 32667, Key 32793, Port 281, Port Priority 32768 | |
| State Flags [Aggregation, Synchronization, Collecting, Distributing] | |
| Partner Information TLV (0x02), length 20 | |
| System 90:e2:ba:78:33:ec, System Priority 65534, Key 1, Port 1, Port Priority 65535 | |
| State Flags [Activity, Aggregation, Synchronization, Collecting, Distributing] | |
| 20:37:58.869885 90:e2:ba:78:33:ec > 01:80:c2:00:00:02, ethertype Slow Protocols (0x8809), length 124: LACPv1, length 110 | |
| Actor Information TLV (0x01), length 20 | |
| System 90:e2:ba:78:33:ec, System Priority 65534, Key 1, Port 1, Port Priority 65535 | |
| State Flags [Activity, Aggregation, Synchronization, Collecting, Distributing] | |
| Partner Information TLV (0x02), length 20 | |
| System 00:23:04:ee:be:cc, System Priority 32667, Key 32793, Port 281, Port Priority 32768 | |
| State Flags [Aggregation, Synchronization, Collecting, Distributing] | |
| However, when I bring eth1 back up and it starts it's LACP initialization, OVS sends a rogue LACPDU out eth0, missing all of the "Synchronization, Collecting, Distributing" flags. | |
| 20:38:16.979437 90:e2:ba:78:33:ec > 01:80:c2:00:00:02, ethertype Slow Protocols (0x8809), length 124: LACPv1, length 110 | |
| Actor Information TLV (0x01), length 20 | |
| System 90:e2:ba:78:33:ec, System Priority 65534, Key 1, Port 1, Port Priority 65535 | |
| State Flags [Activity, Aggregation] | |
| Partner Information TLV (0x02), length 20 | |
| System 00:23:04:ee:be:cc, System Priority 32667, Key 32793, Port 281, Port Priority 32768 | |
| State Flags [Aggregation, Synchronization, Collecting, Distributing] | |
| When the switch sees the change in its partner's state flags, it responds with the timeout state to restart the LACP negotiation. | |
| 20:38:16.979924 f8:c2:88:44:ff:60 > 01:80:c2:00:00:02, ethertype Slow Protocols (0x8809), length 124: LACPv1, length 110 | |
| Actor Information TLV (0x01), length 20 | |
| System 00:23:04:ee:be:cc, System Priority 32667, Key 32793, Port 281, Port Priority 32768 | |
| State Flags [Timeout, Aggregation] | |
| Partner Information TLV (0x02), length 20 | |
| System 90:e2:ba:78:33:ec, System Priority 65534, Key 1, Port 1, Port Priority 65535 | |
| State Flags [Activity, Aggregation] | |
| What's odd is when OVS see's this LACPDU and responds with its state, OVS responds with a different state than it advertised previously. It advertises "Synchronization, Collecting, Distributing" again. | |
| 20:38:16.980909 90:e2:ba:78:33:ec > 01:80:c2:00:00:02, ethertype Slow Protocols (0x8809), length 124: LACPv1, length 110 | |
| Actor Information TLV (0x01), length 20 | |
| System 90:e2:ba:78:33:ec, System Priority 65534, Key 1, Port 1, Port Priority 65535 | |
| State Flags [Activity, Aggregation, Synchronization, Collecting, Distributing] | |
| Partner Information TLV (0x02), length 20 | |
| System 00:23:04:ee:be:cc, System Priority 32667, Key 32793, Port 281, Port Priority 32768 | |
| State Flags [Timeout, Aggregation] | |
| The rest of the exchange is the two resynchronizing the LACP state. | |
| 20:38:17.109650 00:00:00:00:00:00 > 01:80:c2:00:00:02, ethertype Slow Protocols (0x8809), length 124: LACPv1, length 110 | |
| Actor Information TLV (0x01), length 20 | |
| System 00:00:00:00:00:00, System Priority 0, Key 0, Port 0, Port Priority 0 | |
| State Flags [Timeout] | |
| Partner Information TLV (0x02), length 20 | |
| System 00:00:00:00:00:00, System Priority 0, Key 0, Port 0, Port Priority 0 | |
| State Flags [none] | |
| 20:38:17.489756 90:e2:ba:78:33:ec > 01:80:c2:00:00:02, ethertype Slow Protocols (0x8809), length 124: LACPv1, length 110 | |
| Actor Information TLV (0x01), length 20 | |
| System 90:e2:ba:78:33:ec, System Priority 65534, Key 1, Port 1, Port Priority 65535 | |
| State Flags [Activity, Aggregation, Synchronization, Collecting, Distributing] | |
| Partner Information TLV (0x02), length 20 | |
| System 00:00:00:00:00:00, System Priority 0, Key 0, Port 0, Port Priority 0 | |
| State Flags [Timeout] | |
| 20:38:17.490310 f8:c2:88:44:ff:60 > 01:80:c2:00:00:02, ethertype Slow Protocols (0x8809), length 124: LACPv1, length 110 | |
| Actor Information TLV (0x01), length 20 | |
| System 00:23:04:ee:be:cc, System Priority 32667, Key 32793, Port 281, Port Priority 32768 | |
| State Flags [Timeout, Aggregation] | |
| Partner Information TLV (0x02), length 20 | |
| System 90:e2:ba:78:33:ec, System Priority 65534, Key 1, Port 1, Port Priority 65535 | |
| State Flags [Activity, Aggregation, Collecting, Distributing] | |
| 20:38:17.586266 90:e2:ba:78:33:ec > 01:80:c2:00:00:02, ethertype Slow Protocols (0x8809), length 124: LACPv1, length 110 | |
| Actor Information TLV (0x01), length 20 | |
| System 90:e2:ba:78:33:ec, System Priority 65534, Key 1, Port 1, Port Priority 65535 | |
| State Flags [Activity, Aggregation, Synchronization, Collecting, Distributing] | |
| Partner Information TLV (0x02), length 20 | |
| System 00:23:04:ee:be:cc, System Priority 32667, Key 32793, Port 281, Port Priority 32768 | |
| State Flags [Timeout, Aggregation] | |
| 20:38:18.599023 90:e2:ba:78:33:ec > 01:80:c2:00:00:02, ethertype Slow Protocols (0x8809), length 124: LACPv1, length 110 | |
| Actor Information TLV (0x01), length 20 | |
| System 90:e2:ba:78:33:ec, System Priority 65534, Key 1, Port 1, Port Priority 65535 | |
| State Flags [Activity, Aggregation, Synchronization, Collecting, Distributing] | |
| Partner Information TLV (0x02), length 20 | |
| System 00:23:04:ee:be:cc, System Priority 32667, Key 32793, Port 281, Port Priority 32768 | |
| State Flags [Timeout, Aggregation] | |
| 20:38:19.499912 f8:c2:88:44:ff:60 > 01:80:c2:00:00:02, ethertype Slow Protocols (0x8809), length 124: LACPv1, length 110 | |
| Actor Information TLV (0x01), length 20 | |
| System 00:23:04:ee:be:cc, System Priority 32667, Key 32793, Port 281, Port Priority 32768 | |
| State Flags [Timeout, Aggregation, Synchronization] | |
| Partner Information TLV (0x02), length 20 | |
| System 90:e2:ba:78:33:ec, System Priority 65534, Key 1, Port 1, Port Priority 65535 | |
| State Flags [Activity, Aggregation, Synchronization, Collecting, Distributing] | |
| 20:38:19.609038 90:e2:ba:78:33:ec > 01:80:c2:00:00:02, ethertype Slow Protocols (0x8809), length 124: LACPv1, length 110 | |
| Actor Information TLV (0x01), length 20 | |
| System 90:e2:ba:78:33:ec, System Priority 65534, Key 1, Port 1, Port Priority 65535 | |
| State Flags [Activity, Aggregation, Synchronization, Collecting, Distributing] | |
| Partner Information TLV (0x02), length 20 | |
| System 00:23:04:ee:be:cc, System Priority 32667, Key 32793, Port 281, Port Priority 32768 | |
| State Flags [Timeout, Aggregation, Synchronization] | |
| 20:38:19.864590 f8:c2:88:44:ff:60 > 01:80:c2:00:00:02, ethertype Slow Protocols (0x8809), length 124: LACPv1, length 110 | |
| Actor Information TLV (0x01), length 20 | |
| System 00:23:04:ee:be:cc, System Priority 32667, Key 32793, Port 281, Port Priority 32768 | |
| State Flags [Timeout, Aggregation, Synchronization, Collecting] | |
| Partner Information TLV (0x02), length 20 | |
| System 90:e2:ba:78:33:ec, System Priority 65534, Key 1, Port 1, Port Priority 65535 | |
| State Flags [Activity, Aggregation, Synchronization, Collecting, Distributing] | |
| 20:38:19.871692 f8:c2:88:44:ff:60 > 01:80:c2:00:00:02, ethertype Slow Protocols (0x8809), length 124: LACPv1, length 110 | |
| Actor Information TLV (0x01), length 20 | |
| System 00:23:04:ee:be:cc, System Priority 32667, Key 32793, Port 281, Port Priority 32768 | |
| State Flags [Aggregation, Synchronization, Collecting, Distributing] | |
| Partner Information TLV (0x02), length 20 | |
| System 90:e2:ba:78:33:ec, System Priority 65534, Key 1, Port 1, Port Priority 65535 | |
| State Flags [Activity, Aggregation, Synchronization, Collecting, Distributing] | |
| 20:38:20.619056 90:e2:ba:78:33:ec > 01:80:c2:00:00:02, ethertype Slow Protocols (0x8809), length 124: LACPv1, length 110 | |
| Actor Information TLV (0x01), length 20 | |
| System 90:e2:ba:78:33:ec, System Priority 65534, Key 1, Port 1, Port Priority 65535 | |
| State Flags [Activity, Aggregation, Synchronization, Collecting, Distributing] | |
| Partner Information TLV (0x02), length 20 | |
| System 00:23:04:ee:be:cc, System Priority 32667, Key 32793, Port 281, Port Priority 32768 | |
| State Flags [Aggregation, Synchronization, Collecting, Distributing] |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment