Last active
April 9, 2017 23:29
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Two machines with 10G NIC directly connect each other | |
nn22 and nn23 | |
step 1: create one VF on nn23 | |
echo 1 > /sys/bus/pci/devices/0000:07:00.0/sriov_numvfs | |
Before bind VF to vfid-pci, try to ping this VF from nn22 (IP 12.0.0.22) | |
zhangwei1984@nimbnode23:~/bifurcation/dpdk-17.02$ sudo ./usertools/dpdk-devbind.py --status | |
Network devices using DPDK-compatible driver | |
============================================ | |
<none> | |
Network devices using kernel driver | |
=================================== | |
0000:05:00.0 '82576 Gigabit Network Connection' if=em1 drv=igb unused=igb_uio,vfio-pci *Active* | |
0000:05:00.1 '82576 Gigabit Network Connection' if=em2 drv=igb unused=igb_uio,vfio-pci | |
0000:07:00.0 '82599ES 10-Gigabit SFI/SFP+ Network Connection' if=p2p1 drv=ixgbe unused=igb_uio,vfio-pci | |
0000:07:00.1 '82599ES 10-Gigabit SFI/SFP+ Network Connection' if=p2p2 drv=ixgbe unused=igb_uio,vfio-pci | |
0000:07:10.0 '82599 Ethernet Controller Virtual Function' if=p2p1_0 drv=ixgbevf unused=igb_uio,vfio-pci *Active* | |
0000:07:10.1 '82599 Ethernet Controller Virtual Function' if=p2p2_0 drv=ixgbevf unused=igb_uio,vfio-pci | |
step 2: assign IP to this VF p2p1_0 12.0.0.23 | |
p2p1 Link encap:Ethernet HWaddr 90:e2:ba:4a:e8:08 | |
inet6 addr: fe80::92e2:baff:fe4a:e808/64 Scope:Link | |
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 | |
RX packets:1847 errors:0 dropped:0 overruns:0 frame:0 | |
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0 | |
collisions:0 txqueuelen:1000 | |
RX bytes:111124 (111.1 KB) TX bytes:0 (0.0 B) | |
p2p1_0 Link encap:Ethernet HWaddr 2a:6b:c5:99:c1:e7 | |
inet addr:12.0.0.23 Bcast:12.255.255.255 Mask:255.0.0.0 | |
inet6 addr: fe80::286b:c5ff:fe99:c1e7/64 Scope:Link | |
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 | |
RX packets:1836 errors:0 dropped:0 overruns:0 frame:0 | |
TX packets:1844 errors:0 dropped:0 overruns:0 carrier:0 | |
collisions:0 txqueuelen:1000 | |
RX bytes:110160 (110.1 KB) TX bytes:77760 (77.7 KB) | |
From the tcpdump -i p2p1_0, it gave the reply. | |
tcpdump output for p2p1_0 | |
19:32:26.495373 ARP, Request who-has 12.0.0.23 tell 12.0.0.22, length 46 | |
19:32:26.495386 ARP, Reply 12.0.0.23 is-at 2a:6b:c5:99:c1:e7 (oui Unknown), length 28 | |
19:32:27.493077 ARP, Request who-has 12.0.0.23 tell 12.0.0.22, length 46 | |
19:32:27.493098 ARP, Reply 12.0.0.23 is-at 2a:6b:c5:99:c1:e7 (oui Unknown), length 28 | |
Also ifconfig counter increases. It seems that the arp reply should be sent out. | |
But nn22 does not receive the packets. ifconfig drop counter is 0. | |
If using ethtool checks the counter, tx counter is 0. | |
NIC statistics: | |
rx_packets: 29 | |
tx_packets: 0 | |
rx_bytes: 1740 | |
tx_bytes: 0 | |
tx_busy: 0 | |
tx_restart_queue: 0 | |
tx_timeout_count: 1 | |
multicast: 0 | |
rx_csum_offload_errors: 0 | |
rx_bp_poll_yield: 0 | |
rx_bp_cleaned: 0 | |
rx_bp_misses: 0 | |
tx_bp_napi_yield: 0 | |
tx_bp_cleaned: 0 | |
tx_bp_misses: 0 | |
why ifconfig and ethtool does not match each other. ifconfig shows tx increases. | |
but ethtool shows tx is 0. p2p1 tx is also 0 from ethtool stats. | |
Do you have any idea how to debug this problem? | |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment