Hi !
I have just added an Intel x550T2 10Gb NIC on my OMV server, and noticed that I have a lot of rx_dropped errors for all NICs (2 Intel i210 on the motherboard, 2 x550 on the card).
The card is on a correct PCIe extension slot (PCIe3.0 x8).
At the moment, they are all linked to a Netgear GS108e switch.
Here are some infos for the i210 NIC :
Code
# ethtool -i enp35s0
driver: igb
version: 5.10.0-0.bpo.9-amd64
firmware-version: 3.16, 0x800004d6
expansion-rom-version:
bus-info: 0000:23:00.0
supports-statistics: yes
supports-test: yes
supports-eeprom-access: yes
supports-register-dump: yes
supports-priv-flags: yesAnd for the x550 NIC
Code
# ethtool -i enp1s0f0
driver: ixgbe
version: 5.10.0-0.bpo.9-amd64
firmware-version: 0x80000c67, 1.1276.0
expansion-rom-version:
bus-info: 0000:01:00.0
supports-statistics: yes
supports-test: yes
supports-eeprom-access: yes
supports-register-dump: yes
supports-priv-flags: yes
# ethtool -S enp1s0f0 | grep rx_dropped
rx_dropped: 238664I have already tried to max the buffer ring but still have the error:
Code
# ethtool -g enp1s0f0
Ring parameters for enp1s0f0:
Pre-set maximums:
RX: 4096
RX Mini: 0
RX Jumbo: 0
TX: 4096
Current hardware settings:
RX: 1024
RX Mini: 0
RX Jumbo: 0
TX: 1024
# ethtool -G enp1s0f0 rx 4096 tx 4096
# ethtool -g enp1s0f0
Ring parameters for enp1s0f0:
Pre-set maximums:
RX: 4096
RX Mini: 0
RX Jumbo: 0
TX: 4096
Current hardware settings:
RX: 4096
RX Mini: 0
RX Jumbo: 0
TX: 4096Here are some Iperf tests too :
Code
Connecting to host 192.168.1.100, port 5201
[ 4] local 192.168.1.31 port 63926 connected to 192.168.1.100 port 5201
[ ID] Interval Transfer Bandwidth
[ 4] 0.00-1.00 sec 111 MBytes 934 Mbits/sec
[ 4] 1.00-2.00 sec 111 MBytes 928 Mbits/sec
[ 4] 2.00-3.00 sec 112 MBytes 942 Mbits/sec
[ 4] 3.00-4.00 sec 112 MBytes 940 Mbits/sec
[ 4] 4.00-5.00 sec 109 MBytes 911 Mbits/sec
[ 4] 5.00-6.00 sec 108 MBytes 904 Mbits/sec
[ 4] 6.00-7.00 sec 112 MBytes 942 Mbits/sec
[ 4] 7.00-8.00 sec 112 MBytes 940 Mbits/sec
[ 4] 8.00-9.00 sec 112 MBytes 942 Mbits/sec
[ 4] 9.00-10.00 sec 110 MBytes 919 Mbits/sec
[ 4] 10.00-11.00 sec 112 MBytes 941 Mbits/sec
[ 4] 11.00-12.00 sec 112 MBytes 942 Mbits/sec
[ 4] 12.00-13.00 sec 112 MBytes 942 Mbits/sec
[ 4] 13.00-14.00 sec 112 MBytes 936 Mbits/sec
[ 4] 14.00-15.00 sec 112 MBytes 939 Mbits/sec
[ 4] 15.00-16.00 sec 112 MBytes 942 Mbits/sec
[ 4] 16.00-17.00 sec 112 MBytes 937 Mbits/sec
[ 4] 17.00-18.00 sec 112 MBytes 941 Mbits/sec
[ 4] 18.00-19.00 sec 112 MBytes 942 Mbits/sec
[ 4] 19.00-20.00 sec 112 MBytes 942 Mbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bandwidth
[ 4] 0.00-20.00 sec 2.18 GBytes 935 Mbits/sec sender
[ 4] 0.00-20.00 sec 2.18 GBytes 935 Mbits/sec receiver
iperf Done.
Connecting to host 192.168.1.200, port 5201
[ 4] local 192.168.1.31 port 63921 connected to 192.168.1.200 port 5201
[ ID] Interval Transfer Bandwidth
[ 4] 0.00-1.00 sec 112 MBytes 942 Mbits/sec
[ 4] 1.00-2.00 sec 111 MBytes 933 Mbits/sec
[ 4] 2.00-3.00 sec 112 MBytes 940 Mbits/sec
[ 4] 3.00-4.00 sec 112 MBytes 940 Mbits/sec
[ 4] 4.00-5.00 sec 111 MBytes 933 Mbits/sec
[ 4] 5.00-6.00 sec 111 MBytes 930 Mbits/sec
[ 4] 6.00-7.00 sec 112 MBytes 942 Mbits/sec
[ 4] 7.00-8.00 sec 112 MBytes 942 Mbits/sec
[ 4] 8.00-9.00 sec 112 MBytes 937 Mbits/sec
[ 4] 9.00-10.00 sec 112 MBytes 941 Mbits/sec
[ 4] 10.00-11.00 sec 109 MBytes 914 Mbits/sec
[ 4] 11.00-12.00 sec 112 MBytes 940 Mbits/sec
[ 4] 12.00-13.00 sec 112 MBytes 938 Mbits/sec
[ 4] 13.00-14.00 sec 112 MBytes 942 Mbits/sec
[ 4] 14.00-15.00 sec 112 MBytes 936 Mbits/sec
[ 4] 15.00-16.00 sec 112 MBytes 942 Mbits/sec
[ 4] 16.00-17.00 sec 112 MBytes 938 Mbits/sec
[ 4] 17.00-18.00 sec 112 MBytes 942 Mbits/sec
[ 4] 18.00-19.00 sec 112 MBytes 942 Mbits/sec
[ 4] 19.00-20.00 sec 111 MBytes 934 Mbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bandwidth
[ 4] 0.00-20.00 sec 2.18 GBytes 937 Mbits/sec sender
[ 4] 0.00-20.00 sec 2.18 GBytes 937 Mbits/sec receiver
iperf Done.
I can see that ALL my NIC have some dropped packets :
Code
# ifconfig -a
enp1s0f0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 192.168.1.100 netmask 255.255.255.0 broadcast 192.168.1.255
ether b4:96:91:8d:8f:c8 txqueuelen 1000 (Ethernet)
RX packets 1155390517 bytes 1252988750330 (1.1 TiB)
RX errors 0 dropped 241402 overruns 0 frame 0
TX packets 1272400729 bytes 1513362541979 (1.3 TiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
enp1s0f1: flags=4099<UP,BROADCAST,MULTICAST> mtu 1500
ether b4:96:91:8d:8f:c9 txqueuelen 1000 (Ethernet)
RX packets 4520 bytes 875489 (854.9 KiB)
RX errors 0 dropped 1105 overruns 0 frame 0
TX packets 231 bytes 54376 (53.1 KiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
enp35s0: flags=4099<UP,BROADCAST,MULTICAST> mtu 1500
ether d0:50:99:d8:dd:5f txqueuelen 1000 (Ethernet)
RX packets 176942580 bytes 109978029619 (102.4 GiB)
RX errors 0 dropped 10104 overruns 0 frame 0
TX packets 416419193 bytes 546134806422 (508.6 GiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
device memory 0xf7500000-f757ffff
enp36s0: flags=4099<UP,BROADCAST,MULTICAST> mtu 1500
ether d0:50:99:d8:dd:60 txqueuelen 1000 (Ethernet)
RX packets 1262473 bytes 1255531882 (1.1 GiB)
RX errors 0 dropped 1760 overruns 0 frame 0
TX packets 1457957 bytes 1779356814 (1.6 GiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0So :
- It's probably not the cables (all the cables are directly connected to the switch), even the test PC... => Not the cables in the walls !
- It's probably not the NICs, as 2 are on the motherboard (ASRock Rack x470D4U) and 2 are on the PCie x550T2 Card,
- Switch ?
- Server configuration... maybe/maybe not ? Intel drivers ?
Any help or idea is welcome as I'm a little lost at the moment
