Intel X520 recognized as 10G, but only 1G is used?

daAndy · Mar 6th 2022

Hello,

I already searched around for my problem, but most of the time it is unsupported modules or a problem with the ixgbe driver. For me both seems to be fine.

Here is the ixgbe log:

Code

root@XXX:~ # dmesg | grep ixgbe
[    0.956277] ixgbe: Intel(R) 10 Gigabit PCI Express Network Driver
[    0.956280] ixgbe: Copyright (c) 1999-2016 Intel Corporation.
[    1.121063] ixgbe 0000:07:00.0: Multiqueue Enabled: Rx Queue count = 12, Tx Queue count = 12 XDP Queue count = 0
[    1.121345] ixgbe 0000:07:00.0: 32.000 Gb/s available PCIe bandwidth (5.0 GT/s PCIe x8 link)
[    1.121464] ixgbe 0000:07:00.0: MAC: 2, PHY: 16, SFP+: 5, PBA No: Unknown
[    1.121465] ixgbe 0000:07:00.0: XX:XX:XX:XX:XX:00
[    1.142798] ixgbe 0000:07:00.0: Intel(R) 10 Gigabit Network Connection
[    1.142910] libphy: ixgbe-mdio: probed
[    2.329113] ixgbe 0000:07:00.1: Multiqueue Enabled: Rx Queue count = 12, Tx Queue count = 12 XDP Queue count = 0
[    2.329395] ixgbe 0000:07:00.1: 32.000 Gb/s available PCIe bandwidth (5.0 GT/s PCIe x8 link)
[    2.329514] ixgbe 0000:07:00.1: MAC: 2, PHY: 1, PBA No: Unknown
[    2.329515] ixgbe 0000:07:00.1: XX:XX:XX:XX:XX:01
[    2.330423] ixgbe 0000:07:00.1: Intel(R) 10 Gigabit Network Connection
[    2.330471] libphy: ixgbe-mdio: probed
[    2.331635] ixgbe 0000:07:00.1 enp7s0f1: renamed from eth1
[    2.346244] ixgbe 0000:07:00.0 enp7s0f0: renamed from eth0
[    5.165248] ixgbe 0000:07:00.1: registered PHC device on enp7s0f1
[    6.011916] ixgbe 0000:07:00.0: registered PHC device on enp7s0f0
[    6.197617] ixgbe 0000:07:00.0 enp7s0f0: detected SFP+: 5
[    6.337682] ixgbe 0000:07:00.0 enp7s0f0: NIC Link is Up 10 Gbps, Flow Control: RX/TX
[  550.516867] ixgbe 0000:07:00.0 enp7s0f0: NIC Link is Down
[  578.961106] ixgbe 0000:07:00.0 enp7s0f0: NIC Link is Up 10 Gbps, Flow Control: RX/TX

Display More

Here the "qlen 1000" (I do think this is the problem - it is somehow recognized as 10G driverwise, also in the switch, but it is not used somehow:

Code

root@XXX:~ # ip a
2: enp5s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000
    link/ether 0XX:XX:XX:XX:XX:f1 brd ff:ff:ff:ff:ff:ff
    inet XXX.XXX.XXX.192/24 brd XXX.XXX.XXX.255 scope global dynamic enp5s0
       valid_lft 84731sec preferred_lft 84731sec
3: enp7s0f0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000
    link/ether XX:XX:XX:XX:XX:00 brd ff:ff:ff:ff:ff:ff
    inet XXX.XXX.XXX.190/24 brd XXX.XXX.XXX.255 scope global dynamic enp7s0f0
       valid_lft 84731sec preferred_lft 84731sec
4: enp7s0f1: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq state DOWN group default qlen 1000
    link/ether XX:XX:XX:XX:XX:01 brd ff:ff:ff:ff:ff:ff

Display More

ipfer3 log:

Code

root@thevault:~ # iperf3 -s
-----------------------------------------------------------
Server listening on 5201
-----------------------------------------------------------
Accepted connection from XXX.XXX.XXX.141, port 54564
[  5] local XXX.XXX.XXX.190 port 5201 connected to XXX.XXX.XXX.141 port 54565
[ ID] Interval           Transfer     Bitrate
[  5]   0.00-1.00   sec   106 MBytes   892 Mbits/sec
[  5]   1.00-2.00   sec   118 MBytes   991 Mbits/sec
[  5]   2.00-3.00   sec   117 MBytes   986 Mbits/sec
[  5]   3.00-4.00   sec   114 MBytes   955 Mbits/sec
[  5]   4.00-5.00   sec   118 MBytes   992 Mbits/sec
[  5]   5.00-6.00   sec   112 MBytes   938 Mbits/sec
[  5]   6.00-7.00   sec   115 MBytes   967 Mbits/sec
[  5]   7.00-8.00   sec   113 MBytes   944 Mbits/sec
[  5]   8.00-9.00   sec   118 MBytes   987 Mbits/sec
[  5]   9.00-10.00  sec   105 MBytes   881 Mbits/sec
[  5]  10.00-10.04  sec  5.05 MBytes  1.02 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate
[  5]   0.00-10.04  sec  1.11 GBytes   954 Mbits/sec                  receiver

Display More

Another iperf3 log (2 clients: Windows to Windows):

Code

Connecting to host XXX.XXX.XXX.152, port 5201
[  4] local XXX.XXX.XXX.141 port 54290 connected to XXX.XXX.XXX.152 port 5201
[ ID] Interval           Transfer     Bandwidth
[  4]   0.00-1.00   sec   253 MBytes  2.12 Gbits/sec
[  4]   1.00-2.00   sec   277 MBytes  2.32 Gbits/sec
[  4]   2.00-3.00   sec   273 MBytes  2.29 Gbits/sec
[  4]   3.00-4.00   sec   289 MBytes  2.43 Gbits/sec
[  4]   4.00-5.00   sec   283 MBytes  2.37 Gbits/sec
[  4]   5.00-6.00   sec   302 MBytes  2.53 Gbits/sec
[  4]   6.00-7.00   sec   282 MBytes  2.37 Gbits/sec
[  4]   7.00-8.00   sec   290 MBytes  2.43 Gbits/sec
[  4]   8.00-9.00   sec   286 MBytes  2.40 Gbits/sec
[  4]   9.00-10.00  sec   278 MBytes  2.33 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bandwidth
[  4]   0.00-10.00  sec  2.75 GBytes  2.36 Gbits/sec                  sender
[  4]   0.00-10.00  sec  2.75 GBytes  2.36 Gbits/sec                  receiver

Display More

The module is compatible, the driver and the network card seems fine. The network in general can also do >1G (wonder why it is only 2,5G since they are both 10G, but whatever). Has the

"qlen 1000" anything to do with that or am I on a wrong path there?

Thank you in advance!

Greetings

Andy

palerob · Mar 9th 2022

Andy,

I have the same on my 10GbE interface on OMV5. It runs fine to another 10GbE host on the same switch. I see something similar to you when attempting to reach a 10GbE devices which on only connected via cat5E over a long distance, I get ~3.98Gbps. I know my devices are good as when I switch to a 10G-baseT SFP+ that is 80M I get the full 10Gbps.

So it could be that you have a less than great cable causing the issue. 10GbE needs Cat 6/6E depending on the length. How are the devices connected?

daAndy · Mar 9th 2022

Quote from palerob

Andy,
I have the same on my 10GbE interface on OMV5. It runs fine to another 10GbE host on the same switch. I see something similar to you when attempting to reach a 10GbE devices which on only connected via cat5E over a long distance, I get ~3.98Gbps. I know my devices are good as when I switch to a 10G-baseT SFP+ that is 80M I get the full 10Gbps.

So it could be that you have a less than great cable causing the issue. 10GbE needs Cat 6/6E depending on the length. How are the devices connected?

Thank you for the hint!

All devices are connected via fiber - there is no "normal" cable involved. To the 2 Windows machines it is 15m, to the new OMV6 server 2m and the old server 5m.

conectar · Mar 10th 2022

Quote from daAndy

Hola,

Ya busqué mi problema, pero la mayoría de las veces son módulos no compatibles o un problema con el controlador ixgbe. Para mí, ambos parecen estar bien.

Aquí está el registro de ixgbe:

Code

raíz@XXX:~ # dmesg | grep
[ 0.956277] ixgbe: controlador de red Intel(R) 10 Gigabit PCI Express
[ 0.956280] ixgbe: Copyright (c) 1999-2016 Intel Corporation.
[ 1.121063] ixgbe 0000:07:00.0: multicola habilitada: Recuento de cola Rx = 12, Recuento de cola Tx = 12 Recuento de cola XDP = 0
[ 1.121345] ixgbe 0000:07:00.0: ancho de banda PCIe disponible de 32 000 Gb/s (enlace PCIe x8 de 5,0 GT/s)
[ 1.121464] ixgbe 0000:07:00.0: MAC: 2, PHY: 16, SFP+: 5, PBA No: Desconocido
[1.121465] ixgbe 0000:07:00.0: XX:XX:XX:XX:XX:00
[ 1.142798] ixgbe 0000:07:00.0: Conexión de red Intel(R) 10 Gigabit
[ 1.142910] libphy: ixgbe-mdio: probado
[ 2.329113] ixgbe 0000:07:00.1: multicola habilitada: Recuento de cola Rx = 12, Recuento de cola Tx = 12 Recuento de cola XDP = 0
[ 2.329395] ixgbe 0000:07:00.1: 32.000 Gb/s de ancho de banda PCIe disponible (5.0 GT/s enlace PCIe x8)
[ 2.329514] ixgbe 0000:07:00.1: MAC: 2, PHY: 1, PBA No: Desconocido
[2.329515] ixgbe 0000:07:00.1: XX:XX:XX:XX:XX:01
[ 2.330423] ixgbe 0000:07:00.1: Conexión de red Intel(R) 10 Gigabit
[ 2.330471] libphy: ixgbe-mdio: probado
[ 2.331635] ixgbe 0000:07:00.1 enp7s0f1: renombrado de eth1
[ 2.346244] ixgbe 0000:07:00.0 enp7s0f0: renombrado de eth0
[ 5.165248] ixgbe 0000:07:00.1: dispositivo PHC registrado en enp7s0f1
[6.011916] ixgbe 0000:07:00.0: dispositivo PHC registrado en enp7s0f0
[6.197617] ixgbe 0000:07:00.0 enp7s0f0: detectado SFP+: 5
[ 6.337682] ixgbe 0000:07:00.0 enp7s0f0: el enlace NIC está arriba de 10 Gbps, control de flujo: RX/TX
[550.516867] ixgbe 0000:07:00.0 enp7s0f0: el enlace NIC está inactivo
[ 578.961106] ixgbe 0000:07:00.0 enp7s0f0: NIC Link es de hasta 10 Gbps, control de flujo: RX/TX

Display More

Aquí el "qlen 1000" (creo que este es el problema; de alguna manera se reconoce como 10G en el controlador, también en el interruptor, pero no se usa de alguna manera:

Code

raíz@XXX:~ # ip a
2: enp5s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000
    enlace/éter 0XX:XX:XX:XX:XX:f1 brd ff:ff:ff:ff:ff:ff
    inet XXX.XXX.XXX.192/24 brd XXX.XXX.XXX.255 ámbito global dinámico enp5s0
       válido_lft 84731 seg preferido_lft 84731 seg
3: enp7s0f0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP grupo predeterminado qlen 1000
    enlace/éter XX:XX:XX:XX:XX:00 brd ff:ff:ff:ff:ff:ff
    inet XXX.XXX.XXX.190/24 brd XXX.XXX.XXX.255 alcance dinámica global enp7s0f0
       válido_lft 84731 seg preferido_lft 84731 seg
4: enp7s0f1: <SIN PORTADORA, DIFUSIÓN, MULTIDIFUSIÓN, ARRIBA> mtu 1500 qdisc mq estado ABAJO grupo predeterminado qlen 1000
    enlace/éter XX:XX:XX:XX:XX:01 brd ff:ff:ff:ff:ff:ff

Display More

registro ipfer3:

Code

root@labóveda:~ # iperf3 -s
-------------------------------------------------- ---------
Servidor escuchando en 5201
-------------------------------------------------- ---------
Conexión aceptada desde XXX.XXX.XXX.141, puerto 54564
[ 5] local XXX.XXX.XXX.190 puerto 5201 conectado a XXX.XXX.XXX.141 puerto 54565
[ ID] Tasa de bits de transferencia de intervalo
[ 5] 0.00-1.00 seg 106 MBytes 892 Mbits/seg
[ 5] 1,00-2,00 s 118 MBytes 991 Mbits/s
[ 5] 2,00-3,00 s 117 MBytes 986 Mbits/s
[ 5] 3,00-4,00 s 114 MBytes 955 Mbits/seg
[ 5] 4.00-5.00 seg 118 MBytes 992 Mbits/seg
[ 5] 5,00-6,00 s 112 MBytes 938 Mbits/s
[ 5] 6,00-7,00 s 115 MBytes 967 Mbits/seg
[ 5] 7,00-8,00 s 113 MBytes 944 Mbits/s
[ 5] 8,00-9,00 s 118 MBytes 987 Mbits/s
[ 5] 9.00-10.00 seg 105 MBytes 881 Mbits/seg
[ 5] 10,00-10,04 s 5,05 MBytes 1,02 Gbits/s
- - - - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Tasa de bits de transferencia de intervalo
[ 5] 0.00-10.04 seg 1.11 GBytes 954 Mbits/seg receptor

Display More

Otro registro iperf3 (2 clientes: Windows a Windows):

Code

Connecting to host XXX.XXX.XXX.152, port 5201
[  4] local XXX.XXX.XXX.141 port 54290 connected to XXX.XXX.XXX.152 port 5201
[ ID] Interval           Transfer     Bandwidth
[  4]   0.00-1.00   sec   253 MBytes  2.12 Gbits/sec
[  4]   1.00-2.00   sec   277 MBytes  2.32 Gbits/sec
[  4]   2.00-3.00   sec   273 MBytes  2.29 Gbits/sec
[  4]   3.00-4.00   sec   289 MBytes  2.43 Gbits/sec
[  4]   4.00-5.00   sec   283 MBytes  2.37 Gbits/sec
[  4]   5.00-6.00   sec   302 MBytes  2.53 Gbits/sec
[  4]   6.00-7.00   sec   282 MBytes  2.37 Gbits/sec
[  4]   7.00-8.00   sec   290 MBytes  2.43 Gbits/sec
[  4]   8.00-9.00   sec   286 MBytes  2.40 Gbits/sec
[  4]   9.00-10.00  sec   278 MBytes  2.33 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bandwidth
[  4]   0.00-10.00  sec  2.75 GBytes  2.36 Gbits/sec                  sender
[ 4] 0,00-10,00 s 2,75 GBytes 2,36 Gbits/seg receptor

Display More

El módulo es compatible, el controlador y la tarjeta de red parecen estar bien. La red en general también puede hacer> 1G (me pregunto por qué es solo 2,5G ya que ambos son 10G, pero lo que sea). Tiene el

"qlen 1000" tiene algo que ver con eso o estoy en el camino equivocado?

¡Gracias de antemano!

Saludos

Andy

Display More

conectar · Mar 10th 2022

A mi me habia pasado con un disco de 20GB y dos de 18GB que el OMV me reconocia solo 2GB de c*/u..

Ls Solucione Creando Particion en otra PC con Win10 en formato ReFS... Cree las Carpetas,

Cargue los contenidos Multimedias y Luego los Monte en el NAS OMV y funciono de maravilla..

geaves · Mar 10th 2022

Quote from conectar

A mi me habia pasado con un disco de 20GB y dos de 18GB que el OMV me reconocia solo 2GB de c*/u..
Ls Solucione Creando Particion en otra PC con Win10 en formato ReFS... Cree las Carpetas,
Cargue los contenidos Multimedias y Luego los Monte en el NAS OMV y funciono de maravilla..

This is a networking issue, nothing to do with drives, and please the forum uses English or German

Zoki · Mar 10th 2022

Do you have a 10GBit switch with all ports beeing 10GBit?

daAndy · Mar 10th 2022

Quote from Zoki

Do you have a 10GBit switch with all ports beeing 10GBit?

Yes, I am using an Unifi Aggregation Switch and all ports are on Autonegotiation and they show 10G:

I probably just found my problem. Did some more testing and the two OMV-Servers are reaching 10 GBit when connected to each other. So I went a bit deeper and thought about VLANs and then I found this: https://community.ui.com/quest…a5-48e9-8575-806d7c86d366 (I am not done reading yet, but it seems to be the same: my Aggregation Switch is a L2 Switch).

Everything relevant should be 10G connected:

I need to read that thread first - pretty sure at this point that my problem is not OMV related, but network related! Thank you all for your help

Intel X520 recognized as 10G, but only 1G is used?

daAndy Mar 10th 2022

Participate now!

Tags