Using additional ethernet ports for switching

    Hi :)


    I've bought a Topton Fanless Mini PC at AliExpress. It's based on the Celeron N5105, it has 4x2.5Gb ethernet ports and usually it's meant for a pfsense/opnsense firewall but I've decided to use it as a small NAS running 24/7.


    Because the NAS only needs a single ethernet port and 2.5Gb switches are still rather pricy, I thought I could use the additional 3 ports for switching. After some digging around I've found that I need to bridge all the ports together and I was able to do that through OMV UI.


    The NAS and its shares are still running fine but unfortunately when I plug in a different PC to one of the previously unused ports, it does not recieve an IP address from DHCP. If I assign a static IP to that PC, I can communicate with the NAS but not with any other machines in the network.


    I don't know what I'm doing incorrectly, so I would be grateful for any help or pointers into how I can achieve my goal :)


    Best regards!

    If your bridge is configured to get an IP via DHCP is up, and all four NICs are up and are members of the bridge, then I believe this should work as desired. Please post the output of using "ip a" at the CLI and the contents of the bridge's associated netplan yaml file under "/etc/netplan".

    Krisbee: I'm very sorry for the late response but unfortunately I didn't get a notifcation... The bridge is configured to have a static IP but I did also try with DHCP, with no luck...


    Here is the result of ip a:

    And here are the two files in /etc/netplan:

    * /etc/netplan/10-openmediavault-default.yaml


    Code
    network:
  version: 2
  renderer: networkd


    * /etc/netplan/60-openmediavault-br0.yaml



    I'm also attaching screenshots from the UI, I hope this information will be useful as I've run out of ideas...

    Krisbee Yes, there is a DHCP server running on the router and all the other machines on the local network get their IPs just fine...


    It looks like something is blocking passing the network packets through the bridge - as I said, if I set a static IP (say 192.168.0.4) I can communicate with 192.168.0.5 (the mini pc) but not with 192.168.0.1 (the router).

    Stupid question: Did you enable package forwarding?

    If you got help in the forum and want to give something back to the project click here (omv) or here (scroll down) (plugins) and write up your solution for others.

    OK, so I had yet another senior moment. A linux bridge acting as a switch is a layer 2 device, so it's using mac address to forward traffic. In which case, the bridge doesn't need an IP. An IP address is required if you want your bridge to take part in layer 3 routing of IP packets and you would enable IP forwarding in /etc/sysctl.conf. So is OMV6 working as a layer3 switch in this case?

    You can not make a non switch a switch. You will have to use L3 switching / routing.

    If you got help in the forum and want to give something back to the project click here (omv) or here (scroll down) (plugins) and write up your solution for others.

    Krisbee  Zoki Hi and thank you for you responses :) Would you please care to explain more precisely what I should try? I'm not very good at networking, but from what I've read an ethernet bridge (layer 2) is better because it's basically transparent to the netwoirking protocols, while ip forwarding (layer 3) isn't.


    I'm not sure if this is what you mean by "enabling ip forwarding" but from sysctl -a:

    Code
    net.ipv4.ip_forward = 1


    Something else that might be useful... I've found following commands to enable and configure a bridge from command line:

    Code
    # ip link add br0 type bridge
# ip link set enp3s0 master br0
# ip link set enp4s0 master br0
# ip link set enp5s0 master br0
# ip link set enp6s0 master br0
# dhclient br0


    First I've tried it with a Live USB of Manjaro Linux and it worked! Machines connected to the 3 additional ports would receive IP addresses from DHCP. ip addr was showing:


    Then I've launched OMV, removed all interfaces from UI and run the very same commands. Unfortunately, this behaved the same way as a bridge configured through the UI - machines would not receive IPs from DHCP. Results of ip addr in OMV:

    The only real difference that I can see are the 2 additional Docker related interfaces. Maybe this is what's causing problems? Or maybe there's some other configuration difference between OMV and Manjaro?

    I missread what you want, you are trying to put all ethernet cards into one brridge, not all but one and use OMV as router.

    For this your config looks ok, but all but enp3s0 are DOWN.


    If I read #1 again you say: " If I assign a static IP to that PC, I can communicate with the NAS but not with any other machines in the network." So your problem is not DHCP, but general connectivity. On starup the dhcpo client sends a broadcast message to the network DHCPDISCOVER and expects to get an answer from the dhcp server. If these packages are not exchanged, the client does not get an IP.


    Can you sho the output of brctl show and brctl showmacs br0


    If you got help in the forum and want to give something back to the project click here (omv) or here (scroll down) (plugins) and write up your solution for others.

    Crocodil Ports enp4s0 to enp6s0 will be down if nothing is plugged into them . But have you tested each port individually?


    The brtcl command is part of the bridge-utils package which IIRC is not installed on OMV6 by default.


    The ip link commands are ip link show type bridge and ip link show master <bridgename>


    I can only say when I've tried to create a bridge within OMV6 with no IP by setting the method to disabled, it screwed things up . Yet to sort this out. A bridge with an IP is not a problem.

    Zoki  Krisbee Hi guys :) I'm sorry for the late response but I had little time and I wanted to try a few things...


    So try I did and things look even more complicated :) When the additional PC (PC1) is connected to any of the ports when the "Topton" is booting, the bridge works and DHCP assigns an IP number. When I connect another machine after boot (PC2) - it doesn't pickup an IP. Same when I disconnect and reconnect the PC1, regardless if I use the same port or a different one. This looks like either a physcial problem, a kernel bug or some power saving problem.


    Here are the requested commands and their outputs (enp3s0 is the "uplink", enp4s0 is PC1 that gets the connection on boot and enp6s0 is PC2 that I connected after boot and doesn't get a connection).



    I don't know if this is of any use but here is the output of dmesg, when I plug in the second PC (after reboot):

    Code
    [  331.330254] igc 0000:06:00.0 enp6s0: NIC Link is Up 1000 Mbps Full Duplex, Flow Control: RX/TX
[  331.440487] br0: port 1(enp6s0) entered blocking state
[  331.440510] br0: port 1(enp6s0) entered forwarding state

    Crocodil Those messages re: blocking/forwarding state are normal. You want all bridge ports to be in a forwarding state. As you have installed bridge-utils, use the brctl showstp br0 command to check. For example:



    What puzzles me is that things worked with your Manjaro test, which logically tends to discount your idea about power saving. True it has probably got a more recent kernel than OMV6 and possibly more up to date drivers for those 2.5gb ethernet ports, something you could easily verify. When you say PC2 doesn't get an IP as above, does a ping get a response?


    In any test I can do, OMV6 acting as bridge works as expected.


    PS I forgot to say before, this not something daft like running out of DHCP leases?

    Einmal editiert, zuletzt von Krisbee () aus folgendem Grund: addendum

    Krisbee I've checked and in fact the latest OMV 6 has a kernel newer than the Manjaro Live USB that I've used (5.16 vs 5.15). PC2 cannot ping anything when trying to get IP from DHCP (it has no IP after all) and when I assign a static IP, I can only ping the topton mini pc.


    I did play around a little with a fresh OMV installation and the bridge seemed to be working fine. I think that br0 stopped passing traffic when I installed Docker (from OMV Extras) and it added its own interfaces, including its own bridge...


    Code
    8: docker0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN group default 
    link/ether 02:42:03:d9:67:ee brd ff:ff:ff:ff:ff:ff
    inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0
       valid_lft forever preferred_lft forever
9: br-ec3631db49d9: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN group default 
    link/ether 02:42:3c:ca:2a:48 brd ff:ff:ff:ff:ff:ff
    inet 172.18.0.1/16 brd 172.18.255.255 scope global br-ec3631db49d9
       valid_lft forever preferred_lft forever

    It's difficult to verify this 100% as I don't know how to completely remove docker. Clicking on "Remove" in UI does nothing.


    Do you know if this can somehow be fixed? I would prefer to keep using Docker ;)

    Crocodil Installing docker should not cause a problem in itself. The other bridge is an additional network device created by whatever container you are running. You can list all the docker networks using  docker network ls  at the CLI. Stop/remove that container and observe what happens.


    To completely remove docker, use apt at the CLI: apt remove --purge docker-ce docker-ce-cl

    Zoki The UI shows no firewall rules and here is the result of iptables -L:

    what happens when you stop the docker daemon with

    systemctl stop docker


    Does the bridge start working?

    If you got help in the forum and want to give something back to the project click here (omv) or here (scroll down) (plugins) and write up your solution for others.

    Crocodil I think the stuff about docker is a red herring. After reading again your #12 & #14 I think you should check the 2.5Gb nic drivers. Are they intel i-225 models? People appear to have had various problems with these, as a quick google would show.


    Install the ethtool package, and ethool -i enp3s0 will show you the driver and version in use on OMV6.


    Incidently, what OS is running on your PCs and are they equipped with 2.5Gbe or 1GBe nic?

    Einmal editiert, zuletzt von Krisbee () aus folgendem Grund: addendum

Jetzt mitmachen!

Sie haben noch kein Benutzerkonto auf unserer Seite? Registrieren Sie sich kostenlos und nehmen Sie an unserer Community teil!

Benutzerkonto erstellen Anmelden