nfs causes server to reboot after upgrade

  • Not sure if this is the right place, but I'd like to share my experience with migration from OMV4 to OMV5.

    Initially, my migration failed, because of UPS plugin; thx to Ryecoaaron, we have been able to fix that by uninstalling this ups plugin prior to the migration.

    Then with OMV5, I faced 2 issues:

    #1 erratic reboot of my NAS

    #2 remote mount failed to mount automatically after reboot. I have to mount it manually in the GUI, after each reboot

    As I had Plex, JDownloader, OwnCloud installed on my NAS with OMV4, I suspected they polluted my system and the migration was not so good, so I decided to do a fresh installation of OMV5.

    Thx to that fresh install, issue #2 is fixed: my advise here, as expressed by OMV experts, USE DOCKER :love: As soon as your container is built, it's super easy to run and it does not pollute your base system.

    But issue #1 was still there with 2 or 3 reboot per day. In my syslog, I've seen that just before reboot sequence I get systematically the following error:

    Code
    blkmapd[327]: open pipe file /run/rpc_pipefs/nfs/blocklayout failed: No such file or directory

    I'm using extensively NFS in my home network and I've also noticed that my NAS reboots when my Linux client resumes from hybernate and setups the NFS mount.

    I've found the following thread in the forum: OMV4->5 residue cleanup blkmapd[275]: open pipe file /run/rpc_pipefs/nfs/blocklayout

    It seems others are facing the same issue, but I'm surprised it's not so common as I have a fresh install and it fails systematically.

    As advised in this last thread, I've disabled the nfs-blkmap service by renaming it, and after reboot the issue #1 just vanished; my system is completely stable and I don't get anymore this issue in my syslog.


    My apologies for this long thread... :sleeping: but I'd like to get feedbacks from expert on this issue, at least to know if I did right and also to warn other users.

  • finally I'm still facing erractic reboot of my NAS... ;(

    I don't have any issue with coming back from hybernate with my Ubuntu laptop, but if I reboot the machine, my NAS reboots.

    I confirm this is related with NFS, as the last lines in my syslog are the NFS unmount, then few minutes later it starts the boot sequence without any error message: my ubuntu laptop on his side is not able to mount the NFS disks on my NAS.

    The error related with blkmapd does not show up anymore in my syslog.

    This is related only with OMV5, as I've never got this problem with OMV4.

    Your help will be appreciated.

  • I wonder if this is similar to my issue where my server reboots but the Dell server halts post to tell me that the watchdog service rebooted the server. I thought it was due to my putting the server in uefi mode when I built the os (hardware I migrated from was omv4 on bios) but I've just rebuilt the os drive again switching the server back to bios mode, will monitor see if it reboots.

  • I wonder if this is similar to my issue where my server reboots but the Dell server halts post to tell me that the watchdog service rebooted the server. I thought it was due to my putting the server in uefi mode when I built the os (hardware I migrated from was omv4 on bios) but I've just rebuilt the os drive again switching the server back to bios mode, will monitor see if it reboots.

    so its actually just done this now and restarted all i was doing was watching a film on one of the raspberry pis which connect to the server via nfs (from orginal pi days of nfs was better to use then smb) and the film just hung when i tried any services from the server they was not avalible and the iDrac reports the watchdog timer reset the system, not sure how os monitoring is turned off in the dell bios, but because of this it hangs at post waiting for me to continue the boot. the syslog from the server only shows the below and i cant tell if theres anythign useful there or not. it was around 17:35 time it occoured. the ip it mentions 192.168.2.4 is the pi i was using which runs openelec at the time.


    so defo not a UEFI issue as this is a rebuilt os while it was set to BIOS, so will have to switch that back and rebuild again.


    https://pastebin.com/6uzQ0Ls3

  • I didn't change BIOS settings between OMV4 and OMV5, so I would have been surprised that UEFI mode could generate such issue.

    Same for me, there is nothing in the syslog before the reboot.

    I was also wondering if there is something wrong in my Linux client (Ubuntu) to mount nfs shares with my NAS (like deprecated options), but I'm not using any option: I just specify nfs type, that's it. This config was fine with OMV4.

  • I didn't change BIOS settings between OMV4 and OMV5, so I would have been surprised that UEFI mode could generate such issue.

    Same for me, there is nothing in the syslog before the reboot.

    I was also wondering if there is something wrong in my Linux client (Ubuntu) to mount nfs shares with my NAS (like deprecated options), but I'm not using any option: I just specify nfs type, that's it. This config was fine with OMV4.

    Same with omv4 was fine. The only devices j use that use NFS is the Kodi pis. The rest of my devices are windows devices using SMB.


    I have noticed tonight after the reboot thought my raid is reporting active: resyncing. But as far as I can see the drives are all good smart says passed on each drive. Not sure if it's done this as I told drac to cold reboot instead of turn off and on when the error happened.

  • same for me all disks are clean.

    I'm currently wondering if there is no side effects with Docker.

    My RAID5 volume contains all the NFS shares. With OMV5, I've created multiple docker containers, which mount volumes within the RAID5; I didn't use Docker with OMV4, and that's the only change in my config since the upgrade. In particular, config directories for my docker containers are in my RAID5; I'm currently moving all my config folders outside of RAID5 to see if it changes something. I'll keep you posted.

  • same for me all disks are clean.

    I'm currently wondering if there is no side effects with Docker.

    My RAID5 volume contains all the NFS shares. With OMV5, I've created multiple docker containers, which mount volumes within the RAID5; I didn't use Docker with OMV4, and that's the only change in my config since the upgrade. In particular, config directories for my docker containers are in my RAID5; I'm currently moving all my config folders outside of RAID5 to see if it changes something. I'll keep you posted.

    Ive been using dockers for sometime now on omv 4 and 5 maybe 3 so I doubt it's a docker issue.

  • I confirm it does not change anything. My NAS still reboots from time to time when my ubuntu laptop resumes from hybernate or restart.

    Last crash I had, after my Ubuntu resumes from hybernate, I get in syslog information that there is an autenthicated munt request received by rpc.mountd, and after 3min the reboot sequence starts (there is nothing in between).

    I'm really stuck; knowing this is a fresh install, there is nothing in the logs, I really don't know what to do here...

    Any idea?

  • I confirm it does not change anything. My NAS still reboots from time to time when my ubuntu laptop resumes from hybernate or restart.

    Last crash I had, after my Ubuntu resumes from hybernate, I get in syslog information that there is an autenthicated munt request received by rpc.mountd, and after 3min the reboot sequence starts (there is nothing in between).

    I'm really stuck; knowing this is a fresh install, there is nothing in the logs, I really don't know what to do here...

    Any idea?

    If you want your problem to get the visibility it deserves I suggest you open a new thread in the appropriate support section. This thread has drifted so far off topic that the people who could help you probably no longer read it.

    --
    Google is your friend and Bob's your uncle!


    OMV AMD64 5.x on ASRock Rack C2550D4I C0 Stepping - 16GB ECC - Silverstone DS380 + Silverstone DS380 DAS Box.

  • Since my comments above have been extracted from another thread, I give below the full details of the NFS issues I'm facing...



    After running OMV4 since years, it was time for me to migrate to OMV5 and implement Docker for Plex, Jdownloader and Duplicati.

    I started with an upgrade from OMV4 to OMV5, but I was facing many unstabilities, so I did a fresh install last week end.

    My configuration is the following: 1 system disk + 1 RAID5 volume (4 disks 2TB). I've installed Docker with the 3 containers for Plex, duplicati, jdownloader. I'm using the plugins backup and RemoteMount (NFS mount of Synology array to store rsnapshot images of my RAID5 volume).

    I'm using extensively NFS shares with my RAID5 volume; they are mounted by Ubuntu 18.04 clients on my internal network.

    I emphasize that (except Docker) I had exactly the same configuration with OMV4 and it was very stable.

    Since I moved to OMV5, my NAS reboots regularly (2-3 times a day) and I've found out that it happens when my Ubuntu clients resumes from hybernate or restart; they are mounting the NFS shares and it causes the NAS to reboot.


    1/ Looking at the syslog, I've found out that there is an error message caused by blkmapd service, juste when my ubuntu clients tries to mount the NFS shares:

    Code

    Code
    Jul 25 21:01:56 Gemenos blkmapd[327]: open pipe file /run/rpc_pipefs/nfs/blocklayout failed: No such file or directory

    then it seems there are tentative to recover the system and finally system reboots. You have the syslog extract with few comments in file syslog20200725.txt.

    syslog20200725.txt

    2/ I decided to get rid of blkmapd service (the thread OMV4->5 residue cleanup blkmapd[275]: open pipe file /run/rpc_pipefs/nfs/blocklayout relates about this issue and seems to say that this service is not required).

    Since then, I do not get any error message in the syslog, my system just reboots few minutes after my ubuntu client tries to mount (unsuccessfully) the NFS shares.

    You will find the syslog with my comments in file syslog20200726.txt.

    syslog20200726.txt


    Honestly, I'm completely stuck as I don't find any error or issue in the logs.

    I wish someone can guide me shortly to identify the root cause of this issue, as I need to rely on my NAS for all my personal data.


    Thx in advance for your support.

  • Im also seeing issues where server is rebooting randomly all I can see in the logs is messages relating to my raspberry pi librelec units they currently connect to the server using NFS. As a test I'm currently changing all my connections on them from NFS to SMB and then turning NFS off on the server to see what happens.

  • this is what I did also today as I don't see other option to increminate NFS... I've kept NFS shares on OMV, but I've removed my NFS mounts on my Ubuntu client and I'm using Samba.

    So far so good, today no reboot. I'll see in the coming days if this is confirmed.

  • this is what I did also today as I don't see other option to increminate NFS... I've kept NFS shares on OMV, but I've removed my NFS mounts on my Ubuntu client and I'm using Samba.

    So far so good, today no reboot. I'll see in the coming days if this is confirmed.

    Good to know I've changed the paths so far on the pis there currently rescanning to the new paths annoyingly. My issue isn't as excessive as yours mine can be a few days to a week before it does it the soonest was 1-2 days.

  • zevleu

    Added the Label OMV 5.x
  • this is what I did also today as I don't see other option to increminate NFS... I've kept NFS shares on OMV, but I've removed my NFS mounts on my Ubuntu client and I'm using Samba.

    So far so good, today no reboot. I'll see in the coming days if this is confirmed.

    Interesting. Let me see if I'm understanding this correctly. You're accessing NFS shares as Samba (Windows shares) at the Ubuntu client?

  • yes I've maintained NFS shares active in OMV, but don't use them anymore. I'm using Samba instead.

    Since 2 days, no more crash so the issue is clearly coming from NFS. Maybe there is a conflict between NFS client on my Ubuntu 18 and the NFS server in OMV...

  • Yes I have both NFS and Samba shares enabled on OMV.

    I'm using mainly Ubuntu clients and they were (until now) connected with OMV using NFS. I was using Samba only when I have to use Windows from time to time.

    Now, I'm not mounting anymore the NFS shares on my Ubuntu Client, and I'm using the Samba Shares only.

    Since then, OMV does not reboot erratically.

  • I use SMB for my Android devices. And NFS between my OMV servers and between OMV servers and Ubuntu Clients.


    The NFS shares are all created in OMV, but I mount all the NFS shares in the network, on all the Linux clients and servers, using autofs and on my Ubuntu MATE laptop I also cache NFS using fscache and 128GB NVMe storage.


    Works fine.

    Be smart - be lazy. Clone your rootfs.
    OMV 5: 9 x Odroid HC2 + 1 x Odroid HC1 + 1 x Raspberry Pi 4

Participate now!

Don’t have an account yet? Register yourself now and be a part of our community!