OMV6 Crashes when applying changes

1. offizieller Beitrag

    Hi Guys,


    I am re-experiencing a problem that i have had before - there are two other threads that seem to touch on it


    Thema
    OMV6 crashes when mounting filesystem - 502 Gateway Error
    Long-time user of OMV, first-time poster.

    SuperMicro server running OMV6 latest version. I have tried mounting a new drive multiple times and the system always crashes with a 502 Gateway Error when applying changes. The server then is completely fubar and I have to forcibly reboot the server. What gives?

    I can create the filesystem no problem. In this case it was a 2TB NVME drive on an add-in card. I cleaned it first. Then I went to mount it and when I clicked submit changes, the server freaks…
    BigChiefS4


    Thema
    Applying configuration changes breaks system after latest upgrades
    Hello all,

    After upgrading to omv 6.0.41 and sharerootfs 6.0.1 (I'm fairly certain I had applied the config at least once after 6.0.40, but I cannot be sure), applying the config completely crashes my system.

    This morning, after applying the config, I received this notification:
    (Quelltext, 9 Zeilen)


    A bunch of processes then crashed, so ssh and webui access was impossible. I forced a restart with the physical button.

    After the reboot, I tried to apply the config again and experienced the same…
    nonmaterialistically


    I have previously chimed in on both of these.


    This problem has come back and bitten me today - i note that i probably forgot about it/left it after the last round - the server prior to today had been up for 301 days.


    This is remote server and has not had an issue in that time - it just sits there and churns away in the background.




    I went back to my notes and realise i had not solved this the last time and just worked around it.


    I have 2 x USB external drives i wish to add to the system.


    From the CLI i can fdisk to find the disks, then mount them (they have an EXT4 filesystem) with no problems - obviously then OMV knows nothing about them in file system terms. They do show up in the GUI under disks.


    I can mount them at the CLI and write to them on an extended basis (they have approx 1TB of data written to them once per week) as a form of secondary backup storage. They have had this working fine with commandline for the last approx 10 months.


    Today i tried to go back and access them through the GUI and set them up in the OMV way - Filesystem, then Shared Folders to be mounted etc.


    As soon as i do this - i get a series of error messages in the gui - culminating in 502 Bad Gateway.


    The CLI is then unable to do anything with reports of systemctl not running.


    Currently, the content of /var/lib/openmediavault/dirtymodules.json is


    Code
    root@omv-afsydney:/var/lib/openmediavault# ls
apt  borgbackup  cron.d  dirtymodules.json  dirtymodules.json.bkp  engined.sock  fstab_tasks.json  rrd  rsnapshot.d  salt  ssh  workbench
root@omv-afsydney:/var/lib/openmediavault# cat dirtymodules.json
[
    "collectd",
    "fstab",
    "monit",
    "quota",
    "nginx"
]root@omv-afsydney:/var/lib/openmediavault#



    What other information would you like or what can you suggest i do


    I currently have the pending changes notification on the screen


    I had to get someone to do a hard power off of the machine to retake control and the screenshots etc above are after that restart.


    Craig

    Doing some more reading and there are a couple of other threads where people have raised this issue


    Thema
    Applying Pending Configuration Changes Occasionally Causes 502 Bad Gateway Error
    Hello OMV Community,

    The last few times I've installed software updates and received the "Pending configuration changes. You must apply these changes in order for them to take effect." pop-up, I've encountered an annoying issue with trying to apply the changes. When I click "Apply", the update attempt fails with a 502 Bad Gateway, and the system becomes totally unusable. Each time it has happened, I haven't even been able to open a new SSH session. One time when it happened, I had an existing…
    David B


    I am working through the steps in there and will add them to this thread as i go for any one else with this



    Continuuing on with this - i have gone into the plugins section and removed all the optional plugins


    I have rebooted after that and am now trying the whole process of integrating the drives again - still the same thing happening.


    Of interest though - from the CLI before i make any changes i do a shutdown -r 10 - which delays a reboot for 10 minutes. Even though i lose connection to the server and can not do anything at the command line the reboot works and i get control back.


    Craig

    OK still no change as i continue to try things - still getting the error when applying config changes


    Code
    Http failure response for http://192.168.0.40/rpc.php: 502 Bad Gateway

    Only recovery method is either through the power down manual switch or to have previously scheduled a shutdown/reboot from the CLI


    This is what i am doing each time before trying something


    shutdown -r 10


    Seems to work as the box comes back each time but still none of the config changes have been applied.


    Craig

    Thanks for taking the time to respond.


    I have just gone through the following manual actions at the CLI (as i needed it in place for this weekend)


    1) gdisk both USB drives and placed a standard linux fd00 partition on them using all the defaults

    2) ran mkfs.ext4 on both drives and created the filesystems tables and inodes etc

    3) created a subdirectory for each drive to mount to in the export folder (which is where i want them to end up when i get the GUI working)

    4) mounted the /dev/sdx1 partition from both drives in /export/USB-1 and /export/USB-2

    5) manually edited the exports file in /etc and added the correct NFS sharing for the drives

    6) shared the NFS export through exportfs -a


    Went to my ESXi console and successfully mounted both drives as Datastores and am now writing VMware Replication jobs to both drives concurrently - so far each drives has written approx 900GB with no issues.


    I think this proves that all the hardware etc is OK


    I just ran journalctl -f and the only thing of note is some postfix errors in there for a misconfigured forwarder


    I am reluctant to restart the server until Monday once i have these copy jobs finalised.


    Is there something else i can try/provide you with in the meantime ?


    Craig

    • Offizieller Beitrag

    According to that it seems you did not use OMV at anything or am i missing something? You should check the journal log while using the UI to reproduce the error you mentioned above.

    Yes all of those steps i listed above i perfomed from the CLI i.e. no involvement of OMV. After this weekend when the lastest set of replications is done i can redo them through the GUI and will monitor Journactl


    The point of the above steps was to show there are no hardware issues with the drives or the physical setup of the system.


    The UI is currently sitting there saying the following modules need updating



    Will report back as to the outcome of journalctl once i process those changes


    Craig

    Here is the pending configuration changes and this is what happens when i apply them


    And the system then sits there with the 502 bad gateway.


    Any previously established SSH sessions remain open - but can not do anything - however having a shutdown -r 10 command stacked prior to performing the updates does allow the system to restart and returns control.


    Any ideas ?


    Craig

    You have linked my thread and in the meanwhile I have been posting in this other one. Check your fstab_tasks.json to see what it contains.

    I think that's what causes it, I just don't know how to fix the config yet.

    My fstab_tasks.json contains this entry related to deleting the root filesystem:

    My mountpoint config:

    My blkid, that shows that the root filesystem is actually there and mounted on /dev/sdf2:

    Can you check the content of your fstab_tasks.json and post your blkid? Let's see if it's the same issue.

    2 Mal editiert, zuletzt von nonmaterialistically () aus folgendem Grund: I had mistakenly thought that the UUID in fstab_tasks.jon was the disk or partition UUID. The filesystem exists and is my current root filesystem.

    After troubleshooting (and migrating data) all day, I may have found the issue. I had to change the disks in my server and my boot drive is an SSD attached by USB.

    After moving the disks around and booting the server, I found the / filesystem labeled as MISSING in OMV because it was looking for it on /dev/sdf2 (see omv-confdbadm output above). The device was actually /dev/sde2 now.

    I have manually edited the OMV config.xml entry like this:

    To use the UUID instead. What probably happened last time, I instead deleted the FS from the WebUI and mounted it again. So somewhere, OMV wrote that fstab task to delete the FS. By the way, /etc/fstab was already using uuid references.

    Anyway, I home that referencing the disk by uuid means the whole trying-to-delete-/ thing won't happen again.

    Zitat von nonmaterialistically

    You have linked my thread and in the meanwhile I have been posting in this other one. Check your fstab_tasks.json to see what it contains.

    I think that's what causes it, I just don't know how to fix the config yet.

    My fstab_tasks.json contains this entry related to deleting the root filesystem:

    Code

    Can you check the content of your [tt]fstab_tasks.json[/tt] and post your blkid? Let's see if it's the same issue.

    Here is the content of mine - not sure why it is deleting filesystems ?



    Thanks for taking the time to post - i will read through the other thread and see where that leads me also and post back


    Here is my actual fstab


Jetzt mitmachen!

Sie haben noch kein Benutzerkonto auf unserer Seite? Registrieren Sie sich kostenlos und nehmen Sie an unserer Community teil!

Benutzerkonto erstellen Anmelden