Applying Pending Configuration Changes Occasionally Causes 502 Bad Gateway Error

1st Official Post

    Hello OMV Community,


    The last few times I've installed software updates and received the "Pending configuration changes. You must apply these changes in order for them to take effect." pop-up, I've encountered an annoying issue with trying to apply the changes. When I click "Apply", the update attempt fails with a 502 Bad Gateway, and the system becomes totally unusable. Each time it has happened, I haven't even been able to open a new SSH session. One time when it happened, I had an existing SSH session opened, but trying to issue shutdown now resulted in an error. The only way for me to get around this issue has been to outright crash the computer, reboot, and abort the change request. This would be tolerable if it was a one-off issue, but since it's quite common now, it's annoying and I would like to properly fix it. Has anyone else seen this, or know what I can do to stop it?

    Hi David,

    I faced the same problem with the last 3 updates.

    Applying configuration changes after the update brings me to an unusable system. 502 Bad gateway error from the web interface. No more ssh access.

    Rebooting and "undoing" the changes fixed it.

    But actually no revert was done, the required update was still present. And working fine.

    With the latest update (to version 6.0.43-1) I tried directly to select "undo changes" just after the update completion and this just closed the message. The system is updated. No more messages, no errors, no problemes.


    Maybe some more experienced user has an explanation for this behaviour?

    Seems to me... not very logical.

    FD Node 304, BN207, ASRock J3455-itx, 16GB RAM, OMV5

    I have the same problem on my installation and it is not nice that during pending commits there are tries to unmount "/". Why? Only Power Cycle is possible after this, because "/" is unmounted and nothing is possible to do after this, because everything is "gone".


    omv-rpc -u admin "Config" "applyChanges" "{\"modules\": $(cat /var/lib/openmediavault/dirtymodules.json), \"force\": true}"

    {"response":null,"error":{"code":0,"message":"Removing the directory '\/' has been aborted, the resource

    is busy.","trace":"OMV\\Exception: Removing the directory '\/' has been aborted, the resource is busy. in \/usr\/share\/openmediavault\/engined\/module\/fstab.inc:65\nStack trace:\n#0 [internal function]: Engined\\Module\\FSTab->deleteEntry(Array)\n#1 \/usr\/share\/php\/openmediavault\/engine\/module\/moduleabstract.inc(157): call_user_func_array(Array, Array)\n#2 \/usr\/share\/openmediavault\/engined\/module\/fstab.inc(31): OMV\\Engine\\Module\\ModuleAbstract->execTasks('delete')\n#3 \/usr\/share\/openmediavault\/engined\/rpc\/config.inc(167): Engined\\Module\\FSTab->preDeploy()\n#4 [internal function]: Engined\\Rpc\\Config->applyChanges(Array, Array)\n#5 \/usr\/share\/php\/openmediavault\/rpc\/serviceabstract.inc(123): call_user_func_array(Array, Array)\n#6 \/usr\/share\/php\/openmediavault\/rpc\/rpc.inc(86): OMV\\Rpc\\ServiceAbstract->callMethod('applyChanges', Array, Array)\n#7 \/usr\/sbin\/omv-engined(537): OMV\\Rpc\\Rpc::call('Config', 'applyChanges', Array, Array, 1)\n#8 {main}"}}

    The problem exists during the last three updates of omv. Before there where no problems. If I do "revert changes" this works, but what is done, because I commited no changes for now for the last three times as nothing is shown, why some sort of remount is done.

    • Official Post
    Quote from Peppa123

    I have the same problem on my installation and it is not nice that during pending commits there are tries to unmount "/". Why? Only Power Cycle is possible after this, because "/" is unmounted and nothing is possible to do after this, because everything is "gone".


    omv-rpc -u admin "Config" "applyChanges" "{\"modules\": $(cat /var/lib/openmediavault/dirtymodules.json), \"force\": true}"

    {"response":null,"error":{"code":0,"message":"Removing the directory '\/' has been aborted, the resource

    is busy.","trace":"OMV\\Exception: Removing the directory '\/' has been aborted, the resource is busy. in \/usr\/share\/openmediavault\/engined\/module\/fstab.inc:65\nStack trace:\n#0 [internal function]: Engined\\Module\\FSTab->deleteEntry(Array)\n#1 \/usr\/share\/php\/openmediavault\/engine\/module\/moduleabstract.inc(157): call_user_func_array(Array, Array)\n#2 \/usr\/share\/openmediavault\/engined\/module\/fstab.inc(31): OMV\\Engine\\Module\\ModuleAbstract->execTasks('delete')\n#3 \/usr\/share\/openmediavault\/engined\/rpc\/config.inc(167): Engined\\Module\\FSTab->preDeploy()\n#4 [internal function]: Engined\\Rpc\\Config->applyChanges(Array, Array)\n#5 \/usr\/share\/php\/openmediavault\/rpc\/serviceabstract.inc(123): call_user_func_array(Array, Array)\n#6 \/usr\/share\/php\/openmediavault\/rpc\/rpc.inc(86): OMV\\Rpc\\ServiceAbstract->callMethod('applyChanges', Array, Array)\n#7 \/usr\/sbin\/omv-engined(537): OMV\\Rpc\\Rpc::call('Config', 'applyChanges', Array, Array, 1)\n#8 {main}"}}

    IMO this is caused by a misconfigured mount point. I assume you are using the openmediavault-sharerootfs plugin. The latest plugin version has fixed that problem. Do you have installed it?


    Please post the output of

    Code
    # omv-confdbadm read --prettify conf.system.filesystem.mountpoint

    Here is the output:


    [

    {

    "comment": "Seagate Festplatte 4TB",

    "dir": "/srv/dev-disk-by-uuid-94537b05-9875-4501-9a8c-3aa6cbda18db",

    "freq": 0,

    "fsname": "/dev/disk/by-uuid/94537b05-9875-4501-9a8c-3aa6cbda18db",

    "hidden": false,

    "opts": "defaults,nofail,user_xattr,usrjquota=aquota.user,grpjquota=aquota.group,jqfmt=vfsv0,acl",

    "passno": 2,

    "type": "ext4",

    "usagewarnthreshold": 95,

    "uuid": "86d9bed5-fbb2-49ad-a7ce-4e773428f874"

    },

    {

    "comment": "Samsung EVO SSD 2 TB",

    "dir": "/srv/dev-disk-by-uuid-35b74cc0-e206-49a2-ab0a-eb64c2e2d317",

    "freq": 0,

    "fsname": "/dev/disk/by-uuid/35b74cc0-e206-49a2-ab0a-eb64c2e2d317",

    "hidden": false,

    "opts": "defaults,nofail,user_xattr,usrjquota=aquota.user,grpjquota=aquota.group,jqfmt=vfsv0,acl",

    "passno": 2,

    "type": "ext4",

    "usagewarnthreshold": 90,

    "uuid": "ec518a26-7a45-4577-9fb4-42d352b089eb"

    },

    {

    "comment": "WD SATA 5TB",

    "dir": "/srv/dev-disk-by-uuid-cd15dd88-5d5b-4390-8553-91abadbeb1cb",

    "freq": 0,

    "fsname": "/dev/disk/by-uuid/cd15dd88-5d5b-4390-8553-91abadbeb1cb",

    "hidden": false,

    "opts": "defaults,nofail,user_xattr,usrjquota=aquota.user,grpjquota=aquota.group,jqfmt=vfsv0,acl",

    "passno": 2,

    "type": "ext4",

    "usagewarnthreshold": 95,

    "uuid": "6286d700-1285-47c5-9694-cfc18069c416"

    },

    {

    "comment": "",

    "dir": "/export/Filme",

    "freq": 0,

    "fsname": "/srv/dev-disk-by-uuid-cd15dd88-5d5b-4390-8553-91abadbeb1cb/Filme/",

    "hidden": false,

    "opts": "bind,nofail",

    "passno": 0,

    "type": "none",

    "usagewarnthreshold": 0,

    "uuid": "af6c19b3-a056-4ae9-a52b-5f8587a17758"

    },

    {

    "comment": "",

    "dir": "/",

    "freq": 0,

    "fsname": "/dev/disk/by-uuid/4fee3c4a-d782-47f1-8972-ef8ba821ac52",

    "hidden": true,

    "opts": "errors=remount-ro",

    "passno": 1,

    "type": "ext4",

    "usagewarnthreshold": 0,

    "uuid": "79684322-3eac-11ea-a974-63a080abab18"

    }

    ]

    • Official Post

    Why do you try to delete the root filesystem via UI? Could you please tell me how this was possible? I'm asking because this should never be possible because the UI disables the delete button for the root filesystem. I'm asking to understand how it was possible to bypass this security measure.


    Could you please post the output of


    Code
    # omv-rpc -u admin 'FileSystemMgmt' 'getList' '{"start":0,"limit":-1}' | jq

    Actually I am doing and did nothing about deleting the root filesystem. I also don't know why during "pending tasks" there is a job trying to dismount the root filesystem. The only things I did was testing with the "kvm" plugin and this was weeks before. And the "kvm" plugin is installing the "shareroot" plugin, so deinstalling it, also deinstalls the "kvm" plugin.


    Here is the output:

    And here are the open tasks. I changed the root disks some weeks before and duplicated the installation to the new disk. Before it was a 300 GB Western Digital Hard Drive and now it is a Kingston NVMe Disk.


    As I see there is a tasks that tries to delete the entry "/", but the corresponding UUID does not exists anymore and is now different for the "/" entry.


    I am not sure, but after changing and cloning the root disk there was an old entry shown in the OMV GUI and I think I tried to delete this entry. Maybe this is a hint and this happens not often to change the installation disk.

    Quote from votdev

    IMO this is caused by a misconfigured mount point. I assume you are using the openmediavault-sharerootfs plugin. The latest plugin version has fixed that problem. Do you have installed it?


    Please post the output of

    Code
    # omv-confdbadm read --prettify conf.system.filesystem.mountpoint

    I do indeed use openmediavault-sharerootfs! In case it helps provide any extra information to you, here's my output from running the command:


    And omv-rpc -u admin 'FileSystemMgmt' 'getList' '{"start":0,"limit":-1}' | jq: https://pastebin.com/5BmYh0US


    The forum was complaining about the character limit when I tried to copy it in to a code block, hence the use of Pastebin.

Participate now!

Don’t have an account yet? Register yourself now and be a part of our community!

Register Yourself Login