systemctl restart srv-mergerfs-Red.mount' with exit code '1'

Ranieri94 · 17. April 2023

Hi,

I run OMV 6.3.8-1 (Shaitan) with kernel Linux 6.1.0-0.deb11.5-amd64 and as the title says, after an upgrade, mergerfs stopped working and exited with the following error code:

Code

Failed to execute command 'export PATH=/bin:/sbin:/usr/bin:/usr/sbin:/usr/local/bin:/usr/local/sbin; export LANG=C.UTF-8; export LANGUAGE=; systemctl restart srv-mergerfs-Red.mount' with exit code '1': 

OMV\ExecException: Failed to execute command 'export PATH=/bin:/sbin:/usr/bin:/usr/sbin:/usr/local/bin:/usr/local/sbin; export LANG=C.UTF-8; export LANGUAGE=; systemctl restart srv-mergerfs-Red.mount' with exit code '1':  in /usr/share/php/openmediavault/system/process.inc:220
Stack trace:
#0 /usr/share/openmediavault/engined/rpc/mergerfs.inc(197): OMV\System\Process->execute(Array, 1)
#1 [internal function]: OMVRpcServiceMergerfs->restartPool(Array, Array)
#2 /usr/share/php/openmediavault/rpc/serviceabstract.inc(123): call_user_func_array(Array, Array)
#3 /usr/share/php/openmediavault/rpc/rpc.inc(86): OMV\Rpc\ServiceAbstract->callMethod('restartPool', Array, Array)
#4 /usr/sbin/omv-engined(537): OMV\Rpc\Rpc::call('Mergerfs', 'restartPool', Array, Array, 1)
#5 {main}

I red here in the forum similar issiues, related to the not-empty mouting point. I tried to manage to cancel the files/folder in the mounting path and refresh the perimssions, but had no luck. I don't really know exactly what I tried, because time has passed

So... after about a month full of work, my setup is still bricked because about every service relay on the mergerfs drive. The drives work fine and an other thing I noticed that isn't normal is the missing file system in the dashboard (see attachment).

Hope someone can help me figure out what's the problem.

ryecoaaron · 17. April 2023

What is the output of:

dpkg -l | grep mergerfs

sudo journalctl -u srv-mergerfs-Red.mount

Ranieri94 · 20. April 2023

Hi,

that's the output

Code

root@fenice:~# dpkg -l | grep mergerfs

ii  mergerfs                               2.35.1~debian-bullseye                                                                     amd64        another FUSE union filesystem
ii  openmediavault-mergerfs                6.3.5                                                                                      all          mergerfs plugin for openmediavault.
root@fenice:~# sudo journalctl -u srv-mergerfs-Red.mount
-- Journal begins at Thu 2023-04-20 09:48:48 CEST, ends at Thu 2023-04-20 16:33:30 CEST. --
-- No entries --

ryecoaaron · 20. April 2023

You need to try to restart the mount again so that something might be in journalctl

sudo systemctl restart srv-mergerfs-Red.mount

sudo journalctl -u srv-mergerfs-Red.mount

Ranieri94 · 22. April 2023

Something went wront 😔

Code

sudo systemctl restart srv-mergerfs-Red.mount

sudo journalctl -u srv-mergerfs-Red.mount
Job failed. See "journalctl -xe" for details.
-- Journal begins at Sat 2023-04-22 14:09:22 CEST, ends at Sat 2023-04-22 20:48>
apr 22 20:46:46 fenice systemd[1]: srv-mergerfs-Red.mount: Directory /srv/merge>
apr 22 20:46:46 fenice systemd[1]: Mounting MergerFS mount for Red...
apr 22 20:46:46 fenice mount[387559]: fuse: unknown option ` allow_other'
apr 22 20:46:46 fenice systemd[1]: srv-mergerfs-Red.mount: Mount process exited>
apr 22 20:46:46 fenice systemd[1]: srv-mergerfs-Red.mount: Failed with result '>
apr 22 20:46:46 fenice systemd[1]: Failed to mount MergerFS mount for Red.
apr 22 20:46:55 fenice systemd[1]: srv-mergerfs-Red.mount: Directory /srv/merge>
apr 22 20:46:55 fenice systemd[1]: Mounting MergerFS mount for Red...
apr 22 20:46:55 fenice mount[387570]: fuse: unknown option ` allow_other'
apr 22 20:46:55 fenice systemd[1]: srv-mergerfs-Red.mount: Mount process exited>
apr 22 20:46:55 fenice systemd[1]: srv-mergerfs-Red.mount: Failed with result '>
apr 22 20:46:55 fenice systemd[1]: Failed to mount MergerFS mount for Red.
apr 22 20:48:29 fenice systemd[1]: srv-mergerfs-Red.mount: Directory /srv/merge>
apr 22 20:48:29 fenice systemd[1]: Mounting MergerFS mount for Red...
apr 22 20:48:29 fenice mount[387633]: fuse: unknown option ` allow_other'
apr 22 20:48:29 fenice systemd[1]: srv-mergerfs-Red.mount: Mount process exited>
apr 22 20:48:29 fenice systemd[1]: srv-mergerfs-Red.mount: Failed with result '>
apr 22 20:48:29 fenice systemd[1]: Failed to mount MergerFS mount for Red.
lines 1-19/19 (END)...skipping...
-- Journal begins at Sat 2023-04-22 14:09:22 CEST, ends at Sat 2023-04-22 20:48:29 CEST. --
apr 22 20:46:46 fenice systemd[1]: srv-mergerfs-Red.mount: Directory /srv/mergerfs/Red to mount over is not empty, mounting anyway.
apr 22 20:46:46 fenice systemd[1]: Mounting MergerFS mount for Red...
apr 22 20:46:46 fenice mount[387559]: fuse: unknown option ` allow_other'
apr 22 20:46:46 fenice systemd[1]: srv-mergerfs-Red.mount: Mount process exited, code=exited, status=1/FAILURE
apr 22 20:46:46 fenice systemd[1]: srv-mergerfs-Red.mount: Failed with result 'exit-code'.
apr 22 20:46:46 fenice systemd[1]: Failed to mount MergerFS mount for Red.
apr 22 20:46:55 fenice systemd[1]: srv-mergerfs-Red.mount: Directory /srv/mergerfs/Red to mount over is not empty, mounting anyway.
apr 22 20:46:55 fenice systemd[1]: Mounting MergerFS mount for Red...
apr 22 20:46:55 fenice mount[387570]: fuse: unknown option ` allow_other'
apr 22 20:46:55 fenice systemd[1]: srv-mergerfs-Red.mount: Mount process exited, code=exited, status=1/FAILURE
apr 22 20:46:55 fenice systemd[1]: srv-mergerfs-Red.mount: Failed with result 'exit-code'.
apr 22 20:46:55 fenice systemd[1]: Failed to mount MergerFS mount for Red.
apr 22 20:48:29 fenice systemd[1]: srv-mergerfs-Red.mount: Directory /srv/mergerfs/Red to mount over is not empty, mounting anyway.
apr 22 20:48:29 fenice systemd[1]: Mounting MergerFS mount for Red...
apr 22 20:48:29 fenice mount[387633]: fuse: unknown option ` allow_other'
apr 22 20:48:29 fenice systemd[1]: srv-mergerfs-Red.mount: Mount process exited, code=exited, status=1/FAILURE
apr 22 20:48:29 fenice systemd[1]: srv-mergerfs-Red.mount: Failed with result 'exit-code'.
apr 22 20:48:29 fenice systemd[1]: Failed to mount MergerFS mount for Red.

Alles anzeigen

ryecoaaron · 22. April 2023

It should ignore allow_other but I would try editing your pool and removing that option. Also post the output of: dpkg -l | grep mergerfs

Ranieri94 · 22. April 2023

So, I edited the pool's option, removing "allow_other" and tried to save, but exited with the following error:

Code

Failed to execute command 'export PATH=/bin:/sbin:/usr/bin:/usr/sbin:/usr/local/bin:/usr/local/sbin; export LANG=C.UTF-8; export LANGUAGE=; omv-salt deploy run --no-color mergerfs 2>&1' with exit code '1': debian:
----------
          ID: configure_pool_dir
    Function: file.directory
        Name: /srv/mergerfs
      Result: True
     Comment: The directory /srv/mergerfs is in the correct state
     Started: 21:27:22.001779
    Duration: 10.571 ms
     Changes:
----------
          ID: remove_mergerfs_mount_files_/etc/systemd/system
    Function: module.run
      Result: True
     Comment: file.find: ['/etc/systemd/system/srv-mergerfs-Red.mount']
     Started: 21:27:22.013482
    Duration: 8.047 ms
     Changes:
              ----------
              file.find:
                  - /etc/systemd/system/srv-mergerfs-Red.mount
----------
          ID: systemd_remove_dead_symlinks
    Function: cmd.run
        Name: find /etc/systemd/system/multi-user.target.wants -xtype l -print -delete
      Result: True
     Comment: Command "find /etc/systemd/system/multi-user.target.wants -xtype l -print -delete" run
     Started: 21:27:22.022716
    Duration: 11.907 ms
     Changes:
              ----------
              pid:
                  4501
              retcode:
                  0
              stderr:
              stdout:
                  /etc/systemd/system/multi-user.target.wants/srv-mergerfs-Red.mount
----------
          ID: configure_mergerfs_Red
    Function: file.managed
        Name: /etc/systemd/system/srv-mergerfs-Red.mount
      Result: True
     Comment: File /etc/systemd/system/srv-mergerfs-Red.mount updated
     Started: 21:27:22.035220
    Duration: 151.895 ms
     Changes:
              ----------
              diff:
                  New file
              mode:
                  0644
----------
          ID: systemd-reload_Red
    Function: cmd.run
        Name: systemctl daemon-reload
      Result: True
     Comment: Command "systemctl daemon-reload" run
     Started: 21:27:22.187555
    Duration: 638.64 ms
     Changes:
              ----------
              pid:
                  4523
              retcode:
                  0
              stderr:
              stdout:
----------
          ID: enable_Red_mergerfs
    Function: service.enabled
        Name: srv-mergerfs-Red.mount
      Result: True
     Comment: Service srv-mergerfs-Red.mount has been enabled, and is in the desired state
     Started: 21:27:22.858572
    Duration: 852.914 ms
     Changes:
              ----------
              srv-mergerfs-Red.mount:
                  True
----------
          ID: restart_Red_mergerfs
    Function: cmd.run
        Name: systemctl restart srv-mergerfs-Red.mount
      Result: False
     Comment: Command "systemctl restart srv-mergerfs-Red.mount" run
     Started: 21:27:23.712231
    Duration: 106.923 ms
     Changes:
              ----------
              pid:
                  4574
              retcode:
                  1
              stderr:
                  Job failed. See "journalctl -xe" for details.
              stdout:

Summary for debian
------------
Succeeded: 6 (changed=6)
Failed:    1
------------
Total states run:     7
Total run time:   1.781 s

OMV\ExecException: Failed to execute command 'export PATH=/bin:/sbin:/usr/bin:/usr/sbin:/usr/local/bin:/usr/local/sbin; export LANG=C.UTF-8; export LANGUAGE=; omv-salt deploy run --no-color mergerfs 2>&1' with exit code '1': debian:
----------
          ID: configure_pool_dir
    Function: file.directory
        Name: /srv/mergerfs
      Result: True
     Comment: The directory /srv/mergerfs is in the correct state
     Started: 21:27:22.001779
    Duration: 10.571 ms
     Changes:
----------
          ID: remove_mergerfs_mount_files_/etc/systemd/system
    Function: module.run
      Result: True
     Comment: file.find: ['/etc/systemd/system/srv-mergerfs-Red.mount']
     Started: 21:27:22.013482
    Duration: 8.047 ms
     Changes:
              ----------
              file.find:
                  - /etc/systemd/system/srv-mergerfs-Red.mount
----------
          ID: systemd_remove_dead_symlinks
    Function: cmd.run
        Name: find /etc/systemd/system/multi-user.target.wants -xtype l -print -delete
      Result: True
     Comment: Command "find /etc/systemd/system/multi-user.target.wants -xtype l -print -delete" run
     Started: 21:27:22.022716
    Duration: 11.907 ms
     Changes:
              ----------
              pid:
                  4501
              retcode:
                  0
              stderr:
              stdout:
                  /etc/systemd/system/multi-user.target.wants/srv-mergerfs-Red.mount
----------
          ID: configure_mergerfs_Red
    Function: file.managed
        Name: /etc/systemd/system/srv-mergerfs-Red.mount
      Result: True
     Comment: File /etc/systemd/system/srv-mergerfs-Red.mount updated
     Started: 21:27:22.035220
    Duration: 151.895 ms
     Changes:
              ----------
              diff:
                  New file
              mode:
                  0644
----------
          ID: systemd-reload_Red
    Function: cmd.run
        Name: systemctl daemon-reload
      Result: True
     Comment: Command "systemctl daemon-reload" run
     Started: 21:27:22.187555
    Duration: 638.64 ms
     Changes:
              ----------
              pid:
                  4523
              retcode:
                  0
              stderr:
              stdout:
----------
          ID: enable_Red_mergerfs
    Function: service.enabled
        Name: srv-mergerfs-Red.mount
      Result: True
     Comment: Service srv-mergerfs-Red.mount has been enabled, and is in the desired state
     Started: 21:27:22.858572
    Duration: 852.914 ms
     Changes:
              ----------
              srv-mergerfs-Red.mount:
                  True
----------
          ID: restart_Red_mergerfs
    Function: cmd.run
        Name: systemctl restart srv-mergerfs-Red.mount
      Result: False
     Comment: Command "systemctl restart srv-mergerfs-Red.mount" run
     Started: 21:27:23.712231
    Duration: 106.923 ms
     Changes:
              ----------
              pid:
                  4574
              retcode:
                  1
              stderr:
                  Job failed. See "journalctl -xe" for details.
              stdout:

Summary for debian
------------
Succeeded: 6 (changed=6)
Failed:    1
------------
Total states run:     7
Total run time:   1.781 s in /usr/share/php/openmediavault/system/process.inc:220
Stack trace:
#0 /usr/share/php/openmediavault/engine/module/serviceabstract.inc(62): OMV\System\Process->execute()
#1 /usr/share/openmediavault/engined/rpc/config.inc(174): OMV\Engine\Module\ServiceAbstract->deploy()
#2 [internal function]: Engined\Rpc\Config->applyChanges(Array, Array)
#3 /usr/share/php/openmediavault/rpc/serviceabstract.inc(123): call_user_func_array(Array, Array)
#4 /usr/share/php/openmediavault/rpc/serviceabstract.inc(149): OMV\Rpc\ServiceAbstract->callMethod('applyChanges', Array, Array)
#5 /usr/share/php/openmediavault/rpc/serviceabstract.inc(619): OMV\Rpc\ServiceAbstract->OMV\Rpc\{closure}('/tmp/bgstatusM3...', '/tmp/bgoutput1O...')
#6 /usr/share/php/openmediavault/rpc/serviceabstract.inc(159): OMV\Rpc\ServiceAbstract->execBgProc(Object(Closure))
#7 /usr/share/openmediavault/engined/rpc/config.inc(195): OMV\Rpc\ServiceAbstract->callMethodBg('applyChanges', Array, Array)
#8 [internal function]: Engined\Rpc\Config->applyChangesBg(Array, Array)
#9 /usr/share/php/openmediavault/rpc/serviceabstract.inc(123): call_user_func_array(Array, Array)
#10 /usr/share/php/openmediavault/rpc/rpc.inc(86): OMV\Rpc\ServiceAbstract->callMethod('applyChangesBg', Array, Array)
#11 /usr/sbin/omv-engined(537): OMV\Rpc\Rpc::call('Config', 'applyChangesBg', Array, Array, 1)
#12 {main}

Alles anzeigen

The dpkg -l | grep mergerfs outputs:

Code

ii  mergerfs                               2.35.1~debian-bullseye                                       amd64        another FUSE union filesystem
ii  openmediavault-mergerfs                6.3.5                                                        all          mergerfs plugin for openmediavault.

ryecoaaron · 22. April 2023

Zitat von Ranieri94

I edited the pool's option, removing "allow_other" and tried to save, but exited with the following error:

Still failing to start the pool. Post the pool's config.

Ranieri94 · 23. April 2023

These?

ryecoaaron · 23. April 2023

I guess I need to trim the whitespace off the options but until then, you need to remove the spaces around the commas in the options. use_ino is also not needed since it is a default option in mergerfs now. Once you remove the spaces, it will be removed.

ryecoaaron · 23. April 2023

trim options · OpenMediaVault-Plugin-Developers/openmediavault-mergerfs@ff86765

github.com

Ranieri94 · 24. April 2023

It worked! Pool is now working again! Thank you so much!😀

One last problem: I'm unable to write a folder through samba and FTP. I tried to reset permissions in service menu, giving everyone read/write but was unable to complete the task.

Any advice? Thanks again 💪

ryecoaaron · 24. April 2023

Zitat von Ranieri94

One last problem: I'm unable to write a folder through samba and FTP. I tried to reset permissions in service menu, giving everyone read/write but was unable to complete the task.

This should be a new thread but reset perms worked but the aquota files weren't change. This happens when you create a shared folder in the root of the filesystem. There are plenty of guides on how to setup samba that should help. ftp should just work after that too. I don't use either myself.

Ranieri94 · 24. April 2023

I understand.

Thanks again 👍👍

systemctl restart srv-mergerfs-Red.mount' with exit code '1'

Jetzt mitmachen!

Ähnliche Themen

Couldn't extract UUID (mergerfs)

mergerfs - help!

Remote mount failed to execute error

OMV in inconsistent state after MergerFS harddisk died

Trying to RemoteMount a HDD connected to a remote FritzBox

Tags