If the docker root is on ther mergerfs you should stop or better remove docker, move the docker root to a single disk and start all over.
We can help to save the containers confguration if you tell us how you are managing your containers.
If the docker root is on ther mergerfs you should stop or better remove docker, move the docker root to a single disk and start all over.
We can help to save the containers confguration if you tell us how you are managing your containers.
Docker stored in this path
/var/lib/docker
No need, I guess.
But if you pointed the docker root to mergerfs, I don't know what it will cause.
Just try to remount the mergerfs now.
Tried to remount and still same error
srv-mergerfs-Data.mount - MergerFS mount for Data
Loaded: loaded (/etc/systemd/system/srv-mergerfs-Data.mount; enabled; vend>
Active: failed (Result: exit-code) since Wed 2022-05-25 23:23:38 +03; 17s >
Where: /srv/mergerfs/Data
What: Data:67d2f591-8552-4d71-91ab-41343fa26a6c
CPU: 10ms
May 25 23:23:38 Tank systemd[1]: Mounting MergerFS mount for Data...
May 25 23:23:38 Tank mount[8002]: fuse: mountpoint is not empty
May 25 23:23:38 Tank mount[8002]: fuse: if you are sure this is safe, use the '>
May 25 23:23:38 Tank systemd[1]: srv-mergerfs-Data.mount: Mount process exited,>
May 25 23:23:38 Tank systemd[1]: srv-mergerfs-Data.mount: Failed with result 'e>
May 25 23:23:38 Tank systemd[1]: Failed to mount MergerFS mount for Data.
Alles anzeigen
OK, found the root cause in #15
May 25 22:20:34 Tank systemd[1]: Mounting MergerFS mount for Data...
May 25 22:20:34 Tank mount[609377]: fuse: mountpoint is not empty
May 25 22:20:34 Tank mount[609377]: fuse: if you are sure this is safe, use
Can you check what files are in /srv/mergerfs/Data? To me it looks like some process (probably a docker container) as written data there when the mergerfs was not mounted. So now the directory is not empty and can not be used as a mount point for fuse.
Move the files in /srv/mergerfs/Data to some other place, mount the mergerfs and try to copy the data back (if you need it).
Be carefull to decide what to do with the data and consider wisely if it is safe to merge.
Alles anzeigenOK, found the root cause in #15
CodeMay 25 22:20:34 Tank systemd[1]: Mounting MergerFS mount for Data... May 25 22:20:34 Tank mount[609377]: fuse: mountpoint is not empty May 25 22:20:34 Tank mount[609377]: fuse: if you are sure this is safe, use
Can you check what files are in /srv/mergerfs/Data? To me it looks like some process (probably a docker container) as written data there when the mergerfs was not mounted. So now the directory is not empty and can not be used as a mount point for fuse.
Move the files in /srv/mergerfs/Data to some other place, mount the mergerfs and try to copy the data back (if you need it).
Be carefull to decide what to do with the data and consider wisely if it is safe to merge.
These are the folders in /srv/mergerfs/Data
total 32
drwxr-xr-x 8 root root 4096 May 22 20:18 .
drwxrwxrwx 3 root root 4096 Apr 1 03:00 ..
drwxr-xr-x 2 root root 4096 May 22 20:18 Anime
drwxr-xr-x 10 root root 4096 May 22 20:18 Config
drwxr-xr-x 5 admin users 4096 May 22 20:18 Downloads
drwxr-xr-x 2 root root 4096 May 22 20:18 Movies
drwxr-xr-x 2 root root 4096 May 22 20:18 Music
drwxr-xr-x 2 root root 4096 May 22 20:18 Shows
How move the files from there? how can access it?
Find out how much space you need:
cd /srv/mergerfs/Data
du -h -d1 .
Do you have a preffered place where to move it?
mv /srv/mergerfs/Data/* /som/path/made/to/temporarily/store/the/data
Does this means my files are gone?
Edit:
I used this command to move Music folder at least to local folder
mv /srv/mergerfs/Data/Music/* /F:\recovery
but it says
This means yourfiles are stored elsewhere. F: is a Windws drive.
Assuming your os disk is large enough for additional 15MB:
mkdir -p /root/Backup/Data
systemctl stop docker
systemctl disable docker
systemctl mask docker
mv /srv/mergerfs/Data/* /root/Backup/Data
in the Web UI apply
ls -l /srv/mergerfs/Data
If your files are there, reboot and
systemctl unmask docker
systemctl enable docker
systemctl start docker
be happy
If one command gives an error, do not continue, but post, will be back tomorrow.
Alles anzeigenThis means yourfiles are stored elsewhere. F: is a Windws drive.
Assuming your os disk is large enough for additional 15MB:
mkdir -p /root/Backup/Data
systemctl stop docker
systemctl disable docker
systemctl mask docker
mv /srv/mergerfs/Data/* /root/Backup/Data
in the Web UI apply
ls -l /srv/mergerfs/Data
If your files are there, reboot and
systemctl unmask docker
systemctl enable docker
systemctl start docker
be happy
If one command gives an error, do not continue, but post, will be back tomorrow.
Now I can access my files, you're a life saver
didn't apply this command
in the Web UI apply
ls -l /srv/mergerfs/Data
cause I don't know how to apply it from Web UI, anyhow, I rather backup my data and do fresh install to avoid any future so if there is any advice for my fresh install then let me know, Thank you very much and thank you too Soma.
no need to do a fresh install. I meant "hit appl" in the web ui to apply pending changes. You are back to business.
Your files have never been in danger.
Backing up is always a good idea.
After restarting the server and restarting Docker the problem has returned, good thing I backed up the important files, so is it save to use mergerfs or better avoid it ? and Soma said to not use it for docker containers config files.
Post the logs again.
Did you move the containers away from the mergers?
I didn't do anything with containers, just restarted OMV and Docker, every time I restart OMV it won't run unless I press ctrl+d on keyboard ever since this issue started.
Here the logs:
Couldn't extract an UUID from the provided path '/srv/mergerfs/Data'.
OMV\Exception: Couldn't extract an UUID from the provided path '/srv/mergerfs/Data'. in /usr/share/php/openmediavault/system/filesystem/backend/mergerfs.inc:98
Stack trace:
#0 /usr/share/php/openmediavault/system/filesystem/backend/mergerfs.inc(64): OMV\System\Filesystem\Backend\Mergerfs::extractUuidFromMountPoint('/srv/mergerfs/D...')
#1 /usr/share/php/openmediavault/system/filesystem/filesystem.inc(799): OMV\System\Filesystem\Backend\Mergerfs->getImpl('/srv/mergerfs/D...')
#2 /usr/share/openmediavault/engined/rpc/filesystemmgmt.inc(182): OMV\System\Filesystem\Filesystem::getFilesystems()
#3 [internal function]: Engined\Rpc\OMVRpcServiceFileSystemMgmt->enumerateFilesystems(NULL, Array)
#4 /usr/share/php/openmediavault/rpc/serviceabstract.inc(123): call_user_func_array(Array, Array)
#5 /usr/share/openmediavault/engined/rpc/filesystemmgmt.inc(380): OMV\Rpc\ServiceAbstract->callMethod('enumerateFilesy...', NULL, Array)
#6 [internal function]: Engined\Rpc\OMVRpcServiceFileSystemMgmt->getList(Array, Array)
#7 /usr/share/php/openmediavault/rpc/serviceabstract.inc(123): call_user_func_array(Array, Array)
#8 /usr/share/php/openmediavault/rpc/serviceabstract.inc(149): OMV\Rpc\ServiceAbstract->callMethod('getList', Array, Array)
#9 /usr/share/php/openmediavault/rpc/serviceabstract.inc(588): OMV\Rpc\ServiceAbstract->OMV\Rpc\{closure}('/tmp/bgstatusz4...', '/tmp/bgoutputMH...')
#10 /usr/share/php/openmediavault/rpc/serviceabstract.inc(159): OMV\Rpc\ServiceAbstract->execBgProc(Object(Closure))
#11 /usr/share/openmediavault/engined/rpc/filesystemmgmt.inc(519): OMV\Rpc\ServiceAbstract->callMethodBg('getList', Array, Array)
#12 [internal function]: Engined\Rpc\OMVRpcServiceFileSystemMgmt->getListBg(Array, Array)
#13 /usr/share/php/openmediavault/rpc/serviceabstract.inc(123): call_user_func_array(Array, Array)
#14 /usr/share/php/openmediavault/rpc/rpc.inc(86): OMV\Rpc\ServiceAbstract->callMethod('getListBg', Array, Array)
#15 /usr/sbin/omv-engined(537): OMV\Rpc\Rpc::call('FileSystemMgmt', 'getListBg', Array, Array, 1)
#16 {main}
Alles anzeigen
lsblk
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
sda 8:0 0 3.6T 0 disk
└─sda1 8:1 0 3.6T 0 part
sdb 8:16 0 3.6T 0 disk
└─sdb1 8:17 0 3.6T 0 part /srv/dev-disk-by-uuid-cdab85bc-8264-45a9-9cd9-d
sdc 8:32 0 298.1G 0 disk
└─sdc1 8:33 0 298.1G 0 part /srv/dev-disk-by-uuid-cb4bb7d3-ee94-430f-8bd0-b
sdd 8:48 0 238.5G 0 disk
├─sdd1 8:49 0 237.5G 0 part /
├─sdd2 8:50 0 1K 0 part
└─sdd5 8:53 0 976M 0 part [SWAP]
Alles anzeigen
blkid
/dev/sdc1: UUID="cb4bb7d3-ee94-430f-8bd0-b512d24ed5b6" BLOCK_SIZE="4096" TYPE="ext4" PARTUUID="1f9a0d9b-4c6d-48a8-91e5-23764f891a8d"
/dev/sda1: UUID="ff77aeaf-2037-426d-9eca-996de8bb0b08" BLOCK_SIZE="4096" TYPE="ext4" PARTUUID="bebcd1c7-bc9a-4707-a083-898c2efc409a"
/dev/sdd1: UUID="b6081524-e919-47a2-b2e6-1e62fc51cf8e" BLOCK_SIZE="4096" TYPE="ext4" PARTUUID="7820d684-01"
/dev/sdd5: UUID="ad30def0-045f-44d4-84f6-4310f039f254" TYPE="swap" PARTUUID="7820d684-05"
/dev/sdb1: UUID="cdab85bc-8264-45a9-9cd9-d198bfa7091f" BLOCK_SIZE="4096" TYPE="ext4" PARTUUID="fd44920d-97c8-45c7-9a45-81a644cc0f33"
Btw, last thing I was doing before this happened I was transferring data from OMV to my synology, I didn't make any upgrade, not sure if I install any updates, last thing I did before a few days ago was making some Cloudflare tunnels like in this video:
I mean the logs you posted in #15. Something is happening during boot which does not allow the mergerfs to be mounted.
Here
ls -lah /srv/dev-disk-by-uuid-cb4bb7d3-ee94-430f-8bd0-b512d24ed5b6
total 40K
drwxr-xr-x 3 root root 4.0K Apr 1 03:01 .
drwxr-xr-x 9 root root 4.0K Apr 1 02:54 ..
-rw------- 1 root root 6.0K May 26 12:18 aquota.group
-rw------- 1 root root 6.0K May 26 12:18 aquota.user
drwx------ 2 root root 16K Apr 1 02:52 lost+found
ls -lah /srv/dev-disk-by-uuid-cdab85bc-8264-45a9-9cd9-d198bfa7091f
total 56K
drwxr-xr-x 7 root root 4.0K Apr 1 09:52 .
drwxr-xr-x 9 root root 4.0K Apr 1 02:54 ..
-rw------- 1 root root 7.0K May 26 12:19 aquota.group
-rw------- 1 root root 8.0K May 26 12:18 aquota.user
drwxrwsr-x 12 root users 4.0K Apr 6 16:05 Config
drwx------ 2 root root 16K Apr 1 02:50 lost+found
drwxrwsrwx 4 root users 4.0K Apr 21 19:33 Movies
drwxrwsrwx 5 root users 4.0K Apr 21 19:33 Music
drwxrwsrwx 2 root users 4.0K Apr 1 09:39 yhf
Alles anzeigen
Sie haben noch kein Benutzerkonto auf unserer Seite? Registrieren Sie sich kostenlos und nehmen Sie an unserer Community teil!