omv-extras plugins - porting progress to OMV 6.x (done)

chente · 1. Dezember 2021

Zitat von ryecoaaron

It looks like it is a 255 character limit

I guess you mean this. https://www.freedesktop.org/so…md/man/systemd.unit.html#

Zitat

Valid unit names consist of a "name prefix" and a dot and a suffix specifying the unit type. The "unit prefix" must consist of one or more valid characters (ASCII letters, digits, ":", "-", "_", ".", and "\"). The total length of the unit name including the suffix must not exceed 256 characters. The type suffix must be one of ".service", ".socket", ".device", ".mount", ".automount", ".swap", ".target", ".path", ".timer", ".slice", or ".scope".

What I don't understand is how this affects. I suppose that first the units that make up the mergerfs unit are defined, and then the mergerfs unit is mounted with these dependencies. The unit is given a name and assembly is done. The character limit should apply to the mergerfs unit name, and should not exceed this limit even if it is uuid.

So what is the reason that no more than 3 are allowed? Something escapes me...

chente · 1. Dezember 2021

Zitat

Units names can be parameterized by a single argument called the "instance name". The unit is then constructed based on a "template file" which serves as the definition of multiple services or other units. A template unit must have a single "@" at the end of the name (right before the type suffix). The name of the full unit is formed by inserting the instance name between "@" and the unit type suffix. In the unit file itself, the instance parameter may be referred to using "%i" and other specifiers, see below.

I do not know if I understand well what I read, but this paragraph after the above may help you...

ryecoaaron · 1. Dezember 2021

Zitat von chente

I guess you mean this. https://www.freedesktop.org/so…md/man/systemd.unit.html#

Actually it is this - https://www.freedesktop.org/so…md/man/systemd.mount.html

Zitat von chente

What I don't understand is how this affects. I suppose that first the units that make up the mergerfs unit are defined, and then the mergerfs unit is mounted with these dependencies. The unit is given a name and assembly is done. The character limit should apply to the mergerfs unit name, and should not exceed this limit even if it is uuid.
So what is the reason that no more than 3 are allowed? Something escapes me...

The unit name is not the problem. The "What=" line is too long when using long paths that make up the mergerfs branches. See line below. People using sharedfolders with filesystems mounted by OMV are hitting the limit because the sharedfolder's parent filesystem is mounted on an extremely long path (/srv/dev-disk-by-uuid-2d3093f0-1829-4616-9822-c4db793cdf17). Because this is so long and the sharedfolder subdirectory also gets added, about 3 branches are all that you can add before hitting the 255 character limit. If you pool other paths like I was testing with /srv/test1, /srv/test2, etc, you can have many more than 3 branches before hitting the 255 character limit.

What = /srv/dev-disk-by-uuid-2d3093f0-1829-4616-9822-c4db793cdf17/sf2/:/srv/dev-disk-by-uuid-f4986fb7-838b-41ab-bc30-cfd22e29a4d1/sf1/

ryecoaaron · 1. Dezember 2021

Zitat von chente

I do not know if I understand well what I read, but this paragraph after the above may help you...

I'm not having a problem. If people use long paths, the systemd mount file can only handle so many. There is nothing I can do to change the 255 character limit.

ryecoaaron · 1. Dezember 2021

If people name their shared folders in a similar way as show below, they can use a wildcard.

/srv/dev-disk-by-uuid-2d3093f0-1829-4616-9822-c4db793cdf17/poolfolder1

/srv/dev-disk-by-uuid-2d3093f0-1829-4616-9822-c4db793cdf17/poolfolder2

/srv/dev-disk-by-uuid-2d3093f0-1829-4616-9822-c4db793cdf17/poolfolder3

/srv/dev-disk-by-uuid-2d3093f0-1829-4616-9822-c4db793cdf17/poolfolder4

/srv/dev-disk-by-uuid-2d3093f0-1829-4616-9822-c4db793cdf17/poolfolder5

Then they could just use this one line in the plugin

/srv/dev-disk-by-uuid-2d3093f0-1829-4616-9822-c4db793cdf17/poolfolder*

chente · 1. Dezember 2021

Couldn't the plugin rename those mounts?

Or create a second montage on top of the previous one by choosing the name. Then the mergerfs unit with those dependencies is mounted. That way the original name for the user is also kept.

If I'm talking nonsense, make me shut up.

ryecoaaron · 1. Dezember 2021

Zitat von chente

Couldn't the plugin rename those mounts?

Um, that would undo everything OMV is doing.

Zitat von chente

Or create a second montage on top of the previous one by choosing the name. Then the mergerfs unit with those dependencies is mounted. That way the original name for the user is also kept.

I'm not sure what you mean.

chente · 1. Dezember 2021

Zitat von ryecoaaron

I'm not sure what you mean.

In my mind this means building a unit from each of those that make up the union. We can call this mount whatever we want and mount it in a directory that doesn't "bother". So we have all the mergerfs unit dependencies with a short name, and we can mount it with systemd.

This remains to be done, if it can be done.

ryecoaaron · 1. Dezember 2021

Zitat von chente

In my mind this means building a unit from each of those that make up the union. We can call this mount whatever we want and mount it in a directory that doesn't "bother". So we have all the mergerfs unit dependencies with a short name, and we can mount it with systemd.
This remains to be done, if it can be done.

Other than not wanting to code this, the only way I know to do this is with bind mounts. Bind mounts were how the /sharedfolder paths worked and they are disabled for a reason. The mergerfs mount still needs to have paths (not units or mounts) in it.

mkgriesinger · 1. Dezember 2021

Zitat von ryecoaaron

If people name their shared folders in a similar way as show below, they can use a wildcard.

/srv/dev-disk-by-uuid-2d3093f0-1829-4616-9822-c4db793cdf17/poolfolder1
/srv/dev-disk-by-uuid-2d3093f0-1829-4616-9822-c4db793cdf17/poolfolder2
/srv/dev-disk-by-uuid-2d3093f0-1829-4616-9822-c4db793cdf17/poolfolder3
/srv/dev-disk-by-uuid-2d3093f0-1829-4616-9822-c4db793cdf17/poolfolder4
/srv/dev-disk-by-uuid-2d3093f0-1829-4616-9822-c4db793cdf17/poolfolder5

Then they could just use this one line in the plugin

/srv/dev-disk-by-uuid-2d3093f0-1829-4616-9822-c4db793cdf17/poolfolder*

Alles anzeigen

I have the issue with >255 on a brand new install (Last night at 10pm.) The problem is all my disks are now labeled with the extremely long uuid (4 data disks). Just my 4 data disks puts the plugin over the limit.

/srv/dev-disk-by-uuid-1eb9af7f-9bc7-4e78-b5f6-9c6bad0a9194\poolshare

/srv/dev-disk-by-uuid-bb1703fc-c16b-457a-913e-16d507143158\poolshare

/srv/dev-disk-by-uuid-2665b842-0f8d-4679-8b98-7e4ad67b3822\poolshare

/srv/dev-disk-by-uuid-757c864a-9e52-4d95-a56e-dfbda4c90824\poolshare

This means people will only be able to pool 3 disks before hitting the limit.

ryecoaaron · 1. Dezember 2021

Zitat von mkgriesinger

I have the issue with >255 on a brand new install (Last night at 10pm.) The problem is all my disks are now labeled with the extremely long uuid (4 data disks). Just my 4 data disks puts the plugin over the limit.
/srv/dev-disk-by-uuid-1eb9af7f-9bc7-4e78-b5f6-9c6bad0a9194\poolshare
/srv/dev-disk-by-uuid-bb1703fc-c16b-457a-913e-16d507143158\poolshare
/srv/dev-disk-by-uuid-2665b842-0f8d-4679-8b98-7e4ad67b3822\poolshare
/srv/dev-disk-by-uuid-757c864a-9e52-4d95-a56e-dfbda4c90824\poolshare

This means people will only be able to pool 3 disks before hitting the limit.

Alles anzeigen

Yep, the last few posts have been discussing this. In your situation, you could probably just add one line of /srv/dev-disk-by-uuid-*/poolshare

mkgriesinger · 1. Dezember 2021

Zitat von ryecoaaron

Yep, the last few posts have been discussing this. In your situation, you could probably just add one line of /srv/dev-disk-by-uuid-*/poolshare

That did the trick. Thanks!

kdmn · 2. Dezember 2021

As I have 7 drives in my pool, that requires ~418 chars I would love to with "/srv/dev-disk-by-uuid-*/", but I also have 2 drives that I don't want to pool (parity drives for Snapraid). Is there a way to go with "/srv/dev-disk-by-uuid-*/" option while still excluding 2 drives? TIA

ryecoaaron · 2. Dezember 2021

Zitat von kdmn

Is there a way to go with "/srv/dev-disk-by-uuid-*/" option while still excluding 2 drives

No. You will have to create sharedfolders on each drive that you can pool with a wildcard until some solution is come up with.

kdmn · 2. Dezember 2021

So basically move everything to "/everything" and go with "/srv/dev-disk-by-uuid-*/everything"? Got it, thank you.

CrowleyAJ · 2. Dezember 2021

Zitat von kdmn

As I have 7 drives in my pool, that requires ~418 chars I would love to with "/srv/dev-disk-by-uuid-*/", but I also have 2 drives that I don't want to pool (parity drives for Snapraid). Is there a way to go with "/srv/dev-disk-by-uuid-*/" option while still excluding 2 drives? TIA

You can use character classes in square brackets:
These are my drives:

dev-disk-by-uuid-16578662-b8be-4ded-a43b-bed36de32f6b

dev-disk-by-uuid-468c061d-4943-44db-9d6e-7efad6fcad8f

dev-disk-by-uuid-b000e4b4-0947-4345-b0e3-010472ab7c5b

dev-disk-by-uuid-b31200c5-2906-4a2e-b8ed-bc428ef76da2

dev-disk-by-uuid-bc99bd8a-854c-426a-96e7-a6187ead074a

dev-disk-by-uuid-e1695f6d-370d-4ac2-aa49-bb3f9ed5bd75

If I only want to include the first four I use this expression:

/srv/dev-disk-by-uuid-[14b][603]*/

This will include all drives which UUIDs start with 1, 4 or b and have a 6, 0 or 3 in the second position, which matches the first four UUIDs, and only those.

molnart · 2. Dezember 2021

I have updated my install to 6.x and need some help with mergerFS.

- update went trough, i have noticed all my omv-extras plugins updated as well, except unionfilesystems

- so i have manually uninstall uninstalled unionfilesystems from the webui and installed mergerfs

- now my file systems page looks like this: - i cannot remove or edit the missing mergerFS mount from the web

- no shared folders are visible in the webUI

- logging in into ssh, my /srv folder contains a merged filesystem of all my disks

- on the mergerfs page in the webui i cannot do anything, as no shared folders are available for selection

next i have uninstalled mergerfs, but OMV booted into maintanance mode due to missing shares. so i have removed all the mergers related entries from fstab. then OMV started up, but my file systems page gives this error:

500 - Internal Server Error No file system backend exists for 'fuse.mergerfs'.

ryecoaaron · 2. Dezember 2021

Zitat von molnart

- update went trough, i have noticed all my omv-extras plugins updated as well, except unionfilesystems
- so i have manually uninstall uninstalled unionfilesystems from the webui and installed mergerfs
- now my file systems page looks like this: - i cannot remove or edit the missing mergerFS mount from the web

You shouldn't have removed unionfilesystems. The mergerfs plugin replaces it the correct way and will import your old pools (unless you remove the old plugin). You will need to remove the mergerfs entries from the unionfilesystem plugin from the mntent section of /etc/openmediavault/config.xml. If people insist on removing the plugin, you will need to delete the pools first.

molnart · 2. Dezember 2021

yes, it has crossed my mind that removing the unionfs plugin was a bad idea. anyhow I have removed the unionfilesystem entries and all shares from config.xml and tried to setup merger fs from scratch, now getting this error:

Code

 500 - Internal Server Error
Failed to execute command 'export PATH=/bin:/sbin:/usr/bin:/usr/sbin:/usr/local/bin:/usr/local/sbin; export LANG=C.UTF-8; omv-salt deploy run --no-color mergerfs 2>&1' with exit code '1': HP-OMV.workgroup: ---------- ID: configure_pool_dir Function: file.directory Name: /srv/mergerfs Result: True Comment: The directory /srv/mergerfs is in the correct state Started: 22:38:00.755475 Duration: 7.261 ms Changes: ---------- ID: remove_mergerfs_mount_files Function: module.run Result: True Comment: file.find: ['/etc/systemd/system/srv-mergerfs-Data\\x2dall.mount'] Started: 22:38:00.763633 Duration: 5.383 ms Changes: ---------- file.find: - /etc/systemd/system/srv-mergerfs-Data\x2dall.mount ---------- ID: configure_mergerfs_Data-all Function: file.managed Name: /etc/systemd/system/srv-mergerfs-Data\x2dall.mount Result: True Comment: File /etc/systemd/system/srv-mergerfs-Data\x2dall.mount updated Started: 22:38:00.769153 Duration: 55.637 ms Changes: ---------- diff: New file mode: 0644 ---------- ID: systemd-reload_Data-all Function: cmd.run Name: systemctl daemon-reload Result: True Comment: Command "systemctl daemon-reload" run Started: 22:38:00.825576 Duration: 371.799 ms Changes...

i have configured mergerfs like this:

i have no idea why my drives are listed by label instead of uuid in the /srv folder.... maybe this is a remnant of OMV 3.x where my system was originally set up..

ryecoaaron · 2. Dezember 2021

I think you cut off some of the message because I don't see what failed there. Have you rebooted since all of your manual fixes?

Jetzt mitmachen!