Looking to extend the size of a File System

    • OMV 4.x
    • Looking to extend the size of a File System

      Thanks in advance for the advice. I've read some threads on this, but none that directly mirror my situation, so I figured I would ask.

      First the hardware:

      R510 running OMV all by itself.It has an H200 so it see's larger drives.

      It has 3 146gb dell drives in a software RAID in OMV.

      It has had an 8TB WD (shucked easystore) for a while. And its got basically all of our "stuff" on it.
      I was using another 8tb easystore still in the enclosure for backup and cold storage.

      I picked up a 10tb easystore now for cold storage, so I shucked the other 8tb drive and put it in the r510.


      I want to extend the current 8tb to a full 16tb by combining the two drives. It would be great if I can do this without loosing any of the data on the primary drive. But if I need to I could back it all up and then write it back again.
    • I don't quite understand "what" is "where" however:

      You can combine two drives using UnionFS, without disturbing their contents. You should investigate the differences between the storage policies, used by UnionFS. The primary 2 polices for same sized drives are, "Existing Path, most free space" or "Most free space". The former is the easiest to back out of but, given the policy, it's possible to completely fill the first drive before using the second drive. The later (most free space) distributes files evenly, from the point where the policy is implemented, across the 2 drives. Since files are split evenly between the member drives, this policy tends to commit one to the use of UnionFS

      While the individual drives can be accessed direct at their mount points, after the drives are in the "Union", you should use the UnionFs mount point only.
      _____________________________________

      Since you're reordering your storage, perhaps you might give UnionFS+SNAPRAID a look. There are a lot of benefits when compared to an mdadm (software) RAID array.

      Video Guides :!: New User Guide :!: Docker Guides :!: Pi-hole in Docker
      Good backup takes the "drama" out of computing.
      ____________________________________
      Primary: OMV 3.0.99, ThinkServer TS140, 12GB ECC, 32GB USB boot, 4TB+4TB zmirror, 3TB client backup.
      OMV 4.1.13, Intel Server SC5650HCBRP, 32GB ECC, 16GB USB boot, UnionFS+SNAPRAID
      Backup: OMV 4.1.9, Acer RC-111, 4GB, 32GB USB boot, 3TB+3TB zmirror, 4TB Rsync'ed disk

      The post was edited 2 times, last by flmaxey: edit2 ().

    • flmaxey wrote:

      I don't quite understand "what" is "where" however:

      You can combine two drives using UnionFS, without disturbing their contents. You should investigate the differences between the storage policies, used by UnionFS. The primary 2 polices for same sized drives are, "Existing Path, most free space" or "Most free space". The former is the easiest to back out of but, given the policy, it's possible to completely fill the first drive before using the second drive. The later (most free space) distributes files evenly, from the point where the policy is implement, across the 2 drives. Since files are split evenly between the member drives, this policy tends to commit one to the use of UnionFS

      While the individual drives can be accessed direct at their mount points, after the drives are in the "Union", you should use the UnionFs mount point only.
      _____________________________________

      Since you're reordering your storage, perhaps you might give UnionFS+SNAPRAID a look. There are a lot of benefits when compared to and mdadm (software) RAID array.


      Thanks for the help!

      So my action plan is to:

      Format the second 8tb drive.
      Install the UnionFS plugin
      Install the SnapRAID plugin

      Looks like for UnionFS I need to create a filesystem and mount the second drive.

      Do I do SnapRAID on the two drives first? Or UnionFS them together, and then SnapRAID them as one?
    • UnionFs requires 2 drives (obviously - it wouldn't be needed otherwise). And SNAPRAID requires another, or a 3rd drive, for parity. So, a UnionFS+SNAPRAID setup requires a total of 3 disks. Also, a hard requirement is that the SNAPRAID drive must be the largest in the array. This is necessary for the parity calculation.

      When setting up SNAPRAID note that both of your data drives should have a "content file". A good copy of the content file is required for recovery. (In the event of a data drive failure, to insure a good copy is available, the content file would need to be on both data drives).
      ____________________________________________________________________

      The freedom you'll have, on the other hand, is that SNAPRAID can be added or removed at anytime with no impact to UnionFS' data drives. (But, with no protection, you'd be taking a substantial risk...)

      Take a look at the SNAPRAID page for further details.

      Video Guides :!: New User Guide :!: Docker Guides :!: Pi-hole in Docker
      Good backup takes the "drama" out of computing.
      ____________________________________
      Primary: OMV 3.0.99, ThinkServer TS140, 12GB ECC, 32GB USB boot, 4TB+4TB zmirror, 3TB client backup.
      OMV 4.1.13, Intel Server SC5650HCBRP, 32GB ECC, 16GB USB boot, UnionFS+SNAPRAID
      Backup: OMV 4.1.9, Acer RC-111, 4GB, 32GB USB boot, 3TB+3TB zmirror, 4TB Rsync'ed disk

      The post was edited 1 time, last by flmaxey: edit ().

    • flmaxey wrote:

      UnionFs requires 2 drives (obviously - it wouldn't be needed otherwise). And SNAPRAID requires another, or a 3rd drive, for parity. So, a UnionFS+SNAPRAID setup requires a total of 3 disks. Also, a hard requirement is that the SNAPRAID drive must be the largest in the array. This is necessary for the parity calculation.

      When setting up SNAPRAID note that both of your data drives should have a "content file". A good copy of the content file is required for recovery. (In the event of a data drive failure, to insure a good copy is available, the content file would need to be on both data drives).
      ____________________________________________________________________

      The freedom you'll have, on the other hand, is that SNAPRAID can be added or removed at anytime with no impact to UnionFS' data drives. (But, with no protection, you'd be taking a substantial risk...)

      Take a look at the SNAPRAID page for further details.
      Thank you for the advice and the link to the Doc. I'll get reading.
    • flmaxey wrote:

      UnionFs requires 2 drives (obviously - it wouldn't be needed otherwise). And SNAPRAID requires another, or a 3rd drive, for parity. So, a UnionFS+SNAPRAID setup requires a total of 3 disks. Also, a hard requirement is that the SNAPRAID drive must be the largest in the array. This is necessary for the parity calculation.

      When setting up SNAPRAID note that both of your data drives should have a "content file". A good copy of the content file is required for recovery. (In the event of a data drive failure, to insure a good copy is available, the content file would need to be on both data drives).
      ____________________________________________________________________

      The freedom you'll have, on the other hand, is that SNAPRAID can be added or removed at anytime with no impact to UnionFS' data drives. (But, with no protection, you'd be taking a substantial risk...)

      Take a look at the SNAPRAID page for further details.


      I have a question. Here is what I did.


      Format the second 8tb drive
      Create a new Filesystem for the second drive
      Install the UnionFS plugin
      Merge the two file systems to create one large file system
      Enable the new file system as a SMB share, and remove the old single 8tb drive share.

      I connected to the smb version of the new share folder and it connects fine. Shows the full file size, and the used portion of the first drive.

      But when I open the folder there are no folders/files inside of it.
    • esteid wrote:

      I connected to the smb version of the new share folder and it connects fine. Shows the full file size, and the used portion of the first drive.

      But when I open the folder there are no folders/files inside of it.
      Using SMB to put a "new shared folder" on the network is not going to put data in it.

      Are you sure you're using the right mount point?

      The following (in the red box) is my mount point for Union1. (The union consists of the 2 drives in the green box. And, while off this topic, the parity drive is for SNAPRAID.)


      _________________________________________________________________________________________________________

      If you want to create a new shared folder for existing data, the following is the process.

      1. Add a new shared folder
      2. Name the share
      In "Device" - use the Label of your combined Union drive. (In my case, it's Union1)
      3. Click on the folder Icon.
      4. In the pop-up, navigate to the appropriate folder and highlight it. (While it doesn't show here, if it's highlighted, the folder will have a yellow background.)
      Click OK.
      5. The path to the folder selected should transfer to Path box. (Ignore what you see in this example - I didn't actually make a selection.)
      Save the Shared folder.



      Now, you can go to SMB/CIF and put the new share (with old data) on the network.

      Video Guides :!: New User Guide :!: Docker Guides :!: Pi-hole in Docker
      Good backup takes the "drama" out of computing.
      ____________________________________
      Primary: OMV 3.0.99, ThinkServer TS140, 12GB ECC, 32GB USB boot, 4TB+4TB zmirror, 3TB client backup.
      OMV 4.1.13, Intel Server SC5650HCBRP, 32GB ECC, 16GB USB boot, UnionFS+SNAPRAID
      Backup: OMV 4.1.9, Acer RC-111, 4GB, 32GB USB boot, 3TB+3TB zmirror, 4TB Rsync'ed disk

      The post was edited 1 time, last by flmaxey: edit ().

    • flmaxey wrote:

      esteid wrote:

      I connected to the smb version of the new share folder and it connects fine. Shows the full file size, and the used portion of the first drive.

      But when I open the folder there are no folders/files inside of it.
      Using SMB to put a "new shared folder" on the network is not going to put data in it.
      Are you sure you're using the right mount point?



      /snipped

      Now, you can go to SMB/CIF and put the new share (with old data) on the network.

      It was my understanding that using UnionFS was going to create one large file system which included my old data and made room for new data. I expected that the old folders would display inside the new folder I create from that file system.

      You've helped me create a larger share, and retain my data, but now I have two mount points. One old, One new.
      Maybe the only way to have everything I want is to remove the data and re-write it on a new joined drive.
    • esteid wrote:

      It was my understanding that using UnionFS was going to create one large file system which included my old data and made room for new data.
      That's exactly what it did. But it didn't erase or otherwise remove/hide the old mount points.

      esteid wrote:

      I expected that the old folders would display inside the new folder I create from that file system.
      OK, it's about definitions. A "folder" is a directory - pure and simple. A "shared folder", in OMV, is nothing more changing the permissions on a given folder from root:root (read/write) to something else which includes, and allows, user access root:users (read/write). AND the shared folder process adds the folder to a database for OMV's management purposes. And SMB to the shared folder puts that folder on the network, again, with permissions that allow it to be accessed on a LAN.

      When you merged two drives with UnionFS, all of your old folders are there along with the extra space on the drive under a new common mount point "Union1" (or whatever you named it). But, again, the Union did not erase or hide the original drive mount points. It simply created an new, single, mount point for two different drives.
      ______________________________________________________________

      If you had a functioning OMV server before hand, it is also possible to "repoint" a shared folder from the original drive, to the Union drive, by editing the shared folder and changing the device (to the Union). But that's another topic.

      esteid wrote:

      Maybe the only way to have everything I want is to remove the data and re-write it on a new joined drive.
      Everything that was on the drive with data before, is on the joined drive, with the extra space of the second drive added. It's just a matter of either re-pointing your old shared folders to the new Union device (where the associated SMB network shares will follow),,, or creating new shared folders and using SMB to put them on the network.

      But you're free to do what you want to do. Sometimes, doing things one's own way makes it easier.
      ______________________________________________________________

      Have you heard of WinSCP? Install it on a Windows client and SSH into your server. You can use it to look over the filesystem, the data drives, and the new Union drive in a manner that's familiar. Maybe that will help you to come to a decision. Also, when it comes to other basic process, useful utilities, etc., maybe there's something in this Guide you'll find useful.

      Regards

      Video Guides :!: New User Guide :!: Docker Guides :!: Pi-hole in Docker
      Good backup takes the "drama" out of computing.
      ____________________________________
      Primary: OMV 3.0.99, ThinkServer TS140, 12GB ECC, 32GB USB boot, 4TB+4TB zmirror, 3TB client backup.
      OMV 4.1.13, Intel Server SC5650HCBRP, 32GB ECC, 16GB USB boot, UnionFS+SNAPRAID
      Backup: OMV 4.1.9, Acer RC-111, 4GB, 32GB USB boot, 3TB+3TB zmirror, 4TB Rsync'ed disk
    • Hi,
      I think I am having the same questions. May I ask if I understand correctly?
      Here I go:

      I have 1 discs in my setup.
      This is running out of space.
      So I put in another disc.
      I format the new disc.
      I install the UnionFS-Plugin.
      I join the old and the new disc together.
      I have one big filesystem with my old data.
      I delete the old SMB/CIF share and create a new one pointing to the UnionFS filesystem.
      I end up with the same thing as before just with more space.

      After all this I may ad another drive to the system and create a Snapraid with the 2 old HDDs (together in a unionFS).
      The 3rd drive should act as a parity drive. As I know this should be the largest drive in the setup. But should it be the largest of the 3 or as large as the 2 old ones together as a union?

      Ok, I just found this...
      How to Setup Snapraid
      This explains my last question ;) Thanks @flmaxey!!!

      The last thing I am unsure is the thing with the content files of snapraid. Do I have to create them? Some explanation please...or snapraid doing this by itself?


      Thanks and sorry for entering this thread.

      Best.

      Fabian
      MoBo: Fujitsu D3417-B
      CPU: Intel Celeron G3900
      RAM: Samsung DDR4 - 8 GB ECC
      Case: Fractal Design Node 80
      HDD: SSD 64GB (System), 2*3TB (Data)
      OMV 4
    • FFrank wrote:

      But should it be the largest of the 3 or as large as the 2 old ones together as a union?
      SNAPRAID is unaware of the drive Union. The parity calculation is done against physical drives so the parity drive should be the largest of the 3. (When selecting drives to add to SNAPRAID when setting it up, select physical drives only. (Do not select the Union.)


      You have it right. The only change you'll need to make when adding a drive is selecting the UnionFS storage "policy". The default policy is "Existing Path, Most Free Space". Since all existing paths currently lead to your filled drive, you'd want all new data to go to the empty drive so you'll need to change the policy to "Most free space".

      FFrank wrote:

      I delete the old SMB/CIF share and create a new one pointing to the UnionFS filesystem.
      Actually, you can "edit" your existing shared folders. Open any of your shared folders (on the data drive that's now inside the Union). In the dialog box; on the Device Line, click the drop down arrow and select the UnionFS device and save it. The Samba share and other services, layered on the shared folder, will follow it to the Union drive. (This saves a lot of work on settings, permissions, etc.)

      FFrank wrote:

      The last thing I am unsure is the thing with the content files of snapraid. Do I have to create them? Some explanation please...or snapraid doing this by itself?
      You don't have to create content files. SNAPRAID will do that during the first SYNC. You're simply telling SNAPRAID where to store them. You'll need at least two copies to recover, in the event of a drive failure, which means enabling "content" on at least two data drives.

      While you can do this manually in the GUI, I would set up a Scheduled Job, with a command line of: snapraid sync
      I'd set it to run after-hours, because the operation consumes CPU resources. An interval of once a week or so should be fine. Your call.

      Check out the SNAPRAID page for info on what the various operations do.

      Video Guides :!: New User Guide :!: Docker Guides :!: Pi-hole in Docker
      Good backup takes the "drama" out of computing.
      ____________________________________
      Primary: OMV 3.0.99, ThinkServer TS140, 12GB ECC, 32GB USB boot, 4TB+4TB zmirror, 3TB client backup.
      OMV 4.1.13, Intel Server SC5650HCBRP, 32GB ECC, 16GB USB boot, UnionFS+SNAPRAID
      Backup: OMV 4.1.9, Acer RC-111, 4GB, 32GB USB boot, 3TB+3TB zmirror, 4TB Rsync'ed disk

      The post was edited 1 time, last by flmaxey: edit ().