Filesystem setup

  • Dear OMV Community,


    Thanks to your help received in these last days, I understood that the RAID is not the way I have to go for my personal NAS and use. At the moment, I have 2 x 6 TB (just arrived, EXT4, just mounted in the OMV) and 2 x 4 TB (in RAID 1, which data I am copying to one drive of 6 TB). Once all the data from the RAID1 are transferred, I will remove the RAID 1 and I will mount the disks again. At the moment I still do not have any needs to pool the disks, as at the moment I have just 3.5 TB used.


    I am writing because I still have some doubt....I am reading and more I read and more I understand that I know less and less. My first plan was to create a data disk and backup with the 6tb pair and a data disk and backup with the 4tb pair. Now I have some doubt, maybe I can pool 6+4+4 and keep one 6tb for backup for important documents and files.


    Reading about snapraid and margerfs, there is something that I am not fully sure:

    1) Can I use snapraid without parity? Or it does not make any sense? I am asking because my real question is: does snapraid has some kind of advantage over the "normal" mounted disks? Advantage like checksum, more disk/data protection, more speed, more reliability... I could not understand from the snapraid site

    2) if sanpraid has some advantage over normal mounted disk, does it make any sense to to have one disk for data in snapraid and one normal disk for backup (just mounted)?

    3) Reading some example of snapraid (which I could misunderstood) I saw for example 3 disks of 6 tb for data and one disk of 6 tb for parity to restore the data in case any one of the three disks is gone? Did I understood well? How 6 tb of parity can restore any of 6tb data? If this is true, how big is the capacity of the data pool? 6+6+6=18tb?

    4) Should I use this command to format my disks: mkfs.ext4 -m 0 -T largefile4 -L LabelXYZ /dev/sdX1? I just create the EXT4 in OMV by GUI, something wrong with this?

    5) I still have enough space, so I do not need to merge disks and folder at the moment. However I was wondering if the problem with the disappeared directories in the sharefolders when using mergerfs is sorted out? Or still mounting MergerFS shares at startup by commenting out the AssertPathIsDirectory is the best solution?

    6) Where can I find this file to edit (to solve the disappeared shared directories issue)?

    7) Should I use mergerFS by GUI or from the SSH by command? Any disadvantage with GUI?


    My apologies for the long thread and too many questions. Hopefully you can find the time to read and help.


    Thank you Guys!!!

  • ^^ thread too long or silly questions ^^


    To make shorter:


    Is snapraid to prefer to normal data disk and backup, considering checksum and scrubbing? Is it giving any advantage over the normal data disk and backup?


    Furthermore how parity space works? If I have 3 data disks of 6+4+4tb (14tb), is 6 tb of parity enough? The parity is like a backup or it take less space than normal backup with equal amount of data?


    Thanks

  • Thank you!!! I use my server for media (photos and music and movies). Furthermore I keep software's and documents important for me. Documents are a lot as they are personal documents and work documents. At the moment I have 4 tb used. Furthermore I use a lot docker for my containers. One of my containers is qBittorrent to download files. I do not know if it is an important info but I have 16 gb of ECC ram. Thanks for the support!!


  • I use my server for media (photos and music and movies)

    This you could use mergerfs and snapraid for

    Furthermore I keep software's and documents important for me

    This not so much.


    Looking at your your available storage with the 2x6Tb and 2x4Tb I would stick with using one of each for data with the second as a backup to the first using rsync.


    This would take some time to set this up especially running rsync for the first time, the other downside is your going to have resolve your dockers to point to their new share/mount point, in essence you'll need to reconfigure most of your shares within the GUI.

    Raid is not a backup! Would you go skydiving without a parachute?

  • For the new docker point position I should have clear what to do. I think I can manage it without any issue.

    At the moment rsync is close to finish its job. I completed the backup from the raid to one 6 tb disk and I am running the second back up from the raid to the second 6tb disk. So I am close to wipe the 2 x 4tb disks.


    I would like to summarize to see if I understood properly what you kindly suggested:

    1) marge 6+4tb to use for data and marge 6+4tb for the backup

    2) In order to marge and manage this setup, I need to use snapraid and mergerfs

    3) Use rysinc to continue to back up the data


    Is snapraid a better solution than normal data disk + backup for the safety of the data and disks? For safety I mean checksum, more disk/data protection, more speed, more reliability, data integrity and protaction from silent corruption.


    Thank you one more time!!!

  • You're not merging anything,


    1x6TB as data rsync to 1x6TB as your backup

    1x4TB as data rsync to 1x4TB as your backup


    for you this is the simplest option


    To do what you are talking about;


    2x4TB to create a mergerfs pool, you then add those 2x4TB to snapraid + 1x6TB as the parity drive, then 1x6TB setup to rsync as a backup.

    Raid is not a backup! Would you go skydiving without a parachute?

  • All right, I think the best option is to go with a simple mounted disk, in order to not lose too much space. I am planning to do the secono backup now and than with an external disk. If I really need to join the 2 disks in order to map only one partition (to make easier to find the files), I could still use mergerfs, isn't it?


    My only concern is the safety of the data, as for checksum, more reliability (if any), data integrity and protection from silent corruption that normal mounted disks do not have. Is this a critical aspect or is this just a fear of mine without any reason? I cannot understand this part.

  • Is this a critical aspect or is this just a fear of mine without any reason? I cannot understand this part.

    I've had server/s now for over 2 decades, yes I've had to replace drives, I've never experienced file corruption, errors etc, probably because I have always used SMART to keep an eye on the integrity of my drives.

    As I don't have issues with power cuts I don't use a UPS, file corruption occurs when something is being written to drive when a power cut happens.

    But you could backup your backups to an external drive.

    Raid is not a backup! Would you go skydiving without a parachute?

  • Thanks for the good news :)!!!!


    I have started to use SMART just from the last week. I just mounted the HDDs in SMART, I enabled it and I created scheduled tests. Are you checking the SMART by scheduled tests report or in logs->SMART. What exactly should I check? There are so many info in the reports that I cannot understand what I have to look at? What is you best strategy, once you see something which you do not like, do you replace the HDD and you drop it in the bin? I am sorry for all my questions but I would like to learn to make prevention and avoid to cry in the future, you know what I mean. I know that I always have the backups but I would like to be more active. Thanks for your kind patience!!!

  • This is exactly what I would do.

    Air Conditioners are a lot like PC's... They work great until you open Windows.


  • The output of 5, 187 and 198, those three will alert you to a potential failure, you can also install scrunity in docker and add it to heimdall interface.

    Interesting. I always wondered if any of those were more important than the other, and never thought to ask.


    5 has shown pre-fail on my drives for quite a while... 187/198 have both only showed "old-age" for some time.. which I know. (virtually all of mine are "old-age", as the drives are several years old)

    Air Conditioners are a lot like PC's... They work great until you open Windows.


  • Interesting. I always wondered if any of those were more important than the other, and never thought to ask.

    There's an excellent article here note the 187 reference from Backblaze, it states that if that goes above 0 the drive gets scheduled for replacement

    Raid is not a backup! Would you go skydiving without a parachute?

  • Thank you Geaves, As you suggested I have installed scrutiny, in fact it is much easier to to read to me than SMART in OMV.


    I have read the link uploaded above and it seems to reflect what I see for my drivers (pictures attached).


    WD disks seems to be good. Seagate ironwolf seems to have some errors, but based on the article it should be not real errors. Please, could you confirm? Are my disks seagate in good shape for you?


    As general attitude that I need to keep, is there any way too see what file was corrupted, I am afraid that I could back up a corrupted file with rsync and so corrupt a good file in the back up hard disk.


    Thanks!!

  • Please, could you confirm? Are my disks seagate in good shape for you?

    They're fine, I have 2 Seagates both show a warning and a failed (3 & 7) nothing to worry about, as from that link 187 is the one to keep an eye on.

    As general attitude that I need to keep, is there any way too see what file was corrupted, I am afraid that I could back up a corrupted file with rsync and so corrupt a good file in the back up hard disk

    Once an initial rsync is run, rsync will check a file to see if it has been changed before updating the destination, I would assume if a file is corrupt i.e. cannot be opened from the data drive then rsync will not update the destination. You could in the extra options add --checksum or -c but I never have.

    Raid is not a backup! Would you go skydiving without a parachute?

  • Thanks you again. I am using -c. If rsync will not copy corrupted files to the destination, it is a really great news to me :). It is just an assumption or it is a real fact?

  • Thanks you again. I am using -c. If rsync will not copy corrupted files to the destination, it is a really great news to me :). It is just an assumption or it is a real fact?

    If you use -c, rsync will copy a file if the checksum does NOT match. So if the file is corrupted on the source, it will copy the file because the checksum does not match.

    One use case for -c is to check if a first sync was completed without issues.

    So first run without -c.

    Second run with -c. At the second run, NO file should be copied. If that is the case, the first run completed without issues.

Participate now!

Don’t have an account yet? Register yourself now and be a part of our community!